✨ Add find-project-anomalies skill and helper

Check a milestone against the Main project board with
scripts/project-anomalies.py (check writes
tmp/<MILESTONE>-ANOMALIES.md): open issues with merged PRs,
milestone mismatches either way, needs triage labels and
unassigned non-community issues. Outsiders resolve in bulk;
only a missing milestone auto-fixes, on confirmation.

Teach scripts/gh.py batched issue lookup (issue subcommand),
milestone on prs output, plus assignees and projects fields,
and retry transient HTTP 504s.

AI-assisted-by: muse-spark-1.3

.agents/README.md

@@ -508,6 +508,7 @@ JSON, REPL access, and so on.
 | [`ste`](skills/ste/SKILL.md) | Rewrites prose in Simplified Technical English. Loads only when you name it. |
 | [`refine-prompt`](skills/refine-prompt/SKILL.md) | Rewrites a rough prompt into a clearer one. Never runs the prompt. |
 | [`update-changelog`](skills/update-changelog/SKILL.md) | Regenerates `CHANGES.md` from a GitHub milestone. |
+| [`find-project-anomalies`](skills/find-project-anomalies/SKILL.md) | Checks a milestone against the Main project board and reports anomalies to `tmp/<MILESTONE>-ANOMALIES.md`. |
 
 ### A typical round
 

.agents/skills/find-project-anomalies/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: find-project-anomalies
+description: Check a GitHub milestone against the Main project board, report the five anomaly types to tmp/<MILESTONE>-ANOMALIES.md, and fix missing milestone assignments on request.
+---
+
+# Skill: find-project-anomalies
+
+Check every issue and PR in a given GitHub milestone against the Main
+project board. Report the five anomaly types below, then fix the ones a
+human confirms.
+
+## Getting the version ($ARGUMENTS)
+
+The milestone version comes from the skill arguments (`$ARGUMENTS`), the
+same way other skills receive theirs. Accept `2.17.0`, `v2.17.0`,
+`milestone 2.17.0`, or any text clearly containing one `X.Y.Z` version.
+
+If no version is given and none can be deduced from the arguments or the
+conversation context, **stop and ask the user** which milestone to check.
+Do not guess, do not default to the latest release.
+
+## When to Use
+
+- Before a release, to catch board/milestone drift while there is still time
+  to fix it
+- After triage sessions, to verify every milestone item is tracked, owned,
+  and correctly assigned
+- Whenever someone asks "is milestone X.Y.Z clean on Main?"
+
+## Prerequisites
+
+- `gh` CLI authenticated (`gh auth status`)
+- Python 3.8+
+- `scripts/gh.py` for GitHub queries, `scripts/project-anomalies.py` for
+  detection and reporting (`python3 scripts/project-anomalies.py check --help`)
+
+## Workflow
+
+### 1. Resolve the version (see above), then run the check
+
+```bash
+python3 scripts/project-anomalies.py check "2.17.0"
+# explicit path:
+python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md
+```
+
+This fetches milestone issues and PRs in bulk (one GraphQL call per 50
+items), resolves outsiders the same way, and overwrites
+`tmp/<MILESTONE>-ANOMALIES.md`. Every number renders as a full
+`[#N](https://github.com/penpot/penpot/issues/N)` or
+`[#N](https://github.com/penpot/penpot/pull/N)` link.
+
+### 2. Read the report and judge each anomaly
+
+The five types, in report order:
+
+1. **OPEN issue with a MERGED PR** — the fix landed but the issue never
+   closed (or it reopened). Fix: close the issue, or move it out of the
+   milestone if the fix did not actually land here.
+2. **Milestone issue, PR elsewhere or issue off Main** — a *merged* closing
+   PR in another milestone (or none), or the issue itself not on the Main
+   board. An unmerged PR's milestone means nothing (it landed nowhere), so
+   it never triggers this type. Fix: align the milestones (see step 3), or
+   add the issue to Main.
+3. **Milestone PR, issue elsewhere or off Main** — the mirror view over
+   *merged* PRs only: a PR released here closes an issue tracked elsewhere
+   (or nowhere on Main). Unmerged PRs never trigger this type — their
+   milestone means nothing yet. Fix: align the milestones (see step 3), or
+   add the issue to Main.
+4. **MERGED PR on a `needs triage` issue** — landed without triage. Only a
+   human can triage; never auto-remove the label.
+5. **MERGED PR on an unassigned, non-community issue** — no owner and no
+   `community contribution` label (checked on both issue and PR). Either
+   assign an owner or confirm it is community work. Never invent an owner.
+
+Scope notes (deliberate, not bugs):
+- Project membership is checked **on the issue side only**. PRs are not the
+  unit tracked on Main, so an unprojected PR alone is never an anomaly.
+- An issue with *no* milestone closed by a milestone PR **is** reported
+  here (type 3): unlike the changelog flow, this pairing always needs a
+  human look.
+
+### 3. Fix missing milestone assignments (only these, only on confirmation)
+
+The only mechanical fix in this skill is assigning a missing milestone.
+The report already prints the exact command next to each such case:
+
+```bash
+gh pr edit <PR> --milestone "<MILESTONE>"
+gh issue edit <ISSUE> --milestone "<MILESTONE>"
+```
+
+Rules:
+- Apply **only** when one side lacks a milestone. When both sides have
+  *different* milestones, a human decides which one moves — never pick a
+  side yourself.
+- Confirm **each fix (or each small batch)** with the user before running
+  it. Read back what the command will change.
+- Everything else (closing issues, triaging, assigning owners, adding
+  items to the Main board) is human work: point at it, do not do it.
+
+### 4. Re-run until clean
+
+After fixes land, re-run step 1 and confirm the report shows zero
+anomalies. The `tmp/` report is scratch output (gitignored): quote or
+paste entries into chat when reporting back, do not commit it.
+
+## Key Principles
+
+- **No version, no run.** Stop and ask when the milestone is unknown.
+- **Report first, touch nothing.** No board or milestone changes without
+  explicit per-item confirmation.
+- **Milestone gaps auto-fix; judgment calls do not.** Only a missing
+  milestone is mechanical. triage, ownership, and close/reopen decisions
+  belong to humans.
+- **Every number clickable.** The report is useless without full GitHub links.

.gitignore

@@ -108,5 +108,6 @@ opencode.json
 /.opencode/reports
 /.opencode/prompts
 /.ci-logs
+/tmp/
 /.codex/
 /tools/__pycache__

.serena/memories/critical-info.md

@@ -55,6 +55,7 @@ The memory is structured in a way that you can get the critical information abou
 - `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`.
 - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`.
 - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`.
+- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`.
 - `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`.
 - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`.
 - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `<repo>/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`.

.serena/memories/scripts/gh.md

@@ -49,6 +49,8 @@ python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md
 - Issues with type "Task" are excluded (`--include-tasks` to keep them).
 - Issues with "Rejected" project status are excluded (`--include-rejected` to keep them).
 
+Every issue entry carries `assignees` (logins) and `projects` (board titles); every PR entry carries `assignees`.
+
 **Output**: JSON array to stdout; progress to stderr.
 
 ### `prs`

.serena/memories/scripts/project-anomalies.md

@@ -0,0 +1,17 @@
+# Project anomalies helper
+
+`scripts/project-anomalies.py` implements the `find-project-anomalies` skill: `check <MILESTONE>` fetches milestone issues/PRs with `--state all` via `scripts/gh.py`, resolves outsiders in bulk (one GraphQL call per 50 items), and writes `tmp/<MILESTONE>-ANOMALIES.md` (gitignored scratch output, every number a full GitHub link).
+
+## Anomaly types (all scoped to the milestone)
+
+- Open-with-merged-PR: OPEN issue with a MERGED closing PR (close it or move it out).
+- Issue-PR mismatch: milestone issue whose *merged* closing PR is elsewhere/nowhere (an unmerged PR's milestone is irrelevant), or issue off the Main board.
+- PR-issue mismatch: *merged* milestone PR closing an issue elsewhere/nowhere (milestone-less issues ARE reported here, unlike the changelog flow), or off the Main board. Unmerged PRs never count anywhere: their milestone is meaningless.
+- Needs-triage: MERGED PR on a `needs triage` issue (human triage only, never auto-unlabel).
+- Unassigned: MERGED PR on an issue with no assignees and no `community contribution` label on either side (valid states: assigned, or unassigned+community).
+
+## Conventions
+
+- Project membership is checked on the issue side only (`projects` list from `gh.py`); PRs are not the Main-tracked unit, so an unprojected PR alone is never an anomaly.
+- Detection (`find_anomalies`) and rendering (`render_report`) are pure functions over pre-resolved dicts; outsiders must be merged into the lookup maps first (missing numbers are skipped, never flagged).
+- The only mechanical fix is assigning a missing milestone (`gh pr/issue edit --milestone`); everything else is human judgment with per-item confirmation.

scripts/gh.py

@@ -319,6 +319,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) {
               state
               issueType { name }
               labels(first: 20) { nodes { name } }
+              assignees(first: 10) { nodes { login } }
               closedByPullRequestsReferences(first: 5) { nodes { number } }
               projectItems(first: 10) {
                 nodes {
@@ -357,6 +358,7 @@ query($query: String!, $cursor: String) {
         milestone { title }
         issueType { name }
         labels(first: 20) { nodes { name } }
+        assignees(first: 10) { nodes { login } }
         closedByPullRequestsReferences(first: 5) { nodes { number } }
         projectItems(first: 10) {
           nodes {
@@ -375,6 +377,32 @@ query($query: String!, $cursor: String) {
 """
 
 
+def node_assignees(node: dict) -> list[str]:
+    """Extract assignee logins from a GraphQL issue/PR node."""
+    return [
+        a["login"] for a in (node.get("assignees") or {}).get("nodes") or []
+    ]
+
+
+def node_projects(node: dict) -> list[str]:
+    """Extract project board titles from a GraphQL issue node."""
+    return [
+        (pi.get("project") or {}).get("title")
+        for pi in (node.get("projectItems") or {}).get("nodes") or []
+        if (pi.get("project") or {}).get("title")
+    ]
+
+
+def node_main_status(node: dict) -> str | None:
+    """Extract the "Main" project board status from a GraphQL issue node."""
+    for pi in (node.get("projectItems") or {}).get("nodes") or []:
+        project = pi.get("project") or {}
+        if project.get("title") == "Main":
+            status_field = pi.get("fieldValueByName") or {}
+            return status_field.get("name")
+    return None
+
+
 def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[dict]:
     """
     Fetch all issues that belong to NO milestone via paginated GraphQL search.
@@ -384,7 +412,7 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di
         labels: optional comma-separated labels to include (built into the search query)
 
     Returns:
-        List of {number, title, state, milestone, issue_type, labels, closing_prs, project_status}
+        List of {number, title, state, milestone, issue_type, labels, assignees, closing_prs, project_status, projects}
     """
     all_nodes: list[dict] = []
     cursor: str | None = None
@@ -415,13 +443,6 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di
                 continue
             issue_type = node.get("issueType")
             ms = node.get("milestone")
-            project_status = None
-            for pi in (node.get("projectItems") or {}).get("nodes") or []:
-                project = pi.get("project") or {}
-                if project.get("title") == "Main":
-                    status_field = pi.get("fieldValueByName") or {}
-                    project_status = status_field.get("name")
-                    break
             all_nodes.append({
                 "number": node["number"],
                 "title": node["title"],
@@ -429,8 +450,10 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di
                 "milestone": ms["title"] if ms else None,
                 "issue_type": issue_type["name"] if issue_type else None,
                 "labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
+                "assignees": node_assignees(node),
                 "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]],
-                "project_status": project_status,
+                "project_status": node_main_status(node),
+                "projects": node_projects(node),
             })
 
         total = len(all_nodes)
@@ -452,7 +475,7 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]:
         states: GraphQL states enum array literal, e.g. ``"[CLOSED]"`` or ``"[OPEN CLOSED]"``
 
     Returns:
-        List of {number, title, state, issue_type: str|None, labels: [str], closing_prs: [int]}
+        List of {number, title, state, issue_type: str|None, labels: [str], assignees: [str], closing_prs: [int], project_status, projects}
     """
     query = GQL_ISSUES_QUERY.replace("__STATES__", states)
     all_nodes: list[dict] = []
@@ -473,22 +496,16 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]:
             if node is None:
                 continue
             issue_type = node.get("issueType")
-            # Extract project status from the "Main" project board (if present)
-            project_status = None
-            for pi in (node.get("projectItems") or {}).get("nodes") or []:
-                project = pi.get("project") or {}
-                if project.get("title") == "Main":
-                    status_field = pi.get("fieldValueByName") or {}
-                    project_status = status_field.get("name")
-                    break
             all_nodes.append({
                 "number": node["number"],
                 "title": node["title"],
                 "state": node["state"],
                 "issue_type": issue_type["name"] if issue_type else None,
                 "labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
+                "assignees": node_assignees(node),
                 "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]],
-                "project_status": project_status,
+                "project_status": node_main_status(node),
+                "projects": node_projects(node),
             })
 
         total = len(all_nodes)
@@ -592,6 +609,7 @@ GQL_PRS_QUERY_ITEM = """\
       createdAt
       milestone {{ title }}
       author {{ login }}
+      assignees(first: 10) {{ nodes {{ login }} }}
       labels(first: 20) {{ nodes {{ name }} }}
       closingIssuesReferences(first: 5) {{ nodes {{ number }} }}
     }}
@@ -640,6 +658,7 @@ def fetch_prs_batch(pr_numbers: list[int]) -> list[dict]:
             "created_at": pr.get("createdAt"),
             "milestone": (pr.get("milestone") or {}).get("title"),
             "author": pr["author"]["login"] if pr["author"] else None,
+            "assignees": node_assignees(pr),
             "labels": [lbl["name"] for lbl in pr["labels"]["nodes"]],
             "closing_issues": [iss["number"] for iss in pr["closingIssuesReferences"]["nodes"]],
         })
@@ -663,6 +682,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) {
             createdAt
             headRefName
             author { login }
+            assignees(first: 10) { nodes { login } }
             labels(first: 20) { nodes { name } }
             files(first: 100) { nodes { path } }
             closingIssuesReferences(first: 5) { nodes { number } }
@@ -685,7 +705,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]:
 
     Returns:
         List of {number, title, body, state, merged_at, created_at,
-                head_ref_name, author, labels: [str], files: [str],
+                head_ref_name, author, assignees, labels: [str], files: [str],
                 closing_issues: [int]}
     """
     query = GQL_MILESTONE_PRS_QUERY.replace("__STATES__", states)
@@ -715,6 +735,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]:
                 "created_at": node.get("createdAt"),
                 "head_ref_name": node.get("headRefName"),
                 "author": node["author"]["login"] if node["author"] else None,
+                "assignees": node_assignees(node),
                 "labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
                 "files": [file["path"] for file in node["files"]["nodes"]],
                 "closing_issues": [iss["number"] for iss in node["closingIssuesReferences"]["nodes"]],
@@ -781,6 +802,7 @@ GQL_ISSUE_BY_NUMBER_QUERY_ITEM = """\
       milestone {{ title }}
       issueType {{ name }}
       labels(first: 20) {{ nodes {{ name }} }}
+      assignees(first: 10) {{ nodes {{ login }} }}
       closedByPullRequestsReferences(first: 5) {{ nodes {{ number }} }}
       projectItems(first: 10) {{
         nodes {{
@@ -800,13 +822,6 @@ def issue_node_to_dict(node: dict) -> dict:
     """Convert a GraphQL issue node to the shared issue dict shape."""
     issue_type = node.get("issueType")
     ms = node.get("milestone")
-    project_status = None
-    for pi in (node.get("projectItems") or {}).get("nodes") or []:
-        project = pi.get("project") or {}
-        if project.get("title") == "Main":
-            status_field = pi.get("fieldValueByName") or {}
-            project_status = status_field.get("name")
-            break
     return {
         "number": node["number"],
         "title": node["title"],
@@ -814,8 +829,10 @@ def issue_node_to_dict(node: dict) -> dict:
         "milestone": ms["title"] if ms else None,
         "issue_type": issue_type["name"] if issue_type else None,
         "labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
+        "assignees": node_assignees(node),
         "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]],
-        "project_status": project_status,
+        "project_status": node_main_status(node),
+        "projects": node_projects(node),
     }
 
 

scripts/project-anomalies.py

@@ -0,0 +1,468 @@
+#!/usr/bin/env python3
+"""
+project-anomalies.py — Find anomalies on the Main project board for a milestone.
+
+Used by the ``find-project-anomalies`` skill. Fetches milestone issues and
+PRs in bulk via ``scripts/gh.py`` (one GraphQL call per 50 items, never one
+call per item) and writes a clickable report to
+``tmp/<MILESTONE>-ANOMALIES.md``.
+
+Anomaly types (all scoped to the given milestone):
+  1. open-with-merged-pr — OPEN issue with a MERGED closing PR.
+  2. issue-pr-mismatch — milestone issue whose closing PR is in a different
+     milestone (or none), or issue not associated to the Main project.
+  3. pr-issue-mismatch — milestone PR whose closed issue is in a different
+     milestone (or none), or not associated to the Main project.
+  4. needs-triage — MERGED PR whose closed issue has the `needs triage` label.
+  5. unassigned — MERGED PR whose closed issue has no assignees and no
+     `community contribution` label (on the issue or the PR).
+
+Project membership is only checked on the issue side: PRs are not the unit
+tracked on the Main board, so an unprojected PR alone is never an anomaly.
+
+Usage:
+  python3 scripts/project-anomalies.py check "2.17.0"
+  python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md
+
+Prerequisites:
+  - gh CLI authenticated (gh auth status)
+  - Python 3.8+
+"""
+
+import argparse
+import json
+import subprocess
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+
+REPO = "penpot/penpot"
+GH_PY = Path(__file__).resolve().parent / "gh.py"
+MAIN_PROJECT = "Main"
+COMMUNITY_LABEL = "community contribution"
+TRIAGE_LABEL = "needs triage"
+
+
+# ─────────────────────────────────────────────
+#  Shared helpers
+# ─────────────────────────────────────────────
+
+
+def run_gh_py(*args: str) -> str:
+    """Run ``scripts/gh.py`` with the given arguments, return stdout."""
+    cmd = [sys.executable, str(GH_PY), *args]
+    result = subprocess.run(cmd, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"gh.py error: {result.stderr}", file=sys.stderr)
+        sys.exit(1)
+    return result.stdout
+
+
+def issue_url(n: int) -> str:
+    return f"https://github.com/{REPO}/issues/{n}"
+
+
+def pr_url(n: int) -> str:
+    return f"https://github.com/{REPO}/pull/{n}"
+
+
+def issue_link(n: int) -> str:
+    return f"[#{n}]({issue_url(n)})"
+
+
+def pr_link(n: int) -> str:
+    return f"[#{n}]({pr_url(n)})"
+
+
+def issue_link_title(n: int, title: str) -> str:
+    url = issue_url(n)
+    if title:
+        return f"[#{n}]({url}) — [{title}]({url})"
+    return f"[#{n}]({url})"
+
+
+def pr_link_title(n: int, title: str) -> str:
+    url = pr_url(n)
+    if title:
+        return f"[#{n}]({url}) — [{title}]({url})"
+    return f"[#{n}]({url})"
+
+
+def in_main_project(item: dict) -> bool:
+    """Whether an issue/PR dict is associated to the Main project board."""
+    return MAIN_PROJECT in (item.get("projects") or [])
+
+
+# ─────────────────────────────────────────────
+#  Anomaly detection (pure: no network inside)
+# ─────────────────────────────────────────────
+
+
+def find_anomalies(
+    milestone: str,
+    issues: list[dict],
+    prs: list[dict],
+    pr_by_num: dict[int, dict],
+    issue_by_num: dict[int, dict],
+) -> dict[str, list[dict]]:
+    """Detect the five anomaly types for a milestone.
+
+    Callers pre-resolve outsiders: any closing PR/issue not in the milestone
+    must already sit in ``pr_by_num``/``issue_by_num`` (with at least
+    ``milestone``, ``state``, ``labels``, ``assignees`` and ``projects``).
+    Unknown numbers are skipped, never flagged.
+    """
+    t1_open_merged: list[dict] = []
+    t2_issue_pr: list[dict] = []
+    t3_pr_issue: list[dict] = []
+    t4_needs_triage: list[dict] = []
+    t5_unassigned: list[dict] = []
+
+    for issue in issues:
+        if issue.get("state") != "OPEN":
+            continue
+        merged = [
+            n for n in issue.get("closing_prs", [])
+            if (pr_by_num.get(n) or {}).get("state") == "MERGED"
+        ]
+        if merged:
+            t1_open_merged.append({
+                "issue": issue["number"],
+                "issue_title": issue.get("title", ""),
+                "prs": sorted(merged),
+            })
+
+    for issue in issues:
+        for pr_num in issue.get("closing_prs", []):
+            pr = pr_by_num.get(pr_num)
+            if pr is None:
+                continue
+            # An unmerged PR has landed nowhere: its milestone says nothing
+            # about any release, so only merged PRs count here.
+            if pr.get("state") != "MERGED":
+                continue
+            if pr.get("milestone") != milestone:
+                t2_issue_pr.append({
+                    "issue": issue["number"],
+                    "issue_title": issue.get("title", ""),
+                    "pr": pr_num,
+                    "pr_milestone": pr.get("milestone"),
+                })
+        if not in_main_project(issue):
+            t2_issue_pr.append({
+                "issue": issue["number"],
+                "issue_title": issue.get("title", ""),
+                "pr": None,
+                "pr_milestone": None,
+                "projects": issue.get("projects") or [],
+            })
+
+    for pr in prs:
+        # Like type 2: only a merged PR has landed somewhere, so only
+        # merged PRs take part in milestone pairing. (Whether the issue is
+        # on the Main board is checked regardless.)
+        if pr.get("state") != "MERGED":
+            continue
+        for issue_num in pr.get("closing_issues", []):
+            issue = issue_by_num.get(issue_num)
+            if issue is None:
+                continue
+            if issue.get("milestone", milestone) != milestone or not in_main_project(issue):
+                t3_pr_issue.append({
+                    "pr": pr["number"],
+                    "pr_title": pr.get("title", ""),
+                    "issue": issue_num,
+                    "issue_title": issue.get("title", ""),
+                    "issue_milestone": issue.get("milestone", milestone),
+                    "projects": issue.get("projects") or [],
+                })
+
+    for pr in prs:
+        if pr.get("state") != "MERGED":
+            continue
+        for issue_num in pr.get("closing_issues", []):
+            issue = issue_by_num.get(issue_num)
+            if issue is None:
+                continue
+            if TRIAGE_LABEL in (issue.get("labels") or []):
+                t4_needs_triage.append({
+                    "pr": pr["number"],
+                    "pr_title": pr.get("title", ""),
+                    "issue": issue_num,
+                    "issue_title": issue.get("title", ""),
+                })
+            if not issue.get("assignees") and COMMUNITY_LABEL not in (
+                (issue.get("labels") or []) + (pr.get("labels") or [])
+            ):
+                t5_unassigned.append({
+                    "pr": pr["number"],
+                    "pr_title": pr.get("title", ""),
+                    "issue": issue_num,
+                    "issue_title": issue.get("title", ""),
+                })
+
+    return {
+        "open_merged": t1_open_merged,
+        "issue_pr": t2_issue_pr,
+        "pr_issue": t3_pr_issue,
+        "needs_triage": t4_needs_triage,
+        "unassigned": t5_unassigned,
+    }
+
+
+# ─────────────────────────────────────────────
+#  Report rendering (pure)
+# ─────────────────────────────────────────────
+
+
+def milestone_fix(kind: str, number: int, milestone: str) -> str:
+    """Render the one-line `gh` command that assigns a missing milestone."""
+    return f"Fix: `gh {kind} edit {number} --milestone \"{milestone}\"`"
+
+
+def render_report(
+    milestone: str,
+    anomalies: dict[str, list[dict]],
+    total_issues: int,
+    closed_issues: int,
+    total_prs: int,
+    merged_prs: int,
+) -> str:
+    """Render the full anomalies report as Markdown."""
+    t1 = anomalies["open_merged"]
+    t2 = anomalies["issue_pr"]
+    t3 = anomalies["pr_issue"]
+    t4 = anomalies["needs_triage"]
+    t5 = anomalies["unassigned"]
+    total = len(t1) + len(t2) + len(t3) + len(t4) + len(t5)
+
+    lines = [
+        f"# Project Anomalies — {milestone}",
+        "",
+        f"Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}",
+        "",
+        "---",
+        "",
+        "## Summary",
+        "",
+        f"- **OPEN issue with a MERGED PR:** {len(t1)}",
+        f"- **Milestone issue, PR elsewhere/unprojected:** {len(t2)}",
+        f"- **Milestone PR, issue elsewhere/unprojected:** {len(t3)}",
+        f"- **MERGED PR on a `needs triage` issue:** {len(t4)}",
+        f"- **MERGED PR on an unassigned, non-community issue:** {len(t5)}",
+        f"- **Total anomalies:** {total}",
+        "",
+    ]
+
+    if total == 0:
+        lines += [
+            "✅ No anomalies found. The milestone and the Main project board agree.",
+            "",
+        ]
+    else:
+        lines += ["## Anomalies", ""]
+        if t1:
+            lines += [
+                "### OPEN issue with a MERGED PR",
+                "",
+                "The fix already landed but the issue never closed (or it was "
+                "reopened). Close the issue or move it out of the milestone.",
+                "",
+            ]
+            for e in t1:
+                lines.append(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}")
+                for n in e["prs"]:
+                    lines.append(f"  - 🔴 Merged {pr_link(n)}")
+                lines.append("")
+
+        if t2:
+            lines += [
+                "### Milestone issue, PR elsewhere or issue off the Main board",
+                "",
+            ]
+            by_issue: dict[int, list[dict]] = {}
+            for e in t2:
+                by_issue.setdefault(e["issue"], []).append(e)
+            for issue_num in sorted(by_issue):
+                entries = by_issue[issue_num]
+                lines.append(
+                    f"- {issue_link_title(issue_num, entries[0]['issue_title'][:80])}"
+                )
+                for e in entries:
+                    if e["pr"] is None:
+                        projects = ", ".join(e["projects"]) or "_none_"
+                        lines.append(
+                            "  - 🔴 Issue is not on the Main project board "
+                            f"(projects: {projects})"
+                        )
+                    else:
+                        ms = e["pr_milestone"] or "_none_"
+                        badge = "🔴" if e["pr_milestone"] is None else "⚠️"
+                        lines.append(
+                            f"  - {badge} Closing {pr_link(e['pr'])} is in milestone "
+                            f"**{ms}** (expected: {milestone})"
+                        )
+                        if e["pr_milestone"] is None:
+                            lines.append(
+                                f"    {milestone_fix('pr', e['pr'], milestone)}"
+                            )
+                lines.append("")
+
+        if t3:
+            lines += [
+                "### Milestone PR, issue elsewhere or off the Main board",
+                "",
+            ]
+            by_pr: dict[int, list[dict]] = {}
+            for e in t3:
+                by_pr.setdefault(e["pr"], []).append(e)
+            for pr_num in sorted(by_pr):
+                entries = by_pr[pr_num]
+                lines.append(
+                    f"- {pr_link_title(pr_num, entries[0]['pr_title'][:80])}"
+                )
+                for e in entries:
+                    ms = e["issue_milestone"] or "_none_"
+                    if e["issue_milestone"] != milestone:
+                        lines.append(
+                            f"  - ⚠️ Closing {issue_link(e['issue'])} is in milestone "
+                            f"**{ms}** (expected: {milestone})"
+                        )
+                        if e["issue_milestone"] is None:
+                            lines.append(
+                                f"    {milestone_fix('issue', e['issue'], milestone)}"
+                            )
+                    else:
+                        projects = ", ".join(e["projects"]) or "_none_"
+                        lines.append(
+                            f"  - 🔴 Closing {issue_link(e['issue'])} is not on the "
+                            f"Main project board (projects: {projects})"
+                        )
+                lines.append("")
+
+        if t4:
+            lines += [
+                "### MERGED PR on a `needs triage` issue",
+                "",
+                "The fix landed but the issue was never triaged. Human triage needed.",
+                "",
+            ]
+            for e in t4:
+                lines.append(
+                    f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}"
+                    f" closes {issue_link(e['issue'])}"
+                )
+            lines.append("")
+
+        if t5:
+            lines += [
+                "### MERGED PR on an unassigned, non-community issue",
+                "",
+                "No owner and no community label. Assign an owner or confirm it "
+                "is a community contribution.",
+                "",
+            ]
+            for e in t5:
+                lines.append(
+                    f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}"
+                    f" closes {issue_link(e['issue'])}"
+                )
+            lines.append("")
+
+    lines += [
+        "---",
+        "",
+        "## Context",
+        "",
+        f"- Milestone: **{milestone}**",
+        f"- Milestone issues (all states): {total_issues}",
+        f"- Open issues in milestone: {total_issues - closed_issues}",
+        f"- Milestone PRs (all states): {total_prs}",
+        f"- Merged PRs in milestone: {merged_prs}",
+        "",
+    ]
+    return "\n".join(lines)
+
+
+# ─────────────────────────────────────────────
+#  CLI entrypoint
+# ─────────────────────────────────────────────
+
+
+def cmd_check(args: argparse.Namespace) -> None:
+    """Fetch milestone data in bulk, detect anomalies, write the report."""
+    milestone = args.milestone
+
+    all_issues = json.loads(run_gh_py("issues", milestone, "--state", "all"))
+    issue_by_num = {i["number"]: i for i in all_issues}
+
+    all_prs = json.loads(run_gh_py("prs", "--milestone", milestone, "--state", "all"))
+    pr_by_num = {p["number"]: p for p in all_prs}
+
+    # Resolve outsiders in bulk (one GraphQL call per 50 items).
+    unknown_prs = set()
+    for issue in all_issues:
+        unknown_prs.update(issue.get("closing_prs", []))
+    unknown_prs -= set(pr_by_num)
+    if unknown_prs:
+        print(
+            f"Resolving {len(unknown_prs)} PRs outside {milestone}...",
+            file=sys.stderr,
+        )
+        for pr in json.loads(run_gh_py("prs", *[str(n) for n in sorted(unknown_prs)])):
+            pr_by_num[pr["number"]] = pr
+
+    unknown_issues = set()
+    for pr in all_prs:
+        unknown_issues.update(pr.get("closing_issues", []))
+    unknown_issues -= set(issue_by_num)
+    if unknown_issues:
+        print(
+            f"Resolving {len(unknown_issues)} issues outside {milestone}...",
+            file=sys.stderr,
+        )
+        for issue in json.loads(
+            run_gh_py("issue", *[str(n) for n in sorted(unknown_issues)])
+        ):
+            if "error" not in issue:
+                issue_by_num[issue["number"]] = issue
+
+    anomalies = find_anomalies(milestone, all_issues, all_prs, pr_by_num, issue_by_num)
+    report = render_report(
+        milestone,
+        anomalies,
+        total_issues=len(all_issues),
+        closed_issues=sum(1 for i in all_issues if i.get("state") == "CLOSED"),
+        total_prs=len(all_prs),
+        merged_prs=sum(1 for p in all_prs if p.get("state") == "MERGED"),
+    )
+
+    output = Path(args.output or f"tmp/{milestone}-ANOMALIES.md")
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(report)
+    total = sum(len(v) for v in anomalies.values())
+    print(f"{total} anomalies written to {output}")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Find Main-project anomalies for a milestone"
+    )
+    sub = parser.add_subparsers(dest="command", required=True, title="subcommands")
+
+    p_check = sub.add_parser("check", help="Detect anomalies and write the report")
+    p_check.add_argument("milestone", help="Milestone title (e.g. '2.17.0')")
+    p_check.add_argument(
+        "--output",
+        default=None,
+        help="Report path (default: tmp/<MILESTONE>-ANOMALIES.md)",
+    )
+    p_check.set_defaults(func=cmd_check)
+
+    args = parser.parse_args()
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()

scripts/test_project_anomalies.py

@@ -0,0 +1,278 @@
+#!/usr/bin/env python3
+"""Tests for scripts/project-anomalies.py (pure logic, no network).
+
+Run with:
+
+    python3 scripts/test_project_anomalies.py
+"""
+
+import importlib.machinery
+import importlib.util
+import pathlib
+import sys
+import unittest
+
+
+# Loading scripts/project-anomalies.py should not emit scripts/__pycache__/.
+sys.dont_write_bytecode = True
+
+SCRIPT_PATH = pathlib.Path(__file__).resolve().parent / "project-anomalies.py"
+
+
+def load_pa():
+    """Load scripts/project-anomalies.py as a module without running its CLI."""
+    loader = importlib.machinery.SourceFileLoader("project_anomalies", str(SCRIPT_PATH))
+    spec = importlib.util.spec_from_loader("project_anomalies", loader)
+    module = importlib.util.module_from_spec(spec)
+    loader.exec_module(module)
+    return module
+
+
+pa = load_pa()
+
+M = "2.17.0"
+
+
+def issue(n, **kw):
+    base = {
+        "number": n,
+        "title": f"issue {n}",
+        "state": "CLOSED",
+        "milestone": M,
+        "labels": [],
+        "assignees": ["someone"],
+        "closing_prs": [],
+        "project_status": "Done",
+        "projects": ["Main"],
+    }
+    base.update(kw)
+    return base
+
+
+def pr(n, **kw):
+    base = {
+        "number": n,
+        "title": f"pr {n}",
+        "state": "MERGED",
+        "milestone": M,
+        "labels": [],
+        "assignees": ["someone"],
+        "closing_issues": [],
+    }
+    base.update(kw)
+    return base
+
+
+def run(issues, prs):
+    return pa.find_anomalies(
+        M, issues, prs,
+        {p["number"]: p for p in prs},
+        {i["number"]: i for i in issues},
+    )
+
+
+class OpenMergedTests(unittest.TestCase):
+    def test_open_issue_with_merged_pr_is_flagged(self):
+        out = run(
+            [issue(1, state="OPEN", closing_prs=[11])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(len(out["open_merged"]), 1)
+        self.assertEqual(out["open_merged"][0]["prs"], [11])
+
+    def test_open_issue_with_unmerged_pr_is_clean(self):
+        out = run(
+            [issue(1, state="OPEN", closing_prs=[11])],
+            [pr(11, state="OPEN", closing_issues=[1])],
+        )
+        self.assertEqual(out["open_merged"], [])
+
+    def test_closed_issue_with_merged_pr_is_clean(self):
+        out = run(
+            [issue(1, closing_prs=[11])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertTrue(all(v == [] for v in out.values()))
+
+
+class IssuePrMismatchTests(unittest.TestCase):
+    def test_pr_in_other_milestone_is_flagged(self):
+        out = run(
+            [issue(1, closing_prs=[11])],
+            [pr(11, milestone="2.18.0")],
+        )
+        self.assertEqual(len(out["issue_pr"]), 1)
+        self.assertEqual(out["issue_pr"][0]["pr_milestone"], "2.18.0")
+
+    def test_pr_without_milestone_is_flagged(self):
+        out = run(
+            [issue(1, closing_prs=[11])],
+            [pr(11, milestone=None)],
+        )
+        self.assertEqual(len(out["issue_pr"]), 1)
+
+    def test_unmerged_pr_milestone_is_ignored(self):
+        # An OPEN (unmerged) PR has landed nowhere: its milestone is
+        # irrelevant, even when it differs.
+        out = run(
+            [issue(1, closing_prs=[11])],
+            [pr(11, state="OPEN", milestone="2.18.0")],
+        )
+        self.assertEqual(out["issue_pr"], [])
+
+    def test_off_board_issue_flagged_despite_unmerged_pr(self):
+        # The project check does not depend on PR merge state.
+        out = run(
+            [issue(1, projects=["Other"], closing_prs=[11])],
+            [pr(11, state="OPEN", milestone="2.18.0")],
+        )
+        self.assertEqual(len(out["issue_pr"]), 1)
+        self.assertIsNone(out["issue_pr"][0]["pr"])
+
+    def test_issue_off_main_board_is_flagged_without_pr(self):
+        out = run([issue(1, projects=["Other"], project_status=None)], [])
+        self.assertEqual(len(out["issue_pr"]), 1)
+        self.assertIsNone(out["issue_pr"][0]["pr"])
+
+    def test_aligned_pair_is_clean(self):
+        out = run(
+            [issue(1, closing_prs=[11])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(out["issue_pr"], [])
+        self.assertEqual(out["pr_issue"], [])
+
+
+class PrIssueMismatchTests(unittest.TestCase):
+    def test_issue_in_other_milestone_is_flagged(self):
+        other = issue(2, milestone="2.16.0")
+        out = pa.find_anomalies(
+            M, [issue(1)], [pr(11, closing_issues=[2])],
+            {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other},
+        )
+        self.assertEqual(len(out["pr_issue"]), 1)
+
+    def test_issue_without_milestone_is_flagged(self):
+        # Unlike the changelog flow, here a milestone-less issue closed by a
+        # milestone PR is reported: the pairing needs human judgment.
+        other = issue(2, milestone=None, projects=[], project_status=None)
+        out = pa.find_anomalies(
+            M, [issue(1)], [pr(11, closing_issues=[2])],
+            {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other},
+        )
+        self.assertEqual(len(out["pr_issue"]), 1)
+
+    def test_issue_off_main_board_is_flagged(self):
+        other = issue(2, projects=["Other"], project_status=None)
+        out = pa.find_anomalies(
+            M, [issue(1)], [pr(11, closing_issues=[2])],
+            {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other},
+        )
+        self.assertEqual(len(out["pr_issue"]), 1)
+
+    def test_milestone_issue_keeps_milestone_in_record(self):
+        # Milestone issues carry no "milestone" key (known by construction);
+        # an off-board one must still render its own milestone, never _none_.
+        mine = issue(1, projects=["Other"], project_status=None)
+        del mine["milestone"]
+        out = pa.find_anomalies(
+            M, [mine], [pr(11, closing_issues=[1])],
+            {11: pr(11, closing_issues=[1])}, {1: mine},
+        )
+        self.assertEqual(len(out["pr_issue"]), 1)
+        self.assertEqual(out["pr_issue"][0]["issue_milestone"], M)
+        report = pa.render_report(M, out, 1, 0, 1, 1)
+        self.assertNotIn("gh issue edit 1", report)
+
+    def test_unmerged_pr_is_ignored(self):
+        # Closed-unmerged (or open) milestone PRs take no part in milestone
+        # pairing: only merged PRs landed somewhere.
+        other = issue(2, milestone="2.16.0")
+        for state in ("OPEN", "CLOSED"):
+            out = pa.find_anomalies(
+                M, [issue(1)], [pr(11, state=state, closing_issues=[2])],
+                {11: pr(11, state=state, closing_issues=[2])},
+                {1: issue(1), 2: other},
+            )
+            self.assertEqual(out["pr_issue"], [], f"state={state}")
+
+
+class NeedsTriageTests(unittest.TestCase):
+    def test_merged_pr_on_triaged_issue_is_clean(self):
+        out = run(
+            [issue(1, labels=["bug"])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(out["needs_triage"], [])
+
+    def test_merged_pr_on_needs_triage_issue_is_flagged(self):
+        out = run(
+            [issue(1, labels=["needs triage"])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(len(out["needs_triage"]), 1)
+
+    def test_unmerged_pr_on_needs_triage_issue_is_clean(self):
+        out = run(
+            [issue(1, labels=["needs triage"])],
+            [pr(11, state="OPEN", closing_issues=[1])],
+        )
+        self.assertEqual(out["needs_triage"], [])
+
+
+class UnassignedTests(unittest.TestCase):
+    def test_assigned_issue_is_clean(self):
+        out = run(
+            [issue(1, assignees=["owner"])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(out["unassigned"], [])
+
+    def test_unassigned_community_issue_is_clean(self):
+        out = run(
+            [issue(1, assignees=[], labels=["community contribution"])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(out["unassigned"], [])
+
+    def test_unassigned_community_pr_is_clean(self):
+        out = run(
+            [issue(1, assignees=[])],
+            [pr(11, closing_issues=[1], labels=["community contribution"])],
+        )
+        self.assertEqual(out["unassigned"], [])
+
+    def test_unassigned_non_community_issue_is_flagged(self):
+        out = run(
+            [issue(1, assignees=[])],
+            [pr(11, closing_issues=[1])],
+        )
+        self.assertEqual(len(out["unassigned"]), 1)
+
+    def test_unmerged_pr_is_clean(self):
+        out = run(
+            [issue(1, assignees=[])],
+            [pr(11, state="CLOSED", closing_issues=[1])],
+        )
+        self.assertEqual(out["unassigned"], [])
+
+
+class RenderTests(unittest.TestCase):
+    def test_clean_report_says_so(self):
+        out = run([issue(1, closing_prs=[11])], [pr(11, closing_issues=[1])])
+        report = pa.render_report(M, out, 1, 0, 1, 1)
+        self.assertIn("Total anomalies:** 0", report)
+        self.assertIn("✅ No anomalies", report)
+
+    def test_every_number_is_a_clickable_link(self):
+        out = run(
+            [issue(1, state="OPEN", closing_prs=[11])],
+            [pr(11, closing_issues=[1])],
+        )
+        report = pa.render_report(M, out, 1, 0, 1, 1)
+        self.assertIn("[#1](https://github.com/penpot/penpot/issues/1)", report)
+        self.assertIn("[#11](https://github.com/penpot/penpot/pull/11)", report)
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)