diff --git a/.agents/README.md b/.agents/README.md index 1f4e2364c5..657ab6eac3 100644 --- a/.agents/README.md +++ b/.agents/README.md @@ -508,6 +508,7 @@ JSON, REPL access, and so on. | [`ste`](skills/ste/SKILL.md) | Rewrites prose in Simplified Technical English. Loads only when you name it. | | [`refine-prompt`](skills/refine-prompt/SKILL.md) | Rewrites a rough prompt into a clearer one. Never runs the prompt. | | [`update-changelog`](skills/update-changelog/SKILL.md) | Regenerates `CHANGES.md` from a GitHub milestone. | +| [`find-project-anomalies`](skills/find-project-anomalies/SKILL.md) | Checks a milestone against the Main project board and reports anomalies to `tmp/-ANOMALIES.md`. | ### A typical round diff --git a/.agents/skills/find-project-anomalies/SKILL.md b/.agents/skills/find-project-anomalies/SKILL.md new file mode 100644 index 0000000000..208db481a9 --- /dev/null +++ b/.agents/skills/find-project-anomalies/SKILL.md @@ -0,0 +1,116 @@ +--- +name: find-project-anomalies +description: Check a GitHub milestone against the Main project board, report the five anomaly types to tmp/-ANOMALIES.md, and fix missing milestone assignments on request. +--- + +# Skill: find-project-anomalies + +Check every issue and PR in a given GitHub milestone against the Main +project board. Report the five anomaly types below, then fix the ones a +human confirms. + +## Getting the version ($ARGUMENTS) + +The milestone version comes from the skill arguments (`$ARGUMENTS`), the +same way other skills receive theirs. Accept `2.17.0`, `v2.17.0`, +`milestone 2.17.0`, or any text clearly containing one `X.Y.Z` version. + +If no version is given and none can be deduced from the arguments or the +conversation context, **stop and ask the user** which milestone to check. +Do not guess, do not default to the latest release. + +## When to Use + +- Before a release, to catch board/milestone drift while there is still time + to fix it +- After triage sessions, to verify every milestone item is tracked, owned, + and correctly assigned +- Whenever someone asks "is milestone X.Y.Z clean on Main?" + +## Prerequisites + +- `gh` CLI authenticated (`gh auth status`) +- Python 3.8+ +- `scripts/gh.py` for GitHub queries, `scripts/project-anomalies.py` for + detection and reporting (`python3 scripts/project-anomalies.py check --help`) + +## Workflow + +### 1. Resolve the version (see above), then run the check + +```bash +python3 scripts/project-anomalies.py check "2.17.0" +# explicit path: +python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md +``` + +This fetches milestone issues and PRs in bulk (one GraphQL call per 50 +items), resolves outsiders the same way, and overwrites +`tmp/-ANOMALIES.md`. Every number renders as a full +`[#N](https://github.com/penpot/penpot/issues/N)` or +`[#N](https://github.com/penpot/penpot/pull/N)` link. + +### 2. Read the report and judge each anomaly + +The five types, in report order: + +1. **OPEN issue with a MERGED PR** — the fix landed but the issue never + closed (or it reopened). Fix: close the issue, or move it out of the + milestone if the fix did not actually land here. +2. **Milestone issue, PR elsewhere or issue off Main** — a *merged* closing + PR in another milestone (or none), or the issue itself not on the Main + board. An unmerged PR's milestone means nothing (it landed nowhere), so + it never triggers this type. Fix: align the milestones (see step 3), or + add the issue to Main. +3. **Milestone PR, issue elsewhere or off Main** — the mirror view over + *merged* PRs only: a PR released here closes an issue tracked elsewhere + (or nowhere on Main). Unmerged PRs never trigger this type — their + milestone means nothing yet. Fix: align the milestones (see step 3), or + add the issue to Main. +4. **MERGED PR on a `needs triage` issue** — landed without triage. Only a + human can triage; never auto-remove the label. +5. **MERGED PR on an unassigned, non-community issue** — no owner and no + `community contribution` label (checked on both issue and PR). Either + assign an owner or confirm it is community work. Never invent an owner. + +Scope notes (deliberate, not bugs): +- Project membership is checked **on the issue side only**. PRs are not the + unit tracked on Main, so an unprojected PR alone is never an anomaly. +- An issue with *no* milestone closed by a milestone PR **is** reported + here (type 3): unlike the changelog flow, this pairing always needs a + human look. + +### 3. Fix missing milestone assignments (only these, only on confirmation) + +The only mechanical fix in this skill is assigning a missing milestone. +The report already prints the exact command next to each such case: + +```bash +gh pr edit --milestone "" +gh issue edit --milestone "" +``` + +Rules: +- Apply **only** when one side lacks a milestone. When both sides have + *different* milestones, a human decides which one moves — never pick a + side yourself. +- Confirm **each fix (or each small batch)** with the user before running + it. Read back what the command will change. +- Everything else (closing issues, triaging, assigning owners, adding + items to the Main board) is human work: point at it, do not do it. + +### 4. Re-run until clean + +After fixes land, re-run step 1 and confirm the report shows zero +anomalies. The `tmp/` report is scratch output (gitignored): quote or +paste entries into chat when reporting back, do not commit it. + +## Key Principles + +- **No version, no run.** Stop and ask when the milestone is unknown. +- **Report first, touch nothing.** No board or milestone changes without + explicit per-item confirmation. +- **Milestone gaps auto-fix; judgment calls do not.** Only a missing + milestone is mechanical. triage, ownership, and close/reopen decisions + belong to humans. +- **Every number clickable.** The report is useless without full GitHub links. diff --git a/.gitignore b/.gitignore index 50c8d19494..a0d9844a01 100644 --- a/.gitignore +++ b/.gitignore @@ -108,5 +108,6 @@ opencode.json /.opencode/reports /.opencode/prompts /.ci-logs +/tmp/ /.codex/ /tools/__pycache__ diff --git a/.serena/memories/critical-info.md b/.serena/memories/critical-info.md index 5f1b52c81b..5e84c9834e 100644 --- a/.serena/memories/critical-info.md +++ b/.serena/memories/critical-info.md @@ -55,6 +55,7 @@ The memory is structured in a way that you can get the critical information abou - `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`. - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`. - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`. +- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`. - `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`. - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`. diff --git a/.serena/memories/scripts/gh.md b/.serena/memories/scripts/gh.md index d0546e5f13..7182120524 100644 --- a/.serena/memories/scripts/gh.md +++ b/.serena/memories/scripts/gh.md @@ -49,6 +49,8 @@ python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md - Issues with type "Task" are excluded (`--include-tasks` to keep them). - Issues with "Rejected" project status are excluded (`--include-rejected` to keep them). +Every issue entry carries `assignees` (logins) and `projects` (board titles); every PR entry carries `assignees`. + **Output**: JSON array to stdout; progress to stderr. ### `prs` diff --git a/.serena/memories/scripts/project-anomalies.md b/.serena/memories/scripts/project-anomalies.md new file mode 100644 index 0000000000..9d0bfe6ecc --- /dev/null +++ b/.serena/memories/scripts/project-anomalies.md @@ -0,0 +1,17 @@ +# Project anomalies helper + +`scripts/project-anomalies.py` implements the `find-project-anomalies` skill: `check ` fetches milestone issues/PRs with `--state all` via `scripts/gh.py`, resolves outsiders in bulk (one GraphQL call per 50 items), and writes `tmp/-ANOMALIES.md` (gitignored scratch output, every number a full GitHub link). + +## Anomaly types (all scoped to the milestone) + +- Open-with-merged-PR: OPEN issue with a MERGED closing PR (close it or move it out). +- Issue-PR mismatch: milestone issue whose *merged* closing PR is elsewhere/nowhere (an unmerged PR's milestone is irrelevant), or issue off the Main board. +- PR-issue mismatch: *merged* milestone PR closing an issue elsewhere/nowhere (milestone-less issues ARE reported here, unlike the changelog flow), or off the Main board. Unmerged PRs never count anywhere: their milestone is meaningless. +- Needs-triage: MERGED PR on a `needs triage` issue (human triage only, never auto-unlabel). +- Unassigned: MERGED PR on an issue with no assignees and no `community contribution` label on either side (valid states: assigned, or unassigned+community). + +## Conventions + +- Project membership is checked on the issue side only (`projects` list from `gh.py`); PRs are not the Main-tracked unit, so an unprojected PR alone is never an anomaly. +- Detection (`find_anomalies`) and rendering (`render_report`) are pure functions over pre-resolved dicts; outsiders must be merged into the lookup maps first (missing numbers are skipped, never flagged). +- The only mechanical fix is assigning a missing milestone (`gh pr/issue edit --milestone`); everything else is human judgment with per-item confirmation. diff --git a/scripts/gh.py b/scripts/gh.py index e0db6b4ac7..600c96f243 100755 --- a/scripts/gh.py +++ b/scripts/gh.py @@ -319,6 +319,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) { state issueType { name } labels(first: 20) { nodes { name } } + assignees(first: 10) { nodes { login } } closedByPullRequestsReferences(first: 5) { nodes { number } } projectItems(first: 10) { nodes { @@ -357,6 +358,7 @@ query($query: String!, $cursor: String) { milestone { title } issueType { name } labels(first: 20) { nodes { name } } + assignees(first: 10) { nodes { login } } closedByPullRequestsReferences(first: 5) { nodes { number } } projectItems(first: 10) { nodes { @@ -375,6 +377,32 @@ query($query: String!, $cursor: String) { """ +def node_assignees(node: dict) -> list[str]: + """Extract assignee logins from a GraphQL issue/PR node.""" + return [ + a["login"] for a in (node.get("assignees") or {}).get("nodes") or [] + ] + + +def node_projects(node: dict) -> list[str]: + """Extract project board titles from a GraphQL issue node.""" + return [ + (pi.get("project") or {}).get("title") + for pi in (node.get("projectItems") or {}).get("nodes") or [] + if (pi.get("project") or {}).get("title") + ] + + +def node_main_status(node: dict) -> str | None: + """Extract the "Main" project board status from a GraphQL issue node.""" + for pi in (node.get("projectItems") or {}).get("nodes") or []: + project = pi.get("project") or {} + if project.get("title") == "Main": + status_field = pi.get("fieldValueByName") or {} + return status_field.get("name") + return None + + def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[dict]: """ Fetch all issues that belong to NO milestone via paginated GraphQL search. @@ -384,7 +412,7 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di labels: optional comma-separated labels to include (built into the search query) Returns: - List of {number, title, state, milestone, issue_type, labels, closing_prs, project_status} + List of {number, title, state, milestone, issue_type, labels, assignees, closing_prs, project_status, projects} """ all_nodes: list[dict] = [] cursor: str | None = None @@ -415,13 +443,6 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di continue issue_type = node.get("issueType") ms = node.get("milestone") - project_status = None - for pi in (node.get("projectItems") or {}).get("nodes") or []: - project = pi.get("project") or {} - if project.get("title") == "Main": - status_field = pi.get("fieldValueByName") or {} - project_status = status_field.get("name") - break all_nodes.append({ "number": node["number"], "title": node["title"], @@ -429,8 +450,10 @@ def fetch_no_milestone_issues(states: str, labels: str | None = None) -> list[di "milestone": ms["title"] if ms else None, "issue_type": issue_type["name"] if issue_type else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], - "project_status": project_status, + "project_status": node_main_status(node), + "projects": node_projects(node), }) total = len(all_nodes) @@ -452,7 +475,7 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]: states: GraphQL states enum array literal, e.g. ``"[CLOSED]"`` or ``"[OPEN CLOSED]"`` Returns: - List of {number, title, state, issue_type: str|None, labels: [str], closing_prs: [int]} + List of {number, title, state, issue_type: str|None, labels: [str], assignees: [str], closing_prs: [int], project_status, projects} """ query = GQL_ISSUES_QUERY.replace("__STATES__", states) all_nodes: list[dict] = [] @@ -473,22 +496,16 @@ def fetch_milestone_issues(milestone_num: int, states: str) -> list[dict]: if node is None: continue issue_type = node.get("issueType") - # Extract project status from the "Main" project board (if present) - project_status = None - for pi in (node.get("projectItems") or {}).get("nodes") or []: - project = pi.get("project") or {} - if project.get("title") == "Main": - status_field = pi.get("fieldValueByName") or {} - project_status = status_field.get("name") - break all_nodes.append({ "number": node["number"], "title": node["title"], "state": node["state"], "issue_type": issue_type["name"] if issue_type else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], - "project_status": project_status, + "project_status": node_main_status(node), + "projects": node_projects(node), }) total = len(all_nodes) @@ -592,6 +609,7 @@ GQL_PRS_QUERY_ITEM = """\ createdAt milestone {{ title }} author {{ login }} + assignees(first: 10) {{ nodes {{ login }} }} labels(first: 20) {{ nodes {{ name }} }} closingIssuesReferences(first: 5) {{ nodes {{ number }} }} }} @@ -640,6 +658,7 @@ def fetch_prs_batch(pr_numbers: list[int]) -> list[dict]: "created_at": pr.get("createdAt"), "milestone": (pr.get("milestone") or {}).get("title"), "author": pr["author"]["login"] if pr["author"] else None, + "assignees": node_assignees(pr), "labels": [lbl["name"] for lbl in pr["labels"]["nodes"]], "closing_issues": [iss["number"] for iss in pr["closingIssuesReferences"]["nodes"]], }) @@ -663,6 +682,7 @@ query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) { createdAt headRefName author { login } + assignees(first: 10) { nodes { login } } labels(first: 20) { nodes { name } } files(first: 100) { nodes { path } } closingIssuesReferences(first: 5) { nodes { number } } @@ -685,7 +705,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]: Returns: List of {number, title, body, state, merged_at, created_at, - head_ref_name, author, labels: [str], files: [str], + head_ref_name, author, assignees, labels: [str], files: [str], closing_issues: [int]} """ query = GQL_MILESTONE_PRS_QUERY.replace("__STATES__", states) @@ -715,6 +735,7 @@ def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]: "created_at": node.get("createdAt"), "head_ref_name": node.get("headRefName"), "author": node["author"]["login"] if node["author"] else None, + "assignees": node_assignees(node), "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], "files": [file["path"] for file in node["files"]["nodes"]], "closing_issues": [iss["number"] for iss in node["closingIssuesReferences"]["nodes"]], @@ -781,6 +802,7 @@ GQL_ISSUE_BY_NUMBER_QUERY_ITEM = """\ milestone {{ title }} issueType {{ name }} labels(first: 20) {{ nodes {{ name }} }} + assignees(first: 10) {{ nodes {{ login }} }} closedByPullRequestsReferences(first: 5) {{ nodes {{ number }} }} projectItems(first: 10) {{ nodes {{ @@ -800,13 +822,6 @@ def issue_node_to_dict(node: dict) -> dict: """Convert a GraphQL issue node to the shared issue dict shape.""" issue_type = node.get("issueType") ms = node.get("milestone") - project_status = None - for pi in (node.get("projectItems") or {}).get("nodes") or []: - project = pi.get("project") or {} - if project.get("title") == "Main": - status_field = pi.get("fieldValueByName") or {} - project_status = status_field.get("name") - break return { "number": node["number"], "title": node["title"], @@ -814,8 +829,10 @@ def issue_node_to_dict(node: dict) -> dict: "milestone": ms["title"] if ms else None, "issue_type": issue_type["name"] if issue_type else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], + "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], - "project_status": project_status, + "project_status": node_main_status(node), + "projects": node_projects(node), } diff --git a/scripts/project-anomalies.py b/scripts/project-anomalies.py new file mode 100644 index 0000000000..120681234a --- /dev/null +++ b/scripts/project-anomalies.py @@ -0,0 +1,468 @@ +#!/usr/bin/env python3 +""" +project-anomalies.py — Find anomalies on the Main project board for a milestone. + +Used by the ``find-project-anomalies`` skill. Fetches milestone issues and +PRs in bulk via ``scripts/gh.py`` (one GraphQL call per 50 items, never one +call per item) and writes a clickable report to +``tmp/-ANOMALIES.md``. + +Anomaly types (all scoped to the given milestone): + 1. open-with-merged-pr — OPEN issue with a MERGED closing PR. + 2. issue-pr-mismatch — milestone issue whose closing PR is in a different + milestone (or none), or issue not associated to the Main project. + 3. pr-issue-mismatch — milestone PR whose closed issue is in a different + milestone (or none), or not associated to the Main project. + 4. needs-triage — MERGED PR whose closed issue has the `needs triage` label. + 5. unassigned — MERGED PR whose closed issue has no assignees and no + `community contribution` label (on the issue or the PR). + +Project membership is only checked on the issue side: PRs are not the unit +tracked on the Main board, so an unprojected PR alone is never an anomaly. + +Usage: + python3 scripts/project-anomalies.py check "2.17.0" + python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md + +Prerequisites: + - gh CLI authenticated (gh auth status) + - Python 3.8+ +""" + +import argparse +import json +import subprocess +import sys +from datetime import datetime, timezone +from pathlib import Path + + +REPO = "penpot/penpot" +GH_PY = Path(__file__).resolve().parent / "gh.py" +MAIN_PROJECT = "Main" +COMMUNITY_LABEL = "community contribution" +TRIAGE_LABEL = "needs triage" + + +# ───────────────────────────────────────────── +# Shared helpers +# ───────────────────────────────────────────── + + +def run_gh_py(*args: str) -> str: + """Run ``scripts/gh.py`` with the given arguments, return stdout.""" + cmd = [sys.executable, str(GH_PY), *args] + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + print(f"gh.py error: {result.stderr}", file=sys.stderr) + sys.exit(1) + return result.stdout + + +def issue_url(n: int) -> str: + return f"https://github.com/{REPO}/issues/{n}" + + +def pr_url(n: int) -> str: + return f"https://github.com/{REPO}/pull/{n}" + + +def issue_link(n: int) -> str: + return f"[#{n}]({issue_url(n)})" + + +def pr_link(n: int) -> str: + return f"[#{n}]({pr_url(n)})" + + +def issue_link_title(n: int, title: str) -> str: + url = issue_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def pr_link_title(n: int, title: str) -> str: + url = pr_url(n) + if title: + return f"[#{n}]({url}) — [{title}]({url})" + return f"[#{n}]({url})" + + +def in_main_project(item: dict) -> bool: + """Whether an issue/PR dict is associated to the Main project board.""" + return MAIN_PROJECT in (item.get("projects") or []) + + +# ───────────────────────────────────────────── +# Anomaly detection (pure: no network inside) +# ───────────────────────────────────────────── + + +def find_anomalies( + milestone: str, + issues: list[dict], + prs: list[dict], + pr_by_num: dict[int, dict], + issue_by_num: dict[int, dict], +) -> dict[str, list[dict]]: + """Detect the five anomaly types for a milestone. + + Callers pre-resolve outsiders: any closing PR/issue not in the milestone + must already sit in ``pr_by_num``/``issue_by_num`` (with at least + ``milestone``, ``state``, ``labels``, ``assignees`` and ``projects``). + Unknown numbers are skipped, never flagged. + """ + t1_open_merged: list[dict] = [] + t2_issue_pr: list[dict] = [] + t3_pr_issue: list[dict] = [] + t4_needs_triage: list[dict] = [] + t5_unassigned: list[dict] = [] + + for issue in issues: + if issue.get("state") != "OPEN": + continue + merged = [ + n for n in issue.get("closing_prs", []) + if (pr_by_num.get(n) or {}).get("state") == "MERGED" + ] + if merged: + t1_open_merged.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "prs": sorted(merged), + }) + + for issue in issues: + for pr_num in issue.get("closing_prs", []): + pr = pr_by_num.get(pr_num) + if pr is None: + continue + # An unmerged PR has landed nowhere: its milestone says nothing + # about any release, so only merged PRs count here. + if pr.get("state") != "MERGED": + continue + if pr.get("milestone") != milestone: + t2_issue_pr.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "pr": pr_num, + "pr_milestone": pr.get("milestone"), + }) + if not in_main_project(issue): + t2_issue_pr.append({ + "issue": issue["number"], + "issue_title": issue.get("title", ""), + "pr": None, + "pr_milestone": None, + "projects": issue.get("projects") or [], + }) + + for pr in prs: + # Like type 2: only a merged PR has landed somewhere, so only + # merged PRs take part in milestone pairing. (Whether the issue is + # on the Main board is checked regardless.) + if pr.get("state") != "MERGED": + continue + for issue_num in pr.get("closing_issues", []): + issue = issue_by_num.get(issue_num) + if issue is None: + continue + if issue.get("milestone", milestone) != milestone or not in_main_project(issue): + t3_pr_issue.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + "issue_milestone": issue.get("milestone", milestone), + "projects": issue.get("projects") or [], + }) + + for pr in prs: + if pr.get("state") != "MERGED": + continue + for issue_num in pr.get("closing_issues", []): + issue = issue_by_num.get(issue_num) + if issue is None: + continue + if TRIAGE_LABEL in (issue.get("labels") or []): + t4_needs_triage.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + }) + if not issue.get("assignees") and COMMUNITY_LABEL not in ( + (issue.get("labels") or []) + (pr.get("labels") or []) + ): + t5_unassigned.append({ + "pr": pr["number"], + "pr_title": pr.get("title", ""), + "issue": issue_num, + "issue_title": issue.get("title", ""), + }) + + return { + "open_merged": t1_open_merged, + "issue_pr": t2_issue_pr, + "pr_issue": t3_pr_issue, + "needs_triage": t4_needs_triage, + "unassigned": t5_unassigned, + } + + +# ───────────────────────────────────────────── +# Report rendering (pure) +# ───────────────────────────────────────────── + + +def milestone_fix(kind: str, number: int, milestone: str) -> str: + """Render the one-line `gh` command that assigns a missing milestone.""" + return f"Fix: `gh {kind} edit {number} --milestone \"{milestone}\"`" + + +def render_report( + milestone: str, + anomalies: dict[str, list[dict]], + total_issues: int, + closed_issues: int, + total_prs: int, + merged_prs: int, +) -> str: + """Render the full anomalies report as Markdown.""" + t1 = anomalies["open_merged"] + t2 = anomalies["issue_pr"] + t3 = anomalies["pr_issue"] + t4 = anomalies["needs_triage"] + t5 = anomalies["unassigned"] + total = len(t1) + len(t2) + len(t3) + len(t4) + len(t5) + + lines = [ + f"# Project Anomalies — {milestone}", + "", + f"Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}", + "", + "---", + "", + "## Summary", + "", + f"- **OPEN issue with a MERGED PR:** {len(t1)}", + f"- **Milestone issue, PR elsewhere/unprojected:** {len(t2)}", + f"- **Milestone PR, issue elsewhere/unprojected:** {len(t3)}", + f"- **MERGED PR on a `needs triage` issue:** {len(t4)}", + f"- **MERGED PR on an unassigned, non-community issue:** {len(t5)}", + f"- **Total anomalies:** {total}", + "", + ] + + if total == 0: + lines += [ + "✅ No anomalies found. The milestone and the Main project board agree.", + "", + ] + else: + lines += ["## Anomalies", ""] + if t1: + lines += [ + "### OPEN issue with a MERGED PR", + "", + "The fix already landed but the issue never closed (or it was " + "reopened). Close the issue or move it out of the milestone.", + "", + ] + for e in t1: + lines.append(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}") + for n in e["prs"]: + lines.append(f" - 🔴 Merged {pr_link(n)}") + lines.append("") + + if t2: + lines += [ + "### Milestone issue, PR elsewhere or issue off the Main board", + "", + ] + by_issue: dict[int, list[dict]] = {} + for e in t2: + by_issue.setdefault(e["issue"], []).append(e) + for issue_num in sorted(by_issue): + entries = by_issue[issue_num] + lines.append( + f"- {issue_link_title(issue_num, entries[0]['issue_title'][:80])}" + ) + for e in entries: + if e["pr"] is None: + projects = ", ".join(e["projects"]) or "_none_" + lines.append( + " - 🔴 Issue is not on the Main project board " + f"(projects: {projects})" + ) + else: + ms = e["pr_milestone"] or "_none_" + badge = "🔴" if e["pr_milestone"] is None else "⚠️" + lines.append( + f" - {badge} Closing {pr_link(e['pr'])} is in milestone " + f"**{ms}** (expected: {milestone})" + ) + if e["pr_milestone"] is None: + lines.append( + f" {milestone_fix('pr', e['pr'], milestone)}" + ) + lines.append("") + + if t3: + lines += [ + "### Milestone PR, issue elsewhere or off the Main board", + "", + ] + by_pr: dict[int, list[dict]] = {} + for e in t3: + by_pr.setdefault(e["pr"], []).append(e) + for pr_num in sorted(by_pr): + entries = by_pr[pr_num] + lines.append( + f"- {pr_link_title(pr_num, entries[0]['pr_title'][:80])}" + ) + for e in entries: + ms = e["issue_milestone"] or "_none_" + if e["issue_milestone"] != milestone: + lines.append( + f" - ⚠️ Closing {issue_link(e['issue'])} is in milestone " + f"**{ms}** (expected: {milestone})" + ) + if e["issue_milestone"] is None: + lines.append( + f" {milestone_fix('issue', e['issue'], milestone)}" + ) + else: + projects = ", ".join(e["projects"]) or "_none_" + lines.append( + f" - 🔴 Closing {issue_link(e['issue'])} is not on the " + f"Main project board (projects: {projects})" + ) + lines.append("") + + if t4: + lines += [ + "### MERGED PR on a `needs triage` issue", + "", + "The fix landed but the issue was never triaged. Human triage needed.", + "", + ] + for e in t4: + lines.append( + f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}" + f" closes {issue_link(e['issue'])}" + ) + lines.append("") + + if t5: + lines += [ + "### MERGED PR on an unassigned, non-community issue", + "", + "No owner and no community label. Assign an owner or confirm it " + "is a community contribution.", + "", + ] + for e in t5: + lines.append( + f"- ⚠️ {pr_link_title(e['pr'], e['pr_title'][:80])}" + f" closes {issue_link(e['issue'])}" + ) + lines.append("") + + lines += [ + "---", + "", + "## Context", + "", + f"- Milestone: **{milestone}**", + f"- Milestone issues (all states): {total_issues}", + f"- Open issues in milestone: {total_issues - closed_issues}", + f"- Milestone PRs (all states): {total_prs}", + f"- Merged PRs in milestone: {merged_prs}", + "", + ] + return "\n".join(lines) + + +# ───────────────────────────────────────────── +# CLI entrypoint +# ───────────────────────────────────────────── + + +def cmd_check(args: argparse.Namespace) -> None: + """Fetch milestone data in bulk, detect anomalies, write the report.""" + milestone = args.milestone + + all_issues = json.loads(run_gh_py("issues", milestone, "--state", "all")) + issue_by_num = {i["number"]: i for i in all_issues} + + all_prs = json.loads(run_gh_py("prs", "--milestone", milestone, "--state", "all")) + pr_by_num = {p["number"]: p for p in all_prs} + + # Resolve outsiders in bulk (one GraphQL call per 50 items). + unknown_prs = set() + for issue in all_issues: + unknown_prs.update(issue.get("closing_prs", [])) + unknown_prs -= set(pr_by_num) + if unknown_prs: + print( + f"Resolving {len(unknown_prs)} PRs outside {milestone}...", + file=sys.stderr, + ) + for pr in json.loads(run_gh_py("prs", *[str(n) for n in sorted(unknown_prs)])): + pr_by_num[pr["number"]] = pr + + unknown_issues = set() + for pr in all_prs: + unknown_issues.update(pr.get("closing_issues", [])) + unknown_issues -= set(issue_by_num) + if unknown_issues: + print( + f"Resolving {len(unknown_issues)} issues outside {milestone}...", + file=sys.stderr, + ) + for issue in json.loads( + run_gh_py("issue", *[str(n) for n in sorted(unknown_issues)]) + ): + if "error" not in issue: + issue_by_num[issue["number"]] = issue + + anomalies = find_anomalies(milestone, all_issues, all_prs, pr_by_num, issue_by_num) + report = render_report( + milestone, + anomalies, + total_issues=len(all_issues), + closed_issues=sum(1 for i in all_issues if i.get("state") == "CLOSED"), + total_prs=len(all_prs), + merged_prs=sum(1 for p in all_prs if p.get("state") == "MERGED"), + ) + + output = Path(args.output or f"tmp/{milestone}-ANOMALIES.md") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(report) + total = sum(len(v) for v in anomalies.values()) + print(f"{total} anomalies written to {output}") + + +def main() -> None: + parser = argparse.ArgumentParser( + description="Find Main-project anomalies for a milestone" + ) + sub = parser.add_subparsers(dest="command", required=True, title="subcommands") + + p_check = sub.add_parser("check", help="Detect anomalies and write the report") + p_check.add_argument("milestone", help="Milestone title (e.g. '2.17.0')") + p_check.add_argument( + "--output", + default=None, + help="Report path (default: tmp/-ANOMALIES.md)", + ) + p_check.set_defaults(func=cmd_check) + + args = parser.parse_args() + args.func(args) + + +if __name__ == "__main__": + main() diff --git a/scripts/test_project_anomalies.py b/scripts/test_project_anomalies.py new file mode 100644 index 0000000000..ca781c58d2 --- /dev/null +++ b/scripts/test_project_anomalies.py @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 +"""Tests for scripts/project-anomalies.py (pure logic, no network). + +Run with: + + python3 scripts/test_project_anomalies.py +""" + +import importlib.machinery +import importlib.util +import pathlib +import sys +import unittest + + +# Loading scripts/project-anomalies.py should not emit scripts/__pycache__/. +sys.dont_write_bytecode = True + +SCRIPT_PATH = pathlib.Path(__file__).resolve().parent / "project-anomalies.py" + + +def load_pa(): + """Load scripts/project-anomalies.py as a module without running its CLI.""" + loader = importlib.machinery.SourceFileLoader("project_anomalies", str(SCRIPT_PATH)) + spec = importlib.util.spec_from_loader("project_anomalies", loader) + module = importlib.util.module_from_spec(spec) + loader.exec_module(module) + return module + + +pa = load_pa() + +M = "2.17.0" + + +def issue(n, **kw): + base = { + "number": n, + "title": f"issue {n}", + "state": "CLOSED", + "milestone": M, + "labels": [], + "assignees": ["someone"], + "closing_prs": [], + "project_status": "Done", + "projects": ["Main"], + } + base.update(kw) + return base + + +def pr(n, **kw): + base = { + "number": n, + "title": f"pr {n}", + "state": "MERGED", + "milestone": M, + "labels": [], + "assignees": ["someone"], + "closing_issues": [], + } + base.update(kw) + return base + + +def run(issues, prs): + return pa.find_anomalies( + M, issues, prs, + {p["number"]: p for p in prs}, + {i["number"]: i for i in issues}, + ) + + +class OpenMergedTests(unittest.TestCase): + def test_open_issue_with_merged_pr_is_flagged(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["open_merged"]), 1) + self.assertEqual(out["open_merged"][0]["prs"], [11]) + + def test_open_issue_with_unmerged_pr_is_clean(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, state="OPEN", closing_issues=[1])], + ) + self.assertEqual(out["open_merged"], []) + + def test_closed_issue_with_merged_pr_is_clean(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertTrue(all(v == [] for v in out.values())) + + +class IssuePrMismatchTests(unittest.TestCase): + def test_pr_in_other_milestone_is_flagged(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, milestone="2.18.0")], + ) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertEqual(out["issue_pr"][0]["pr_milestone"], "2.18.0") + + def test_pr_without_milestone_is_flagged(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, milestone=None)], + ) + self.assertEqual(len(out["issue_pr"]), 1) + + def test_unmerged_pr_milestone_is_ignored(self): + # An OPEN (unmerged) PR has landed nowhere: its milestone is + # irrelevant, even when it differs. + out = run( + [issue(1, closing_prs=[11])], + [pr(11, state="OPEN", milestone="2.18.0")], + ) + self.assertEqual(out["issue_pr"], []) + + def test_off_board_issue_flagged_despite_unmerged_pr(self): + # The project check does not depend on PR merge state. + out = run( + [issue(1, projects=["Other"], closing_prs=[11])], + [pr(11, state="OPEN", milestone="2.18.0")], + ) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertIsNone(out["issue_pr"][0]["pr"]) + + def test_issue_off_main_board_is_flagged_without_pr(self): + out = run([issue(1, projects=["Other"], project_status=None)], []) + self.assertEqual(len(out["issue_pr"]), 1) + self.assertIsNone(out["issue_pr"][0]["pr"]) + + def test_aligned_pair_is_clean(self): + out = run( + [issue(1, closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["issue_pr"], []) + self.assertEqual(out["pr_issue"], []) + + +class PrIssueMismatchTests(unittest.TestCase): + def test_issue_in_other_milestone_is_flagged(self): + other = issue(2, milestone="2.16.0") + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_issue_without_milestone_is_flagged(self): + # Unlike the changelog flow, here a milestone-less issue closed by a + # milestone PR is reported: the pairing needs human judgment. + other = issue(2, milestone=None, projects=[], project_status=None) + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_issue_off_main_board_is_flagged(self): + other = issue(2, projects=["Other"], project_status=None) + out = pa.find_anomalies( + M, [issue(1)], [pr(11, closing_issues=[2])], + {11: pr(11, closing_issues=[2])}, {1: issue(1), 2: other}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + + def test_milestone_issue_keeps_milestone_in_record(self): + # Milestone issues carry no "milestone" key (known by construction); + # an off-board one must still render its own milestone, never _none_. + mine = issue(1, projects=["Other"], project_status=None) + del mine["milestone"] + out = pa.find_anomalies( + M, [mine], [pr(11, closing_issues=[1])], + {11: pr(11, closing_issues=[1])}, {1: mine}, + ) + self.assertEqual(len(out["pr_issue"]), 1) + self.assertEqual(out["pr_issue"][0]["issue_milestone"], M) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertNotIn("gh issue edit 1", report) + + def test_unmerged_pr_is_ignored(self): + # Closed-unmerged (or open) milestone PRs take no part in milestone + # pairing: only merged PRs landed somewhere. + other = issue(2, milestone="2.16.0") + for state in ("OPEN", "CLOSED"): + out = pa.find_anomalies( + M, [issue(1)], [pr(11, state=state, closing_issues=[2])], + {11: pr(11, state=state, closing_issues=[2])}, + {1: issue(1), 2: other}, + ) + self.assertEqual(out["pr_issue"], [], f"state={state}") + + +class NeedsTriageTests(unittest.TestCase): + def test_merged_pr_on_triaged_issue_is_clean(self): + out = run( + [issue(1, labels=["bug"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["needs_triage"], []) + + def test_merged_pr_on_needs_triage_issue_is_flagged(self): + out = run( + [issue(1, labels=["needs triage"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["needs_triage"]), 1) + + def test_unmerged_pr_on_needs_triage_issue_is_clean(self): + out = run( + [issue(1, labels=["needs triage"])], + [pr(11, state="OPEN", closing_issues=[1])], + ) + self.assertEqual(out["needs_triage"], []) + + +class UnassignedTests(unittest.TestCase): + def test_assigned_issue_is_clean(self): + out = run( + [issue(1, assignees=["owner"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_community_issue_is_clean(self): + out = run( + [issue(1, assignees=[], labels=["community contribution"])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_community_pr_is_clean(self): + out = run( + [issue(1, assignees=[])], + [pr(11, closing_issues=[1], labels=["community contribution"])], + ) + self.assertEqual(out["unassigned"], []) + + def test_unassigned_non_community_issue_is_flagged(self): + out = run( + [issue(1, assignees=[])], + [pr(11, closing_issues=[1])], + ) + self.assertEqual(len(out["unassigned"]), 1) + + def test_unmerged_pr_is_clean(self): + out = run( + [issue(1, assignees=[])], + [pr(11, state="CLOSED", closing_issues=[1])], + ) + self.assertEqual(out["unassigned"], []) + + +class RenderTests(unittest.TestCase): + def test_clean_report_says_so(self): + out = run([issue(1, closing_prs=[11])], [pr(11, closing_issues=[1])]) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertIn("Total anomalies:** 0", report) + self.assertIn("✅ No anomalies", report) + + def test_every_number_is_a_clickable_link(self): + out = run( + [issue(1, state="OPEN", closing_prs=[11])], + [pr(11, closing_issues=[1])], + ) + report = pa.render_report(M, out, 1, 0, 1, 1) + self.assertIn("[#1](https://github.com/penpot/penpot/issues/1)", report) + self.assertIn("[#11](https://github.com/penpot/penpot/pull/11)", report) + + +if __name__ == "__main__": + unittest.main(verbosity=2)