From 0efa550a99bbc5676f795552e5ed2cbee64ba6cb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?andr=C3=A9s=20gonz=C3=A1lez?= Date: Wed, 11 Nov 2026 13:00:06 +0100 Subject: [PATCH] :sparkles: Improve the changelog flow: dev-only bugs and feature sections (#12981) * :sparkles: Keep bugs that never reached production out of the changelog The team rarely adds `no changelog` to bugs found while building a feature of the same release, so they end up in the changelog and make Penpot look less stable than it is. Add `changelog.py dev-only-bugs`. Bugs whose parent is a Feature or Enhancement of the same milestone are sure: `--apply` labels them `no changelog` on GitHub. Bugs under an epic of the milestone, with the `regression` label, or whose version field names only hourly, develop, staging, PRE or this milestone, are candidates for a human to confirm; the anomaly report lists them as warnings. `gh.py issue` now returns body and parent. The flow memory documents it as step 3. Closes #12747 AI-assisted-by: claude-opus-5-5 * :sparkles: Split features and enhancements in the changelog flow Features were listed together with enhancements under ":sparkles: New features & Enhancements", so they did not stand out. Map the Feature issue type to its own ":tada: Features" section, keep Enhancement under ":sparkles: Enhancements", rename the bug section to ":bug: Bugs" and set the order: boom, rocket, lock, tada, sparkles, bug. The new layout applies from 2.20.0: 2.19.0 and 2.19.1 are already out with the old one, so 2.19.x patches keep it. Closes #12741 AI-assisted-by: claude-opus-5-5 --- .agents/skills/update-changelog/SKILL.md | 5 +- .serena/memories/scripts/changelog.md | 3 +- .serena/memories/scripts/gh.md | 2 +- .../memories/workflow/updating-changelog.md | 52 ++++--- scripts/changelog.py | 131 ++++++++++++++++++ scripts/gh.py | 9 ++ scripts/test_changelog.py | 56 ++++++++ 7 files changed, 235 insertions(+), 23 deletions(-) diff --git a/.agents/skills/update-changelog/SKILL.md b/.agents/skills/update-changelog/SKILL.md index c83ab8d1f7..87c493e7c4 100644 --- a/.agents/skills/update-changelog/SKILL.md +++ b/.agents/skills/update-changelog/SKILL.md @@ -31,7 +31,7 @@ memory; it does not repeat it. layout, entry rules, the `## Breaking change note` comment contract, the advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls. 2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`, - `report`. + `dev-only-bugs`, `report`. 3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`. ## Commands @@ -39,6 +39,8 @@ memory; it does not repeat it. ```bash python3 scripts/gh.py issues "" --exclude "release blocker,no changelog" python3 scripts/gh.py advisories --patched-version "" +python3 scripts/changelog.py dev-only-bugs "" # list bugs that never reached production +python3 scripts/changelog.py dev-only-bugs "" --apply # label the sure ones `no changelog` python3 scripts/changelog.py check-merged python3 scripts/changelog.py boom "" # print the :boom: section python3 scripts/changelog.py boom "" --write # replace it in CHANGES.md @@ -54,4 +56,5 @@ python3 scripts/changelog.py report "" - Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline. - `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command. - Verify PR merge status before writing; only merged PRs are referenced. +- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself. - Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid. diff --git a/.serena/memories/scripts/changelog.md b/.serena/memories/scripts/changelog.md index 7134377fa5..6785692d82 100644 --- a/.serena/memories/scripts/changelog.md +++ b/.serena/memories/scripts/changelog.md @@ -8,7 +8,8 @@ - `boom [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### ` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`. - `security [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix () (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default. - `cross-ref [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design. -- `report [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place. +- `dev-only-bugs [--apply]` — list closed milestone bugs that never reached production, without `no changelog`. Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or this milestone (`only_unreleased_envs`; any production hint or other version keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup. +- `report [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place. ## Conventions diff --git a/.serena/memories/scripts/gh.md b/.serena/memories/scripts/gh.md index c2a10d8f6b..6b546e578c 100644 --- a/.serena/memories/scripts/gh.md +++ b/.serena/memories/scripts/gh.md @@ -81,7 +81,7 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts. -**Output**: JSON array to stdout; progress to stderr. +**Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both. ### `comments` diff --git a/.serena/memories/workflow/updating-changelog.md b/.serena/memories/workflow/updating-changelog.md index 66d1220fa7..1c233312b3 100644 --- a/.serena/memories/workflow/updating-changelog.md +++ b/.serena/memories/workflow/updating-changelog.md @@ -24,11 +24,12 @@ Editing `CHANGES.md` happens only on explicit request or as part of the release | Is it a breaking change? | `breaking change` label | | Is it a highlight? | `release highlight` label | -- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them. +- **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them. - **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug). - **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry. - **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge. - **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status. +- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3). - **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own. - **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly. @@ -39,7 +40,7 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside ```markdown # CHANGELOG -## 2.19.0 (Unreleased) +## 2.20.0 (Unreleased) ### :boom: Breaking changes & Deprecations @@ -57,21 +58,27 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside - Fix () (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) -### :bug: Bugs fixed - -- Fix [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) - -### :sparkles: New features & Enhancements +### :tada: Features - Add [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) + +### :sparkles: Enhancements + +- Improve [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) + +### :bug: Bugs + +- Fix [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) ``` -- `:boom:` first, then `:rocket:`, `:lock:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions. +- `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update. +- **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it. +- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions. - Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories. - One blank line between the last entry of a section and the next heading. - No manual line wrapping: one line per bullet or paragraph, however long. -## 4. Regular entries (`:bug:`, `:sparkles:`) +## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`) The entry shape is: @@ -152,15 +159,18 @@ Generated from the advisories that declare the version in `patched_versions`; ne 1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious. 2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`. -3. **Fetch PR details when needed**: `python3 scripts/gh.py prs ` (also `--milestone`). Use it for the PR author, labels and merge state. -4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4). -5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged `. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it. -6. **Reconcile the existing text** (section 9). -7. **Generate `:boom:`**: `python3 scripts/changelog.py boom ""` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent. -8. **Generate `:rocket:`** from the `release highlight` label (section 6). -9. **Generate `:lock:`**: `python3 scripts/changelog.py security ""` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7. -10. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref ""`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design. -11. **Generate the anomaly report**: `python3 scripts/changelog.py report ""`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10). +3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs ""` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration. + - **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these. + - **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or a version of this milestone. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10). +4. **Fetch PR details when needed**: `python3 scripts/gh.py prs ` (also `--milestone`). Use it for the PR author, labels and merge state. +5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4). +6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged `. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it. +7. **Reconcile the existing text** (section 9). +8. **Generate `:boom:`**: `python3 scripts/changelog.py boom ""` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent. +9. **Generate `:rocket:`** from the `release highlight` label (section 6). +10. **Generate `:lock:`**: `python3 scripts/changelog.py security ""` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7. +11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref ""`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design. +12. **Generate the anomaly report**: `python3 scripts/changelog.py report ""`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10). ## 9. Pre-flight: reconcile before writing @@ -171,7 +181,7 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` 3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry. 4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry. 5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently. -6. **Issue Type changed to `Task`** → remove the entry. +6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section. 7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry. 8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it. 9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section. @@ -188,6 +198,8 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` - **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights). - **D — `:rocket:` entry without issue AND PR references** (gap). +**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total. + Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run. ## 11. Pitfalls @@ -202,6 +214,6 @@ Anything else is a rule violation, not a report item: fix it in the pre-flight a ## 12. Where the code is - `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`. -- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `report`. Details: `mem:scripts/changelog`. +- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`. - Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`. - The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**. diff --git a/scripts/changelog.py b/scripts/changelog.py index 280ffa1f85..e09cfa1218 100644 --- a/scripts/changelog.py +++ b/scripts/changelog.py @@ -10,6 +10,7 @@ Subcommands: boom Generate the :boom: section from `breaking change` issues security Generate the :lock: section from advisories fixed in a version cross-ref Compare merged milestone PRs against the changelog section + dev-only-bugs List milestone bugs that never reached production report Generate the anomaly report (CHANGES-ISSUES.md) Usage: @@ -20,6 +21,7 @@ Usage: python3 scripts/changelog.py security "2.19.1" python3 scripts/changelog.py security "2.19.1" --write python3 scripts/changelog.py cross-ref "2.16.0" + python3 scripts/changelog.py dev-only-bugs "2.16.0" [--apply] python3 scripts/changelog.py report "2.16.0" python3 scripts/changelog.py report "2.16.0" --changes CHANGES.md --output CHANGES-ISSUES.md @@ -615,6 +617,103 @@ def cmd_cross_ref(args: argparse.Namespace) -> None: print(f" #{pr['number']} {pr['title'][:80]}") +# ───────────────────────────────────────────── +# Subcommand: dev-only-bugs +# ───────────────────────────────────────────── + +# Bugs found and fixed while building a feature of the same, unreleased +# milestone never reached end users and stay out of the changelog. +AUTO_PARENT_TYPES = {"Feature", "Enhancement"} + +VERSION_FIELD_RE = re.compile( + r"^#{2,4}\s*(?:affected versions?|versions?|environment)[^\n]*\n(.*?)(?=^#{2,4}\s|\Z)", + re.IGNORECASE | re.MULTILINE | re.DOTALL, +) +DEV_ENV_RE = re.compile(r"\b(?:hourly|develop|staging|pre|unreleased)\b", re.IGNORECASE) +PROD_ENV_RE = re.compile( + r"\b(?:production|pro|main|saas|design\.penpot\.app|self[- ]?hosted?)\b", + re.IGNORECASE, +) +VERSION_RE = re.compile(r"\b(\d+)\.(\d+)(?:\.\d+)?") + + +def version_field(body: str | None) -> str: + """Return the "Affected versions" / "Environment" text of an issue body.""" + match = VERSION_FIELD_RE.search(body or "") + return " ".join(match.group(1).split()) if match else "" + + +def only_unreleased_envs(text: str, milestone: str) -> bool: + """True when the version text names unreleased environments only: + hourly, develop, staging, PRE or a version of this milestone. Any + production hint or other version number keeps the bug in the changelog.""" + current = tuple(milestone.split(".")[:2]) + versions = [m for m in VERSION_RE.findall(text)] + dev = bool(DEV_ENV_RE.search(text)) or current in versions + prod = bool(PROD_ENV_RE.search(text)) or any(v != current for v in versions) + return dev and not prod + + +def find_dev_only_bugs(milestone: str, issues: list[dict]) -> tuple[list[dict], list[dict]]: + """Split the milestone's changelog bugs that never reached production. + + ``issues`` come from ``gh.py issue`` (they carry ``body`` and + ``parent``). Returns ``(auto, suggested)``: ``auto`` bugs are + sub-issues of a Feature or Enhancement of this milestone and get the + `no changelog` label; ``suggested`` ones need a human to confirm.""" + auto, suggested = [], [] + for issue in issues: + if issue.get("issue_type") != "Bug" or issue.get("state") != "CLOSED": + continue + if {"no changelog", "release blocker"} & set(issue.get("labels", [])): + continue + parent = issue.get("parent") or {} + entry = {"issue": issue["number"], "issue_title": issue.get("title", "")} + if parent.get("milestone") == milestone: + entry["reason"] = f"sub-issue of {parent.get('issue_type')} #{parent['number']} in {milestone}" + (auto if parent.get("issue_type") in AUTO_PARENT_TYPES else suggested).append(entry) + continue + # `regression` says something that worked broke, not where: it + # also marks regressions users hit in production. + if "regression" in issue.get("labels", []): + entry["reason"] = "labelled `regression`" + suggested.append(entry) + continue + text = version_field(issue.get("body")) + if only_unreleased_envs(text, milestone): + entry["reason"] = f'only seen on "{text[:80]}"' + suggested.append(entry) + return auto, suggested + + +def fetch_milestone_bugs(milestone: str) -> list[dict]: + """Closed milestone bugs with body and parent (two batched lookups).""" + issues = json.loads(run_gh_py("issues", milestone)) + numbers = [str(i["number"]) for i in issues if i.get("issue_type") == "Bug"] + return json.loads(run_gh_py("issue", *numbers)) if numbers else [] + + +def cmd_dev_only_bugs(args: argparse.Namespace) -> None: + """List bugs that never reached production; --apply labels the sure ones.""" + auto, suggested = find_dev_only_bugs(args.milestone, fetch_milestone_bugs(args.milestone)) + + print(f"Label `no changelog` ({len(auto)}):") + for e in auto: + print(f" #{e['issue']} {e['issue_title'][:70]} ({e['reason']})") + print(f"Candidates to confirm ({len(suggested)}):") + for e in suggested: + print(f" #{e['issue']} {e['issue_title'][:70]} ({e['reason']})") + + if args.apply: + for e in auto: + subprocess.run( + ["gh", "issue", "edit", str(e["issue"]), "--repo", REPO, + "--add-label", "no changelog"], + check=True, capture_output=True, + ) + print(f"Labelled {len(auto)} issues `no changelog`", file=sys.stderr) + + # ───────────────────────────────────────────── # Subcommand: report # ───────────────────────────────────────────── @@ -872,6 +971,9 @@ def cmd_report(args: argparse.Namespace) -> None: ): anomalies_d.append({"version": ver, "line": line[:100]}) + # Bugs that may never have reached production: warnings, not anomalies. + dev_auto, dev_suggested = find_dev_only_bugs(milestone, fetch_milestone_bugs(milestone)) + def fmt_ms(ms: str | None) -> str: return ms if ms else "_none_" @@ -1033,6 +1135,26 @@ def cmd_report(args: argparse.Namespace) -> None: "✅ No highlight gaps found. All released X.Y.0 versions have properly referenced :rocket: entries.\n\n" ) + if dev_auto or dev_suggested: + f.write("## Bugs that may never have reached production\n\n") + f.write( + "These are warnings, not anomalies. Bugs found while building " + "a feature of this release never affected end users. Add the " + "`no changelog` label to the ones you confirm and re-run.\n\n" + ) + if dev_auto: + f.write( + "### Sub-issues of a feature of this milestone (run `dev-only-bugs --apply`)\n\n" + ) + for e in dev_auto: + f.write(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}: {e['reason']}\n") + f.write("\n") + if dev_suggested: + f.write("### Candidates to confirm\n\n") + for e in dev_suggested: + f.write(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}: {e['reason']}\n") + f.write("\n") + f.write("---\n\n") f.write("## Context\n\n") f.write(f"- Milestone: **{milestone}**\n") @@ -1100,6 +1222,15 @@ def main() -> None: help="Path to the changelog file (default: CHANGES.md)") p_cross.set_defaults(func=cmd_cross_ref) + p_dev = sub.add_parser( + "dev-only-bugs", + help="List milestone bugs that never reached production", + ) + p_dev.add_argument("milestone", help="Milestone title (e.g. '2.16.0')") + p_dev.add_argument("--apply", action="store_true", + help="Label the sub-issues of a milestone feature `no changelog`") + p_dev.set_defaults(func=cmd_dev_only_bugs) + p_report = sub.add_parser( "report", help="Generate the anomaly report (CHANGES-ISSUES.md)" ) diff --git a/scripts/gh.py b/scripts/gh.py index eebd5155fd..30c12e58f2 100755 --- a/scripts/gh.py +++ b/scripts/gh.py @@ -752,8 +752,10 @@ GQL_ISSUE_BY_NUMBER_QUERY_ITEM = """\ number title state + body milestone {{ title }} issueType {{ name }} + parent {{ number issueType {{ name }} milestone {{ title }} }} labels(first: 20) {{ nodes {{ name }} }} assignees(first: 10) {{ nodes {{ login }} }} closedByPullRequestsReferences(first: 5) {{ nodes {{ number }} }} @@ -775,12 +777,19 @@ def issue_node_to_dict(node: dict) -> dict: """Convert a GraphQL issue node to the shared issue dict shape.""" issue_type = node.get("issueType") ms = node.get("milestone") + parent = node.get("parent") return { "number": node["number"], "title": node["title"], "state": node["state"], + "body": node.get("body"), "milestone": ms["title"] if ms else None, "issue_type": issue_type["name"] if issue_type else None, + "parent": { + "number": parent["number"], + "issue_type": (parent.get("issueType") or {}).get("name"), + "milestone": (parent.get("milestone") or {}).get("title"), + } if parent else None, "labels": [lbl["name"] for lbl in node["labels"]["nodes"]], "assignees": node_assignees(node), "closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]], diff --git a/scripts/test_changelog.py b/scripts/test_changelog.py index e27d3de1d2..8eb0311c9f 100644 --- a/scripts/test_changelog.py +++ b/scripts/test_changelog.py @@ -657,5 +657,61 @@ class CmdSecurityTests(unittest.TestCase): self.assertIn("will not be listed", err) +def bug(number, body="", parent=None, labels=()): + return { + "number": number, + "title": f"Bug {number}", + "state": "CLOSED", + "issue_type": "Bug", + "labels": list(labels), + "body": body, + "parent": parent, + } + + +class OnlyUnreleasedEnvsTests(unittest.TestCase): + def check(self, text): + return ch.only_unreleased_envs(text, "2.19.0") + + def test_dev_environments(self): + for text in ["HOURLY", "https://hourly.penpot.dev/", "PRE & hourly", + "staging (current)", "develop (2.19.0 unreleased)", + "2.19.0-RC1 (develop)", "2.19.0"]: + with self.subTest(text=text): + self.assertTrue(self.check(text)) + + def test_production_hints(self): + for text in ["PRO", "pre/pro/hourly", "develop / staging / main", + "2.18.0-RC5", "2.18.1, 2.19.0-RC1", "self-hosted: 2.17.2", + "design.penpot.app and self-host"]: + with self.subTest(text=text): + self.assertFalse(self.check(text)) + + def test_no_signal(self): + for text in ["", "_No response_", "-", "development build"]: + with self.subTest(text=text): + self.assertFalse(self.check(text)) + + +class FindDevOnlyBugsTests(unittest.TestCase): + def test_splits_auto_and_suggested(self): + issues = [ + bug(1, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"}), + bug(2, parent={"number": 11, "issue_type": "Enhancement", "milestone": "2.19.0"}), + bug(3, parent={"number": 12, "issue_type": "EPIC", "milestone": "2.19.0"}), + bug(4, parent={"number": 13, "issue_type": "Feature", "milestone": "2.18.0"}), + bug(5, body="### Affected versions\n\nhourly\n"), + bug(6, body="### Affected versions\n\n2.18.2\n"), + bug(7, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"}, + labels=["no changelog"]), + {**bug(8, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"}), + "issue_type": "Enhancement"}, + bug(9, body="### Affected versions\n\n2.18.2\n", labels=["regression"]), + ] + auto, suggested = ch.find_dev_only_bugs("2.19.0", issues) + self.assertEqual([e["issue"] for e in auto], [1, 2]) + self.assertEqual([e["issue"] for e in suggested], [3, 5, 9]) + + if __name__ == "__main__": unittest.main(verbosity=2)