Add improvements to 'update-changelog' skill

This commit is contained in:
Andrey Antukh 2026-05-27 12:38:00 +02:00
parent 40ce360c99
commit 243798f458
2 changed files with 246 additions and 7 deletions

View File

@ -45,10 +45,17 @@ python3 tools/gh.py issues "2.16.0" --state all
python3 tools/gh.py issues "2.16.0" --exclude "release blocker,no changelog"
```
**Exclusion rules:**
**Exclusion rules (issue-level):**
- `no changelog` label — Chore/refactor work that doesn't need a changelog entry
- `release blocker` label — Blocked issues not yet ready for changelog
- `Task` issue type — Internal chores are not user-facing; filter these out after fetching
**Exclusion rules (PR-level):**
In addition to issue-level exclusions, PRs with these labels should be
excluded regardless of their linked issue's labels:
- `release blocker` — PR is part of a pending release blocker batch
- `no issue required` — Trivial fix not tracked as an issue
The script outputs JSON with each entry containing `number`, `title`, `state`,
`issue_type`, `labels`, and `closing_prs` (the PRs that fix each issue).
@ -63,6 +70,10 @@ python3 tools/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --c
This returns a filtered JSON array with only the missing issues.
> **Note:** The `--compare` flag checks **issues** only (via issue number
> references in the changelog). To find merged **PRs** not yet referenced,
> use the milestone PR cross-reference described in step 10 below.
### 4. Fetch additional PR details when needed
When you need more context for specific PRs (e.g. to find the PR author for
@ -80,9 +91,20 @@ python3 tools/gh.py prs --file prs.txt
cat prs.txt | python3 tools/gh.py prs --stdin
```
The `prs` command also supports listing all PRs in a milestone in one call:
```bash
# All merged PRs in a milestone (default)
python3 tools/gh.py prs --milestone "2.16.0"
# All states (merged, open, closed)
python3 tools/gh.py prs --milestone "2.16.0" --state all
```
The `prs` command returns JSON with `number`, `title`, `body`, `state`,
`merged_at`, `author`, `labels`, and `closing_issues`. PRs are fetched in
batches of 50 via GraphQL to stay within API limits.
batches of 50 via GraphQL to stay within API limits (milestone mode uses
paginated GraphQL on the milestone's `pullRequests` connection).
### 5. Categorize entries — strictly by issue type, never by labels or emoji
@ -134,6 +156,37 @@ tracked in the milestone.
| PR exists with no linked issue | If a corresponding closed issue exists in the same milestone, link the issue. Otherwise, skip the entry (the issue must be the changelog unit). |
| Closed issue with no fix PR in milestone | Link the issue directly, without a PR reference. |
> **False-positive associations:** A PR may incorrectly claim to close an issue
> from a different context (e.g., a very old PR referencing a modern issue, or a
> cross-project reference). If the PR title and issue title are clearly unrelated,
> or the PR was created years before the issue, treat it as a data glitch and
> skip it. PR [#3](https://github.com/penpot/penpot/pull/3) (ancient License PR
> claiming to close a plugin API issue) is a known example.
### 5a. ⚠️ Verify PR merge status before writing
A closed issue may list closing PRs that were **closed without merging**
(e.g., a community PR that was superseded by another). The changelog must
only reference **merged** PRs. Verify before writing:
```bash
# Collect all PR numbers from the candidate entries and check them
python3 tools/gh.py prs <ALL_PR_NUMBERS> | python3 -c "
import json, sys
for pr in json.load(sys.stdin):
if pr['state'] != 'MERGED':
print(f'WARNING: #{pr[\"number\"]} is {pr[\"state\"]} (not merged)')
"
```
If a closing PR is closed-unmerged, find the actual merged PR that
superseded it:
1. Check the issue's closing PRs list for other PRs (there may be multiple)
2. Look for other PRs with similar titles or descriptions referencing the same issue
3. Inspect the closed PR's conversation timeline for a pointer to the replacement
Replace the reference in the changelog entry with the correct merged PR number.
### 6. Read the current CHANGES.md
Read the top of `CHANGES.md` to understand the existing format and find the
@ -198,6 +251,72 @@ Read the top of `CHANGES.md` and confirm:
- The section ordering is correct (newest first)
- Formatting matches the surrounding entries
### 10. Cross-reference milestone PRs against the changelog
Issues can be fixed by PRs that aren't in the milestone, and merged PRs in
the milestone may not close any tracked issue. After writing, run a full
cross-reference to catch gaps:
```bash
# List all merged PRs in the milestone
python3 tools/gh.py prs --milestone "<MILESTONE>" --state merged > /tmp/milestone-prs.json
# Extract PR numbers from the changelog section
python3 -c "
import json, re
with open('CHANGES.md') as f:
content = f.read()
# Extract the version section (adjust regex to match the actual version)
match = re.search(r'## <MILESTONE> \(Unreleased\)\n(.*?)(?:\n## |\Z)', content, re.DOTALL)
section = match.group(1)
# Collect all PR numbers referenced
changelog_prs = set()
for m in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section):
changelog_prs.add(int(m))
# Collect all milestone PRs (filtered)
with open('/tmp/milestone-prs.json') as f:
milestone_prs = json.load(f)
milestone_merged = {pr['number'] for pr in milestone_prs}
# PRs in milestone but not in changelog
missing = sorted(milestone_merged - changelog_prs)
print(f'Milestone merged PRs: {len(milestone_merged)}')
print(f'Changelog referenced PRs: {len(changelog_prs)}')
print(f'PRs in milestone but NOT in changelog: {len(missing)}')
for num in missing:
pr = next(p for p in milestone_prs if p['number'] == num)
print(f' #{num} {pr[\"title\"][:80]}')
"
```
For each missing PR found, decide whether it should be added to the
changelog or is legitimately excluded (check its labels).
Also verify that no closed-unmerged PRs remain in the changelog:
```bash
python3 tools/gh.py prs --milestone "<MILESTONE>" --state all | python3 -c "
import json, sys
data = json.load(sys.stdin)
closed = [p for p in data if p['state'] == 'CLOSED']
if closed:
print('WARNING: CLOSED (unmerged) PRs in milestone:')
for p in closed:
print(f' #{p[\"number\"]} {p[\"title\"][:80]}')
"
```
**Post-edit audit checklist:**
- ✅ All referenced PRs are merged (no closed-unmerged artifacts)
- ✅ Every merged milestone PR is either in the changelog or excluded by label
- ✅ PR and issue counts are internally consistent
- ✅ No false-positive PR-to-issue associations
## Version section template
```markdown
@ -244,3 +363,17 @@ Read the top of `CHANGES.md` and confirm:
- **Use `tools/gh.py`.** Prefer the helper script over raw `gh api` calls for
milestone issue listing and PR detail fetching. It handles GraphQL
pagination, batching, and label filtering automatically.
- **Verify PR merge status.** Not all closing PRs are merged — community PRs
can be superseded and closed without merging. Always check that every PR
referenced in the changelog has `state: MERGED`.
- **PR-level exclusions apply.** A PR can carry its own exclusion labels
(`release blocker`, `no issue required`) independent of its linked issue's
labels. Check both.
- **Cross-reference milestone PRs, not just issues.** The `--compare` flag on
the `issues` command only compares issue numbers. Merged PRs not linked to
any milestone issue can be missed. Use `python3 tools/gh.py prs --milestone`
for a full PR cross-reference.
- **False-positive PR-to-issue associations.** A PR may claim to close an
issue from a different project or context. If the PR title and issue title
are clearly unrelated, or the PR predates the issue by years, treat it as a
data glitch and skip it.

View File

@ -6,7 +6,7 @@ Uses GitHub GraphQL and REST APIs via the authenticated ``gh`` CLI.
Subcommands:
issues List issues in a milestone
prs Fetch details for one or more PRs
prs Fetch details for one or more PRs (or all PRs in a milestone)
Usage:
python3 tools/gh.py issues <milestone-title> (default: state=closed)
@ -16,6 +16,8 @@ Usage:
python3 tools/gh.py prs 9179 9204 9311
python3 tools/gh.py prs --file prs.txt
cat prs.txt | python3 tools/gh.py prs --stdin
python3 tools/gh.py prs --milestone "2.16.0" (default: state=merged)
python3 tools/gh.py prs --milestone "2.16.0" --state all
Prerequisites:
- gh CLI authenticated (gh auth status)
@ -210,7 +212,86 @@ def cmd_issues(args: argparse.Namespace) -> None:
# ─────────────────────────────────────────────
# Subcommand: prs
# Subcommand: prs — milestone PR listing
# ─────────────────────────────────────────────
GQL_MILESTONE_PRS_QUERY = """\
query($owner: String!, $repo: String!, $milestone: Int!, $cursor: String) {
repository(owner: $owner, name: $repo) {
milestone(number: $milestone) {
pullRequests(first: 50, after: $cursor, states: __STATES__) {
totalCount
pageInfo { hasNextPage endCursor }
nodes {
... on PullRequest {
number
title
state
mergedAt
author { login }
labels(first: 20) { nodes { name } }
closingIssuesReferences(first: 5) { nodes { number } }
}
}
}
}
}
}
"""
def fetch_milestone_prs(milestone_num: int, states: str) -> list[dict]:
"""
Fetch all PRs in a milestone via paginated GraphQL.
Args:
milestone_num: milestone number
states: GraphQL states enum array literal for pullRequests,
e.g. ``"[MERGED]"`` or ``"[OPEN CLOSED MERGED]"``
Returns:
List of {number, title, state, merged_at, author, labels, closing_issues}
"""
query = GQL_MILESTONE_PRS_QUERY.replace("__STATES__", states)
all_nodes: list[dict] = []
cursor: str | None = None
while True:
variables: dict[str, Any] = {
"owner": OWNER,
"repo": REPO_NAME,
"milestone": milestone_num,
"cursor": cursor,
}
data = run_gh_graphql(query, variables)
prs = data["repository"]["milestone"]["pullRequests"]
page_info = prs["pageInfo"]
for node in prs["nodes"]:
if node is None:
continue
all_nodes.append({
"number": node["number"],
"title": node["title"],
"state": node["state"],
"merged_at": node.get("mergedAt"),
"author": node["author"]["login"] if node["author"] else None,
"labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
"closing_issues": [iss["number"] for iss in node["closingIssuesReferences"]["nodes"]],
})
total = len(all_nodes)
print(f" ... fetched {total} PRs so far", file=sys.stderr)
if not page_info["hasNextPage"]:
break
cursor = page_info["endCursor"]
return all_nodes
# ─────────────────────────────────────────────
# Subcommand: prs — batch fetch by number
# ─────────────────────────────────────────────
PRS_BATCH_SIZE = 50
@ -280,7 +361,24 @@ def fetch_prs_batch(pr_numbers: list[int]) -> list[dict]:
def cmd_prs(args: argparse.Namespace) -> None:
"""Handle the ``prs`` subcommand."""
# Collect PR numbers from args / file / stdin
# ── Milestone mode: fetch all PRs in a milestone ──
if args.milestone:
print(f"Looking up milestone \"{args.milestone}\"...", file=sys.stderr)
ms = find_milestone(args.milestone)
print(f"Milestone #{ms['number']}: {ms['open_issues']} open, {ms['closed_issues']} closed",
file=sys.stderr)
state_map = {"open": "[OPEN]", "closed": "[CLOSED]",
"merged": "[MERGED]", "all": "[OPEN CLOSED MERGED]"}
gql_states = state_map[args.state]
print(f"Fetching {args.state} PRs via GraphQL...", file=sys.stderr)
results = fetch_milestone_prs(ms["number"], gql_states)
print(f"Fetched {len(results)} PRs total", file=sys.stderr)
print(json.dumps(results, indent=2))
return
# ── Batch mode: fetch specific PRs by number ──
pr_numbers: list[int] = []
if args.numbers:
@ -300,7 +398,7 @@ def cmd_prs(args: argparse.Namespace) -> None:
pr_numbers.append(int(line))
if not pr_numbers:
print("ERROR: no PR numbers provided (pass numbers, --file, or --stdin)",
print("ERROR: no PR numbers provided (pass numbers, --file, --stdin, or --milestone)",
file=sys.stderr)
sys.exit(1)
@ -350,11 +448,19 @@ def main() -> None:
p_issues.set_defaults(func=cmd_issues)
# --- prs ---
p_prs = sub.add_parser("prs", help="Fetch details for one or more PRs")
p_prs = sub.add_parser("prs", help="Fetch details for one or more PRs (or all PRs in a milestone)")
p_prs.add_argument(
"numbers", type=int, nargs="*",
help="PR numbers to fetch (space-separated)"
)
p_prs.add_argument(
"--milestone", type=str,
help="Milestone title to list all PRs from (e.g. '2.16.0')"
)
p_prs.add_argument(
"--state", choices=["open", "closed", "merged", "all"], default="merged",
help="PR state filter when using --milestone (default: merged)"
)
p_prs.add_argument(
"--file", type=str,
help="File with one PR number per line"