mirror of
https://github.com/penpot/penpot.git
synced 2026-11-16 15:11:17 +00:00
Merge remote-tracking branch 'origin/merge-staging' into staging
This commit is contained in:
commit
92c478b0e2
@ -31,7 +31,7 @@ memory; it does not repeat it.
|
||||
layout, entry rules, the `## Breaking change note` comment contract, the
|
||||
advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
|
||||
2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
|
||||
`report`.
|
||||
`dev-only-bugs`, `report`.
|
||||
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
|
||||
|
||||
## Commands
|
||||
@ -39,6 +39,8 @@ memory; it does not repeat it.
|
||||
```bash
|
||||
python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"
|
||||
python3 scripts/gh.py advisories --patched-version "<VERSION>"
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
|
||||
python3 scripts/changelog.py check-merged <PR numbers...>
|
||||
python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
|
||||
python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
|
||||
@ -54,4 +56,5 @@ python3 scripts/changelog.py report "<VERSION>"
|
||||
- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
|
||||
- `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
|
||||
- Verify PR merge status before writing; only merged PRs are referenced.
|
||||
- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
|
||||
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.
|
||||
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@ -121,7 +121,7 @@ opencode.json
|
||||
/.opencode/reports
|
||||
/.opencode/prompts
|
||||
/.ci-logs
|
||||
/tmp/
|
||||
/tmp
|
||||
/.codex/
|
||||
/scripts/__pycache__/
|
||||
/tools/__pycache__
|
||||
|
||||
@ -3,6 +3,8 @@
|
||||
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
|
||||
|
||||
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
|
||||
- Test DB/Redis come from the `PENPOT_TEST_*` env vars already set in the shell (`PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`, ...); per instance they point at `penpot_test` / `penpot_test_wsN` and valkey DB `6+N` (`mem:devenv/core`). Do not override them with values copied from CI logs or other sources unless the user asks (e.g. "use a temporary database"); the CI env (`redis://redis/1`, `postgresql://postgres/penpot_test`) only works inside the CI docker network.
|
||||
- The test database may have migrations applied from newer versions (it is shared dev infra). Reinitializing it — `dropdb`/`createdb` of the `PENPOT_TEST_*` database only — is explicitly permitted when that blocks a test run; never drop the main `penpot` database or another instance's test database.
|
||||
- All CLI commands must be executed from the `backend/` subdirectory.
|
||||
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
|
||||
- **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand.
|
||||
|
||||
7
.serena/memories/clojure/logging.md
Normal file
7
.serena/memories/clojure/logging.md
Normal file
@ -0,0 +1,7 @@
|
||||
# Clojure Logging Conventions
|
||||
|
||||
Structured logging through `app.common.logging` (`l/trc`/`l/dbg`/`l/info`/`l/wrn`/`l/error`/`l/fatal`): `:hint` carries the human-readable message, every other datum goes in its own keyword field (`:id`, `:container`, `:removed`, ...).
|
||||
|
||||
- `:hint` values start lowercase: `"fix dangling children"`, not `"Fix dangling children"`. Uppercase-first hints exist as legacy (~57 occurrences, the densest in `backend/src`; also `l/trc` in migration `legacy-28` and `log/error` in `app.common.files.repair`) — do not copy them, and do not mass-rewrite them unless the surrounding code needs the change.
|
||||
- A name that requires uppercase (an env var like `PENPOT_MEDIA_PROCESSING_SERVICE_URI`, an acronym like `MCP` or `LDAP`) stays uppercase inside an otherwise lowercase sentence.
|
||||
- Repair work done by a file migration logs at `l/trc`, only when it changes something, and with the ids it touched: the house style of `migrations.cljc` (`legacy-28` precedes; the file has no `warn`/`info` lines). The shipped `log4j2.xml` pins the `app` logger to `info` (static file, `monitorInterval="60"`), so those trace lines appear only after the operator raises the level; the durable record of a repaired file is the migration id in its `:migrations`.
|
||||
@ -6,6 +6,7 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
|
||||
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories.
|
||||
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
|
||||
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
|
||||
- Structured logging conventions (`:hint` casing and fields, log levels): `mem:clojure/logging` — read before adding or changing any log call.
|
||||
- Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
|
||||
- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
|
||||
- Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables.
|
||||
|
||||
@ -8,7 +8,8 @@
|
||||
- `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`.
|
||||
- `security <MILESTONE> [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix <description> (<advisory URL>) (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default.
|
||||
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design.
|
||||
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
|
||||
- `dev-only-bugs <MILESTONE> [--apply]` — list closed milestone bugs that never reached production, without `no changelog`. Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or this milestone (`only_unreleased_envs`; any production hint or other version keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup.
|
||||
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
@ -81,7 +81,7 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
|
||||
|
||||
Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts.
|
||||
|
||||
**Output**: JSON array to stdout; progress to stderr.
|
||||
**Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both.
|
||||
|
||||
### `comments`
|
||||
|
||||
|
||||
@ -24,11 +24,12 @@ Editing `CHANGES.md` happens only on explicit request or as part of the release
|
||||
| Is it a breaking change? | `breaking change` label |
|
||||
| Is it a highlight? | `release highlight` label |
|
||||
|
||||
- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
|
||||
- **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
|
||||
- **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug).
|
||||
- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry.
|
||||
- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge.
|
||||
- **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status.
|
||||
- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3).
|
||||
- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own.
|
||||
- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly.
|
||||
|
||||
@ -39,7 +40,7 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
|
||||
```markdown
|
||||
# CHANGELOG
|
||||
|
||||
## 2.19.0 (Unreleased)
|
||||
## 2.20.0 (Unreleased)
|
||||
|
||||
### :boom: Breaking changes & Deprecations
|
||||
|
||||
@ -57,21 +58,27 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
|
||||
|
||||
- Fix <user-facing description of the vulnerability> (<advisory URL>) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :bug: Bugs fixed
|
||||
|
||||
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :sparkles: New features & Enhancements
|
||||
### :tada: Features
|
||||
|
||||
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :sparkles: Enhancements
|
||||
|
||||
- Improve <user-facing description of the existing behaviour> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :bug: Bugs
|
||||
|
||||
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
```
|
||||
|
||||
- `:boom:` first, then `:rocket:`, `:lock:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
|
||||
- `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update.
|
||||
- **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it.
|
||||
- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
|
||||
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories.
|
||||
- One blank line between the last entry of a section and the next heading.
|
||||
- No manual line wrapping: one line per bullet or paragraph, however long.
|
||||
|
||||
## 4. Regular entries (`:bug:`, `:sparkles:`)
|
||||
## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`)
|
||||
|
||||
The entry shape is:
|
||||
|
||||
@ -153,15 +160,18 @@ Generated from the advisories that declare the version in `patched_versions`; ne
|
||||
|
||||
1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious.
|
||||
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
|
||||
3. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
|
||||
4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
|
||||
5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
|
||||
6. **Reconcile the existing text** (section 9).
|
||||
7. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
|
||||
8. **Generate `:rocket:`** from the `release highlight` label (section 6).
|
||||
9. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
|
||||
10. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
|
||||
11. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
|
||||
3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs "<VERSION>"` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration.
|
||||
- **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these.
|
||||
- **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or a version of this milestone. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10).
|
||||
4. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
|
||||
5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
|
||||
6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
|
||||
7. **Reconcile the existing text** (section 9).
|
||||
8. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
|
||||
9. **Generate `:rocket:`** from the `release highlight` label (section 6).
|
||||
10. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
|
||||
11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
|
||||
12. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
|
||||
|
||||
## 9. Pre-flight: reconcile before writing
|
||||
|
||||
@ -172,7 +182,7 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
|
||||
3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry.
|
||||
4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry.
|
||||
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently.
|
||||
6. **Issue Type changed to `Task`** → remove the entry.
|
||||
6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section.
|
||||
7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry.
|
||||
8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it.
|
||||
9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section.
|
||||
@ -189,6 +199,8 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
|
||||
- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights).
|
||||
- **D — `:rocket:` entry without issue AND PR references** (gap).
|
||||
|
||||
**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total.
|
||||
|
||||
Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run.
|
||||
|
||||
## 11. Pitfalls
|
||||
@ -203,6 +215,6 @@ Anything else is a rule violation, not a report item: fix it in the pre-flight a
|
||||
## 12. Where the code is
|
||||
|
||||
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`.
|
||||
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `report`. Details: `mem:scripts/changelog`.
|
||||
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`.
|
||||
- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`.
|
||||
- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**.
|
||||
|
||||
@ -445,10 +445,9 @@
|
||||
(t/is (thrown? clojure.lang.ExceptionInfo
|
||||
(invoke! cfg (fn [_ _] :never) {}))))
|
||||
|
||||
;; the count came back down instead of staying up for good. Only the
|
||||
;; last publication is asserted, not the exact list: the sampler
|
||||
;; republishes the same count when its first tick lands inside the
|
||||
;; window.
|
||||
;; the count came back down instead of staying up for good. The
|
||||
;; republisher daemon thread can add publications inside the window,
|
||||
;; so what is asserted is the last value, not the whole sequence.
|
||||
(t/is (= 0 (waiting-count climit name)))
|
||||
(t/is (= 0 (waiting-last name)))
|
||||
|
||||
|
||||
@ -2233,44 +2233,49 @@
|
||||
;; been added to enforce this from now on.
|
||||
;; Main instances in variants are a special case because they need to match the variant name.
|
||||
(letfn [(update-shape [container shape-id]
|
||||
(let [shape (ctn/get-shape container shape-id)
|
||||
(let [shape (ctn/get-shape container shape-id)]
|
||||
;; A shape may reference a child that does not exist in the
|
||||
;; container (a dangling reference). There is nothing to update
|
||||
;; on it, and storing it would add a nil entry to the objects
|
||||
;; map, making the whole file invalid.
|
||||
(if (nil? shape)
|
||||
container
|
||||
(let [shape (cond
|
||||
;; Variant container must have a normalized name
|
||||
(ctk/is-variant-container? shape)
|
||||
(let [name (:name shape)
|
||||
clean-name (cpn/clean-path name)
|
||||
clean-name (if (str/blank? clean-name)
|
||||
"(empty)" ;; this may occur if the name contains only separators
|
||||
clean-name)]
|
||||
(if (= name clean-name)
|
||||
shape
|
||||
(assoc shape :name clean-name)))
|
||||
|
||||
shape (cond
|
||||
;; Variant container must have a normalized name
|
||||
(ctk/is-variant-container? shape)
|
||||
(let [name (:name shape)
|
||||
clean-name (cpn/clean-path name)
|
||||
clean-name (if (str/blank? clean-name)
|
||||
"(empty)" ;; this may occur if the name contains only separators
|
||||
clean-name)]
|
||||
(if (= name clean-name)
|
||||
shape
|
||||
(assoc shape :name clean-name)))
|
||||
;; Main instance inside a variant must have the same name as the container
|
||||
(ctk/main-instance? shape)
|
||||
(if (ctk/is-variant? shape)
|
||||
(let [variant-container (ctn/get-shape container (:parent-id shape))]
|
||||
(if (or (= (:name shape) (:name variant-container))
|
||||
(nil? variant-container))
|
||||
shape
|
||||
(assoc shape :name (:name variant-container))))
|
||||
|
||||
;; Main instance inside a variant must have the same name as the container
|
||||
(ctk/main-instance? shape)
|
||||
(if (ctk/is-variant? shape)
|
||||
(let [variant-container (ctn/get-shape container (:parent-id shape))]
|
||||
(if (or (= (:name shape) (:name variant-container))
|
||||
(nil? variant-container))
|
||||
shape
|
||||
(assoc shape :name (:name variant-container))))
|
||||
;; Other main instances must have the same name as the path+name of the component
|
||||
(let [component (ctkl/get-component data (:component-id shape) true)
|
||||
expected-name (when component
|
||||
(cpn/merge-path-item (:path component) (:name component)))]
|
||||
(if (or (= expected-name (:name shape))
|
||||
(nil? expected-name))
|
||||
shape
|
||||
(assoc shape :name expected-name))))
|
||||
|
||||
;; Other main instances must have the same name as the path+name of the component
|
||||
(let [component (ctkl/get-component data (:component-id shape) true)
|
||||
expected-name (when component
|
||||
(cpn/merge-path-item (:path component) (:name component)))]
|
||||
(if (or (= expected-name (:name shape))
|
||||
(nil? expected-name))
|
||||
shape
|
||||
(assoc shape :name expected-name))))
|
||||
;; Any other shape does not need processing
|
||||
:else shape)]
|
||||
|
||||
;; Any other shape does not need processing
|
||||
:else shape)]
|
||||
|
||||
(reduce update-shape
|
||||
(ctt/set-shape container shape)
|
||||
(:shapes shape))))
|
||||
(reduce update-shape
|
||||
(ctt/set-shape container shape)
|
||||
(:shapes shape))))))
|
||||
|
||||
(update-container
|
||||
[container]
|
||||
@ -2322,6 +2327,35 @@
|
||||
(update :pages-index d/update-vals update-container)
|
||||
(d/update-when :components d/update-vals update-container)))))
|
||||
|
||||
(defmethod migrate-data "0035-fix-dangling-children"
|
||||
[data _]
|
||||
;; A shape may list a child id that does not exist in the objects map of
|
||||
;; the page or component that owns it (a dangling reference). Such a
|
||||
;; reference is invalid: any operation that walks the shape tree through
|
||||
;; the children lists trips on it, and the file cannot be migrated nor
|
||||
;; opened. Here we remove the dangling ids from the children lists,
|
||||
;; keeping every shape that really exists.
|
||||
(letfn [(fix-shape [container-id objects shape]
|
||||
(d/update-when shape :shapes
|
||||
(fn [shapes]
|
||||
(let [shapes' (filterv #(contains? objects %) shapes)]
|
||||
(when (not= shapes shapes')
|
||||
(l/trc :hint "fix dangling children"
|
||||
:container container-id
|
||||
:shape-id (:id shape)
|
||||
:removed (remove #(contains? objects %) shapes)))
|
||||
shapes'))))
|
||||
|
||||
(fix-objects [container-id objects]
|
||||
(update-vals objects (partial fix-shape container-id objects)))
|
||||
|
||||
(update-container [container]
|
||||
(d/update-when container :objects #(fix-objects (:id container) %)))]
|
||||
|
||||
(-> data
|
||||
(update :pages-index d/update-vals update-container)
|
||||
(d/update-when :components d/update-vals update-container))))
|
||||
|
||||
(defmethod migrate-data "0035-fix-shape-svg-attrs-and-defs"
|
||||
;; Stored svg attribute keys are kebab-case (issue #11947): the v3 reader
|
||||
;; already outputs kebab and birth normalizes any outside spelling, so
|
||||
@ -2459,4 +2493,5 @@
|
||||
"0032-fix-children-not-in-parent"
|
||||
"0033-normalize-main-instance-names"
|
||||
"0034-fix-path-content-offset"
|
||||
"0035-fix-dangling-children"
|
||||
"0035-fix-shape-svg-attrs-and-defs"]))
|
||||
|
||||
@ -28,6 +28,25 @@
|
||||
(map #(ctcl/get-component data % true))
|
||||
reverse)))
|
||||
|
||||
(defn collect-variant-properties
|
||||
"Map of component-id → variant-properties for clipboard copy.
|
||||
Variant container: every related component. Variant main: that
|
||||
component only. Anything else: empty map."
|
||||
[data objects shape]
|
||||
(cond
|
||||
(ctc/is-variant-container? shape)
|
||||
(into {}
|
||||
(map (juxt :id :variant-properties))
|
||||
(find-variant-components data objects (:id shape)))
|
||||
|
||||
(ctc/is-variant-main? shape)
|
||||
(if-let [component (ctcl/get-component data (:component-id shape) true)]
|
||||
{(:id component) (:variant-properties component)}
|
||||
{})
|
||||
|
||||
:else
|
||||
{}))
|
||||
|
||||
(defn extract-properties-names
|
||||
[shape data]
|
||||
(->> shape
|
||||
|
||||
@ -2766,7 +2766,8 @@
|
||||
page-id
|
||||
nil
|
||||
variant-id
|
||||
props))]
|
||||
props
|
||||
{:apply-changes-local-library? true}))]
|
||||
[root-shape changes]))
|
||||
|
||||
(defn generate-add-component
|
||||
@ -3282,6 +3283,25 @@
|
||||
(and is-component-root? is-component-main?)
|
||||
(regenerate-component new-obj))
|
||||
|
||||
;; Unknown-component paste into a variant set: align props/names
|
||||
;; after regenerate. Reload the shape so we use the new component-id.
|
||||
;; Prefit the component name/path to the container so
|
||||
;; create-new-properties-from-variant does not treat the component
|
||||
;; full name as a former set name and inflate properties (add-name?).
|
||||
changes (cond-> changes
|
||||
(and is-component-root?
|
||||
is-component-main?
|
||||
(ctk/is-variant-container? parent))
|
||||
(as-> $
|
||||
(let [shape (get (pcb/get-objects $) (:id new-obj))
|
||||
[cpath cname] (cpn/split-group-name (:name parent))
|
||||
changes (pcb/update-component
|
||||
$
|
||||
(:component-id shape)
|
||||
#(assoc % :name cname :path cpath)
|
||||
{:apply-changes-local-library? true})]
|
||||
(clvp/generate-make-shapes-variant changes [shape] parent))))
|
||||
|
||||
;; This is needed for the recursive call to find the new object as parent
|
||||
page' (ctst/add-shape (:id new-obj)
|
||||
new-obj
|
||||
|
||||
@ -266,10 +266,20 @@
|
||||
(first
|
||||
(reduce
|
||||
(fn [[changes used-values] shape]
|
||||
(let [component (ctcl/get-component data (:component-id shape) true)]
|
||||
(if (or (zero? num-shapes) ;; do nothing if there are no shapes
|
||||
(and (= variant-id (:variant-id shape)) ;; or we are only moving the shape inside its parent (it is
|
||||
(not (:deleted component)))) ;; the same parent and the component isn't deleted)
|
||||
(let [component (ctcl/get-component data (:component-id shape) true)
|
||||
already-here? (= variant-id (:variant-id shape))
|
||||
prop-names (cfv/extract-properties-names shape data)
|
||||
base-names (map :name base-props)
|
||||
props-ok? (= prop-names base-names)
|
||||
names-ok? (= (cpn/merge-path-item (:path component) (:name component))
|
||||
container-name)
|
||||
variant-ok? (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties component)))
|
||||
aligned? (and props-ok? names-ok? variant-ok?)]
|
||||
(if (or (zero? num-shapes) ;; do nothing if there are no shapes
|
||||
(and already-here? ;; or already a fully aligned child of this container
|
||||
aligned?
|
||||
(not (:deleted component))))
|
||||
[changes used-values]
|
||||
(let [props (-> (create-new-properties shape total-props)
|
||||
(fill-blank-properties used-values))
|
||||
|
||||
49
common/test/common_tests/files/variant_test.cljc
Normal file
49
common/test/common_tests/files/variant_test.cljc
Normal file
@ -0,0 +1,49 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.files.variant-test
|
||||
(:require
|
||||
[app.common.files.variant :as cfv]
|
||||
[app.common.test-helpers.components :as thc]
|
||||
[app.common.test-helpers.files :as thf]
|
||||
[app.common.test-helpers.ids-map :as thi]
|
||||
[app.common.test-helpers.shapes :as ths]
|
||||
[app.common.test-helpers.variants :as thv]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :each thi/test-fixture)
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-main
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
main (ths/get-shape file :m01)
|
||||
comp (thc/get-component file :c01)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) main)]
|
||||
|
||||
(t/is (= {(:id comp) (:variant-properties comp)} collected))))
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-container
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
container (ths/get-shape file :v01)
|
||||
c01 (thc/get-component file :c01)
|
||||
c02 (thc/get-component file :c02)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) container)]
|
||||
|
||||
(t/is (= {(:id c01) (:variant-properties c01)
|
||||
(:id c02) (:variant-properties c02)}
|
||||
collected))))
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-plain-shape
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(ths/add-sample-shape :rect1 :type :rect))
|
||||
page (thf/current-page file)
|
||||
shape (ths/get-shape file :rect1)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) shape)]
|
||||
|
||||
(t/is (= {} collected))))
|
||||
138
common/test/common_tests/files_migrations_0033_test.cljc
Normal file
138
common/test/common_tests/files_migrations_0033_test.cljc
Normal file
@ -0,0 +1,138 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.files-migrations-0033-test
|
||||
"Regression coverage for the shape tree traversal of the
|
||||
`0033-normalize-main-instance-names` migration.
|
||||
|
||||
The migration walks a container from its root down, following the
|
||||
`:shapes` vectors. A file can hold a `:shapes` entry pointing to a
|
||||
shape that is not in the `:objects` map (a dangling reference). The
|
||||
traversal must skip it instead of storing a nil shape into the
|
||||
`:objects` map, and must keep normalizing the names of the shapes
|
||||
that do exist."
|
||||
(:require
|
||||
[app.common.files.migrations :as cfm]
|
||||
[app.common.types.shape :as cts]
|
||||
[app.common.uuid :as uuid]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(def ^:private migration "0033-normalize-main-instance-names")
|
||||
|
||||
(defn- make-root
|
||||
[children]
|
||||
(cts/setup-shape {:id uuid/zero
|
||||
:type :frame
|
||||
:name "Root"
|
||||
:shapes children}))
|
||||
|
||||
(defn- make-main-instance
|
||||
"A frame declared as the main instance of a component."
|
||||
[main-id file-id component-id name]
|
||||
(-> (cts/setup-shape {:id main-id :type :frame :name name})
|
||||
(assoc :main-instance true
|
||||
:component-id component-id
|
||||
:component-file file-id)))
|
||||
|
||||
(defn- page-data
|
||||
"File data with a single page owning `objects`."
|
||||
[page-id objects]
|
||||
{:pages-index {page-id {:id page-id :objects objects}}})
|
||||
|
||||
(defn- component-data
|
||||
"File data with a single component owning `objects`."
|
||||
[component-id objects]
|
||||
{:components {component-id {:id component-id :objects objects}}})
|
||||
|
||||
;; --- The guard: a missing child must not become a nil shape
|
||||
|
||||
(t/deftest missing-child-of-a-page-does-not-create-a-nil-shape
|
||||
(let [page-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
root (make-root [child-id])
|
||||
data (page-data page-id {uuid/zero root})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= 1 (count objects'))
|
||||
"the objects map keeps only the shapes that exist")
|
||||
(t/is (= root (get objects' uuid/zero))
|
||||
"the root is left untouched")))
|
||||
|
||||
(t/deftest missing-child-of-a-component-does-not-create-a-nil-shape
|
||||
(let [component-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
root (make-root [child-id])
|
||||
data (component-data component-id {uuid/zero root})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= 1 (count objects'))
|
||||
"the objects map keeps only the shapes that exist")
|
||||
(t/is (= root (get objects' uuid/zero))
|
||||
"the component root is left untouched")))
|
||||
|
||||
(t/deftest missing-child-deep-in-the-tree-does-not-create-a-nil-shape
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
frame (-> (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
(assoc :shapes [missing-id]))
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= frame (get objects' frame-id))
|
||||
"the parent of the missing child is left untouched")))
|
||||
|
||||
;; --- The behavior: names keep being normalized next to a missing child
|
||||
|
||||
(t/deftest main-instance-of-a-page-is-renamed-next-to-a-missing-child
|
||||
(let [file-id (uuid/next)
|
||||
page-id (uuid/next)
|
||||
component-id (uuid/next)
|
||||
main-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
main-shape (make-main-instance main-id file-id component-id "Group/Name")
|
||||
data (assoc (page-data page-id {uuid/zero (make-root [missing-id main-id])
|
||||
main-id main-shape})
|
||||
:components {component-id {:id component-id
|
||||
:name "Name"
|
||||
:path "Group"
|
||||
:main-instance-id main-id}})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= "Group / Name" (get-in objects' [main-id :name]))
|
||||
"the main instance takes the path and name of its component")))
|
||||
|
||||
(t/deftest main-instance-of-a-component-is-renamed-next-to-a-missing-child
|
||||
(let [file-id (uuid/next)
|
||||
component-id (uuid/next)
|
||||
main-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
main-shape (make-main-instance main-id file-id component-id "Group/Name")
|
||||
data (component-data component-id {uuid/zero (make-root [missing-id main-id])
|
||||
main-id main-shape})
|
||||
data (assoc-in data [:components component-id :name] "Name")
|
||||
data (assoc-in data [:components component-id :path] "Group")
|
||||
data (assoc-in data [:components component-id :main-instance-id] main-id)
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= "Group / Name" (get-in objects' [main-id :name]))
|
||||
"the main instance takes the path and name of its component")))
|
||||
@ -10,10 +10,186 @@
|
||||
[app.common.geom.matrix :as gmt]
|
||||
[app.common.geom.rect :as grc]
|
||||
[app.common.types.file :as ctf]
|
||||
[app.common.types.objects-map :as omap]
|
||||
[app.common.types.shape :as cts]
|
||||
[app.common.uuid :as uuid]
|
||||
[clojure.test :as t]))
|
||||
|
||||
;; ======================================================================
|
||||
;; 0035-fix-dangling-children
|
||||
;; A shape can list a child id that is not present in the `:objects` map
|
||||
;; of the page or component owning it (a dangling reference). Any
|
||||
;; traversal of the shape tree trips on it, and the file becomes
|
||||
;; unopenable. The migration drops those ids and keeps every shape that
|
||||
;; really exists.
|
||||
;; ======================================================================
|
||||
|
||||
(def ^:private migration "0035-fix-dangling-children")
|
||||
|
||||
(defn- make-root
|
||||
[children]
|
||||
(cts/setup-shape {:id uuid/zero
|
||||
:type :frame
|
||||
:name "Root"
|
||||
:shapes children}))
|
||||
|
||||
(defn- make-frame
|
||||
[frame-id child-ids]
|
||||
(-> (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
(assoc :shapes child-ids)))
|
||||
|
||||
(defn- page-data
|
||||
[page-id objects]
|
||||
{:pages-index {page-id {:id page-id :objects objects}}})
|
||||
|
||||
(defn- component-data
|
||||
[component-id objects]
|
||||
{:components {component-id {:id component-id :objects objects}}})
|
||||
|
||||
(t/deftest removes-the-dangling-children-of-a-page
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (make-frame frame-id [child-id])
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed from the root children list")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the dangling id is removed from the frame children list")
|
||||
(t/is (= 2 (count objects'))
|
||||
"every shape that exists is kept")
|
||||
(t/is (some? (get objects' frame-id))
|
||||
"the frame itself is kept")))
|
||||
|
||||
(t/deftest removes-the-dangling-children-of-a-component
|
||||
(let [component-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
data (component-data component-id {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed from the children list")
|
||||
(t/is (= 2 (count objects'))
|
||||
"every shape that exists is kept")
|
||||
(t/is (= frame (get objects' frame-id))
|
||||
"the existing shape is left untouched")))
|
||||
|
||||
(t/deftest migration-works-with-an-objects-map
|
||||
;; The backend stores the page :objects as an ObjectsMap (the lazy
|
||||
;; representation enabled by the fdata/objects-map feature). It is an
|
||||
;; IPersistentMap that does not implement IFn on the JVM: a migration
|
||||
;; must consult it with contains?/get, never by calling it as a
|
||||
;; function.
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (make-frame frame-id [child-id])
|
||||
objects (omap/wrap {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
data (page-data page-id objects)
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed with the ObjectsMap representation")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the frame children list is pruned the same way")))
|
||||
|
||||
(t/deftest keeps-the-order-and-the-shapes-of-a-valid-tree
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
rect-id (uuid/next)
|
||||
frame (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
rect (cts/setup-shape {:id rect-id :type :rect :name "Rect"})
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id rect-id])
|
||||
frame-id frame
|
||||
rect-id rect})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id rect-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the children list is kept as it was")
|
||||
(t/is (= data migrated)
|
||||
"a tree without dangling references is not modified")))
|
||||
|
||||
(t/deftest leaves-shapes-with-empty-children-alone
|
||||
(let [page-id (uuid/next)
|
||||
frame (cts/setup-shape {:id (uuid/next) :type :frame :name "Frame"})
|
||||
data (page-data page-id {uuid/zero (make-root [])
|
||||
(:id frame) frame})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= data migrated)
|
||||
"empty children vectors are left alone")))
|
||||
|
||||
(t/deftest leaves-shapes-without-children-key-alone
|
||||
(let [page-id (uuid/next)
|
||||
rect-id (uuid/next)
|
||||
rect (cts/setup-shape {:id rect-id :type :rect :name "Rect"})
|
||||
data (page-data page-id {uuid/zero (make-root [])
|
||||
rect-id rect})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= data migrated)
|
||||
"shapes without a children key are left alone")
|
||||
(t/is (not (contains? (get-in migrated [:pages-index page-id :objects rect-id]) :shapes))
|
||||
"no children key is created on the shape")))
|
||||
|
||||
(t/deftest migration-is-idempotent
|
||||
(let [page-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
data (page-data page-id {uuid/zero (make-root [child-id])})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= migrated (cfm/migrate-data migrated migration))
|
||||
"applying the migration twice does not change the result")))
|
||||
|
||||
(t/deftest file-with-dangling-children-passes-the-whole-migration
|
||||
(let [file-id (uuid/next)
|
||||
page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
frame (make-frame frame-id [missing-id])
|
||||
base (ctf/make-file-data file-id page-id)
|
||||
data (-> base
|
||||
(assoc-in [:pages-index page-id :objects uuid/zero]
|
||||
(make-root [frame-id missing-id]))
|
||||
(assoc-in [:pages-index page-id :objects frame-id] frame))
|
||||
;; The file reached the state before 0033 ran, so 0033, 0034 and
|
||||
;; 0035 are all pending, exactly as in the reported failure.
|
||||
file {:id file-id
|
||||
:data data
|
||||
:migrations (disj cfm/available-migrations
|
||||
"0033-normalize-main-instance-names"
|
||||
"0034-fix-path-content-offset"
|
||||
migration)}]
|
||||
|
||||
(t/is (cfm/need-migration? file)
|
||||
"the file is scheduled for migration")
|
||||
|
||||
(let [migrated (cfm/migrate-file file {})
|
||||
objects' (get-in migrated [:data :pages-index page-id :objects])]
|
||||
|
||||
(t/is (contains? (:migrations migrated) migration)
|
||||
"the migration is recorded as applied")
|
||||
(t/is (not (contains? objects' nil))
|
||||
"the migrated data has no nil shape entry")
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling reference of the root is removed")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the dangling reference of the frame is removed")
|
||||
(t/is (some? (get objects' frame-id))
|
||||
"the frame that exists is kept"))))
|
||||
|
||||
;; ======================================================================
|
||||
;; 0035-fix-shape-svg-attrs-and-defs
|
||||
;; Stored svg attribute keys are kebab-case (issue #11947). Old birth and
|
||||
;; the cleaner era stored camelCase React prop names, which the v3 reader
|
||||
@ -22,6 +198,7 @@
|
||||
;; through the whitelist+kebab transform, svg-raw `:content` attrs are
|
||||
;; kebab-ized spelling-only. It replaces "0021-fix-shape-svg-attrs" (id
|
||||
;; withdrawn): files that recorded it migrate again through this one.
|
||||
;; ======================================================================
|
||||
|
||||
(def ^:private migration-id
|
||||
"0035-fix-shape-svg-attrs-and-defs")
|
||||
|
||||
202
common/test/common_tests/logic/variants_paste_test.cljc
Normal file
202
common/test/common_tests/logic/variants_paste_test.cljc
Normal file
@ -0,0 +1,202 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.logic.variants-paste-test
|
||||
(:require
|
||||
[app.common.data :as d]
|
||||
[app.common.files.changes-builder :as pcb]
|
||||
[app.common.files.helpers :as cfh]
|
||||
[app.common.files.validate :as cfv]
|
||||
[app.common.geom.point :as gpt]
|
||||
[app.common.logic.libraries :as cll]
|
||||
[app.common.logic.variant-properties :as clvp]
|
||||
[app.common.test-helpers.components :as thc]
|
||||
[app.common.test-helpers.compositions :as tho]
|
||||
[app.common.test-helpers.files :as thf]
|
||||
[app.common.test-helpers.ids-map :as thi]
|
||||
[app.common.test-helpers.shapes :as ths]
|
||||
[app.common.test-helpers.variants :as thv]
|
||||
[app.common.types.component :as ctk]
|
||||
[app.common.types.variant :as ctv]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :each thi/test-fixture)
|
||||
|
||||
(defn- paste-variants-into-container
|
||||
"Paste individual variants from file-a into file-b's variant container.
|
||||
Omit file-a from `libraries` to hit the unknown-component path; include
|
||||
it to hit `duplicate-variant`. Optional `variant-props` mirrors clipboard
|
||||
collection (Task 2)."
|
||||
[file-a file-b source-labels target-container-label
|
||||
& {:keys [libraries variant-props]
|
||||
:or {libraries {(:id file-b) file-b}}}]
|
||||
(let [page-a (thf/current-page file-a)
|
||||
page-b (thf/current-page file-b)
|
||||
container-b (ths/get-shape file-b target-container-label)
|
||||
parent-id (:id container-b)
|
||||
|
||||
source-ids (mapv thi/id source-labels)
|
||||
tree-ids (into []
|
||||
(mapcat #(cfh/get-children-ids-with-self (:objects page-a) %))
|
||||
source-ids)
|
||||
|
||||
pasted (->> tree-ids
|
||||
(map (d/getf (:objects page-a)))
|
||||
(map (fn [shape]
|
||||
(cond-> shape
|
||||
(contains? (set source-ids) (:id shape))
|
||||
(assoc :parent-id parent-id
|
||||
:frame-id parent-id))))
|
||||
(d/index-by :id))
|
||||
|
||||
all-objects (merge (:objects page-b) pasted)]
|
||||
(-> (pcb/empty-changes nil)
|
||||
(pcb/with-page-id (:id page-b))
|
||||
(pcb/with-library-data (:data file-b))
|
||||
(pcb/with-objects all-objects)
|
||||
(cll/generate-duplicate-changes all-objects
|
||||
page-b
|
||||
(set source-ids)
|
||||
(gpt/point 0 0)
|
||||
libraries
|
||||
(:data file-b)
|
||||
(:id file-b)
|
||||
{:variant-props variant-props}))))
|
||||
|
||||
(defn- property-names
|
||||
[component]
|
||||
(mapv :name (:variant-properties component)))
|
||||
|
||||
(defn- variant-mains-in
|
||||
[file container-label]
|
||||
(let [container (ths/get-shape file container-label)]
|
||||
(->> (vals (:objects (thf/current-page file)))
|
||||
(filter ctk/main-instance?)
|
||||
(filter #(= (:id container) (:variant-id %))))))
|
||||
|
||||
(defn- other-set-file
|
||||
[]
|
||||
(-> (thf/sample-file :file-b)
|
||||
(thv/add-variant-two-properties :vb :cb1 :mb1 :cb2 :mb2)
|
||||
(ths/update-shape :vb :name "Other")
|
||||
(ths/update-shape :mb1 :name "Other")
|
||||
(ths/update-shape :mb2 :name "Other")
|
||||
(thc/update-component :cb1 {:name "Other"})
|
||||
(thc/update-component :cb2 {:name "Other"})))
|
||||
|
||||
(t/deftest test-paste-unknown-variant-into-container-validates
|
||||
;; #10704: paste individual variants from file A into file B's set
|
||||
;; when A is not in libraries must leave a valid file (desired behavior).
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
errors (cfv/validate-file file' {})
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %))))]
|
||||
|
||||
(t/is (nil? errors))
|
||||
(doseq [shape pasted]
|
||||
(let [comp (thc/get-component-by-id file' (:component-id shape))]
|
||||
(t/is (= ["Property 1" "Property 2"] (property-names comp)))
|
||||
(t/is (= "Other" (:name shape)))
|
||||
(t/is (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties comp))))))))
|
||||
|
||||
(t/deftest test-duplicate-variant-in-own-container-bumps-last-value
|
||||
;; Same-container duplicate via duplicate-variant: originals keep their
|
||||
;; props; the new main gets a bumped last-property value. Does not cover
|
||||
;; generate-make-shapes-variant's already-aligned early-exit (new mains
|
||||
;; enter with a nil variant-id).
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
c01-props (:variant-properties (thc/get-component file :c01))
|
||||
c02-props (:variant-properties (thc/get-component file :c02))
|
||||
file' (tho/duplicate-shape file :m01)
|
||||
mains (variant-mains-in file' :v01)
|
||||
new-main (->> mains
|
||||
(remove #(#{(thi/id :m01) (thi/id :m02)} (:id %)))
|
||||
first)
|
||||
new-comp (thc/get-component-by-id file' (:component-id new-main))]
|
||||
|
||||
(thf/validate-file! file')
|
||||
(t/is (= 3 (count mains)))
|
||||
(t/is (= c01-props (:variant-properties (thc/get-component file' :c01))))
|
||||
(t/is (= c02-props (:variant-properties (thc/get-component file' :c02))))
|
||||
(t/is (= ["Property 1"] (property-names new-comp)))
|
||||
(t/is (= (str ctv/value-prefix 3)
|
||||
(-> new-comp :variant-properties last :value)))))
|
||||
|
||||
(t/deftest test-make-shapes-variant-skips-when-already-aligned
|
||||
;; Locks the narrowed early-exit: a fully aligned child must not have its
|
||||
;; props or variant-name rewritten.
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
container (ths/get-shape file :v01)
|
||||
shape (ths/get-shape file :m01)
|
||||
before (:variant-properties (thc/get-component file :c01))
|
||||
before-vn (:variant-name shape)
|
||||
changes (-> (pcb/empty-changes nil)
|
||||
(pcb/with-page page)
|
||||
(pcb/with-objects (:objects page))
|
||||
(pcb/with-library-data (:data file))
|
||||
(clvp/generate-make-shapes-variant [shape] container))
|
||||
file' (thf/apply-changes file changes)
|
||||
shape' (ths/get-shape-by-id file' (:id shape))]
|
||||
|
||||
(t/is (= before (:variant-properties (thc/get-component file' :c01))))
|
||||
(t/is (= before-vn (:variant-name shape')))))
|
||||
|
||||
(t/deftest test-paste-known-variant-into-other-container-aligns
|
||||
;; file A present in libraries → duplicate-variant path; props must match
|
||||
;; the target set and the file must validate.
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
libraries {(:id file-a) file-a
|
||||
(:id file-b) file-b}
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb
|
||||
:libraries libraries)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
errors (cfv/validate-file file' {})
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %))))]
|
||||
|
||||
(t/is (nil? errors))
|
||||
(t/is (= 2 (count pasted)))
|
||||
(doseq [shape pasted]
|
||||
(let [comp (thc/get-component-by-id file' (:component-id shape))]
|
||||
(t/is (= ["Property 1" "Property 2"] (property-names comp)))
|
||||
(t/is (= "Other" (:name shape)))
|
||||
(t/is (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties comp))))))))
|
||||
|
||||
(t/deftest test-paste-unknown-preserves-matching-property-values
|
||||
;; Clipboard props (Task 2) whose names exist on the target survive regenerate.
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
props {(:id (thc/get-component file-a :ca1))
|
||||
(:variant-properties (thc/get-component file-a :ca1))
|
||||
(:id (thc/get-component file-a :ca2))
|
||||
(:variant-properties (thc/get-component file-a :ca2))}
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb
|
||||
:variant-props props)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %)))
|
||||
(map #(thc/get-component-by-id file' (:component-id %))))
|
||||
values (->> pasted
|
||||
(mapcat :variant-properties)
|
||||
(filter #(= "Property 1" (:name %)))
|
||||
(map :value)
|
||||
set)]
|
||||
|
||||
(thf/validate-file! file')
|
||||
(t/is (contains? values "Value1"))
|
||||
(t/is (contains? values "Value2"))))
|
||||
@ -22,6 +22,7 @@
|
||||
[common-tests.files-changes-test]
|
||||
[common-tests.files-migrations-0025-test]
|
||||
[common-tests.files-migrations-0026-test]
|
||||
[common-tests.files-migrations-0033-test]
|
||||
[common-tests.files-migrations-0034-test]
|
||||
[common-tests.files-migrations-0035-test]
|
||||
[common-tests.files-migrations-test]
|
||||
@ -32,6 +33,7 @@
|
||||
[common-tests.files.shapes-builder-test]
|
||||
[common-tests.files.tokens-test]
|
||||
[common-tests.files.validate-test]
|
||||
[common-tests.files.variant-test]
|
||||
[common-tests.geom-align-test]
|
||||
[common-tests.geom-bounds-layout-nil-test]
|
||||
[common-tests.geom-bounds-map-test]
|
||||
@ -81,6 +83,7 @@
|
||||
[common-tests.logic.text-touched-test]
|
||||
[common-tests.logic.token-apply-test]
|
||||
[common-tests.logic.token-test]
|
||||
[common-tests.logic.variants-paste-test]
|
||||
[common-tests.logic.variants-reparent-test]
|
||||
[common-tests.logic.variants-switch-test]
|
||||
[common-tests.logic.variants-test]
|
||||
@ -135,6 +138,7 @@
|
||||
'common-tests.files-changes-test
|
||||
'common-tests.files-migrations-0025-test
|
||||
'common-tests.files-migrations-0026-test
|
||||
'common-tests.files-migrations-0033-test
|
||||
'common-tests.files-migrations-0034-test
|
||||
'common-tests.files-migrations-0035-test
|
||||
'common-tests.files-migrations-test
|
||||
@ -145,6 +149,7 @@
|
||||
'common-tests.files.shapes-builder-test
|
||||
'common-tests.files.tokens-test
|
||||
'common-tests.files.validate-test
|
||||
'common-tests.files.variant-test
|
||||
'common-tests.geom-align-test
|
||||
'common-tests.geom-bounds-layout-nil-test
|
||||
'common-tests.geom-bounds-map-test
|
||||
@ -194,6 +199,7 @@
|
||||
'common-tests.logic.text-touched-test
|
||||
'common-tests.logic.token-apply-test
|
||||
'common-tests.logic.token-test
|
||||
'common-tests.logic.variants-paste-test
|
||||
'common-tests.logic.variants-reparent-test
|
||||
'common-tests.logic.variants-switch-test
|
||||
'common-tests.logic.variants-test
|
||||
|
||||
@ -160,10 +160,8 @@
|
||||
(let [page-id (:current-page-id state)
|
||||
data (dsh/lookup-file-data state)
|
||||
objects (-> (dsh/get-page data page-id)
|
||||
(get :objects))
|
||||
|
||||
components (cfv/find-variant-components data objects (:id shape))]
|
||||
(into {} (map (juxt :id :variant-properties) components))))
|
||||
(get :objects))]
|
||||
(cfv/collect-variant-properties data objects shape)))
|
||||
|
||||
|
||||
;; Collects all the items together and split images into a
|
||||
@ -179,7 +177,8 @@
|
||||
(some? images)
|
||||
(update :images into images)
|
||||
|
||||
(ctk/is-variant-container? item)
|
||||
(or (ctk/is-variant-container? item)
|
||||
(ctk/is-variant-main? item))
|
||||
(update :variant-properties merge (collect-variants state item))))
|
||||
|
||||
(maybe-translate [shape objects parent-frame-id]
|
||||
|
||||
@ -10,6 +10,7 @@ Subcommands:
|
||||
boom Generate the :boom: section from `breaking change` issues
|
||||
security Generate the :lock: section from advisories fixed in a version
|
||||
cross-ref Compare merged milestone PRs against the changelog section
|
||||
dev-only-bugs List milestone bugs that never reached production
|
||||
report Generate the anomaly report (CHANGES-ISSUES.md)
|
||||
|
||||
Usage:
|
||||
@ -20,6 +21,7 @@ Usage:
|
||||
python3 scripts/changelog.py security "2.19.1"
|
||||
python3 scripts/changelog.py security "2.19.1" --write
|
||||
python3 scripts/changelog.py cross-ref "2.16.0"
|
||||
python3 scripts/changelog.py dev-only-bugs "2.16.0" [--apply]
|
||||
python3 scripts/changelog.py report "2.16.0"
|
||||
python3 scripts/changelog.py report "2.16.0" --changes CHANGES.md --output CHANGES-ISSUES.md
|
||||
|
||||
@ -615,6 +617,103 @@ def cmd_cross_ref(args: argparse.Namespace) -> None:
|
||||
print(f" #{pr['number']} {pr['title'][:80]}")
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────
|
||||
# Subcommand: dev-only-bugs
|
||||
# ─────────────────────────────────────────────
|
||||
|
||||
# Bugs found and fixed while building a feature of the same, unreleased
|
||||
# milestone never reached end users and stay out of the changelog.
|
||||
AUTO_PARENT_TYPES = {"Feature", "Enhancement"}
|
||||
|
||||
VERSION_FIELD_RE = re.compile(
|
||||
r"^#{2,4}\s*(?:affected versions?|versions?|environment)[^\n]*\n(.*?)(?=^#{2,4}\s|\Z)",
|
||||
re.IGNORECASE | re.MULTILINE | re.DOTALL,
|
||||
)
|
||||
DEV_ENV_RE = re.compile(r"\b(?:hourly|develop|staging|pre|unreleased)\b", re.IGNORECASE)
|
||||
PROD_ENV_RE = re.compile(
|
||||
r"\b(?:production|pro|main|saas|design\.penpot\.app|self[- ]?hosted?)\b",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
VERSION_RE = re.compile(r"\b(\d+)\.(\d+)(?:\.\d+)?")
|
||||
|
||||
|
||||
def version_field(body: str | None) -> str:
|
||||
"""Return the "Affected versions" / "Environment" text of an issue body."""
|
||||
match = VERSION_FIELD_RE.search(body or "")
|
||||
return " ".join(match.group(1).split()) if match else ""
|
||||
|
||||
|
||||
def only_unreleased_envs(text: str, milestone: str) -> bool:
|
||||
"""True when the version text names unreleased environments only:
|
||||
hourly, develop, staging, PRE or a version of this milestone. Any
|
||||
production hint or other version number keeps the bug in the changelog."""
|
||||
current = tuple(milestone.split(".")[:2])
|
||||
versions = [m for m in VERSION_RE.findall(text)]
|
||||
dev = bool(DEV_ENV_RE.search(text)) or current in versions
|
||||
prod = bool(PROD_ENV_RE.search(text)) or any(v != current for v in versions)
|
||||
return dev and not prod
|
||||
|
||||
|
||||
def find_dev_only_bugs(milestone: str, issues: list[dict]) -> tuple[list[dict], list[dict]]:
|
||||
"""Split the milestone's changelog bugs that never reached production.
|
||||
|
||||
``issues`` come from ``gh.py issue`` (they carry ``body`` and
|
||||
``parent``). Returns ``(auto, suggested)``: ``auto`` bugs are
|
||||
sub-issues of a Feature or Enhancement of this milestone and get the
|
||||
`no changelog` label; ``suggested`` ones need a human to confirm."""
|
||||
auto, suggested = [], []
|
||||
for issue in issues:
|
||||
if issue.get("issue_type") != "Bug" or issue.get("state") != "CLOSED":
|
||||
continue
|
||||
if {"no changelog", "release blocker"} & set(issue.get("labels", [])):
|
||||
continue
|
||||
parent = issue.get("parent") or {}
|
||||
entry = {"issue": issue["number"], "issue_title": issue.get("title", "")}
|
||||
if parent.get("milestone") == milestone:
|
||||
entry["reason"] = f"sub-issue of {parent.get('issue_type')} #{parent['number']} in {milestone}"
|
||||
(auto if parent.get("issue_type") in AUTO_PARENT_TYPES else suggested).append(entry)
|
||||
continue
|
||||
# `regression` says something that worked broke, not where: it
|
||||
# also marks regressions users hit in production.
|
||||
if "regression" in issue.get("labels", []):
|
||||
entry["reason"] = "labelled `regression`"
|
||||
suggested.append(entry)
|
||||
continue
|
||||
text = version_field(issue.get("body"))
|
||||
if only_unreleased_envs(text, milestone):
|
||||
entry["reason"] = f'only seen on "{text[:80]}"'
|
||||
suggested.append(entry)
|
||||
return auto, suggested
|
||||
|
||||
|
||||
def fetch_milestone_bugs(milestone: str) -> list[dict]:
|
||||
"""Closed milestone bugs with body and parent (two batched lookups)."""
|
||||
issues = json.loads(run_gh_py("issues", milestone))
|
||||
numbers = [str(i["number"]) for i in issues if i.get("issue_type") == "Bug"]
|
||||
return json.loads(run_gh_py("issue", *numbers)) if numbers else []
|
||||
|
||||
|
||||
def cmd_dev_only_bugs(args: argparse.Namespace) -> None:
|
||||
"""List bugs that never reached production; --apply labels the sure ones."""
|
||||
auto, suggested = find_dev_only_bugs(args.milestone, fetch_milestone_bugs(args.milestone))
|
||||
|
||||
print(f"Label `no changelog` ({len(auto)}):")
|
||||
for e in auto:
|
||||
print(f" #{e['issue']} {e['issue_title'][:70]} ({e['reason']})")
|
||||
print(f"Candidates to confirm ({len(suggested)}):")
|
||||
for e in suggested:
|
||||
print(f" #{e['issue']} {e['issue_title'][:70]} ({e['reason']})")
|
||||
|
||||
if args.apply:
|
||||
for e in auto:
|
||||
subprocess.run(
|
||||
["gh", "issue", "edit", str(e["issue"]), "--repo", REPO,
|
||||
"--add-label", "no changelog"],
|
||||
check=True, capture_output=True,
|
||||
)
|
||||
print(f"Labelled {len(auto)} issues `no changelog`", file=sys.stderr)
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────
|
||||
# Subcommand: report
|
||||
# ─────────────────────────────────────────────
|
||||
@ -872,6 +971,9 @@ def cmd_report(args: argparse.Namespace) -> None:
|
||||
):
|
||||
anomalies_d.append({"version": ver, "line": line[:100]})
|
||||
|
||||
# Bugs that may never have reached production: warnings, not anomalies.
|
||||
dev_auto, dev_suggested = find_dev_only_bugs(milestone, fetch_milestone_bugs(milestone))
|
||||
|
||||
def fmt_ms(ms: str | None) -> str:
|
||||
return ms if ms else "_none_"
|
||||
|
||||
@ -1033,6 +1135,26 @@ def cmd_report(args: argparse.Namespace) -> None:
|
||||
"✅ No highlight gaps found. All released X.Y.0 versions have properly referenced :rocket: entries.\n\n"
|
||||
)
|
||||
|
||||
if dev_auto or dev_suggested:
|
||||
f.write("## Bugs that may never have reached production\n\n")
|
||||
f.write(
|
||||
"These are warnings, not anomalies. Bugs found while building "
|
||||
"a feature of this release never affected end users. Add the "
|
||||
"`no changelog` label to the ones you confirm and re-run.\n\n"
|
||||
)
|
||||
if dev_auto:
|
||||
f.write(
|
||||
"### Sub-issues of a feature of this milestone (run `dev-only-bugs --apply`)\n\n"
|
||||
)
|
||||
for e in dev_auto:
|
||||
f.write(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}: {e['reason']}\n")
|
||||
f.write("\n")
|
||||
if dev_suggested:
|
||||
f.write("### Candidates to confirm\n\n")
|
||||
for e in dev_suggested:
|
||||
f.write(f"- {issue_link_title(e['issue'], e['issue_title'][:80])}: {e['reason']}\n")
|
||||
f.write("\n")
|
||||
|
||||
f.write("---\n\n")
|
||||
f.write("## Context\n\n")
|
||||
f.write(f"- Milestone: **{milestone}**\n")
|
||||
@ -1100,6 +1222,15 @@ def main() -> None:
|
||||
help="Path to the changelog file (default: CHANGES.md)")
|
||||
p_cross.set_defaults(func=cmd_cross_ref)
|
||||
|
||||
p_dev = sub.add_parser(
|
||||
"dev-only-bugs",
|
||||
help="List milestone bugs that never reached production",
|
||||
)
|
||||
p_dev.add_argument("milestone", help="Milestone title (e.g. '2.16.0')")
|
||||
p_dev.add_argument("--apply", action="store_true",
|
||||
help="Label the sub-issues of a milestone feature `no changelog`")
|
||||
p_dev.set_defaults(func=cmd_dev_only_bugs)
|
||||
|
||||
p_report = sub.add_parser(
|
||||
"report", help="Generate the anomaly report (CHANGES-ISSUES.md)"
|
||||
)
|
||||
|
||||
@ -752,8 +752,10 @@ GQL_ISSUE_BY_NUMBER_QUERY_ITEM = """\
|
||||
number
|
||||
title
|
||||
state
|
||||
body
|
||||
milestone {{ title }}
|
||||
issueType {{ name }}
|
||||
parent {{ number issueType {{ name }} milestone {{ title }} }}
|
||||
labels(first: 20) {{ nodes {{ name }} }}
|
||||
assignees(first: 10) {{ nodes {{ login }} }}
|
||||
closedByPullRequestsReferences(first: 5) {{ nodes {{ number }} }}
|
||||
@ -775,12 +777,19 @@ def issue_node_to_dict(node: dict) -> dict:
|
||||
"""Convert a GraphQL issue node to the shared issue dict shape."""
|
||||
issue_type = node.get("issueType")
|
||||
ms = node.get("milestone")
|
||||
parent = node.get("parent")
|
||||
return {
|
||||
"number": node["number"],
|
||||
"title": node["title"],
|
||||
"state": node["state"],
|
||||
"body": node.get("body"),
|
||||
"milestone": ms["title"] if ms else None,
|
||||
"issue_type": issue_type["name"] if issue_type else None,
|
||||
"parent": {
|
||||
"number": parent["number"],
|
||||
"issue_type": (parent.get("issueType") or {}).get("name"),
|
||||
"milestone": (parent.get("milestone") or {}).get("title"),
|
||||
} if parent else None,
|
||||
"labels": [lbl["name"] for lbl in node["labels"]["nodes"]],
|
||||
"assignees": node_assignees(node),
|
||||
"closing_prs": [pr["number"] for pr in node["closedByPullRequestsReferences"]["nodes"]],
|
||||
|
||||
@ -657,5 +657,61 @@ class CmdSecurityTests(unittest.TestCase):
|
||||
self.assertIn("will not be listed", err)
|
||||
|
||||
|
||||
def bug(number, body="", parent=None, labels=()):
|
||||
return {
|
||||
"number": number,
|
||||
"title": f"Bug {number}",
|
||||
"state": "CLOSED",
|
||||
"issue_type": "Bug",
|
||||
"labels": list(labels),
|
||||
"body": body,
|
||||
"parent": parent,
|
||||
}
|
||||
|
||||
|
||||
class OnlyUnreleasedEnvsTests(unittest.TestCase):
|
||||
def check(self, text):
|
||||
return ch.only_unreleased_envs(text, "2.19.0")
|
||||
|
||||
def test_dev_environments(self):
|
||||
for text in ["HOURLY", "https://hourly.penpot.dev/", "PRE & hourly",
|
||||
"staging (current)", "develop (2.19.0 unreleased)",
|
||||
"2.19.0-RC1 (develop)", "2.19.0"]:
|
||||
with self.subTest(text=text):
|
||||
self.assertTrue(self.check(text))
|
||||
|
||||
def test_production_hints(self):
|
||||
for text in ["PRO", "pre/pro/hourly", "develop / staging / main",
|
||||
"2.18.0-RC5", "2.18.1, 2.19.0-RC1", "self-hosted: 2.17.2",
|
||||
"design.penpot.app and self-host"]:
|
||||
with self.subTest(text=text):
|
||||
self.assertFalse(self.check(text))
|
||||
|
||||
def test_no_signal(self):
|
||||
for text in ["", "_No response_", "-", "development build"]:
|
||||
with self.subTest(text=text):
|
||||
self.assertFalse(self.check(text))
|
||||
|
||||
|
||||
class FindDevOnlyBugsTests(unittest.TestCase):
|
||||
def test_splits_auto_and_suggested(self):
|
||||
issues = [
|
||||
bug(1, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"}),
|
||||
bug(2, parent={"number": 11, "issue_type": "Enhancement", "milestone": "2.19.0"}),
|
||||
bug(3, parent={"number": 12, "issue_type": "EPIC", "milestone": "2.19.0"}),
|
||||
bug(4, parent={"number": 13, "issue_type": "Feature", "milestone": "2.18.0"}),
|
||||
bug(5, body="### Affected versions\n\nhourly\n"),
|
||||
bug(6, body="### Affected versions\n\n2.18.2\n"),
|
||||
bug(7, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"},
|
||||
labels=["no changelog"]),
|
||||
{**bug(8, parent={"number": 10, "issue_type": "Feature", "milestone": "2.19.0"}),
|
||||
"issue_type": "Enhancement"},
|
||||
bug(9, body="### Affected versions\n\n2.18.2\n", labels=["regression"]),
|
||||
]
|
||||
auto, suggested = ch.find_dev_only_bugs("2.19.0", issues)
|
||||
self.assertEqual([e["issue"] for e in auto], [1, 2])
|
||||
self.assertEqual([e["issue"] for e in suggested], [3, 5, 9])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user