Compare commits

..

216 Commits

Author SHA1 Message Date
Yamila Moreno
172a3589e8 🔧 Prevent PRs without issue from merging
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-19 09:28:57 +01:00
Andrey Antukh
fd785b8246
⚡ Stop resending deleted component objects on every touch (#12308)
* ⚡ Stop resending deleted component objects on every touch

`pcb/update-component` attached the whole `:objects` map of a deleted
component to every `mod-component` change, so a single component swap
re-sent thousands of shapes that the same batch already carried as
shape changes. In one measured swap the 49 maps with objects were 94%
of the request (6.3 MB), and only 27% of the 4.275 shapes they carried
differed from what the server had.

Now `mod-component` carries component metadata only, and shapes inside
a component travel as `:mod-obj` changes scoped with `:component-id`,
which is what the component sync already emits. An absent `:objects`
key means "leave the stored shapes alone", only an explicit `nil`
drops them, so metadata-only changes cannot wipe a deleted component.

The two repairs that fix shapes inside a component (shape-ref cycle
and duplicated swap slot) now emit one `:mod-obj` per shape; the two
that move or clear the `:objects` key keep the whole-map write behind
the new `:objects? true` option.

Same swap on the measured file: 6.328.307 to 382.571 bytes.

AI-assisted-by: space-bunny

* 🔧 Cover the mod-component objects contract with tests

`pcb/update-component` sends the `:objects` map only when the caller
passes `:objects? true`, and `ctkl/mod-component` reads an absent key as
"leave the stored shapes alone". Two repairs depend on that option, and
dropping it broke nothing visible: the change carried no `:objects`, the
applier kept the invalid component as is, and the suite stayed green.

Add one repair test per error code, including the deleted-component
branch that must keep the map empty, and a round-trip test for the
shape-ref cycle repair. That repair now emits one `:mod-obj` per shape
and had no coverage; the test pins the scoped `:component-id` and that no
shape is lost.

Reword the `update-component` docstring as well: it describes what the
option does to the change instead of listing its current callers, which
goes stale as soon as another caller uses it.

AI-assisted-by: deepseek-v4.1-flash

* 🔧 Harden the mod-component repair tests

Address the review of the previous commit.

The cycle test only rejected a shape-ref pointing to itself, so it would
have passed if the repair repointed the link instead of removing it, or
if it left the component links on the main root untouched. It now
compares the resulting objects map with the expected one.

The four tests only inspected the first change, so a duplicated change
went unnoticed. They now compare the full list of changes: an exact
vector for the two repairs that rewrite the map, and id frequencies for
the cycle repair, which emits one change per shape in no fixed order.

The deleted-component fixture now removes the main instance through
`tho/delete-shape`, the same change generator the editor uses, instead
of a hand-written `:del-obj`. The message that promised a restorable map
now says what the repair does: normalize the value to an empty map.

AI-assisted-by: deepseek-v4.1-flash
2026-11-18 23:18:28 +01:00
Andrey Antukh
7a64142cbc Merge remote-tracking branch 'origin/staging' into develop 2026-11-18 20:19:33 +01:00
Andrey Antukh
db1e2f7c9b Merge branch 'staging-2.19.x' into staging 2026-11-18 20:17:37 +01:00
Andrey Antukh
7916e5e073 📚 Clarify agent memory guidance
Promote Clojure design rules to a required core memory and simplify
the discovery guide. Correct test output redirection and RPC routes.

AI-assisted-by: gpt-6.1-terra
2026-11-18 19:14:00 +00:00
Andrey Antukh
9b24b6c7c9 📚 Improve agent workflow safeguards
Protect staged changes when rewriting commit messages and close plans
after verified commits. Clarify checks and remove the unused Taiga skill.

AI-assisted-by: gpt-6.1-terra
2026-11-18 19:14:00 +00:00
Andrey Antukh
7d5fef711c 📚 Refine plan and code review criteria
Make review criteria proportional to risk and project conventions.
Keep code reviews read-only, use existing test evidence and stable IDs.

AI-assisted-by: gpt-6.1-terra
2026-11-18 19:13:59 +00:00
Andrey Antukh
14a623da08 📚 Apply plan review improvements automatically
Apply clear review improvements to the plan and resolve consequential
decisions through the walkthrough before updating the plan metadata.

AI-assisted-by: gpt-6.1-terra
2026-11-18 19:13:59 +00:00
Andrey Antukh
e7732541c3 📚 Add follow-up verification to code reviews
Include remedies, tests and decisions in review findings. Preserve review
context in plans and offer to resume the reviewer after implementation.
Keep follow-up verification optional and report deferred findings clearly.

AI-assisted-by: gpt-6.1-terra
2026-11-18 19:13:59 +00:00
Andrey Antukh
b76eaa1f5c 📚 Switch agent tooling to @-mentions for skills
Agent skills are invoked directly with @-mentions, so the slash-command
wrappers that exposed them as /commands are no longer needed. Update
.agents/README.md and every skill that references another skill to use the
@ syntax, and delete the six .opencode/commands/ dispatcher files.

AI-assisted-by: hy4-flash-free
2026-11-18 19:13:59 +00:00
Alonso Torres
15f9994022
✨ Add test coverage reports to CI and pull requests (#13156)
Measure the coverage of the backend, common, frontend and render-wasm
test suites in CI, and show it on the workflow run pages and in one
"Coverage Report" comment on each pull request.

- Backend and common run kaocha with the cloverage plugin through a new
  `:coverage` alias, which writes lcov, HTML and an EMMA report. The PR
  report shows lines and Clojure forms (as statements); cloverage
  measures no functions or branches. Plain local runs are unchanged.
- Frontend runs the compiled test bundle with V8 coverage
  (`pnpm run test:coverage`, `frontend/scripts/coverage.js`) and maps it
  back to the ClojureScript sources through the source maps of the test
  build. UI namespaces (`app/main/ui`) are left out.
- Render-wasm runs its native Rust tests under cargo-llvm-cov when
  `RENDER_WASM_COVERAGE` is set. The devenv image lacks the tool, so the
  workflow downloads a pinned release and checks its sha256.
- `scripts/coverage-summary.mjs` renders an lcov file as a markdown table
  by source area for `$GITHUB_STEP_SUMMARY`; each workflow also uploads
  the HTML report as an artifact.
- `scripts/coverage-comment.mjs` keeps a single PR comment, laid out like
  the Vitest Coverage Report Action, with a section per project and the
  coverage of the changed files. Each job updates its own section; the
  data of every project lives in a hidden block of the comment, and a
  job writes again when a concurrent one dropped its data.
- Lines have targets that only colour the report: 70% backend and
  common, 50% frontend, 40% render-wasm.
- The comment step only runs for PRs from branches of this repo: fork
  PRs get a read-only token and keep the summary on the run page.

AI-assisted-by: claude-opus-5-5
2026-11-18 17:53:32 +01:00
Francis Santiago
617e56b71e
Merge pull request #13153 from penpot/fc-issue-done-status-automation
✨ Route completed issues by QA requirement
2026-11-18 16:29:49 +01:00
Francis Santiago
b488f6b2df ✨ Route completed issues by QA requirement
Set issues with the team-qa label to Ready for QA.
Set all other completed issues to Product Review.
2026-11-18 16:15:13 +01:00
Andrey Antukh
eddb824ca7 🐛 Resolve UI CSS copy paths during bundle builds
Resolve the UI CSS source and destination from the package directory so
Vite works when launched outside that directory. Add a regression test and
quiet the bundle's rsync output.

AI-assisted-by: gpt-6-luna
2026-11-18 15:56:07 +01:00
Pablo Alba
8e575894f1
✨ Add migrations to sanitize variant properties and align component names (#12957)
* ✨ Add migration to sanitize variant property text

(cherry picked from commit 77f1767240026df0cc705f4f084725e28b94230e)

* ✨ Add migration to align variant component names

(cherry picked from commit 844993796d2cd16a02d20e10469107a769ace60c)

* ✨ MR fixes

* ✨ MR review 2
2026-11-18 15:46:38 +01:00
Luke Lau
5166b8d5ea
⚡ Avoid iterating over every object in frame-titles* (#13089)
* ⚡ Avoid iterating over every object in frame-titles*

Similarly to #13075, frame-titles* used to iterate over every object to work
out the frames at the root that need titles.

For files with lots of objects this can be slow, but we can just get the root
frames by looking at the root object's children.

We need to remove non-main-component instances to copy the behaviour of
`get-frame-ids :skip-copies? true`. This also now causes the title to be drawn
on a child frame whenever it's focused.

AI-assisted-by: claude-opus-5-5

* 🐛 Don't show frame title on focused frames

Restores the existing behaviour when focused
2026-11-18 15:17:44 +01:00
Yamila Moreno
dedd435db5
🔧 Add PRs to status tech-review (#13152)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 14:59:46 +01:00
Eva Marco
fff44ec758
🐛 Fix shared files 500 on shapes with invalid child ids (#12528)
* 🐛 Fix shared files 500 on shapes with invalid child ids

Old files can list in :shapes ids that are not uuids or that point to
no shape. Migration 0033 walked them and hit the uuid assert in
get-shape, which throws an AssertionError. The shared files list only
caught Exception, so that Error broke the whole list with a 500.

Make migration 0033 skip those ids, and catch Throwable when loading
each shared file, so a broken file is skipped and logged.

Closes #12343

AI-assisted-by: claude-opus-5-5

* 🐛 Repair invalid child ids in migration 0033 v2

Skipping invalid child ids in migration 0033 was not enough: the ids
stayed in :shapes, so the file still failed schema validation and
could not load. Files that already applied the migration would also
never run a changed version of it.

The migration now repairs each container before the naming walk:
string ids are parsed with uuid/coerce, nil and ids that point to no
shape are dropped, the duplicate that 0032 adds when a child is
listed as a string is removed, and a :shapes list becomes a vector.
Only the shapes that change are re-assoc'ed, so a healthy ObjectsMap
is not marked as modified and its page is not rewritten.

It ships as 0033-normalize-main-instance-names-v2 at the same
position in available-migrations, so files that recorded the old id
run it again. That position also makes it parse strings before
0035-fix-dangling-children drops the ids it cannot find.

The shared files listing restores the interrupt flag when the cause
of a skipped file is an InterruptedException.

Tests cover children outside the root walk, legacy components, the
objects-map wrapper of the unrealized save path, a list turned into a
vector, a shape without a :shapes key, a file that applied the old
id, and the end-to-end load of a broken shared file.

Relates to #12343

AI-assisted-by: claude-opus-5-5
AI-assisted-by: inkling-2-small-free
2026-11-18 13:30:40 +01:00
Andrey Antukh
efd95dcf0c Merge remote-tracking branch 'origin/staging' into develop 2026-11-18 13:20:18 +01:00
Andrey Antukh
dad4479b9c Merge remote-tracking branch 'origin/staging' into staging 2026-11-18 13:15:27 +01:00
Andrey Antukh
2377ab0a16 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-18 13:15:10 +01:00
Eva Marco
28a4794d73
🔧 Test the direction of the move shortcuts (#13115)
#13068 fixed the move left and right shortcuts, which moved and showed
the opposite direction. Its test only checks the key binding. These
tests run each move shortcut, in the workspace and the path editor,
and check that it moves in its own direction, fast or by one unit, and
that its tooltip shows the matching arrow.

Without the fix of #13068, 8 assertions fail.

AI-assisted-by: claude-opus-5-5
2026-11-18 13:05:44 +01:00
Eva Marco
eb7860fcd7
🐛 Show the date of a restored autosaved version in the restore notice (#13120)
When a peer restored an autosaved version, the notice said "restored an
earlier version" instead of naming it by its date and time.

The modal wrapper passes the props map to the component without
camelCasing its keys, so the `created-at` prop reached the component as
"created-at" while it read `createdAt`, and the date was always nil.
Named versions were fine because `label` is one word. The prop is now
`date`, and a test renders the registered modal the way the wrapper
does.

Closes #13037

AI-assisted-by: claude-opus-5-5
2026-11-18 13:04:55 +01:00
Andrey Antukh
aa12e52081
⬆️ Update the system dependencies of the devenv and the images (#13144)
* ⬆️ Bump the JDK to 27.0.1.1 on devenv and the backend image

The devenv image compiled the backend bundle with JDK 27 while the
production image ran JDK 26. The runtime starts with --enable-preview,
so a runtime line behind the compiler line is a silent trap rather than
a working setup.

Both pins now use Zulu 27.0.1.1 (zulu27.30.201-ca), the current GA of
the line, with the same sums on both architectures. The jlink module
list in the backend image still resolves on 27, so it does not change.

Sums checked by downloading both tarballs; the jlink JRE built from
them starts and reports 27.0.1.1.

AI-assisted-by: deepseek-v4.1-flash

* 🔥 Remove the unused gitpod setup

No workflow built docker/gitpod and its AdoptOpenJDK 16 install no
longer matches the devenv. The README badge linked to a setup that no
longer exists, so it goes with it.

The historical changelog entry stays: CHANGES.md is generated from the
milestones.

AI-assisted-by: deepseek-v4.1-flash

* ⬆️ Bring the docker/ updates from develop

Move the devenv and the production images to what develop already
carries:

- Base images to the current dhi.io digests.
- Node 24.22.3 and pnpm 12.21.0 in the devenv image and in every
  production Dockerfile that ships pnpm.
- Rust 1.99.0 and Serena 2.1.0, with SERENA_UPDATE_VERSION raised to
  match and a note that it must never be lower than the image pin.
- tini as the exporter ENTRYPOINT.

Devenv additions: PENPOT_DEVENV=true so the frontend compiles with the
admin console disabled by default, xauth and libvulkan1 plus the GPU
compose overlay, the MCP OAuth metadata route in nginx, and clipboard
support in tmux.

The JDK stays at 27.0.1.1 on both pins.

AI-assisted-by: deepseek-v4.1-flash

* ⬆️ Finish the Node 24.22.3 bump outside docker/

The docker/ import brought Node 24.22.3 into the devenv and the image
tags while .nvmrc and the two plugins-deploy workflows stayed on
24.22.2.

Node stays on the 24.x line on every site: the plugins-deploy runtime
pins follow .nvmrc, even though develop moved them to node@26.13.1 in
99db795966.

AI-assisted-by: deepseek-v4.1-flash
2026-11-18 12:51:30 +01:00
Yamila Moreno
89d5a862b5
🔧 Require milestone in PRs (#13129)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 11:54:01 +01:00
Álvaro Tejero-Cantero
a84d76d397
🐛 Stop camelCase svg attributes from breaking files (#13132)
* 🐛 Validate the shapes of a page added by an update-file change

The backend validates every shape an `:add-obj` or a `:mod-obj` change
writes, but an `:add-page` change carries a whole page and its shapes
reached the file unchecked. Duplicate page and the undo of a page
deletion send that page as the client holds it, so a tab that loaded a
file before a migration ran could write the old format back into it.
A tab opened before #12006 deployed holds camelCase svg attr keys such
as `:fillRule`, and the closed `:svg-attrs` schema rejects them. The
file saves, and the next read that migrates it fails the file data
check, so the whole file stops opening (#13107).

`:add-page` now validates each shape of the page on the backend, as
`:add-obj` does for one.

AI-assisted-by: claude-opus-5-5

* 🐛 Rerun the svg attr normalization on files that recorded 0035

Files that recorded migration 0035 never run it again, so camelCase
svg attr keys written after it ran stay in the file. A tab that loaded
a file before 0035 ran could save such keys through an `:add-page`
change, and the closed `:svg-attrs` schema then rejects the whole file
on its next migrating read (#13107).

The svg attr normalization now runs under the id
`0037-fix-shape-svg-attrs-and-defs`, and the 0035 id leaves
`available-migrations`, as 0021 did when 0035 replaced it. Every file
that recorded 0035 migrates again. Kebab input passes through
unchanged, so the rerun touches only the keys written after 0035.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the 0037 migration tests to their own namespace

The svg-attr migration was renamed to 0037, but its tests stayed in
the 0035 test namespace under migration-0035-* names. Move the whole
section to files_migrations_0037_test.cljc, register the new namespace
in the test runner, and rename its tests to match the migration id.

Replace the doseq over the two withdrawn ids with two self-contained
deftests, one for 0021 and one for 0035. No production code changes.

AI-assisted-by: hy4-flash-free

* 🐛 Keep the log watch installed until the rejection record arrives

The logger swaps the record on an executor thread, after the call that
logged it returns. The test removed the watch in the `finally` around
`update-file` and only then polled for the record, so under load the
record landed on nobody and the test read a nil `:attempts`.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Count only update-file calls in the save-test transport log

The mocked transport is installed with `set!` and outlives the fixture: a
timer another module left pending can still reach it mid-test, as a
`:delete-file-object-thumbnails` call did in CI. Filter to the only
command persistence sends, `update-file`, so a stray call neither fails
the emptiness assertion nor releases a `wait-for` on a save.

AI-assisted-by: deepseek-v4.1-flash

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 11:39:32 +01:00
Luke Lau
d563444985
✨ Delay building schemas in check-fn in release builds (#12711)
* ✨ Delay building schemas in check-fn

check-fn eagerly builds its schemas which contributes to the startup time. But
we can delay this to when the check function is actually called, which seems to
be when an edit is made. On the Travel A2B - Locofy Sample project file this
reduced the time to the first viewport render by ~15%.

* ✨ Make schema resolution lazy only on production builds

Add a lazy-delay macro that defers the expression only on builds that
set the app.common.data.macros/lazy-delay? closure define, and use it
in check-fn to resolve the schema.

The previous change made the schema resolution unconditionally lazy,
which hid wrong schema definitions until the first check call. With
this change development builds, tests and storybook resolve the schema
when the namespace loads, so a bad definition fails early, and
production builds keep the deferred resolution and the startup win.

The define is enabled in the release builds of frontend (main and
worker), exporter and library. On the JVM the expression is always
evaluated immediately.

AI-assisted-by: deepseek-v4.1-flash

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 11:36:33 +01:00
David Barragán Merino
240e757586 🐛 Size frontend nginx workers from the container CPU limit
worker_processes auto starts one worker per host CPU, ignoring the
container CPU limit, and since worker_connections went from 2048 to
65535 each worker reserves about 30 MB at startup. A frontend limited
to a few CPUs on a 64-core node needs close to 2 GB before serving a
request and gets OOM killed.

Default the worker count to the CPUs the container can use, its cpuset
and its cgroup CPU quota rounded up, when they are fewer than the CPUs
of the host, and keep auto otherwise. Make both settings configurable
through PENPOT_HTTP_SERVER_WORKER_PROCESSES and
PENPOT_HTTP_SERVER_WORKER_CONNECTIONS, and refuse to start on values
that are not valid.

Refs #13103

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-18 10:51:18 +01:00
Yamila Moreno
dfc2ff6b56
🔧 Improve issue templates (#13126)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 10:00:39 +01:00
Andrey Antukh
3f6c69bb6a Merge remote-tracking branch 'origin/staging' into develop 2026-11-18 09:32:49 +01:00
Álvaro Tejero-Cantero
3b3669b63b
🐛 Drop map-valued svg attrs when migrating legacy files (#13130)
Files written by old Penpot versions hold maps where an svg
attribute value should be, such as a `:filter` that once read
`url(#filter_2)`. Since #12006 the `:svg-attrs` schema is closed and
admits scalars only, and migration 0035 kept those maps, so the file
data check at the end of every migration rejected the file. Such a
file can be neither imported nor opened. Two built-in templates,
Avataaars and Tutorial for beginners, are among them.

`attrs->kebab-props` now drops a value that is not a string, a number,
or a boolean, at the top level and inside `:style`. Migration 0035 and
every birth path go through it, so stored attrs match the schema.

Of the 143 files in penpot/penpot-files, 4 fail to import at
`:svg-attrs :filter` before this change, and all 4 import after it.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:31:39 +01:00
Andrey Antukh
930f74218c Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-18 09:29:54 +01:00
Andrey Antukh
09259623da 🔥 Remove the unused Signed-off-by validator
check_signed_off_by was never registered in the validator list, so it never
ran, and no test exercised it. The DCO sign-off applies to external
contributors through CONTRIBUTING.md, not to every commit, so the local
checker has nothing to enforce there.

Drop the function and the DCO mention in the script docstring.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:28:51 +00:00
Andrey Antukh
5c93ece9b9 📚 Slim the code review skills without losing checks
code-review-criteria and review-code carried weight that changed no
decision: a 16-row rationalizations table, red flags that repeated the axes,
and three sections written for the author of a change rather than for the
reviewer. 338 lines become 251.

Drop that layer, dedupe DRY/KISS/YAGNI between the core principles and the
axes, merge the two instruction lists of the flow, and compress the GitHub
comment format.

The reviewer no longer re-runs the suite or a build: the author's run and CI
already provide that evidence, and repeating it only made reviews longer. It
runs at most the single test file that settles a specific doubt, and states
which evidence it relied on.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:28:28 +00:00
Eva Marco
01ceba34ec
♻️ Give the team hero and recovery page links a real href (#12708)
* ♻️ Give the Team management link in the team hero a real href

"Team management" in the "Team up!" notice of a team's Projects page was
an <a> without href. It now links to the team Members page; a plain click
still runs the in-app navigation (prevent default), and Ctrl/Cmd-click
opens the link in a new tab.

New visual test "Shows the Team management link": 0 px of difference at
rest and on hover.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the Go to login link of the recovery page a real href

"Go to login" on the password recovery page was an <a> without href. It
now links to the login page; a plain click still runs the in-app
navigation (prevent default), and Ctrl/Cmd-click opens the link in a new
tab.

New visual test "Shows the Go to login link": 0 px of difference at rest
and on hover.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:22:07 +01:00
Eva Marco
1a355958a4
♻️ Give the team page section links a real href (#12705)
Members, Invitations, Webhooks and Settings in the team page header
were <a> elements without href: the keyboard could not reach them and
they could not open in a new tab. Each one now links to its section; a
plain click still runs the in-app navigation (prevent default), and
Ctrl/Cmd-click opens the link in a new tab. The current section gets
aria-current="page".

They keep their look: the "Team header menu" visual test gives 0 px of
difference at rest and on hover.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:21:14 +01:00
Andrey Antukh
dce4d8fe56 ✨ Allow human co-authors in the commit checker
The checker rejected any Co-authored-by trailer, which also blocked a real
person. The trailer is meant to credit a human co-author, and git and
GitHub use it for that; only AI providers, agents and bots are unwanted.

Match the trailer value against a blocklist of providers and agents plus
bot identities, and point to the dedicated AI-assisted-by trailer for AI
participation. Co-authored-by joins the trailer exemption of the
80-character body rule.

Update the tests, CONTRIBUTING.md, the create-commit skill and the commit
memory to the new rule.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:20:43 +00:00
Florian Schrödl
cdfbff7c53
🐛 Fix active themes in token export and the empty file import error (#13123)
* 🐛 Fix token export writing the wrong active themes

The export passed only the tokens library, so the active themes and
sets in `$metadata` were the ones of the imported file, kept in the
library, instead of the ones activated in the file. The multi-file
export now needs the file's status, so no caller falls back to them.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

* 🐛 Say no tokens were found when importing a file without tokens

Importing `{}`, or a JSON file without any token, said the token data
was invalid.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

---------

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-18 09:19:55 +01:00
Yamila Moreno
d403a66122
🔧 Require milestone to issues linked to a PR (#13133)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 09:07:22 +01:00
penpot-renovate[bot]
a9972d5b47
⬆️ Update dhi.io/nginx image to 5f75855 (#13136)
- dhi.io/nginx 1.31.8-debian13-dev@5f75855 (docker/images/Dockerfile.frontend)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-18 09:05:41 +01:00
Luke Lau
14abaf93f4
⚡ Cull tile shapes to the tile instead of its whole margin (#13112)
After background blur support was added to render-wasm in 5ba53f7296,
we needed to start rendering shapes in the 256px margins of the tile,
since they contribute to the blur result.

This means that a tile may end up drawing shapes from the tiles around
it, and in the worst case we end up drawing 4x the number of shapes.

We can limit this to only the tiles that have background blur, which
significantly reduces the number of shapes drawn.

Masked groups do something similar so this generalizes and reuses the
`group_reach` stack, which stores how far past the current tile is
painted.

Before we start drawing the shapes in `render_shape_tree_partial`, we
walk the shapes in the tile to check how far the background blur
"reaches" and extend `group_reach` by just that amount, rather than
a 256px radius. As part of the generalization we also need to update
`masked_group_filter_reach` so it works on non-masked groups.

On the mem_benchmark file a full render calls render_shape 77.1% less
often. On the admin dahsboard there's 62% less. This gain shows up
particularly in interactive transforms, e.g. dragging another board
across the heroicons board goes from about 21 to 48 fps.

AI-assisted-by: claude-opus-5-5
2026-11-18 07:07:53 +01:00
Pablo Alba
73e7815817
🐛 Make execute_code wait for save before success (#13027)
* 🐛 Make execute_code wait for save before success

* 🐛 Address review feedback on waitForSave

Reject with an Error carrying a `code`: save-timeout, rolled-back,
save-retrying, or the failure code (save-failed by default). Lower the
default timeout from 90s to 60s, leaving room for the 30s layout wait
and the code under the MCP tool timeout.

When persistence is already in error at the start of the wait, keep the
stored failure code, and tell a failure Penpot keeps retrying apart from
one that needs the file reopened.

Share timeout validation with waitForLayoutUpdate through
app.plugins.utils, and make reject-not-valid reject with an Error, which
also fixes the string rejections of connectLibrary, restore, remove,
pin, export, findVersions and saveVersion.

The MCP execute_code handler now waits for the save only when the
workspace is ready, like the layout wait, so dashboard calls no longer
fail on a save error from an earlier file.

Move the waitForSave tests to frontend-tests.plugins, add tests for the
error codes, the retrying state and a failure during the wait, and add
plugin API suite and MCP handler tests. Update the docs, CHANGELOG and
the generated API files.

AI-assisted-by: claude-opus-5-5
2026-11-17 22:35:37 +01:00
Yamila Moreno
f65242fb00
🔧 Manage renovate PRs (#13131)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-17 19:00:05 +01:00
Elena Torró
14b9d77e9f
🐛 Fix duplicated suffix on single-file WASM exports (#13125) 2026-11-17 17:39:04 +01:00
David Barragán Merino
0915f89419 🐛 Size frontend nginx workers from the container CPU limit
worker_processes auto starts one worker per host CPU, ignoring the
container CPU limit, and since worker_connections went from 2048 to
65535 each worker reserves about 30 MB at startup. A frontend limited
to a few CPUs on a 64-core node needs close to 2 GB before serving a
request and gets OOM killed.

Default the worker count to the CPUs the container can use, its cpuset
and its cgroup CPU quota rounded up, when they are fewer than the CPUs
of the host, and keep auto otherwise. Make both settings configurable
through PENPOT_HTTP_SERVER_WORKER_PROCESSES and
PENPOT_HTTP_SERVER_WORKER_CONNECTIONS, and refuse to start on values
that are not valid.

Refs #13103

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-17 16:49:09 +01:00
Belén Albeza
dd83c668cc
🐛 Fix UX error handling (wasm) (#13100)
* 🐛 Show the error page for critical WASM errors

Since #8651 the :wasm-error handler showed the error page for :panic
only, so a :wasm-critical error (a WASM trap with no error code) fell
through to a toast and the user kept working on a broken renderer.

Only the codes in recoverable-wasm-codes now show a toast: :non-blocking
from Rust, plus :wasm-reload-context-failure and :viewer-canvas-failed,
raised by the frontend. Any other code shows the error page. This covers
errors from the render loop, event handlers, promises and store events;
errors thrown inside React effects still reach the error boundary.

The h/call docstring now describes what it does: it always throws.

Closes #12990

AI-assisted-by: claude-opus-5-5

* ✨ Add a guard for React effects that call WASM

React sends any error thrown in an effect, or in its cleanup, to the
error boundary at the app root, which replaces the whole workspace with
the error page. That holds even for recoverable WASM errors, which
should only show a toast.

errors/wrap-wasm-effect runs an effect fn and the cleanup fn it
returns. A recoverable WASM error goes to on-error (toast); any other
error is rethrown, so it still reaches the boundary. The macro
with-wasm-effect wraps mf/with-effect with it.

The macro lives in its own namespace, app.main.ui.hooks.wasm-effect:
app.main.ui.hooks cannot require app.main.errors, because the renderer
loads it and that makes a circular dependency.

AI-assisted-by: claude-opus-5-5

* 🐛 Show a toast for recoverable WASM errors in the viewport

A recoverable WASM error thrown inside a viewport effect reached the
React error boundary, which replaced the whole workspace with the
error page.

Every viewport effect that calls WASM in its body or cleanup now uses
with-wasm-effect, so a recoverable error shows a toast and the
workspace stays; panics and critical errors still show the error page.
Effects that call WASM only from a listener, ts/asap or ts/raf keep
mf/with-effect: those errors never reach React.

AI-assisted-by: claude-opus-5-5

* 🐛 Show a toast for recoverable WASM errors in the text editor

The WASM text editor calls WASM from two React effects: on mount (select
all, render) and in the first caret blink. A recoverable error there
reached the error boundary and replaced the workspace with the error
page.

Both effects now use with-wasm-effect. The mount effect also guards its
select-all and render step on its own: it opens an undo transaction
with no timeout first, and without that inner guard a recoverable error
would stop the effect before it returned the cleanup that commits the
transaction and disposes the WASM editor.

AI-assisted-by: claude-opus-5-5
2026-11-17 16:37:27 +01:00
Belén Albeza
2a9b9a57cc
🔧 Set as default the flag to enable html paste in v3 (#13114) 2026-11-17 16:15:26 +01:00
Alonso Torres
eb2c8cf359
🐛 Check Host and Origin headers in the MCP server (#13117)
* 🐛 Check Host and Origin headers in the MCP server

The MCP endpoint, the REPL server and the plugin WebSocket accepted
requests with any Host and Origin header, so a web page open in the
user's browser could drive a local MCP server: connect as the plugin
and call its tools.

RequestGuard checks both headers against an allowlist and rejects other
requests with HTTP 403, or WebSocket close code 1008. The list comes
from PENPOT_MCP_ALLOWED_HOSTS, where "*" turns the check off. When it
is unset, local mode allows only loopback names and remote mode allows
any host, since remote mode relies on user tokens and has no file
access.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep the Host and Origin check on in single-user remote mode

The default allowlist followed remote mode, so a single-user server with
PENPOT_MCP_REMOTE_MODE=true accepted any Host and Origin. It requires no
user token either, so a web page could still drive it and run code in the
user's Penpot session. The default now follows multi-user mode: single-user
servers allow only loopback names, and multi-user servers, which rely on
user tokens, allow any host. Startup warns when the check is off in
single-user mode.

PENPOT_MCP_ALLOWED_HOSTS entries with a scheme, port or path never matched,
so every request failed with 403. The server now refuses to start and names
the bad entry.

AI-assisted-by: claude-opus-5-5
2026-11-17 16:01:49 +01:00
Andrey Antukh
e6eb8bcd5a
✨ Add pgjdbc prepared statement cache settings (#13099)
* ✨ Add pgjdbc prepared statement cache settings

Expose pgjdbc's server-side prepared statement cache as three
optional backend variables: PENPOT_DATABASE_PREPARE_THRESHOLD,
PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_QUERIES and
PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_SIZE (bytes, converted
to whole MiB for the driver, rounding up). When unset, the
driver defaults apply (5 executions, 256 entries, 5 MiB), so the
behavior is unchanged. The values reach Hikari as
dataSourceProperties strings, because pgjdbc only reads String
values; flushCacheOnDdl stays at its default.

Closes #13092

AI-assisted-by: inkling-2-small-free

* ♻️ Sync the pool options schema with the code

The integrant pool options schema validated a ::connect-timeout
option that nothing reads, while the keys the code consumes
(::connection-timeout, ::idle-timeout, ::max-lifetime) were
missing from it. The schema now declares the three, matching the
code and the defaults map. The removed key was not referenced
anywhere else.

Follow-up to #13092.

AI-assisted-by: inkling-2-small-free

* ♻️ Rename the prepared statement cache threshold setting

Keep the three knobs under one name in the configuration:

  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_THRESHOLD
  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_QUERIES
  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_SIZE

The old PENPOT_DATABASE_PREPARE_THRESHOLD read as a verb and was the only
one of the three that did not name the prepared statement cache.

Fix the documentation of the values: 0 in either cache value disables the
cache, not only the size, and -1 prepares a statement from its first
execution and forces binary transfer, which is what pgjdbc does.

Cover the -1 threshold end to end: it already decoded and validated, but
nothing proved that it reaches the driver. The tests also pin 0, so a
falsy check cannot silently drop it.

Closes #13092

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Fix the prepared statement cache contracts

The threshold schema accepted any integer, but the driver clamps every
negative to -1 and turns on binary transfer, so a typo like -10 changed
the wire protocol instead of failing at boot. Both schemas now declare
{:min -1}: -1 and the non-negatives stay valid.

The tests compared our own dataSourceProperties literals with our own
literals, which cannot catch the failure the feature fears (a property
the driver ignores). They now read the threshold back from a real
connection: 5 with no options, 1 and 0 when configured. The size ceiling
is covered one byte over a MiB, and queries=0 too.

The docs, the app.db comment and the memory described the threshold 0 as
disabling the cache. It does not: 0 stops the server-side prepared
statements while the driver's cache of parsed queries stays in use,
bounded by the other two values. The same texts lost the
pg_prepared_statements pointer (the view only lists the session that runs
the query) together with the flushCacheOnDdl and PgBouncer sentences, and
the section now carries its version marker.

Closes #13092

AI-assisted-by: deepseek-v4.1-flash

* 💄 Format the prepared statement cache pool test

The continuation arguments of the two with-connection calls were aligned
under the options map instead of two spaces past the form, which cljfmt
rejects.

Local verification missed it because `./scripts/ci backend --test` runs
the tests only: lint and format are separate tasks, so the pre-push check
is the whole `./scripts/ci backend` or an explicit --lint/--fmt.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Address the review of the prepared statement cache

The pool startup line shows the three values, so the configured cache is
visible without reading the environment back; a nil means the driver
default.

bytes->mib-ceil uses Math/ceilDiv: the same rounding in one expression,
and the size schema already rejects negatives.

The cache-limit tests count a closed statement in pg_prepared_statements
instead of comparing the property literals. A closed statement stays
prepared in the server only while the driver's cache holds it, which is
what tells an accepted property from an effective one; the positive case
keeps the cache enabled with a sub-MiB size. The config test drops the
groups that only exercised the decoder and keeps the ones that pin the
accepted range.

The manual and the memory note that a query parameter of the same name in
PENPOT_DATABASE_URI wins over the variable: pgjdbc ignores the Properties
value when the URL carries it.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: claude-opus-5-5
2026-11-17 14:06:04 +01:00
Luke Lau
0bfcaa4109
⚡ Avoid redundant copy of tile atlas (#13104)
61cd9fe88683a4068817f538a2ae54d80aca095e stored a reference to the
last snapshot of the tile atlas, presumably to avoid the call to
image_snapshot when not needed.

But it turns out image_snapshot doesn't actually cost anything since
it just returns a reference to the surface texture. Storing the
reference actually causes a 4096x4096 texture copy because skia has
copy-on-write behaviour. Skia sees the reference stored in Surfaces,
so when the next render tries to draw on the surface it needs to
allocate + copy over the tile atlas texture.

By removing the reference skia can reuse the texture on the next
render which avoids a 64MiB copy on the GPU.

AI-assisted-by: claude-opus-5-5
2026-11-17 13:46:27 +01:00
Miguel de Benito Delgado
6043edb95a
♻️ Share WASM modifier bindings (#13016)
- Extract modifier encoding and FFI operations into common bindings
  for renderer benchmarks and application callers.
- Keep readiness checks and render scheduling in the frontend API.
- Release propagation output after decoding failures.
- Cover payload ownership, heap growth, snapping and interactive
  lifecycle behaviour with shared fixtures and integration tests.

AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-17 13:28:33 +01:00
Florian Schrödl
94b13d08fe
✨ Read and write token files with the tokenscript package (#12481)
Token import and export move to tokenscript.document.read and
tokenscript.document.write in the tokenscript-interpreter package
(penpot/tokenscript-interpreter#4).
tokens-lib keeps an adapter that maps its documents to a TokensLib and
back: penpot's token types, the keys of composite values and the
activation of themes and sets. About 600 lines of format detection,
legacy rewriting and tree walking go.

- Imports read DTCG 2025.10 files: {value, unit} dimensions, color
  objects in their color space (oklch(l, c, h)), the fontFamily and
  fontWeight type names, $root, $extends and $ref aliases. Legacy and
  DTCG nodes can be mixed in one file.
- What penpot does not store is dropped and listed in one warning
  toast: $extensions, $deprecated, group descriptions, values the
  reader cannot use (such as a Rec. 2020 color without hex), groups
  whose $extends fails and tokens of types penpot does not have.
- Sets a theme marks disabled are no longer part of the theme, and
  activating a theme deactivates the other themes of its group.
- Exports are unchanged by default. export-dtcg-json and
  export-dtcg-multi-file take {:profile :dtcg} for 2025.10 values and
  types. Such a file reads back with the DTCG types: spacing and
  sizing tokens come back as dimensions.
- The package moves from frontend/deps.edn to common/deps.edn, since
  the backend and the SDK read and write token files too.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 13:04:45 +01:00
Pablo Alba
e0832335c7
🐛 Keep text mounted until its measurement settles (#13097)
The text measurement in update-text-shape used ->> with p/catch. But
p/catch takes the promise first, so the handler was read as the promise.
The call then returned a promise that resolved to the handler function at
once. Measurement errors were never logged, and a failed measurement was
left as an unhandled rejection.

The caller ran p/finally right away, so the text left the pending set and
its :text-measure reflow task closed before it was measured. The text
could then unmount: the measurement read a detached node, got a zero rect
and skipped resize-text. waitForLayoutUpdate could also resolve before
the text was measured. This bug came in with #9898.

Use p/merr instead, which takes the function first. It logs the error and
resolves to nil, so the promise settles only when the measurement does.

Tests check that the promise stays pending until the measurement settles,
that the measurement then completes, and that a failed measurement
resolves to nil.

AI-assisted-by: claude-opus-5-5
2026-11-17 12:48:19 +01:00
Juan de la Cruz
81c2408470
✨ Add scale options panel to the right sidebar (#12453)
* ✨ Extend combobox icons and input handling

Allow a combobox to show a fallback icon when no option provides one.
Clear delivered input values so unmounting does not emit them again.

AI-assisted-by: gpt-6-luna

* ✨ Add scale options panel to the right sidebar

Add scale controls for selection dimensions, presets, and anchor points.
Scale shape content around the selected anchor and preview canvas resizes.
Reset scale state when selection changes, and exit scale mode with Escape.
Cover scaling and resize previews with frontend tests.

AI-assisted-by: gpt-6-luna

---------

Co-authored-by: Luis de Dios <luis.dedios@kaleidos.net>
2026-11-17 12:33:51 +01:00
Florian Schrödl
8b8def3b6e
🐛 Fix clicking a set in the theme editor (#13067)
Clicking a set's row in the theme editor threw an assertion error and
left the set as it was: the row passed the set's name to the toggle,
which looks the set up by id. The row now uses the checkbox's handler,
so the two can't drift apart again.

The editor's set and group toggles move into pure functions next to
it, with tests. A new spec clicks a set's name to tick and untick it.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 12:11:05 +01:00
Rayen Khammar
fa1249c1a5 🐛 Reap zombie Chromium processes in the exporter container (#12969)
* 🐛 Reap zombie Chromium processes in the exporter container

The exporter runs node as PID 1 with no init process. Chromium helper
processes orphaned when Playwright closes a browser are re-parented to
node, which never reaps them, so every browser-based export left
defunct chrome-headless processes behind.

Enable init for the exporter service so Docker's built-in init runs as
PID 1, reaps the orphans and forwards signals to node.

Closes #10339

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

* 🐛 Move zombie reaping fix into the exporter image

Install tini and run it as the image entrypoint with -s, so orphaned
Chromium helpers are reaped in every deployment, including Kubernetes.
-s keeps tini working without a warning when it runs under docker-init
(init: true). Drop the compose init: true change to keep one mechanism.

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

---------

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>
2026-11-17 11:46:19 +01:00
Rayen Khammar
445d02e012
🐛 Reap zombie Chromium processes in the exporter container (#12969)
* 🐛 Reap zombie Chromium processes in the exporter container

The exporter runs node as PID 1 with no init process. Chromium helper
processes orphaned when Playwright closes a browser are re-parented to
node, which never reaps them, so every browser-based export left
defunct chrome-headless processes behind.

Enable init for the exporter service so Docker's built-in init runs as
PID 1, reaps the orphans and forwards signals to node.

Closes #10339

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

* 🐛 Move zombie reaping fix into the exporter image

Install tini and run it as the image entrypoint with -s, so orphaned
Chromium helpers are reaped in every deployment, including Kubernetes.
-s keeps tini working without a warning when it runs under docker-init
(init: true). Drop the compose init: true change to keep one mechanism.

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

---------

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>
2026-11-17 11:41:43 +01:00
Andrey Antukh
38a9878981
🔥 Remove the unused SQLite-based binfile v2 format (#12947)
The SQLite-based v2 binfile package was never released, and its only
caller was the developer REPL helper `app.srepl.binfile`, which exported
and imported a whole team. With the format gone, the helper, the
functions of `app.binfile.common` without a caller (`get-fonts`,
`get-file-thumbnail`, `get-team`, `get-teams`, `collect-summary`) and
the `sqlite-jdbc` dependency go with it.

The public surface does not change: the import RPC accepts only versions
1 and 3, and the test that pins the rejection of version 2 stays as it
is.

AI-assisted-by: deepseek-v4.1-flash
2026-11-17 10:59:10 +01:00
Andrey Antukh
78cfe7fa58 Merge remote-tracking branch 'origin/staging' into develop 2026-11-17 10:53:02 +01:00
Marina López
e422de62d8
🐛 Fix create team inside organization (#13106) 2026-11-17 10:44:54 +01:00
Andrey Antukh
94155508e5 Merge branch 'staging-2.19.x' into staging 2026-11-17 10:16:51 +01:00
Andrey Antukh
1d60ba65b5 Merge remote-tracking branch 'origin/main' into staging-2.19.x 2026-11-17 10:02:29 +01:00
Pablo Alba
ea0ac02900 🐛 Fix duplicate-file crash on nil keys in deleted component objects (#13045)
* 🐛 Fix duplicate-file crash on nil keys in deleted component objects

* ♻️ Withdraw the legacy-36 migration in favor of 0036

legacy-36 and 0036-clear-nil-keys-in-objects perform the same
cleanup of nil keys in `:objects`. Migrations only clean up
invalid data here, so they carry no order dependency between
them and running both is wasteful.

Remove the legacy-36 defmethod and its id from
`available-migrations`, the same way 0021-fix-shape-svg-attrs
was withdrawn in favor of 0035. Files that recorded legacy-36
re-migrate through 0036; their persisted `:migrations` lists
keep the old id, and the playwright fixtures recording it stay
untouched.

Add a migrate-file test proving a file that recorded legacy-36
is re-armed for 0036 and passes schema validation after.
Document the withdrawal pattern in the migration subtleties
memory.

AI-assisted-by: inkling-2-small-free

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-17 09:45:48 +01:00
Roshan Ramani
32d488e162
🐛 Fix swapped Move left and Move right shortcuts (#13068)
The `move-unit-left` shortcut was bound to the Right arrow and moved the
selection right, and `move-unit-right` the reverse, in both the workspace
and the path editor. Arrow keys behaved correctly, but the shortcuts panel
and the shortcut settings showed "Move left" next to the Right arrow, and
customizing "Move left" rebound the rightward move.

AI-assisted-by: claude-opus-5-5

Signed-off-by: Roshan Ramani <roshanramani.dev@gmail.com>
2026-11-17 09:45:02 +01:00
Pablo Alba
aa35bc78c9
🐛 Fix duplicate-file crash on nil keys in deleted component objects (#13045)
* 🐛 Fix duplicate-file crash on nil keys in deleted component objects

* ♻️ Withdraw the legacy-36 migration in favor of 0036

legacy-36 and 0036-clear-nil-keys-in-objects perform the same
cleanup of nil keys in `:objects`. Migrations only clean up
invalid data here, so they carry no order dependency between
them and running both is wasteful.

Remove the legacy-36 defmethod and its id from
`available-migrations`, the same way 0021-fix-shape-svg-attrs
was withdrawn in favor of 0035. Files that recorded legacy-36
re-migrate through 0036; their persisted `:migrations` lists
keep the old id, and the playwright fixtures recording it stay
untouched.

Add a migrate-file test proving a file that recorded legacy-36
is re-armed for 0036 and passes schema validation after.
Document the withdrawal pattern in the migration subtleties
memory.

AI-assisted-by: inkling-2-small-free

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-17 09:36:40 +01:00
Florian Schrödl
0ae02daae9
♻️ Extract and test how tokens are applied to shapes (#13034)
toggle-token and apply-token-from-input each worked out inside the
event which attributes a token goes on, and repeated most of it. The
decision now lives in pure helpers both events share: `toggle-action`
(remove the token or apply it), `apply-action` (on which attributes)
and `target-shapes` (a selected group stands for its children).
`action->event` turns the result into the event to run. Nothing
changes in behavior; apply-spacing-token-separated loses its `attr`
option, which every caller left empty.

Unit tests cover the decision, with the cases the store tests
lacked: a color token comes off the stroke too, a shape without the
token completes the others, and a spacing token on chosen attributes.
Store tests cover what had none: applying from the fill and stroke
rows, the opacity, position and size fields, the min width of a flex
child and the selection colors, choosing the token a shape already
has, and toggling a token on a group.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 09:31:47 +01:00
Andrey Antukh
114ce6ec03
✨ Make legacy-db file data read-only (#13102)
legacy-db is no longer a write backend: fdata/upsert! rejects it
(:legacy-backend-read-only), handle-persistence stops writing
file.data and file_change.data, and the update-params schema of
fdata no longer accepts it. Reading old payloads keeps working, and
the file-data-backend default stays db.

The configuration guide, the backend config (a startup warning reads
PENPOT_FILE_DATA_BACKEND=legacy-db as db instead of failing every
save) and the storage memories say so, with the upgrade note for
deployments that still set the variable.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: inkling-2-small-free
2026-11-17 09:24:23 +01:00
Elena Torró
1068421d0b
🐛 Keep the drop index when moving a child to the end of a flex layout (#13098)
Closes #12929

AI-assisted-by: claude-opus-5-5
2026-11-17 08:48:20 +01:00
Elena Torró
a050a21fae
🐛 Omit board fill on WASM export when Show in exports is off (#13073)
Send the flag to render-wasm and drop the root's fills while it renders.

Closes #12930

AI-assisted-by: claude-opus-5.5
2026-11-17 07:58:10 +01:00
penpot-renovate[bot]
23468f9e4f
⬆️ Update dhi.io/nginx image to ad079e1 (#13096)
- dhi.io/nginx 1.31.8-alpine3.24@ad079e1 (docker/images/Dockerfile.storybook)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 18:39:36 +01:00
Andrey Antukh
57b9b8463b
🐛 Keep app booting when SES bootstrap fails (#13085)
* 🐛 Keep app booting when SES bootstrap fails

Wrap repairIntrinsics in a guarded bootstrap so a SES Temporal check failure
no longer aborts bundle evaluation and blanks the whole app. When it fails,
plugins and MCP stay unavailable for that session, a telemetry event reports
the error, and trying to open a plugin shows an environment warning.

Add vitest coverage for both bootstrap paths and a CLJS regression test for
the runtime-unavailable toast.

Closes #12924

AI-assisted-by: hy4-flash-free

* 🐛 Fix CI format check and plugin test entry point

Wrap the long Prettier line in bootstrap.spec.ts so `pnpm run format:check`
passes again in the plugins module, and document that the duplicated SES
options are an intentional contract snapshot.

Switch the runtime-unavailable CLJS test to the public open-plugin! entry
point instead of the private load-plugin! function. The M2 test for
signal-runtime-ready was dropped: loading app.plugins in the Node test
runner needs packaging and DOM shim changes outside this fix.

Closes #12924

AI-assisted-by: hy4-flash-free
2026-11-16 17:52:55 +01:00
Andrey Antukh
d97972c81d
🐛 Build the SSO redirect target on the server (#12952)
* 🐛 Build the SSO redirect target on the server

The organization SSO flow took the destination URL from the client and
redirected to it after the identity provider round-trip, so any
authenticated profile could point that redirect at another domain: the
server never checked where the URL came from.

The RPC now takes the route token of the client (?screen=...&params)
and the server builds the destination out of it: the public URI plus
the query string, parsed and rendered back. The destination can no
longer leave the public URI, and the callback checks it again before
redirecting, which covers the state tokens minted before the change.

The state token also carries the profile that started the flow, so
only that session is granted the organization SSO entry: a state
prepared by someone else cannot mark the session of the one that
completes it. Whenever the RPC receives an organization id (it always
comes from the client), it also checks that the profile is a member of
the organization or its owner, instead of letting any profile start a
flow for an organization it does not belong to.

Tests cover the destination builder, a callback that ignores a
destination outside the public URI, a state that does not grant the SSO
entry to another session, and the organization access check.

AI-assisted-by: minimax-m3.1-flash

* 🐛 Check the SSO redirect target against the public URI

The callback compared the destination with a string prefix, so the two
bypasses the advisory documents (a userinfo prefix and a look-alike
host) were closed only by the trailing slash of the public URI. The
check now compares the parsed origin (scheme, host case-insensitive,
port with the default made explicit) and requires the path to be under
the base path of the public URI, so neither bypass passes.

The client sent `location.search`, the history token, whose leading `?`
is URL syntax and no part of a query string. It sends the query string
proper now (`screen=…&params`) through `rt/get-current-query-string`,
and `rt/token->query-string` is the single bridge between the two
shapes: every other consumer concatenates the token over the base path.
The server puts the separator back when it joins the destination, and
trims one that still arrives, so a browser holding the previous bundle
keeps its destination while the new one rolls out.

Building the destination can no longer break the flow: it catches its
own errors and lands on the public URI, which matters in the
deployments where the browser origin differs from the configured one.
A value that is not a route is ignored with a warning, decided by the
keys it parses into (a URL becomes a single key holding the whole URL),
and a route with no `screen`, like the root params of the application
(`template=…`), stays a usable destination.

The organization membership lookup is now shared by the check that
gates the SSO flow and the one that raises, so the two cannot drift
apart, and the docstring of the RPC lists `:no-organization-access`
next to `:no-team-access`.

`rt/nav-raw` returns its `:href`/`:uri` through `@event`, as its
sibling `rt/navigate` already did. Nothing in the app listens to
`::rt/nav-raw`, so the callers behave the same; the SSO test now reads
the destination from the event instead of intercepting the helper.

Tests cover the two bypasses and the base-path rule, the RPC with an
unusable destination, the value that is not a route, the request that
carries both ids, an organization that does not exist, the query string
the frontend sends, and the two router helpers.

AI-assisted-by: minimax-m3.1-flash

* ♻️ Deduplicate SSO destination URL building

The backend had an unreachable blank check and a one-line wrapper around
the URI helper in build-dest-url. On the client, the SSO retry and error
flows each assembled the destination from public-uri and the route params.

Flatten build-dest-url with a cond and call u/query-string->map directly.
Add rt/token->url as the single client helper for absolute route URLs, with
tests for both token shapes and for a blank token. Update the routing memory
with the new helper contract.

AI-assisted-by: hy4-flash-free
2026-11-16 17:34:49 +01:00
Florian Schrödl
699f79c693
♻️ Extract the token group's pill click and open sections (#12795)
Move what the token group works out into pure functions with tests:
whether a type's section is open, whether a text is being edited, and
what clicking a token pill does. Nothing changes in behavior.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-16 16:14:57 +01:00
Eva Marco
82421f9729
🐛 Show resolved value when opening an alias token edit form (#13038)
* 🐛 Show resolved color in alias token edit form swatch

The color token form opened every alias token with an empty swatch.
The resolved value only reached :color-result once the value field
was touched, so the swatch had nothing to show on open.

Keep the resolved color for the swatch when the field is untouched,
without showing the "resolved value" hint. Add an e2e test that loads
a fixture with a color token and an alias to it.

Closes #12672

AI-assisted-by: claude-opus-5-5

* 🐛 Show resolved value when opening an alias token edit form

The token edit form only showed "Resolved value" after typing in the
value field: the single-value inputs dropped the first resolution while
the field was untouched, and the typography/shadow reference field read
its first value from [:value :reference], which an alias token does not
have (its value is the reference string).

Show the hint on open when the resolved value differs from the written
one, so a plain `16` gets no `Resolved value: 16`. Start the reference
field stream with the alias string. Extend the e2e fixture with font
size and typography aliases and add a test for them.

Closes #12788

AI-assisted-by: claude-opus-5-5
2026-11-16 16:00:18 +01:00
Andrey Antukh
d2dfc09495
🐛 Carry the team id in every team scoped navigation (#12963)
* 🐛 Carry the team id in every team scoped navigation

The eleven `go-to-*` helpers of `app.main.data.common` resolved the
team with a plain `or` against `:current-team-id`, or read that key
alone, so an event emitted while no team was initialized built a
dashboard or workspace route without a `team-id`. `team-container*`
raises `:not-found :invalid-team-id` for such a route and
`exception-section*` does not report `:not-found`, so the user landed
on the not-found page and PostHog never saw it.

Add `resolve-team-id`, which falls back to the team of the route
already on screen and then to the profile `:default-team-id`. The
route goes first on purpose: an emitter that runs from a team screen
knows which team the user is looking at, and the personal team would
open their file under another team. The `:default` sentinel keeps its
meaning. With none of the four the helper still navigates with a nil
id, which is the documented last resort.

Harden the team-container test at the same time: it now renders the
same component with the store team the caller wants, captures markup
and errors per render, unmounts the root from a `finally`, and covers
the guard that compares the route id with the store team. React
reports a hook-order violation through `console.error` and recovers,
and only when a render adds hooks, so the harness captures that too
and the "id leaves" direction stays a guard test.

Closes #12962

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Measure the navigation tests by termination

The `emitted` helper of `common_test` waited for the first event, with a
budget of 50 macrotasks, so a helper that emitted anything before its
navigation could be disposed early, and a helper that stopped closing
its stream passed on the first event. Wait for termination instead:
`async/observe` over a subject, and make `go-to-dashboard-search`
complete by pushing the `::rt/navigated` it listens for, which is the
event it waits for in the app to focus the search box. Every helper of
the family is measured the same way now, `-search` included.

`nav-params` returns nil when no navigation matched instead of calling
`deref` on nil, as its docstring promised. Add the mirror test for the
`:default` sentinel of `go-to-dashboard-deleted`, the second copy of
the branch that `-recent` already had covered.

Correct two memories: the resolver decides for every team scoped
helper, not for every `go-to-*` and not alone, since two direct
`rt/nav` emitters are the residual; and `(rx/empty)` is for a stream
that must not emit, not for one that must stay open.

Relates to #12962

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 15:42:43 +01:00
Andrey Antukh
6e4e69c339
🐛 Read the media and thumbnails of a package one at a time (#13024)
The analysis of a package reads the metadata of every media and
thumbnail of its container to attribute the shared objects to each
file. It read them all at once, one promise per entry, and the
container of a big file holds thousands of them: every read opens its
own decompression stream, and Chromium fails the whole analysis under
that load while Firefox keeps up. The failure is masked by zip.js with
a generic "Cannot close a ERRORED writable stream".

The references now go through an observable, one entry at a time, and a
reference that cannot be read leaves its file without a size instead of
failing the package: a number that misses one of the objects of the
file is worse than no number.

Closes #13022

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 15:42:15 +01:00
Andrey Antukh
0fda857426
🐛 Invalidate all sessions on password recovery (#12967)
* 🐛 Invalidate all sessions on password recovery

recover-profile validated the recovery token and updated the
password, but never deleted the existing server-side sessions in
http_session_v2. A previously captured auth-token kept working
after the victim "recovered" the account (GHSA-3j2p-qr45-q365,
CWE-613).

The command now calls session/invalidate-all for the recovered
profile. The command already runs with ::db/transaction true, so
the delete joins the same transaction as the password UPDATE: both
commit or roll back together. There is no current session to
preserve — the RPC is unauthenticated — which is the same contract
the profile deletion path uses.

Tests in rpc_auth_test.clj pin the behavior: the victim's sessions
die, other profiles' sessions survive, and a failed recovery is a
no-op. Both invalidation tests failed before the fix.

AI-assisted-by: inkling-2-small-free

* 🐛 Pin recovery-transaction atomicity after review

Review of 92a65f544b found the tests pinned the end state but not
the rollback guarantee the fix relies on: the session DELETE joins
the password UPDATE in one transaction.

Add recover-profile-rollback-keeps-password-and-sessions: send-template
throws after the update, so the command must fail with password and
sessions both untouched. recover-profile-failure-keeps-sessions now
also asserts the password, matching its docstring. The invalidate-all
docstring mentions the recovery caller.

Addresses review findings M1, M2 and M3.

AI-assisted-by: inkling-2-small-free

* 🔥 Drop recovery test that never reached the fix

The failure test used a token with the :authentication issuer, so
tokens/verify rejected it before any write. It passed with or without
the invalidation fix and exercised no new code. The rollback test
already covers that a failed recovery leaves no partial effects.

AI-assisted-by: hy4-flash-free
2026-11-16 15:40:36 +01:00
penpot-renovate[bot]
99db795966
⬆️ update GitHub Actions (#13071)
- node 26.13.1 (.github/workflows/plugins-deploy-styles-doc.yml)
- node 26.13.1 (.github/workflows/plugins-deploy-api-doc.yml)
- pnpm/setup v3.2.0 (.github/workflows/plugins-deploy-styles-doc.yml)
- pnpm/setup v3.2.0 (.github/workflows/plugins-deploy-api-doc.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 15:38:28 +01:00
María Valderrama
c0f746ee34
🐛 Apply the invite policy to pending invitations (#13080)
* 🐛 Apply the invite policy to pending invitations

* 📎 Code review
2026-11-16 15:38:07 +01:00
Marina López
ba0eab57ab
🐛 Hide team plan from every member when owner has Enterprise (#13079) 2026-11-16 14:55:54 +01:00
Luke Lau
5d88cb3bb7
🐛 Don't copy margins past tile in draw_current_tile_into_backbuffer (#13066)
When dragging a shape each tile that needs repainted is repainted via
draw_current_tile_into_backbuffer in fast mode. So if you drag over a tile it
ends up dropping drop shadows etc.

IIUC, each tile is 512x512px but drawn on a 1024px surface with 256px margins
so that the background blur has something to read. In a regular render we
render the 1024px surface but only copy the center 512px of the tile into the
backbuffer. But in draw_current_tile_into_backbuffer we were copying the whole
1024px surface. This draws over the other tiles with a fast mode render, even
though those other tiles don't need repainted.

This fixes it by reusing draw_surface_src_rect_to_dst which clips the canvas
properly so we only draw the 512x512px of the tile.

AI-assisted-by: claude-fable-5-2
2026-11-16 14:47:27 +01:00
Luke Lau
a224608a0e
⚡ Memoize boards with grids in frame-grid* (#13075)
On a test file `mem_benchmark` with 30k boards on one page, around 11ms is
spent each time the view is panned/zoomed iterating over each board and
checking has-grid?, which after https://github.com/penpot/penpot/pull/12790 is
now slightly slower because it involves an array scan.

We don't need to compute the list of boards with grids on every step of the pan
and can memoize it, which saves 30k calls to has-grid? on each frame.

AI-assisted-by: claude-opus-5-5
2026-11-16 14:32:10 +01:00
Álvaro Tejero-Cantero
bd4af28d78
📚 Name a worker count when running the Playwright suite (#12799)
* 📚 Name a worker count in the frontend testing memory

The memory gave the command and left the count to the config, whose
default of one worker makes the whole suite slow, and CI's two per
shard reads as the number to copy.

Records the count to reach for and what it costs: about 8 GB of
browser memory at four workers, so a 64 GB host takes four beside a
watching devenv, and a 32 GB host takes two, or four on a slim devenv.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Gate the Playwright worker count on cores and memory

Four workers keep about nine hardware threads busy, measured as the
container's CPU seconds over wall time, and hold about 8 GB of browser
processes. States the cost per worker, so a reader on a smaller machine
does not oversubscribe it.

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 13:55:05 +01:00
Luke Lau
bbaebfb146
🐛 Send the team's custom fonts to the worker (#13044)
The worker draws local exports and looks custom fonts up in its own store
to find their files. Only the main thread fetches the team's fonts, so the
worker's store had none, and exports drew text set in a custom font with
the fallback font.

The main thread now sends the worker the team's fonts each time they
change, and the worker keeps them in its store.

AI-assisted-by: claude-opus-5-5
2026-11-16 13:53:43 +01:00
Luke Lau
1ccb9e9cb9
⚡ Move WASM component thumbnail rendering to the worker (#13046)
This PR moves component thumbnail rendering off the main thread
and onto the worker, which avoids the freezes.

The main thread will send the worker over a request containing the component's
shape data, and the worker will render it in batches. Once finished it will
both persist the thumbnails to the server if requested and send back the PNG
data to the main thread.

AI-assisted-by: claude-opus-5-5
2026-11-16 13:48:04 +01:00
Belén Albeza
b0dbbe8285
🐛 Fix blurry file thumbnails rendered with render-wasm (#13026)
The wasm path drew file thumbnails on a 252px wide canvas, wrapped the
PNG in an SVG and sent it to the rasterizer, which scaled it up to its
default 300px width. The result was blurry on the dashboard on high-DPI
screens and in the link preview card.

Render the wasm thumbnail at 600px wide, twice the old size, and upload
the canvas PNG as is, without the rasterizer round trip. The SVG render
path keeps its current size.

Closes #13025

AI-assisted-by: claude-opus-5-5
2026-11-16 13:32:38 +01:00
penpot-renovate[bot]
99497d3b7c
⬆️ Update dependency pnpm to v12.21.0 (#13063)
- pnpm 12.21.0 (docker/images/Dockerfile.media-processor)
- pnpm 12.21.0 (docker/images/Dockerfile.mcp)
- pnpm 12.21.0 (docker/images/Dockerfile.exporter)
- pnpm 12.21.0 (docker/devenv/Dockerfile)
- pnpm 12.21.0 (render-wasm/package.json)
- pnpm 12.21.0 (plugins/package.json)
- pnpm 12.21.0 (plugins/libs/plugins-styles/package.json)
- pnpm 12.21.0 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.21.0 (plugins/libs/plugin-types/package.json)
- pnpm 12.21.0 (plugins/apps/table-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.21.0 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/icons-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/example-styles/package.json)
- pnpm 12.21.0 (plugins/apps/e2e/package.json)
- pnpm 12.21.0 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.21.0 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.21.0 (package.json)
- pnpm 12.21.0 (media-processor/package.json)
- pnpm 12.21.0 (mcp/packages/server/package.json)
- pnpm 12.21.0 (mcp/packages/plugin/package.json)
- pnpm 12.21.0 (mcp/packages/common/package.json)
- pnpm 12.21.0 (mcp/packages/browser-extension/package.json)
- pnpm 12.21.0 (mcp/package.json)
- pnpm 12.21.0 (library/package.json)
- pnpm 12.21.0 (frontend/text-editor/package.json)
- pnpm 12.21.0 (frontend/packages/ui/package.json)
- pnpm 12.21.0 (frontend/packages/mousetrap/package.json)
- pnpm 12.21.0 (frontend/packages/draft-js/package.json)
- pnpm 12.21.0 (frontend/package.json)
- pnpm 12.21.0 (exporter/package.json)
- pnpm 12.21.0 (docs/package.json)
- pnpm 12.21.0 (common/package.json)
- pnpm 12.21.0 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 13:15:48 +01:00
penpot-renovate[bot]
4d91d8f098
⬆️ Update DHI base images (#13062)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@e3e8512 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@2168f2f (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 13:15:17 +01:00
Belén Albeza
27784f70ac
🐛 Fix custom CJK fonts losing glyphs when converted to TTF (#13074)
FontForge opens a CID-keyed OpenType font as several sub-fonts, and
Generate() writes only the active one. Most CJK fonts use this layout,
so the TTF used by the render-wasm renderer kept only a few hundred
glyphs (the Latin sub-font) and lost its family name. Merge the
sub-fonts with CIDFlatten() before generating, both in the backend and
in the media-processor. The call is guarded because it fails on fonts
that are not CID-keyed.

Add a small synthetic CID-keyed font with two sub-fonts as a fixture
and check that the converted TTF keeps all its glyphs.

Closes #10976

AI-assisted-by: claude-opus-5-5
2026-11-16 13:14:40 +01:00
Andrey Antukh
909a29cf50 🔥 Remove unused file 2026-11-16 12:41:05 +01:00
Andrey Antukh
e95007a93b 🐛 Retry the publish of the websocket delivery assertions
The subscription to the bus reaches redis through an asynchronous agent
action, so a message published right after the connection opens can be
lost before redis knows the subscription. The delivery assertions of the
websocket tests published once and bet on winning that race, which fails
on a loaded CI runner.

`poll-published!` publishes, drains a window and publishes again only
when the window saw nothing, up to three attempts. The assertion still
proves that the client receives a message it published, and the negative
assertions keep the single drained window they need. A test pins the
retry so it does not go back to a single publish.

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 11:59:00 +01:00
Andrey Antukh
239291539b 📚 Add the 2.21.0 changelog section
Add the 2.21.0 changelog section, generated from the milestone
issues and the advisory source of truth.

The 🔒 entry references GHSA-m53j-2766-6jqw with the fixing PR
#12189. The advisory now declares patched version 2.21.0 only: the
fix reached develop after 2.20.0-RC1, so the 🔒 section of
2.20.0, which claimed that release fixes the vulnerability, is
removed as well.

Entries cover the closed milestone issues after the dev-only and
exclusion filters; #9263 stays in 2.21.0 by maintainer decision and
is flagged in the anomaly report as milestone-mismatched with PR
#9958.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:58:05 +00:00
Andrey Antukh
18a827d65a Merge remote-tracking branch 'origin/staging' into develop 2026-11-16 11:38:09 +01:00
Andrey Antukh
65b882d044 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-16 11:37:52 +01:00
Andrey Antukh
b39276c148 📚 Update changelog for 2.19.2
Add the 2.19.2 section with the six bug fixes of the milestone, in
merge order, following the old 2.19.x layout: 🐛 Bugs fixed only,
no 💥 (no breaking changes, consistent with patch releases), no
🚀 (patches carry none), no 🔒 (no advisory declares
2.19.2). #12622 carries community attribution. The section stays
marked (Unreleased) until the release commit.

AI-assisted-by: inkling-2-small-free
2026-11-16 11:25:16 +01:00
Andrey Antukh
b8136da06a 🐛 Fix dev-only-bugs counting released same-minor versions as this milestone
`only_unreleased_envs` compared versions as major.minor, so for the
2.19.2 milestone the released 2.19.0 counted as "this milestone" and
bugs naming it as an affected version were listed as dev-only
candidates (e.g. #13019, "only seen on 2.19.0 and later").

Compare full semver instead, with a missing patch read as 0: only the
milestone version itself counts as an unreleased signal, and any other
version is a production hint that keeps the bug in the changelog, as
the documented contract already said.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:14:55 +00:00
Andrey Antukh
86eb0bf805 Merge remote-tracking branch 'origin/staging' into develop 2026-11-16 11:07:05 +01:00
Andrey Antukh
4a9f553c19 📚 Update the 2.20.0 changelog
Regenerate the 2.20.0 section for the new layout: `💥` and
`🔒` are generated from the breaking change notes and the
advisory (patched_versions includes 2.20.0); features and
enhancements move to their own `🎉` and `✨` sections
instead of one mixed section; `🚀` lists the two milestone
issues labelled `release highlight` (#11759 and #11761, no closing
PR in the milestone).

Add the issues closed since the section was written: #10436,
#12052, #12135, #12615, #12728, #12738, #12740, #12948 and #13028
as bugs, and #10465, #12265, #12796, #12902, #13029 and #13030 as
enhancements. Bugs that never reached production stay out per
`dev-only-bugs` (#12860, #12138) and manual review (#12901).

#12977 leaves the section: its fix (PR #12978, merged 2026-11-13
in develop) carries milestone 2.21.0, so the issue moves there and
will be picked up by the 2.21.0 changelog.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:03:54 +00:00
Andrey Antukh
328e41e037 ♻️ Stop excluding release blocker from the changelog flow
The `release blocker` label marks work that blocks the release; it is
not a signal about whether users will notice a change. Drop it from
the changelog exclusions and keep only `no changelog`.

Closed bugs carrying `release blocker` are analyzed by `dev-only-bugs`
like any other bug: a bug found while testing an unreleased feature
never reached production, so the tool is the right filter for them.
Closed non-bug issues with the label are surfaced by a manual review
step in the flow: `gh.py issues --label "release blocker"` lists them
and the user decides between `no changelog` and keeping the entry.

Update the memories, the update-changelog skill and the tests to match.

AI-assisted-by: inkling-2-small-free
2026-11-16 09:17:27 +00:00
Andrey Antukh
7ef8c1cf36 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-16 10:05:00 +01:00
Andrey Antukh
635f7fc914
✨ Disable the admin-console by default in the devenv frontend (#13061)
* ✨ Disable the admin-console by default in the devenv frontend

The devenv does not run the admin-console service, but the frontend
showed its UI by default. The devenv containers now build the
frontend with PENPOT_DEVENV=true, which prepends disable-admin-console
ahead of the developer's config.js flags. Flags parse in order, so
enable-admin-console in config.js re-enables it; builds outside
devenv are unchanged.

Closes #13060

AI-assisted-by: inkling-2-small-free

* 📚 Require English in all code-facing text

Add a hard rule to AGENTS.md: everything in code must be in
English — identifiers, comments, docstrings, test descriptions,
log hints — with tests not excepted. The only allowed place for
another language is product copy rendered through the i18n
system. Also restated in mem:clojure/design-rules and mem:testing.

AI-assisted-by: inkling-2-small-free
2026-11-16 09:54:23 +01:00
Pablo Alba
77c87d16f1
🐛 Keep variant property order past eight properties (#13014) 2026-11-16 09:30:29 +01:00
Eva Marco
fa6af23d75
🐛 Disconnect a library that fails to load instead of breaking the file (#13048)
Opening a file connected to a broken shared library showed the
"Internal Error" page: every connected library was loaded in a single
stream, so one failing get-file broke the whole workspace.

Catch the error per library. When the backend fails to load it (a
server error, never a transport or transient error), a directly
connected library is disconnected and a toast tells the user; an
indirect one is only dropped from the state.

Closes #12429

AI-assisted-by: claude-opus-5-5
2026-11-16 09:02:31 +01:00
Filip Sajdak
557774e4f4 🐛 Call the team hook on every render of the team container (#12920)
team-container* called mf/deref only after returning early for a team id
that is not a uuid. When the same component rendered once without a team
id and then with one, the number of hooks changed between the two renders
and React threw "Rendered more hooks than during the previous render"
(error #310), which sent the user to the internal error page. The reverse
order throws "Rendered fewer hooks than expected".

Call the hook on every render and keep the uuid check in the condition
that builds the providers.

Two tests mount the component in jsdom and render it with a team id that
arrives and with one that leaves; both fail without this change.

Closes #12728

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-16 08:41:22 +01:00
Usama Sadiq
861bb5bb37 🐛 Fix database pool min-size falling back to max-size (#13052)
* 🐛 Fix database pool min-size falling back to max-size

Since 2.16 the pool minimum defaults to the maximum (60), so an idle
backend opens and holds 60 connections at startup. Restore the pre-2.16
default of 0 so the pool grows with load and idle connections expire.

Closes #12622

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

* 📚 Document database connection pool settings

Describe PENPOT_DATABASE_MIN_POOL_SIZE and PENPOT_DATABASE_MAX_POOL_SIZE,
their defaults, and sizing on a PostgreSQL shared with other apps.

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

---------

Signed-off-by: Usama Sadiq <usama7274@gmail.com>
2026-11-16 08:37:49 +01:00
Andrey Antukh
e23ca02aa2
✨ Reference the file data storage object with a foreign key (#12769)
* ✨ Reference the file data storage object with a foreign key

The file data pointer lived in `file_data.metadata.storage-ref-id`, a JSON
key. The storage GC parsed it to find live references, and nothing stopped it
from physically deleting an object a live row still pointed at: a failure in
that scan lost the blob.

Migration 0159 moves the pointer to a nullable `storage_object_id` column on
`file_data`, backfills it from the metadata key, removes the key from the
rows it backfills, indexes the column and rebuilds the foreign key to
`storage_object(id)` as `NO ACTION DEFERRABLE INITIALLY IMMEDIATE`.
`INITIALLY IMMEDIATE` is deliberate: the GC catches the violation inside the
chunk transaction, while a deferred check would fire at commit, outside it,
and the sweep would never move past the blocked chunk.

The column is the only source of truth. The `storage-ref-id` entry is gone
from the file metadata schema, and the single point that serializes the
metadata drops it, so no write can resurrect it. `upsert!` does not query
the row: the pointer of the row being replaced travels in its params, and
the object is touched before the row is written. Fragments and snapshots
pass nil because a modified pointer always gets a new id; the offload passes
its own row; `insert-file!` drops it, since an INSERT never replaces a
persisted object.

`update-file!` takes the file map read from the database plus the fields to
change and merges them, so the pointer comes from the read map and no caller
can lose it by building a partial map. A map without `:data` raises
`:missing-file-data`: it is not `assert`, which is compiled out in the
deployed binary. The params cannot override the pointer or retarget the
write with `:id`/`:file-id`.

`bfc/file->response` strips the pointer from every response (`get-file`,
`create-file`, `duplicate-file`, the view-only bundle, the team access
request, the comments read, the library summary, the snapshot read and the
debug download). A `file-data` object is authentication-only, so a leaked id
would let any logged-in profile fetch the blob from `/assets/by-id`.
`get-file-fragment` resolves through the file data backend, so a fragment on
cold storage no longer hands nil to `blob/decode`, and answers only the keys
of its result schema.

The storage GC reference scan, the restore union and `objects-gc` read the
column. The offload reads `SELECT fd.*`, so its rows go through a
`decode-row` that decodes jsonb metadata and drops a JSON null, which the row
schema rejected; a row without a readable object raises
`:missing-storage-object` with the file and row ids. A chunk blocked by the
foreign key (SQLState 23503) is a real anomaly, since the reference scan
should have frozen the object: it stays an error with its own hint and the
chunk is retried on the next run until the reference is fixed.

The release is a coordinated restart, not a rolling deploy, and not
rollback-safe for the rows it writes: the previous code writes and scans the
metadata pointer, so a key it writes misses the backfill and removing it
loses the reference. `operations/2.20.0-file-data-storage-object-id.md`
builds the column, the index and the foreign key online and runs the
backfill with every instance stopped, then marks the step as applied; a
fresh or small instance does the whole job in the migration. The runbook
keeps the validation scan out of the stopped window and adds a lock timeout
and retry to the online statements.

Tests cover the foreign key shape, the backfill and its corrupt and orphan
guards, the rebuild of a missing index and of a wrong, non-deferrable or
deferred constraint, the deletion refused by a live reference, the metadata
cleanup, the offload row normalizations, the touched objects of the unshare
and the restore, the response stripping on every call site and the chunk
blocked by the foreign key. The backend memories and the developer guide
document the pointer, the GC behavior and the two migration paths.

Closes #12768

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: inkling-2-small-free

* 🐛 Hold the file row lock across the migration write

`get-file` applies the pending migrations in memory and, when it finds
any, persists the result. That write read the row without a lock and
`update-file!` is an unconditional UPDATE, so a save that committed in
between was overwritten: the saved content, `revn` and `modified-at`
went back, the next save from that client failed with `:revn-conflict`
and the object the save left became an orphan the storage GC cannot
revisit once it is frozen.

The read now runs in a transaction of its own (`get-file` calls
`get-file*` inside `db/tx-run!` and accepts a cfg map only), so the
`file` row lock covers the re-check and the write. `get-file*` reads
the row once (`sql:get-file` plus `resolve-applied-migrations`) and
completes it with `attach-file-data`; only when a migration is pending
it reads the file again under `FOR UPDATE`, re-checks `need-migration?`
on that fresh map and persists it (`migrate-file`). The common case
stays at one row read and one migrations resolution, the migration case
keeps a single decoded copy in the heap, and a row that changed in
between is not overwritten. A migration found on the unlocked read is
applied in memory only, for the next read to persist.

`migrate-file` returns the reattached map when it persists, with the
`:storage-object-id` of the row the write left, and the realized map
when it does not; `:realize?` is what returns the read fully plain.
`fdata/upsert!` answers `{:written ... :storage-object-id ...}` instead
of a boolean, `update-file!` propagates it and the migrating read puts
it back in the map it returns, so a caller that persists again touches
the current object instead of the replaced one. `insert-file!` keeps
nil: an INSERT never replaces a persisted object.

The write also compares the stored pointer with the caller's value
(`IS NOT DISTINCT FROM`) and raises `:stale-file-data-pointer` when it
affects no row. Every writer holds the `file` row lock, so a pointer
that no longer matches is a protocol violation, not a race to retry.
The expected pointer is a separate argument because the storage branch
replaces `:storage-object-id` before the upsert, and the check lives in
`upsert-in-database`: the `legacy-db` branches do not use that SQL and
their count can legitimately be zero.

The save, snapshot and restore-backup reads take `FOR UPDATE` instead
of `FOR SHARE`: they inherit the lock, and the migration branch asks
for the exclusive one in the same transaction, so a shared lock that
upgrades deadlocks two concurrent readers. A data read rejects
`:lock-for-share?` for the same reason; only row-only reads accept it.
`sql:get-file-without-data` is a plain select now, not a CTE, which
makes locking legal without loading the data. Realization has a single
owner: `attach-file-data` only resolves and decodes, and `get-file*`
realizes once per path. The applied migrations no longer travel in the
`get-file-with-summary` response, cache hit and cache miss alike.

`embed-assets` reads each library without `:realize?`, so its
`:components` is an unloaded pointer map; `absorb-assets` walks it with
`vals`, `PointerMap.seq` calls `load!` and that raised with no loader
bound, so every export with `:merge-libraries` of a library with
components failed. The absorb runs under the library's own loader now.

The in-place binfile import (`import-file-and-overwrite*`,
`::overwrite`, `::bfc/file-id`) had no product caller and cannot carry
the previous `storage_object_id`, so overwriting a storage-backed row
would leave its object unreclaimed. `import-files!` rejects a cfg with
`::bfc/file-id` (`:not-supported`) and the helpers only that path used
go with it (`invalidate-thumbnails`, `*reference-file*`, the
`::overwrite` id branch of `index-object` and of `save-file!`, the
`on-conflict-do-nothing?` flag of the media insert).

Tests: a new `file-migration-read-test` pins the single read, the
locked re-read, the concurrent save interleaving, the double migration
and the read-only path; the upsert tests cover the return shape and the
stale pointer; the SQL recorder they share lives in
`backend-tests.helpers`, and the save, snapshot, restore and summary
sequences follow the new reads. The backend memories document the read
contract and the pointer.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Harden the get-file guard and skip its read-only transaction

The cfg guard of `get-file` was an `assert`, which `app.config` elides
when `:backend-asserts` is off, so a deployed binary accepted a bare
pool or connection while the docstring promised it raises. It throws
`IllegalArgumentException` now, the class the option guards of the same
namespace already use, and the existing test expects it instead of
`AssertionError`.

A read with `:read-only?` applies the pending migrations in memory and
writes nothing, so the transaction the migration write needs only costs
two round trips there. `get-file` runs it with `db/run!` instead: with
a cfg that carries a connection it still joins the caller's
transaction, and only the reads that would have opened their own skip
it. A lock option needs a read-write transaction from the caller,
because PostgreSQL rejects row locks in a read-only one; the docstring
states that.

Tests: the guard test expects the new class; a cfg marked
`::db/read-only` reads a pending migration without persisting it (the
config flag that fills that key is legacy, but the key is set by hand);
and a read-only read is pinned as not opening a transaction while the
same call without the option does. The backend memory documents the
split and the guard.

AI-assisted-by: deepseek-v4.1-flash
2026-11-14 19:27:18 +01:00
Belén Albeza
d9a9d0c2bc
📚 Update contributing guide with rules about auto-rejecting PRs (#13012) 2026-11-14 18:16:55 +01:00
Usama Sadiq
dc340f74af
🐛 Fix database pool min-size falling back to max-size (#13052)
* 🐛 Fix database pool min-size falling back to max-size

Since 2.16 the pool minimum defaults to the maximum (60), so an idle
backend opens and holds 60 connections at startup. Restore the pre-2.16
default of 0 so the pool grows with load and idle connections expire.

Closes #12622

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

* 📚 Document database connection pool settings

Describe PENPOT_DATABASE_MIN_POOL_SIZE and PENPOT_DATABASE_MAX_POOL_SIZE,
their defaults, and sizing on a PostgreSQL shared with other apps.

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

---------

Signed-off-by: Usama Sadiq <usama7274@gmail.com>
2026-11-14 18:12:22 +01:00
penpot-renovate[bot]
a19fbbc826
⬆️ Update DHI base images (#13050)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@9cef390 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/devenv/Dockerfile)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-14 18:10:55 +01:00
AJ Schmidt
3ba5a0cb15
📚 Fix typo in export presets docs
This PR fixes a typo:

`pressets` => `presets`

Signed-off-by: AJ Schmidt <ajschmidt8@users.noreply.github.com>
2026-11-14 18:10:24 +01:00
Andrey Antukh
9c5260314a
🐛 Revoke websocket subscriptions when access is revoked (#12189)
* 🐛 Revoke websocket subscriptions when access is revoked

GHSA-m53j-2766-6jqw. A websocket subscription was authorized once,
when it opened, and that decision was trusted for the rest of the
connection's life, so a member whose access was revoked kept receiving
live file content as long as the tab stayed open.

The revocation travels over the message bus instead of closing the
socket from the mutation: the connection registry is local to one
backend instance, while the RPC that revokes access runs on whichever
instance received the request. Every instance re-verifies the
subscriptions it owns with the same `has-read-permissions?` predicates
that `:subscribe-file` and `:subscribe-team` authorize with, so a
downgrade to viewer keeps its subscription and a non-member
organization owner keeps read-only access.

Two mechanisms cover each other:

- Access-changing mutations announce after the write. delete-team-
  member, leave-team, update-team-member-role, delete-file,
  delete-project, move-files, move-project and delete-team announce.
  delete-share-link is deliberately excluded: a share-link visitor
  never passes the subscribe permission check, so there is no
  subscription to revoke. Team-wide changes publish one event per
  team; the watcher resolves the members itself, and both publishers
  defer through `db/after-commit` so the watcher never decides from
  pre-commit state.
- Every relay re-checks access every 5 minutes by default
  (`subscription-revalidation-interval`), driven from the relay loop
  with `alts!` over the subscription channel and a one-shot tick
  channel, so a steady stream of edits cannot push the check away
  forever.

Supporting changes:

- The registry holds `:connections` and `:by-profile` in one atom,
  kept consistent by register/unregister in a single swap!; a profile
  lookup is one map access, and a profile whose last connection
  leaves is dropped from the index.
- `close-file-subscription` and `close-team-subscription` announce
  the :leave-* message, close the relay channel and purge the bus
  subscription, and are idempotent. Closing the channel ends the
  relay loop and its re-check task.
- The permission query the re-check uses filters soft-deleted files,
  projects and teams, matching its sibling queries.
- A transient database error fails open: it logs and keeps the
  subscription; the next tick retries.
- The revocation watcher's params schema declared the nitrate client
  as a bare keyword inside [:map ...], which malli resolved as a
  registry lookup and which took the whole system build down under
  enable-backend-asserts. It is now an explicit pair, optional
  because init-key yields nil with the admin-console flag off.

The tests reproduce the leak against the real message bus, including
the negative control for the interval re-check. The written records
live in mem:backend/websocket-subscriptions.

Closes #12155

AI-assisted-by: space-bunny-free
AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Mirror the subscribe gates in the revocation re-checks

Addresses the second review round on PR #12189 (pabloalba).

The re-checks only asked whether the profile could read. The membership
legs of sql:file-permissions filter soft-deleted rows since the F2 fix,
but the organization-owner fallback grants read-only access over HTTP
without looking at deleted_at, so a soft-deleted file or team kept an
expelled org owner subscribed indefinitely. The re-checks now gate on
existence as well: file-readable? and team-readable? require a live row
(db/get*, non-throwing, soft-deleted rows excluded) plus the read
permission, mirroring the :subscribe-file gates and extending them to
teams. Both relays and revalidate-profile-subscriptions use them; the
subscribe handlers keep their exact behavior.

Also from the same review:

- watcher-consumes-revocation-events-from-the-bus no longer races the
  command's after-commit announcement: it takes access away with a
  direct row delete and drives the watcher with a synthetic bus event,
  instead of deleting through the command and publishing a second
  notify-* that would mask a broken command announcement.
- moving-a-project-closes-source-team-subscriptions pins the
  move-project e2e: the file subscription drops, the source-team
  subscription survives (membership unchanged).
- soft-deleted-file/team-closes-org-owner-subscription cover the new
  gate with red-before/green-after tests.

mem:backend/websocket-subscriptions updated.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Refuse dead teams in resolve-team-id (PR #12189 review)

Second review round on PR #12189 (pabloalba).

The file re-check added in 87ca7aca53 stops at the file row, but the
organization-owner fallback grants read-only access to non-member org
owners via Nitrate over HTTP, which knows nothing about Penpot
soft-deletion. resolve-team-id resolved the team from unfiltered SQL,
so an expelled org owner kept the file subscription open for an
indefinitely soft-deleted file, project or team. Now both SQL lookups
filter deleted_at, and resolve-team-id checks the team row itself
(db/get*, which filters soft-deleted rows) for every caller — teams,
projects and files alike.

Also from the same review:

- file-readable? now uses the same sql:get-minimal-file as the
  subscribe gate (non-throwing via throw-if-not-exists false), and
  team-readable? projects id and deleted-at, so db/get* can filter.
- The test suite extracts the thrice-duplicated raw delete into
  delete-team-membership!; a sibling of the fail-open pin now redefs
  the existence probe to throw; memory and docstring aligned with the
  code.
- docs/technical-guide/developer/architecture/backend.md attributed
  the websocket to notifications.clj; it now names app.http.websocket
  as the owner and notifications.clj as the revocation publisher.
- mem:backend/auth-permissions-product-domains documents the fallback
  mechanism and the resolve-team-id contract.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Test resolve-team-id refusal and tidy review findings

Third pass over PR #12189 after review-code on 225eb98de2.

resolve-team-id now has a direct test: all three branches resolve
while rows are live, and file, project and team soft-deletes each
refuse their branch — the two new SQL predicates and the final team-row
check are all pinned by it.

From the same review:

- failing-existence-check-keeps-the-subscription and the sibling
  a-failing-check-keeps-the-subscription are deleted: they could not
  distinguish a live relay from a dead one. The fail-open policy
  remains documented in the docstring and the memory; the memory no
  longer claims the pins exist.
- team-readable? projects ::sql/columns like the rest of the backend;
  the fully-qualified keyword went away with the new alias.
- The dead when-not guard in get-organization-owner-viewer-team is
  removed (resolve-team-id guarantees the row live); the let binding
  in resolve-team-id no longer shadows the destructured team-id.
- docs/technical-guide names rpc/notifications.clj as the publisher
  and the tree lists the real top-level modules.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Make the revocation tests deterministic

Third review round on PR #12189 (pabloalba).

Both tests took access away through a command that announces, so the
live watcher could close the subscriptions before the test looked:

- revocation-closes-subscriptions-that-are-no-longer-authorized
  asserted an exact count of 2 closes after delete-team-member, and
  that count reads 0 whenever the watcher got there first. It now
  removes the member with delete-team-membership!, the raw row delete
  that publishes nothing, and asserts both subscriptions are still
  open before calling revalidate-profile-subscriptions.
- team-event-closes-every-member-subscription ran delete-team and then
  published a second notify-team-permissions-changed, so a missing
  announcement in delete-team still passed. It now soft-deletes the
  team row (sql:team-members joins team_profile_rel and profile only,
  so the watcher still resolves every member), asserts nothing closed,
  and drives one synthetic team event.

The command paths stay covered by the announce and end-to-end tests
for delete-team-member and delete-team. The file split the reviewer
suggested stays the optional follow-up, tracked outside this PR and
waiting for the merge.

Closes #12155

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 18:27:30 +01:00
Andrey Antukh
f3c1230b79
🐛 Allow OIDC registration with password login disabled (#12974)
* 🐛 Allow OIDC registration with password login disabled

The register-profile command rejected every registration without the
login-with-password flag, including SSO ones, while the OIDC callback
already allowed :oidc-registration as an alternative, so new OIDC
users were stuck at the register screen with a 400
registration-disabled error.

The gate now branches on the verified token backend: password tokens
keep the previous checks, SSO tokens only require :registration or
:oidc-registration through a predicate shared with the callback, so
the two gates cannot drift apart again. Flags and invitation checks
are split into single-responsibility helpers, and tokens without a
backend claim are rejected loudly by schema instead of defaulting
silently. Covered by regression tests for the SSO matrix, canceled
invitations, the missing-backend case and the no-password-fallback,
plus a configuration doc note.

Closes #12964

AI-assisted-by: Muse Spark 1.3 Contributor

* 🐛 Gate SSO registration by provider type, accept invitations

Follow up the review on #12974. The SSO registration gate now treats a
team invitation token as a substitute for :registration /
:oidc-registration, in both the OIDC callback and register-profile,
mirroring the password flow: the callback redirects to the register
screen when an invitation is present, and register-profile revalidates
the invitation row under a row lock before creating the profile when
no SSO flag is set, raising :invalid-token instead of creating an
account that cannot join a team.

The SSO branch is restricted to OIDC-typed backends
(oidc-provider-types); a prepared-register token carrying any other
backend is rejected with :invalid-token instead of silently taking
the SSO rules. The flags and invitation checks are shared by one
check-registration helper in register-profile; the prepare command
runs flags, then the email-mismatch restriction, then the
invitation row lookup, so the password login flag fails closed first.
The schema defs are private, the create-profile default uses
penpot-backend, the OIDC callback test decodes the redirect token to
assert the :prepared-register claims, and the docs and the auth
memory state the invitation path for SSO registration.

Regression tests cover invitation-only registration, canceled
invitations with and without SSO flags, unknown backends, and the
double-fault precedence in prepare-register.

AI-assisted-by: inkling-2-small-free

* 🐛 Require invitation email match for SSO registration

Close the hole left by treating any invitation token as a substitute
for the SSO registration flags. register-profile now requires the
invitation member-email to match the prepared-register email, raising
:email-does-not-match-invitation otherwise, mirroring the password
flow. The OIDC callback verifies the team-invitation JWT and the
email before redirecting to register instead of trusting raw
presence. Covered by the requested mismatch regression test plus
callback tests for mismatch and garbage tokens.

AI-assisted-by: Muse Spark 1.3 Contributor

* ♻️ Share registration gate between prepare and register-profile

After the email-match fix, check-registration runs flags, email
and invitation row in the same order as validate-register-attempt!.
The prepare command now calls the shared helper instead of inlining
the trio, and the stale docstring rationale is gone, so future
invitation rule changes land in one place.

AI-assisted-by: Muse Spark 1.3 Contributor
2026-11-13 17:33:42 +01:00
Marina López
bde53b4696
🐛 Hide team plan for standalone enterprise owners (#13047) 2026-11-13 14:16:30 +01:00
Eva Marco
b2c4b58b27
🐛 Keep the pages panel scroll when switching pages (#13004)
Switching pages swaps the workspace content for a loader until the new
page is ready, so the whole left sidebar mounts again. The pages list
started at the top and the selected page scrolled into view at the
bottom of the panel, even when it was already visible.

The sidebar scroll store now lives in workspace-inner*, which stays
mounted across page switches. The pages panel saves its scroll there and
restores it before the selected page scrolls into view. The layers,
assets and tokens panels keep their positions across page switches too.

Closes #13002

AI-assisted-by: claude-opus-5-5
2026-11-13 14:11:30 +01:00
Andrés Moya
0e4be21320
🐛 Ensure component name always matches root instance name (#12794)
* 🐛 Remove svg extension when creating a component from a svg file

* 🐛 Normalize root names to always match the component name

* ✨ Remove any extension ignoring case, not only .svg

* 🔧 Add some fixes and checks

* 🔧 Improve rename detection
2026-11-13 13:53:28 +01:00
Marina López
bb4556b705
🐛 Persist promotional banner collapse (#13040) 2026-11-13 13:07:05 +01:00
andrés gonzález
097ff41e03
✨ Send telemetry for WebGL renderer failures (#12987)
* ✨ Send telemetry for WebGL renderer failures

WebGL renderer failures left no trace in the audit log, so we could not
tell why people switch WebGL off.

- Send webgl-unavailable when the "WebGL is not available" modal shows,
  with the init error.
- Send webgl-context-lost, webgl-context-restored and webgl-reload-failed
  from the WASM renderer.
- Add the active renderer (wasm or svg) to handled-exception,
  unhandled-exception, exception-page and open-workspace-file.
- Add environment details to the WebGL events and to
  enable/disable-webgl-rendering: normalized GPU family and backend, max
  texture size, device memory, CPU cores and whether the app runs inside
  Electron. The full GPU renderer string is not sent, as it can help
  fingerprint a device.

Closes #12984

AI-assisted-by: claude-opus-5-5

* ⚡ Avoid throwaway WebGL contexts in renderer telemetry

WebGL events no longer create a context to read GPU details; the
settings, menu and unavailable modal create one at most once.

AI-assisted-by: claude-opus-5-5

* 🐛 Refresh remembered GPU details on each renderer init

A restored context can run on another GPU, so read it on every init
instead of only the first one.

AI-assisted-by: claude-opus-5-5

* ⚡ Never create a WebGL context for renderer telemetry

WebGL events send GPU details only when the renderer has already read
them on init, instead of probing a throwaway context.

AI-assisted-by: claude-opus-5-5

---------

Co-authored-by: Elena Torro <elenatorro@gmail.com>
2026-11-13 11:59:12 +01:00
Andrey Antukh
8715450f53 Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 09:49:27 +01:00
alonso.torres
976f016481 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-13 09:42:15 +01:00
Belén Albeza
a7b6ef0d41
🐛 Restore the WASM stack pointer after a failed call (#12978)
A failed render-wasm call traps without popping its shadow-stack frames.
Each failure leaked stack until it overwrote static data, and from then on
every call trapped as wasm-critical. h/call now saves the stack pointer
before the call and restores it on failure.

Closes #12977
Related to #12906

AI-assisted-by: claude-opus-5-5
2026-11-13 09:39:43 +01:00
Andrey Antukh
69c33c4877
♻️ Wait on signals instead of event-loop turns on frontend tests (#12986)
* ♻️ Wait on signals instead of event-loop turns

`wait-for` counted its budget in event-loop turns (50 x `setTimeout 0`,
about 54 ms idle), so a delivery that travelled through the RxJS async
scheduler could arrive after the wait gave up. It now accepts a
`:signal`: the predicate is re-checked on every emission and the wait
resolves on the event that changes the state. The ceiling (5000 ms)
only bounds failure when a signal is given, and stays a poll budget for
the waits that have nothing to listen to.

`wait-for-store` wraps the store's input stream, which notifies its
subscribers after applying the update. The five explicit `:max-ticks`
sites move to `:timeout-ms` in the same change, so the old option never
lives in a state where destructuring silently ignores it.

New `frontend-tests.helpers.async-test` pins the contract: a late
delivery resolves, a store signal resolves on the event without a poll,
an already-true predicate does not yield, a never-true predicate records
one failure within the ceiling, and a resolved wait stops listening.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Wait for the persistence requests on their own stream

The persistence tests waited for `(= N (count @requests))` by polling the
atom at every event-loop turn. The mocked transport now pushes each
request into a `requests$` subject, and those 59 waits pass it as
`:signal`, so they resolve on the request that makes the count true. The
two predicates that mix the request count with the store state merge
both signals.

The wait that flaked (`the other file's edit is sent next`, in the
rollback test) is one of these: it no longer depends on a turn budget,
and it cannot lose the delivery.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Wait for the store event in the store backed tests

153 waits whose predicate reads the store now go through
`wait-for-store`, so they resolve on the store event that changes the
state instead of polling at every event-loop turn. The count is 153, not
the 162 the plan estimated: nine of them read the store through a
helper (`fill-color`, `shape-name`, `copy-children`) rather than
`@store`. The five sites that carried an explicit ceiling drop it, since
with an event the ceiling only bounds failure.

Resolving on the event exposes the places where the poll was really a
pause for a side effect to advance:

- `tokens-source-test` emitted `unlink-file-from-library`, a watch that
  chains a second `rp/cmd!` after an async hop. The store signal fires
  before the chain runs (the event itself dissocs the library), so the
  mock scope closed in flight and the second call reached the real HTTP
  layer. The test now waits for that second call, through the mock.
- `persistence-rollback-test` drives the persistence pipeline step by
  step; two of its waits are the pause before the next emit, not an
  effect, so they keep the poll.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Document the wait contract and what still polls

`mem:frontend/testing` described `wait-for` as an immediate check plus a
bounded poll. It now waits on a signal, and the ceiling only bounds
failure; `wait-for-store` and the persistence `requests$` subject are
how a wait resolves on the event that changes the state. The waits with
nothing to listen to keep the poll, and the memory names them.

The two traps that cost a red suite are recorded too: a wait that means
"the side effect advanced" has no event and keeps the poll, and a
store-state wait can close the mock scope before the flow it observes
ends, so the chain's last RPC call has to be awaited through the mock.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Give the wait wrapper a single signal

`wait-for-store` built its options with `assoc`, so it replaced the
`:signal` a caller passed. The two waits whose predicate reads both the
store and the request counter relied on a merged signal that was being
discarded; they now pass it to `wait-for` themselves, and the wrapper
takes no options. That merge was the safety net for a request counted
after the store event, and it was inert.

The helper also reports why a wait failed (ceiling, ended signal, or an
error with its cause) instead of calling all three a timeout, counts
the predicate evaluations in the late-delivery case to prove the signal
path does not poll, reads the clock through `app.common.time` in the
helper and the test, and the test's turn probe cancels its timer. The
copyright header of `helpers/async.cljs` stays as the previous commit
left it, matching the majority spelling.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Give the cycle counter a signal too

Two waits in the retry tests read the store and the `cycles` atom, which
the `slow-cycle-timer` double fills inside the persistence watch. They
passed only the store signal, so a counter change could not wake them:
the same latent race the previous commit removed from the request count.
The double now also emits each new cycle into a `cycles$` subject, and
both waits pass the merged signal to `wait-for`.

The public `wait-for` docstring names the three failure reasons, and the
message for a failed signal bounds the cause text: the exception message
when there is one, otherwise the printed value inside a try, cut at 500
characters.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Stop the error handlers flooding the test log

Four tests in `main-errors-test` run a real error handler end to end.
Those handlers print the cause with `ex/print-throwable` before
notifying, which is what a developer wants in the app; in a test it only
writes console noise. One of them replays 10,000 offline errors, so a run
carried 10,006 dumps: 190,010 of its 192,682 lines, 13.8 MB, well past
the ~4 MB where GitHub truncates a step log and takes the summary and any
failing assertion with it.

The tests stub `ex/print-throwable` with a no-op for the duration of
their mock scope. The handlers still run, the assertions do not change,
and the 10,000-error loop still proves the governor emits once. A full
run now logs 2,583 lines and 384 KB. The single block that remains comes
from the re-entrancy guard, which prints its own notice on purpose.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Silence the deliberate WASM panic in its test

`get-svg-markup` logs a WASM panic with `console.error` before returning
empty, and `get-svg-markup-swallows-wasm-errors` sends two panics through
it on purpose. That was two 35-line dumps in which the exception printed
its own `data`, `arr` and protocol masks, and it reads like a failure that
the suite is hiding.

The test replaces `(.-error js/console)` for its body and restores it in a
`finally`. The assertions and the helper stay as they are: the same
panic answered with a different stack would still show up, because only
this test silences the console.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Use real timer for wait-poll ceiling

wait-poll read the app clock (ct/now), so tests freezing that clock
never reached the deadline and hung instead of failing. Track expiry
with a real setTimeout flag, pass timeout-ms through wait-signal and
wait-for, and cover the frozen-clock case with a regression test.

AI-assisted-by: muse-spark-1.3-contributor
2026-11-13 09:18:43 +01:00
Andrey Antukh
679e70e48c 🐛 Keep file_change kind nullable so rollback keeps writing
Migration 0158 created file_change with kind NOT NULL, which blocks
any emergency rollback to the previous version: the old code inserts
without kind and every save would fail. kind is still a reserve
(reads use label IS NOT NULL), so it stays nullable until the
rollback window closes; the NOT NULL comes in a later migration.
The runbook keeps parity, the schema and migration tests cover the
nullable column and the rollback insert, and the storage memory
records the decision.
AI-assisted-by: muse-spark-1.3-contributor
2026-11-13 08:05:15 +00:00
Alonso Torres
3e48ab96e4
🐛 Limit token propagation to the new instance on swap and instance (#13021)
* 🐛 Limit token propagation to the new instance on swap and instantiate

Switching a variant and adding a component instance re-applied every
active token to every shape of every page. In files with thousands of
token-styled shapes this took over 10 seconds, while only the new
instance needed the file's token values.

propagate-workspace-tokens takes an optional scope (page id and root
ids) that limits the propagation to those shapes and their descendants.
component-swap and instantiate-component pass the new instance's root.
Calls without a scope still cover the whole file.

Closes #13019

AI-assisted-by: claude-opus-5-5

* ✨ Add token tests for instantiate, swap and multi-page propagation

Cover instantiate-component and component-swap with components whose
stored values differ from the resolved tokens, checking that the new
instance gets the file's token values and other shapes are untouched.
Add a second page to the propagation tests to check that unscoped
propagation reaches other pages and scoped propagation does not.

AI-assisted-by: claude-opus-5-5
2026-11-13 09:02:44 +01:00
Andrey Antukh
335c6782a0 Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 08:33:31 +01:00
Andrey Antukh
7dcc0fde7a Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-13 08:33:01 +01:00
Andrey Antukh
a39dee6d79 Merge remote-tracking branch 'origin/main' into staging-2.19.x 2026-11-13 08:32:45 +01:00
Roshan Ramani
7516f1d163 📚 Fix the telemetry flag name in the configuration guide (#13032)
The guide told self-hosters to add enable-telemetries to PENPOT_FLAGS.
flags/parse turns that into :telemetries, which nothing reads; the
telemetry task checks :telemetry, so the documented flag never enables it.

AI-assisted-by: claude-opus-5-5
2026-11-13 08:32:29 +01:00
Roshan Ramani
e202612469
📚 Fix the telemetry flag name in the configuration guide (#13032)
The guide told self-hosters to add enable-telemetries to PENPOT_FLAGS.
flags/parse turns that into :telemetries, which nothing reads; the
telemetry task checks :telemetry, so the documented flag never enables it.

AI-assisted-by: claude-opus-5-5
2026-11-13 08:31:37 +01:00
penpot-renovate[bot]
c19f7a6e5d
⬆️ Update dhi.io/node image to v24.22.3 (#13031)
- dhi.io/node 24.22.3@ (docker/devenv/Dockerfile)
- dhi.io/node 24.22.3@ (.nvmrc)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.3-debian13@84adaf3 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-13 08:30:10 +01:00
Andrey Antukh
5c45c8ddaf Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 08:18:50 +01:00
Andrey Antukh
ad710ed143 📚 Document the squash-merge default and the no-squash opt-out
Squash merge appeared twice in the guide, as a review-process bullet and
in branch naming, with no reason and no way out. Add a Merging
subsection: one PR lands as one atomic commit, so the history stays
readable and git bisect never stops on an intermediate commit that fails
the tests or the linter. A contributor who keeps every commit
self-contained may ask for a regular merge in the first line of the PR
description.

Record the same contract in mem:workflow/creating-prs (merge strategy,
and the Merge line above the AI Note line) and in
mem:workflow/creating-commits (what a self-contained commit means when a
regular merge is requested).

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 07:10:24 +00:00
Andrey Antukh
9bac1d290b 📚 Clarify when to run the file_change partition procedure
The procedure straddles the upgrade to 2.20.0 and the document said
nothing about the order. The preparation (phases 0 and 1) goes before
the version update, the bulk and the delta passes after it, and only
the cutover needs a scheduled maintenance window. The subtitle claimed
the whole thing was online, which is not true.

The copy marks no longer travel as a session GUC with a date typed by
hand. The delta mark is seeded at the bulk with `now()`, and a CHECK
keeps it from ending up empty; the instant of the cutover is recorded
in the same table after the commit, so the checks of phase 5 read it
instead of comparing against a date the operator retypes in every
session.

The procedure was run end to end on a scratch database, with the 23
SQL blocks extracted from the document and the application writes
simulated between the phases.

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 06:58:46 +00:00
deni-ing
d774e197d0
🐛 Make GitHub logo follow the theme on the login page (#12943)
The GitHub icon had a hardcoded white fill, so it was almost
invisible on the light theme. Use currentColor so it follows
the button text color.

Fixes #12938

Signed-off-by: deni-ing <242363192+deni-ing@users.noreply.github.com>
2026-11-12 18:26:57 +01:00
Luis de Dios
24a5fd2043
♻️ Refactor the share link dialog to the design system (#12965)
* ♻️ Split the share link dialog into smaller components

Move the dialog state into a content component and extract the
permissions section, the page selection, the select rows, the
collapsible panel and the destroy confirmation.

The confirmation now keeps the link to destroy instead of matching it by
flags, so deleting a link no longer depends on the options snapshot.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ♻️ Use the design system controls in the share link dialog

Replace the deprecated select, the raw text input and the hand-written
page checkboxes with the design system select, input field and checkbox
components, and drop the deprecated icons and the unused uuid parsing.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ✨ Add dialog semantics to the share link dialog

Give the dialog and the destroy confirmation proper dialog and
alertdialog roles and labels, cancel the confirmation with Escape, and
connect the disclosure button with its panel through aria-controls.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ✨ Toggle the share dialog from the viewer header

Let the share button close the dialog when it is already open, report
the dialog state with aria-expanded and aria-pressed, and mark the
button with data-allow-click-modal so the closing click does not count
as an outside click.

Drop the click-outside guard on close; it is not needed once the button
can reopen the dialog.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* 🐛 Move focus into the share dialog and restore it on close

Move focus into the dialog when it opens and into the Cancel button when
the delete confirmation opens, so Escape reaches the confirmation handler
instead of closing the whole dialog.

Restore focus to Destroy link when the confirmation closes and to the
Share button when the dialog closes.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Apply review follow-ups to the share link dialog

Use aria-haspopup instead of aria-pressed on the Share toggle, drop the
raw z-index, and express the small spacing values with px2rem. Restore
the collapsible pointer cursor and merge its two classes.

Memoize the permission options, expose aria-controls only while the
panel is open, and label the read-only URL field.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 17:53:39 +01:00
Danny Shirely
192e76781a
✨ Add export previews to Design and Inspect (#12059)
* ✨ Add export previews to Design and Inspect

Preview a selected shape's export presets in Design and Inspect, with
output dimensions and a shared persisted expansion preference.

Bound native previews to 240px and debounce subtree updates while keeping
the current image until its replacement is ready. Release replaced URLs
and cancel pending rendering when the preview closes.

Keep this change focused on previews and track native export fill
handling separately.

Closes #7561

AI-assisted-by: gpt-6
Signed-off-by: Danial Shirali <danialshirali16@gmail.com>

* 🌐 Add Spanish translations for the export preview

The export preview added four new keys only to en.po, so Spanish users
saw them in English.

AI-assisted-by: claude-opus-5-5

* 🌐 Improve the Spanish export preview labels

AI-assisted-by: claude-opus-5-5

---------

Signed-off-by: Danial Shirali <danialshirali16@gmail.com>
Co-authored-by: Eva Marco <evamarcod@gmail.com>
2026-11-12 17:23:46 +01:00
Andrey Antukh
ec1e2c9e1a Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-12 15:36:44 +01:00
Andrey Antukh
92c478b0e2 Merge remote-tracking branch 'origin/merge-staging' into staging 2026-11-12 15:32:33 +01:00
Andrey Antukh
dc38eaa843
🐛 Fix font upload crash and preserve font metadata (#11951)
* 🐛 Fix font upload crash and preserve variant metadata

opentype.js 2.0.0 returns no fontSubfamily name for fonts whose
Macintosh name table only holds custom records. That made
parse-font-weight/parse-font-style call re-seq on nil and crash the
upload.

A nil or blank variant is now treated as no match in
app.common.media, defaulting to weight 400 and style normal. The
variant parsing also accepts oblique as italic, and the new
parse-font-family helper derives the family from a filename by
stripping the known weight/style tokens, separating a style token
glued to a weight token ("Roboto-BoldItalic" -> "Roboto").
first-nonblank-string returns the first nonblank candidate, so a
blank preferredFamily/preferredSubfamily no longer shadows a valid
fontFamily/fontSubfamily.

app.main.data.fonts gains resolve-font-metadata and
prepare-font-entry. opentype.js values win per field; any missing
value falls back to the filename-derived family, weight and style.
The parsed and woff2 branches share the same helper, so metadata is
preserved instead of collapsing every face to weight 400, and
fonts with missing vertical-metric tables upload with no height
warning instead of failing.

process-upload reports per-file failures (unknown signature,
unparseable file, preparation error) with a single bad-font
notification instead of silently dropping them, and never errors
for per-file failures.

The upload now sends :variant-name to create-font-variant, which
accepts it as an optional param (ships in 2.20), so the foundry
name survives the upload.

Closes #10436

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: muse-spark-1.3-contributor
Signed-off-by: CHOI JUNSOO <junsoo1172@gmail.com>

* 🐛 Parse glued font style tokens in the font parsers

EvaMarco's review on #11951: parse-font-family and
parse-font-weight handled glued tokens like "BoldItalic", but
parse-font-style did not. For names such as
"opensans-bolditalic-webfont.woff2" the three parsers disagreed:
the italic face resolved to style normal and collided with the
bold-only file on the (family, weight, style) join key. One
binary silently overwrote the other during upload, losing one
font face without any warning.

A new private normalize-font-name helper is applied by all three
parsers. It splits a style token glued to a weight token, now
with a lookahead boundary so "LightItalic_v2" splits too, and
strips a trailing technical suffix (-VF or _v<number>).
parse-font-family, parse-font-weight and parse-font-style now
interpret the name consistently.

Tests cover the three cases from the review, the bold-only
control case, and an end-to-end process-upload check that a bold
and a bold-italic WOFF2 survive as two variants.

Relates to #10436 and PR #11951.

AI-assisted-by: inkling-2-small-free

* Remove duplicate frontend test registration

---------

Signed-off-by: CHOI JUNSOO <junsoo1172@gmail.com>
2026-11-12 15:29:30 +01:00
Marina López
5803f2594c
🐛 Show the organization in the new team form (#13023) 2026-11-12 15:11:43 +01:00
Marina López
029283b1f9
🐛 Hide standalone Enterprise plans for team owners (#13017) 2026-11-12 14:54:56 +01:00
Luis de Dios
185fef889b
🐛 Fix collapsing token set groups after reordering (#12671)
* 🐛 Fix collapsing token set groups after reordering

The handler captured the first render path, and the tree is keyed by
index, so after a reorder it toggled the path of another group.

Closes #12625

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Refresh double click handler of token set groups

Add the start edition callback to the hook deps so the handler does
not keep the initial closure.

AI-assisted-by: deepseek-v4.1-flash

* ✨ Add regression test for folding reordered token set groups

Reorder a group in the sets tree, fold it, and assert that only that
group collapses. The tree is keyed by index, so the test also proves
the collapse handler does not keep a stale path after a reorder.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Refresh double click handler of token set nodes

Add the start edition callback to the hook deps so the handler does not
keep the initial closure.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Key token set tree nodes by identity instead of index

React reuses a node by its key. Keyed by the tree index, a reordered
group moved the instance to another node and its handlers kept the
previous paths. Groups are now keyed by path and sets by id, so each
node keeps its own handlers.

The set being created has no persisted id yet, so the tree walk
generates a new one on every render. Keep one uuid for its whole
creation instead.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 13:49:43 +01:00
Miguel de Benito Delgado
ef725c0cc2
🔧 Enable checking commit messages from files (#12951)
Adds a new flag --message-file to check-commit

AI-assisted-by: deepseek-flash
2026-11-12 13:28:35 +01:00
Eva Marco
d829e47a04
🔧 Run the Playwright tests with a dark color scheme (#12989)
* 🔧 Run the Playwright tests with a dark color scheme

Since #12480 a profile without a theme follows the system one, and
headless Chrome reports a light scheme, so the e2e tests ran with the
light UI. Set colorScheme to dark in the shared Playwright options so
the tests use the default dark UI again. Tests that need another scheme
still set it with page.emulateMedia.

AI-assisted-by: claude-opus-5-5

* 🔧 Drop the dark color scheme overrides now set for every test

The shared Playwright options now set the dark color scheme, so the ds
project override and the emulateMedia call in the selection size badge
test do nothing.

AI-assisted-by: claude-opus-5-5
2026-11-12 13:27:52 +01:00
penpot-renovate[bot]
88e1b29e5c
⬆️ Update DHI base images (#13008)
- dhi.io/debian-base trixie-debian13@c12f605 (docker/imagemagick/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@a5fc6ab (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@fb9c8a5 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-12 13:10:05 +01:00
andrés gonzález
9d42ed29ce
📚 Document how to order variant properties and values (#12996)
Users ask how to choose the order of values in the variant property
dropdown. The values follow the order of the variants in the Layers
panel, and this was not documented. Also document that properties can
be reordered by dragging them in the Design tab.

AI-assisted-by: claude-opus-5-5
2026-11-12 13:04:13 +01:00
JiMyung Lee
61154e6bfe
✨ Keep the swap component search when changing library (#12873)
Changing the library in the Swap component menu cleared the search
field, so checking several connected libraries for the same component
meant typing the name again for each one. Keep the term and reset only
the library id and the group path, so the new library is shown already
filtered.

Closes #12335

AI-assisted-by: claude-opus-5-5

Signed-off-by: JiMyung Lee <lee.ji.myung@gmail.com>
2026-11-12 12:45:36 +01:00
Marina López
b5eab6a949
💄 Update promotional banner (#12944) 2026-11-12 12:35:13 +01:00
Belén Albeza
86a8ac34cd
🎉 Add basic bullet lists to the v3 text editor (#12720)
* 🎉 Add UI for toggling bullet lists (under config flag)

* 🎉 Store list-style-type as a paragraph attribute

The bullet list toggle now saves list-style-type ("none" or "disc") on
the selected paragraphs and reads it back. A paragraph without the attr
counts as "none", so a bullet paragraph next to an older one shows as
mixed.

WASM keeps the value on each paragraph: the paragraph bytes grow from
12 to 16, Enter keeps it on the new paragraph, and the v3 editor reports
it in the current styles, so the sidebar shows it while editing too.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Draw bullets for list paragraphs

A paragraph with list-style-type "disc" now gets a solid round bullet,
sized as Chrome draws a CSS disc marker (measured with Source Sans Pro):
a radius of 0.16 font sizes, centered 0.32 font sizes above the first
baseline, with a gap of 0.33 font sizes before the text. The bullet
hangs outside the text box, before its start edge (after it for RTL),
like CSS list-style-position: outside, so the text layout, caret and
selection stay as they were.

The bullet takes the paint of its line's first style run, so fills,
strokes, shadows and masks apply to it as they do to the glyphs. An
empty item, which has no run, borrows the previous item's paint. The
cached paint, the text extrect and text-to-path share one walker over
the paragraphs, so they always agree: the extrect grows to hold the
bullets, like shadows and blur, and flattening a text keeps them.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Show the list style in Inspect

Text paragraphs with list-style-type "disc" now show a "List Style" row
in the Inspect Styles and Computed views, after Text Transform. Its copy
action gives "list-style-type: disc;". Text without a list shows no row.
The list item check lives in common as list-paragraph?, so every place
that needs it shares one rule.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Generate list HTML and CSS for bullet paragraphs

In the Inspect code, neighbouring paragraphs with list-style-type
"disc" become one <ul> with an <li> each, keeping the paragraph's class
and dir. Other paragraphs stay <p>. Only code mode groups them; text
rendering still uses <p>.

The CSS sets list-style-type on the items and resets the list's margin
and padding, so the browser hangs the bullet before the text as the
canvas does. Paragraph rules set font-size: 0, so each item also gets a
::marker rule with its first span's font size and text color. A
::marker can't paint a gradient, so a gradient fill gives its first
stop.

Refs #12630

AI-assisted-by: claude-opus-5-5
2026-11-12 11:50:05 +01:00
Belén Albeza
00e77a0f53
🐛 Fix slow variant switch with the WASM renderer (#12985)
Changing a component inside a big main component made the WASM renderer
draw that component's thumbnail at full size on the main thread, before
the viewport redraw. On the Fauna file the 2140x1642 thumbnail took about
1.2 s to render and PNG-encode, so the switch showed up late.

- Fit the thumbnail's longest side between 200 and 512 px. Small
  components still scale up; big ones now scale down.
- Draw the thumbnail only on a frame with no viewport render pending

Closes #12968

AI-assisted-by: claude-opus-5-5
2026-11-12 11:49:01 +01:00
Andrés Moya
2c86db80bd 🔧 Unify migration tests 2026-11-12 11:41:27 +01:00
andrés gonzález
1eccb3b4bb
✨ Track library token usage in analytics events (#12958)
Add a from-library property to the apply-tokens event, true when the
file takes its tokens from a linked library.

Emit a link-file-to-library-tokens event with tokens-count when the
linked library has tokens, mirroring link-file-to-library-variants.
Both events share one get-library-usage call.

Closes #12915

AI-assisted-by: claude-opus-5-5
2026-11-12 11:41:11 +01:00
Miguel de Benito Delgado
756541939a
🔧 Add renderer benchmark runner (#12756)
Add a CLI under frontend/performance/benches/render_wasm that builds the
renderer, runs seeded scenes (rects, plus, paths, masks) in headless
Chromium and saves per-case timings.

- build, run: prepare or reuse builds; measure cases with warmups,
  repetitions, deadlines
- ab: build and compare Cargo feature configurations
- compare, merge: median changes with bootstrap CIs between saved runs,
  guarded by scene, build and settings checks
- purge: delete managed builds, runs or comparisons
- GPU runs through `run-devenv --gpu`

Lint, format and pnpm scripts now cover performance/ and its tests.

AI-assisted-by: gpt-6.1-sol, deepseek-flash, muse-spark, glm-5.3, gpt6-astra, claude-5.5-sonnet, claude-haiku-5.5
2026-11-12 11:02:34 +01:00
Andrey Antukh
8b9558b380
♻️ Reduce and speed up the frontend e2e token suite (#13001)
* ✨ Add unit tests for the token name schema

Cover make-token-name-schema, the schema that rejects a token name whose
path clashes with an existing token or group. The lower-level
token-name-path-exists? already had a test; the schema that composes it
did not.

The cases pin an exact duplicate, a token inside a token, a token name
that collides with a group, an existing child path, valid siblings, the
fail-closed nil tree, invalid syntax and the length limit. This is the
unit-test base for reducing the 15 group-name-clash e2e cases to two.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Remove duplicate token tests and fix the spread step

The three top-level tests duplicated the ones inside the creation
describe; only the indentation differed. Keep the originals.

The shadow token test labelled "Valid spread" filled the blur field, so
the token under test ended with blur 0 and the spread never changed.
Fill spread instead and expect the referenced token to resolve with
Blur: 4.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Reduce the token group-name-clash e2e matrix to two tests

The fifteen cases repeated the same setup and only changed the token
type and value. The rule is now covered by the token name schema unit
test, so keep two representatives: thirteen simple types in one test
with a test.step each, and the two composite forms (Typography and
Shadow) in a second test.

The helper no longer re-runs the file setup, so each test loads the
page once: 15 page loads become 2 and the block drops from 433 s to
about 21 s.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Cover the numeric token creation types in one test

The nine creation tests repeated the same validation ladder and only
changed the token label and placeholder. Run the ladder once in the
border radius step, keep the reference token creation, and create the
other eight types with a test.step each.

The new helpers split the flow: open the form, assert the ladder, save
a token. The e2e drops from nine page loads to one.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Reduce the remapping e2e spec to three tests

Keep one remapping flow per token shape: box shadow with alias
references, border radius with a value change, and a group rename with
undo. The two cancel paths lose their own page loads and become steps
inside the shadow test: close keeps the original name, and "don't
remap" renames the token and breaks the reference.

The deleted typography and shadow applied-shape cases exercise the same
data path, which the remapping unit tests already cover.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Shrink the tooltip and numeric-revert e2e tests

The tooltip test asserted five message shapes over eleven layer
selections. Keep the three that have no unit test: a deleted token, a
reference outside the active sets, and a token in an inactive set. Each
is one step with a pill label and its tooltip text.

Three of its five messages were already covered by the token pill unit
test, including both resolved-value cases.

Keep one numeric-revert test and drop the fixed sleeps: await the value
with toHaveValue instead. Division by zero and the min/max clamp are
unit-tested in the numeric input tests.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 10:58:52 +01:00
Andrés Moya
a35dd49b6a Merge branch 'staging-2.19.x' into staging 2026-11-12 10:36:04 +01:00
makesomethingshit
dfc7919f7b
🐛 Preserve HSL values while editing lightness (#12629)
* 🐛 Preserve HSL values while editing lightness

Keep HSL editing components through black, white, and grayscale.
Use the same draft for numeric inputs, sliders, and gradient previews.
Strip the draft before emitting color data and reset it on other edits.

Closes #12615

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
AI-assisted-by: GPT-6

* 🐛 Keep HSL drafts consistent across gradient and ramp edits

Match RGB components to HEX so gradient echoes preserve HSL drafts.
Discard stale HSL values when editing Ramp hue, saturation or value,
while preserving them for opacity-only changes.
Cover grayscale gradient echoes and saturation restoration.

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
AI-assisted-by: GPT-6

---------

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
2026-11-12 10:11:56 +01:00
Andrey Antukh
64a51e86fd
🐛 Fix files blocked by dangling child references in migrations (#12936) (#12979)
The 0033-normalize-main-instance-names migration walks the shape tree
following the :shapes vectors. When a parent listed a child id missing
from the container :objects map (a dangling reference), the traversal
stored the nil child into :objects, creating a nil key that made
check-file-data reject the whole file: get-file answered 500 "invalid
file data" and the file could not be opened or migrated further.

Fix it in two parts:

- 0033 returns the container untouched when a listed child does not
  exist, so the traversal never stores a nil shape.
- The append-only migration 0035-fix-dangling-children prunes from
  every :shapes vector, in pages and components, the ids absent from
  the owning container :objects map, logging each fix at trace level.
  It mirrors 0032-fix-children-not-in-parent, which repairs the
  opposite inconsistency.

Both parts are needed: 0033 runs before the repair migration, so
without the guard the same file fails again in the same migrate call.

Add regression tests for the traversal guard and for the repair
migration, and record in mem:clojure/logging the convention that
:hints start lowercase.

Closes #12936

AI-assisted-by: qwen3.8-flash
2026-11-12 09:41:49 +01:00
Marina López
df0092fc60
🐛 Enforce sso before creating organization teams (#12980) 2026-11-12 08:35:32 +01:00
Andrey Antukh
75b55b567f 📚 Document test database and valkey env discipline
Agents must respect the PENPOT_TEST_* variables already set in the
shell and not override them with values copied from CI logs (the CI
endpoints only exist inside the CI docker network) unless the user
asks otherwise, such as a temporary database.

The shared test database may carry migrations applied by newer
versions; when that blocks a test run, dropping and recreating only
the database from PENPOT_TEST_DATABASE_URI is explicitly permitted.

AI-assisted-by: inkling-2-small-free
2026-11-12 06:27:44 +00:00
Andrey Antukh
12d73d0f63 🐛 Fix flaky assertion in climit waiting-count test
The test expected exactly one publication of the :rpc-climit-waiting
gauge inside the with-redefs window. The republisher daemon thread,
started at component init with no delay and rescheduling every 15 s,
publishes the same counter and can land inside that window, so the
recorded sequence could be [0 0] and the assertion failed with
actual [0 +0] (Backend Tests job in #12979).

Assert on the last published value instead of the whole sequence,
the contract waiting-last documents and the sibling tests use.

AI-assisted-by: inkling-2-small-free
2026-11-12 06:27:40 +00:00
Andrey Antukh
e57d358c53 📎 Update .gitignore 2026-11-12 07:09:42 +01:00
Pablo Alba
34f0d12b99
🐛 Ignore float noise when syncing moved library components (#12976) 2026-11-11 17:09:33 +01:00
penpot-renovate[bot]
b438676d6c
⬆️ Update dhi.io/node image to f6ba5e2 (#12975)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-11 16:55:19 +01:00
andrés gonzález
0d05cfaf5e
✨ Add the file to plugin and MCP audit events (#12972)
* ✨ Add the file and MCP session to plugin audit events

Events sent by plugins and the MCP plugin only carried their origin, so
the multi-file and dashboard MCP workflows could not be measured, nor
duplicated events from several open tabs spotted.

The context of se/event and se/add-event now carries the file-id of the
file the action ran in (absent on the dashboard) and, for the MCP, the
mcp-session-id of the tab connection that ran it.

Closes #12902

AI-assisted-by: claude-opus-5-5

* ♻️ Drop the MCP session id from plugin audit events

Every audit event already carries the session-id of the tab in its
context, and the MCP session id is only a hash of it, so it adds
nothing to group events by tab.

AI-assisted-by: claude-opus-5-5

* 🐛 Use a reified event in the system events test

ptk/data-event does not implement IWithMeta, so add-event failed on it.
Plugin actions emit reified events, which carry metadata.

AI-assisted-by: claude-opus-5-5
2026-11-11 16:18:15 +01:00
andrés gonzález
ca616a39ed
🐛 Send the plugin origin with penpotMgmt dashboard actions (#12971)
Dashboard actions made by a plugin or the MCP through penpotMgmt reached
the backend audit log without event origin, so they looked like actions
made by hand in the dashboard. The backend reads the origin from the
x-event-origin header, which rp/cmd! only sends when the params carry
::ev/origin in their meta.

Tag the params of every penpotMgmt mutation with the same meta that
se/add-event sets (mcp or plugin). The rename, share and team update
events now keep that meta on the RPC params, so setters send it too, and
set-file-variants-shared takes the caller origin instead of a fixed
dashboard one.

Closes #12901

AI-assisted-by: claude-opus-5-5
2026-11-11 16:02:29 +01:00
Pablo Alba
bb672a3723 🐛 Block grid track duplication and paste in component copies 2026-11-11 15:32:10 +01:00
Pablo Alba
3eebe92606 🐛 Keep the swap slot when duplicating a swapped nested copy 2026-11-11 15:32:10 +01:00
Gabriel Ribeiro
7e9a3c3c28 📚 Fix typos in technical and user guides
Fix five misspelled words in the docs: "notifiy" and "occured" in the
integration guide, "enviroment" in the developer guide index, "succesful"
in the authentication subsystem page and "achive" in the layers page.

AI-assisted-by: claude-opus-5-5
Signed-off-by: Gabriel Ribeiro <267068126+gabrielribeiro3105@users.noreply.github.com>
2026-11-11 15:24:13 +01:00
andrés gonzález
a301c2b43f
✨ Keep board names visibility after reloading the workspace (#12872)
Hiding board names from the View menu reset on every reload, while
rulers and comments kept their state. Add the flag to the layout flags
persisted in local storage so it behaves the same way.

Closes #12871

AI-assisted-by: claude-opus-5-5
2026-11-11 14:52:35 +01:00
andrés gonzález
54b1eb5d5f
✨ Add shape count to the apply-tokens audit event (#12970)
The apply-tokens event fires once per apply call, so a plugin or MCP
script that loops over shapes sends one event per shape while the UI
sends one per selection. Add a shape-count prop with the number of
shapes the token was applied to, so analytics can compare origins in
the same unit.

Closes #12900

AI-assisted-by: claude-opus-5-5
2026-11-11 14:43:11 +01:00
Alonso Torres
180b1ef7cb
🐛 Fix default theme for penpot plugins (#12982) 2026-11-11 14:16:20 +01:00
Andrey Antukh
bcac14a4a8
🐛 Skip circular library links on binfile import (#12062)
* 🐛 Skip circular library links on binfile import

File import recreated circular library references that the
link-file-to-library command already rejects. The v3 importer
now skips manifest relations that would close a cycle, or link
a file to itself, and logs each skipped link. Closes #12052.

AI-assisted-by: muse-spark-1.3-contributor-free

🐛 Address review findings on circular import guard

F1 extracts the cycle check into bfc/library-link-status and reuses
it in link-file-to-library and v3 import, keeping UI errors intact.

F3 orders get-files-rels and manifest relations so the first edge
wins deterministically, and documents it.

F2 adds a three-node transitive cycle test; F4 asserts file_library_sync
has no orphan rows for skipped links and matches rels otherwise.

AI-assisted-by: muse-spark-1.3-contributor

🐛 Pass team-id in binfile v3 circular library tests

The four circular library relation tests (import-skips-circular,
import-skips-transitive-cycle, import-skips-self-referencing,
import-keeps-acyclic) called v3/import-files! without ::bfc/team-id.
resolve-and-link-libraries asserts (uuid? team-id), so with
enable-backend-asserts the tests errored instead of running.

Add (assoc ::bfc/team-id (:default-team-id profile)) to each test,
mirroring the production RPC import path and the earlier fix in
a36193e54b for the other binfile v3 import tests.

Verified: 1338 tests, 6910 assertions, 0 failures.

AI-assisted-by: mistral-large-4

* 🐛 Guard the v1 import and restore the RPC check order

The version 1 importer recreated the circular library references that
the version 3 importer already skips, and a user reaches it through
`import-binfile` and the import job. It now asks
`bfc/library-link-status` before inserting each relation, so a self
link or an edge that would close a cycle is skipped and logged, and it
processes the relations in sorted order like v3 does. Closes #12052.

The version 2 importer stays out on purpose: only the developer REPL
reaches it, and a note in the code records the gap.

`link-file-to-library` keeps the shared helper but restores the order
of the base commit: the self link guard, then the edition permission
and team ownership checks, and only then the cycle check. A caller
without permission gets `not-found` again instead of the circular
error, and a rejected request no longer runs the recursive library
query first.

Tests: a v1 import that recreates a cycle, the self link error of the
command, and the permission check running before the cycle check.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Tolerate repeated file entries in the v1 binfile import

The v1 exporter unioned the requested files with their libraries
without deduplicating, so a package exported from a file inside a
library cycle (see #12052) lists that file twice in the metadata. The
v1 import created one file per entry, so the second insert hit the
primary key of `file` and aborted the whole import with "file already
exists".

The importer now reads the pair of objects of every entry to keep the
stream in sync but imports the first occurrence only, warns about the
repeats, and answers with one id per imported file.

The v1 export keeps its behavior: it has no caller in the product, and
only the tests and the developer REPL use it.

Tests: a package with a repeated entry, built with the v1 export from
a cyclic library pair, imports the file once and leaves no duplicated
rows.

AI-assisted-by: deepseek-v4.1-flash
2026-11-11 13:02:57 +01:00
andrés gonzález
0efa550a99
✨ Improve the changelog flow: dev-only bugs and feature sections (#12981)
* ✨ Keep bugs that never reached production out of the changelog

The team rarely adds `no changelog` to bugs found while building a
feature of the same release, so they end up in the changelog and make
Penpot look less stable than it is.

Add `changelog.py dev-only-bugs`. Bugs whose parent is a Feature or
Enhancement of the same milestone are sure: `--apply` labels them
`no changelog` on GitHub. Bugs under an epic of the milestone, with the
`regression` label, or whose version field names only hourly, develop,
staging, PRE or this milestone, are candidates for a human to confirm;
the anomaly report lists them as warnings. `gh.py issue` now returns
body and parent. The flow memory documents it as step 3.

Closes #12747

AI-assisted-by: claude-opus-5-5

* ✨ Split features and enhancements in the changelog flow

Features were listed together with enhancements under "✨ New
features & Enhancements", so they did not stand out. Map the Feature
issue type to its own "🎉 Features" section, keep Enhancement under
"✨ Enhancements", rename the bug section to "🐛 Bugs" and
set the order: boom, rocket, lock, tada, sparkles, bug.

The new layout applies from 2.20.0: 2.19.0 and 2.19.1 are already out
with the old one, so 2.19.x patches keep it.

Closes #12741

AI-assisted-by: claude-opus-5-5
2026-11-11 13:00:06 +01:00
Juanfran
88339e41a5 🐛 Build plugin scripts before Angular copies assets (#12815)
Generate plugin.js before Angular builds each of the eight Angular plugins.
The previous order wrote the script to src/assets after Angular had copied
assets to dist, so clean deploys published manifests without their code.

Verified clean builds include all declared manifest assets for all eight
plugins. Plugin lint, format checks and all 101 tests pass.

AI-assisted-by: gpt-6.1-sol
2026-11-11 12:51:24 +01:00
Alonso Torres
1599e096da
🐛 Fix MCP session discovery for plugins connected from the dashboard (#12865)
* 🐛 Fix MCP session discovery for plugins connected from the dashboard

With Redis-backed routing, execute_code without a sessionId timed out
with SessionDiscoveryError when the plugin ran on the dashboard. The
discovery reply schema required a string fileId and fileName, so replies
listing a session with no open file were dropped, and discovery waited
for its timeout.

Share one session schema between the plugin handshake and discovery
replies. It accepts a null file and keeps workspaceState. An unreadable
reply now fails discovery at once with an error that names the problem
and says a retry will not help. Timeout and Redis failures state their
reason.

Closes #12860

* 🐛 Stop blaming server load when MCP discovery fails on shutdown

Closing the dispatcher rejected pending discoveries with the default advice,
which blamed temporary server load. Shutdown now gives its own advice.

The dispatcher tests now cover each discovery failure path: an unreadable
reply alongside a valid one rejects with InvalidDiscoveryResponseError and
does not dispatch, a late unreadable reply leaves a settled discovery
unchanged, and timeout, Redis failure and shutdown each state their reason.
2026-11-11 12:10:34 +01:00
Alonso Torres
adda9ad908
✨ Restrict global plugins to MCP and localhost (#12949)
Plugins with global scope or a management permission (allow:global,
manage:projects, manage:teams, manage:delete) keep running outside a file
and can drive the management API. That feature is still in progress, so
only the bundled MCP plugin and plugins served from localhost, 127.0.0.1
or [::1] get it. The host check ignores letter case and a trailing dot.

The check runs at three points:

- parse-manifest throws global-not-allowed, so the plugin manager and the
  dashboard install link refuse remote global plugins with a specific
  message. Opening an installed plugin fetches its manifest again, so
  plugins stored before this change are refused too.
- start-plugin! and load-plugin! refuse to run such a plugin and warn the
  user; this also covers bundled plugins without a URL.
- has-permission? withholds the global permissions from stored remote
  plugins, so the management API rejects them.

AI-assisted-by: claude-opus-5-5
2026-11-11 10:28:11 +01:00
andrés gonzález
37f4556b0b
📚 Document MCP sign-in with OAuth (#12950)
Explain how MCP clients with OAuth support connect to the remote MCP
server with a URL that has no key: the user signs in to Penpot, allows
access, and the client receives the MCP key. Keep the key-in-URL path
for clients and instances without sign-in.

Describe the approval page, what allowing access changes, and how
regenerating or expired keys affect signed-in clients. Document
PENPOT_MCP_AUTH_URI for self-hosted instances.

Relates to #12034

AI-assisted-by: claude-opus-5-5
2026-11-11 09:54:27 +01:00
Alonso Torres
f89879cca1
🐛 Fix plugin component lookups for missing and deleted components (#12755)
* 🐛 Fix plugin component lookups for missing and deleted components

Shape.component() threw "Assert failed: (uuid? id)" on a copy whose
library is not available, because the head lookup returns a pair with a
nil component and the proxy was built anyway. It now returns null.

Shape.componentRoot() built its proxy with the component's file and main
page, so a copy on another page or file than its main got a shape that
pointed at nothing, and a copy of a missing library threw. It now
resolves the root on the shape's own file and page, like componentHead().

LibraryComponent.mainInstance() returned a proxy for the root kept in a
deleted component's objects, which is on no page, so every field read
empty and parent threw. It now returns null for a deleted component.

* 📚 Fix the variant example in the MCP instructions

The createVariantContainer example called penpot.currentPage.findAllShapes
and Shape.isMainComponent, which do not exist, so running it as written
failed. Use penpotUtils.findShapes with isComponentMainInstance instead.
2026-11-11 09:39:04 +01:00
Pablo Alba
c466c289de 🐛 Fix integrity error copying variants across files 2026-11-11 09:01:09 +01:00
Andrey Antukh
34c702bc80
✨ Add table partitioning for file-change (#12417)
* ✨ Partition file_change by file_id

Partition `file_change` by `HASH (file_id)` in 16 partitions, with the
same bounds as `file_data`, so a join between them is resolved partition
by partition. The primary key becomes `(file_id, id)`, which a
partitioned table needs, and every access filters by `file_id`.

The new table carries a `kind` column (1 snapshot, 2 revision) as a
reserve: the queries still tell them apart by the label, and `kind`
takes over when they are rewritten. The payload of a snapshot stays in
`file_data`; the `data` column is kept for reading old `legacy-db`
payloads.

The four snapshot commands require `file-id`, because it prunes the
partition. The objects GC locks and deletes by the pair, in batches. The
old table is kept as `file_change_legacy` as the way back.

The automatic migration refuses to run above 10 GiB or 2M rows and
points to the runbook; big instances partition online with that runbook,
which copies by id, follows the `updated_at` watermark in delta passes
and checks that no row of the source is missing from the target.

Closes #12416

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Address the review of the file_change partition

The production runbook promised a rollback it did not have: the section
was a sketch, and its statements could write to the table about to be
dropped. The rollback is removed, and so is every comment that described
it. `file_change_legacy` stays as a frozen copy of the previous layout.

The cutover check now aborts the transaction when a row of the old table
is missing in the new one, instead of leaving the decision to the
operator.

The five secondary indexes of the new table are built after the last
delta pass: the bulk and the delta passes only need the primary key, so
the batches no longer maintain them.

The copy of 0158 is now replayed in a test against the pre-0158 shape
with a labelled and an unlabelled row, plus an empty instance that must
leave no legacy table behind. The updated_at assertion of the snapshot
mutations is bounded by the test clock, and the repair revision stops
writing the `features` of the file, like the normal save.

AI-assisted-by: deepseek-v4.1-flash

* 📎 Fix linter issues

* 🐛 Fix stale helper call in rpc_file_snapshot_test

The suite does not load: `th/run-pending-tasks!` does not exist in
backend-tests.helpers. The pending jobs runner is `run-pending-jobs`
(no arguments), the name every other call site of the suite uses.

AI-assisted-by: inkling-2-small-free

* 📚 Align the partition change with the 2.20.0 milestone

2.19.0 is already shipped; this work targets 2.20.0. The operations
file of the partition procedure carries the 2.20.0 prefix, the
references that point at it (the migration, the runbook table and the
memory) follow, and the RPC doc/changes tags of the snapshot commands
record 2.20 as the release that makes `file-id` required.

AI-assisted-by: inkling-2-small-free

* 🐛 Fix review findings on the partition rebase

The 0157 legacy test was missed by the rebase and inserted file_change
rows without the new mandatory `kind` column, so the backend suite fails
against the migrated schema. The inserts now carry `:kind`
(snapshot-kind for the labelled rows, revision-kind for the rest).

The post-cutover verification block of the runbook terminates the
first query with a semicolon so the block pastes into psql without a
syntax error.

The guard message of the migration names the step
(0158-partition-file-change) instead of "0158", ambiguous since two
migrations share that number.

Closes #12416

AI-assisted-by: inkling-2-small-free

* 🐛 Fix the stale label in the snapshot restore test

The rename block of snapshots-crud runs before the restore block,
so the restored version is already named "label1-updated" when the
restore publishes its message. The command names the version by the
label the row has at that moment, not by the one it had when it was
created, so the test was asserting a stale value.

Assert the current label, and say why in the comment.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Split the version event tests into one test per command

The workspace version events and the plugin proxy send the file-id
of the open file on every snapshot command, because the command
needs it to prune the partition of file_change. Both tests checked
that with a doseq and t/testing blocks, so a failure named the whole
group and not the command that broke.

Give each command its own test. The workspace ones share a helper
that runs the event and returns the command and params it sends; the
plugin ones keep their own mock of rp/cmd!.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 23:27:17 +01:00
Filip Sajdak
58b349fb77 🔧 Keep the CI expect timeout in the default and ds e2e projects (#12922)
The Playwright config gives the retrying expects 40 seconds on CI and 5
seconds elsewhere, but the default and ds projects set their own expect
block for the screenshot tolerance, and a project block replaces the
top-level one instead of merging with it. On CI those projects have run
with Playwright's 5 second default.

That is too short for the first dashboard load on a busy runner, where the
dashboard module is fetched lazily: specs that wait for the dashboard
heading time out and pass on the retry (flaky), or fail after the
retries. In the shard of one recent run that was 7 flaky and 4 failed
specs out of 112, and develop itself failed the same families of specs on
several runs the same day.

Name the timeout once and repeat it in both projects. A probe config that
imports this one and polls a condition that never holds reports 40000 ms
for both projects with this change, and 5000 ms without it. The
render-wasm project already repeats its own value.

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 23:00:14 +01:00
Andrey Antukh
671ff4a05f Merge remote-tracking branch 'origin/staging' into develop 2026-11-10 20:06:46 +01:00
Andrey Antukh
8172059b2e Merge branch 'staging-2.19.x' into staging 2026-11-10 19:53:23 +01:00
Andrey Antukh
5e5722dfe7 📚 Update changelog 2026-11-10 19:24:39 +01:00
Andrey Antukh
a58ace4df4 Merge branch 'staging-2.19.x' 2026-11-10 19:18:41 +01:00
Andrey Antukh
d5c4bfb234 ✨ Add security fixes section to the changelog flow
The 🔒 section is generated from advisories instead of being written
by hand: an advisory becomes an entry when its patched_versions include
the release version, so unpublished (triage/draft) advisories are valid.

- gh.py advisories --patched-version filters on the exact token of
  vulnerabilities[*].patched_versions; the list output keeps the summary
  schema.
- changelog.py security renders the entries sorted by GHSA id, links the
  merged milestone PRs labelled "pr pt" that reference the GHSA id and
  warns when a reference has no matching patched version. The subsection
  follows 🚀 (or 💥, or the version header) and is removed
  when the version has no advisories.
- Tests cover the filter, the entry rendering, PR discovery and the
  subsection placement.
- Memories and the update-changelog skill document the contract.
- CHANGES.md: add 2.19.1 with its security fixes and the remaining bug
  fix.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 18:18:00 +00:00
Andrey Antukh
e4fcc663c0 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-10 18:09:07 +01:00
Andrey Antukh
88b09efd87 📚 Note breaking changes must not be hard-wrapped
The changelog generator copies the note lines verbatim, so a note
hard-wrapped at 80 columns ends up wrapped in CHANGES.md. Record
that the breaking-change note must keep one line per paragraph or
bullet, in both the issue and changelog workflow memories.

AI-assisted-by: inkling-2-small-free
2026-11-10 16:58:21 +00:00
Yamila Moreno
a3f50887b3 📚 Improve 2.19 documentation regarding admin-console
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-10 16:13:06 +01:00
Andrey Antukh
1db1487c7f
🐛 Require thread access when marking threads as read (#12894)
* 🐛 Require thread access when marking threads as read

mark-all-threads-as-read took any list of thread ids and wrote the read
status of the caller on each of them, without checking that the caller
can access the file that owns the thread. It was the only comment
mutation that did not check permissions, and it took the ids as an
unbounded array, so a single request could keep a database connection
and a transaction open for a long time.

The rpc now resolves the threads in one query and checks the comment
permissions of every distinct file, answering not-found when a thread
does not exist or is not accessible. Both cases answer the same error
on purpose, so the response cannot be used to tell existing threads
apart.

The status rows are now written with a single insert, keeping the
modified-at date one second in the future that makes the thread count
as read. Repeated ids are deduped, the command returns nil again like
the sibling commands with no result, and the schema limits the number
of threads per call to 500 while the dashboard sends the ids in batches
of that size.

Tests cover access rejection, unknown threads, mixed batches that write
nothing, repeated ids, empty lists, marking twice, and the frontend
batch split.

Relates to GHSA-j4vc-cmf6-jhr7

AI-assisted-by: minimax-m3.1-flash

* 🐛 Refresh the unread list when a batch fails

The dashboard sends the thread ids in batches of 500 and each batch
commits on the server. A failure in a later batch left the unread list
stale: the re-read only ran when every batch succeeded, so the threads
already marked as read stayed visible as unread.

The re-read now runs in both paths. On failure the event is emitted
before the error is propagated, so the error toast still shows and the
success toast stays limited to a full success.

The backend also answers an empty list without opening a transaction,
since there is nothing to check and nothing to write.

Relates to GHSA-j4vc-cmf6-jhr7

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:54:11 +01:00
Andrey Antukh
8510daef52
🐛 Restrict the team invitation token and list to admins (#12892)
* 🐛 Restrict the team invitation token and list to admins

The token of a team invitation is a bearer credential: whoever holds it
accepts the invitation with the role of the invitation row. The RPC that
mints it only required edition permissions, while the rest of the
invitation commands require admin, so a team editor could get the token of
any pending invitation, including the ones that grant the owner role.

The token now requires admin, and the token of an owner invitation is
restricted to the team owner: an admin cannot create nor promote an owner
invitation, so it cannot keep its token either. It reuses the error code
already used for that rule.

Minting the token now leaves an audit event named as an action, carrying
the granted role and the id of the invitation, so the extraction does not
go unnoticed.

The list of pending invitations is admin only as well: it carries the
email of the invited person and the role offered, and the dashboard was
painting those emails to any member. The frontend catches the permission
error and shows the permission notice instead, and the copy action is not
offered for an owner invitation, which the backend rejects for an admin.

The five invitation commands now answer with the same permission error,
so they speak one language.

Relates to GHSA-4mmc-j8rh-74pf

AI-assisted-by: minimax-m3.1-flash

* 🐛 Clear the cached invitations and gate the list on team admin

The pending invitations are a team admin only list. The dashboard still
offered the entry to every member and keyed the section on the invite
policy of the organization, so a team admin blocked by that policy saw
the no-permission notice although the backend lets them list.

- `fetch-invitations` answers a denied refetch with an
  `invitations-fetched` event carrying an empty list, so a demoted admin
  does not keep the cached emails and the header no longer paints the
  invite button from a stale list.
- The header tabs and the team options dropdown hide the invitations
  entry without `:is-admin`, which already includes the team owner.
- `invitations-section-content` keys the section on `:is-admin`, and the
  invite policy only gates the invite actions: resending keeps the
  `can-invite?` check, deleting and row selection only need team admin.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:53:43 +01:00
Andrey Antukh
0420db1ca9
🐛 Apply the share-link page scope to the comment queries and writes (#12888)
* 🐛 Apply the share-link page scope to the comment queries and writes

A share link can be restricted to a subset of the pages of a file, and
the comment queries enforced that scope. But the RPC that resolves the
profiles of the commenters authorized the request with
check-comment-permissions! and dropped the returned perms, so its query
filtered only by file id. The holder of a page scoped link got the id,
email, fullname and photo of every profile that commented on any page
of the file, including the pages the owner left out of the link.

The query now takes the pages of the share link and only selects the
authors of the threads on them. The profile of the caller is still
added, so the viewer keeps its own avatar.

The scope was missing as well on the comment writes that take a page id
or a thread id: create-comment-thread accepted a page outside the link,
create-comment accepted a thread of an unshared page, and
update-comment-thread, update-comment-thread-position and
update-comment-thread-frame accepted a thread of an unshared page. They
now raise not-found, the same way the comment queries already did.
Replying to a thread of an unshared page also sent the emails to the
thread participants.

The page scope rule now lives in one place: share-link-pages returns
the pages granted by a share link and check-page-scope raises not-found
when a page is outside of it, so get-comment-threads reuses the helper
instead of checking the pages inline.

The docstring of get-profiles-for-file-comments said that it returns
the participants of all the comment threads of the file, which is not
true anymore for a share link. The docs are public, so the text and
the changes list are updated.

The tests of the share link page scope are hardened: the setup asserts
that the comment threads are created, the membership case has a second
commenter so it can no longer pass for the wrong reason, and the link
without pages, the allowed paths of the thread updates and the
who-comment=team denial are covered.

Relates to GHSA-89cq-3wv7-987r

AI-assisted-by: minimax-m3.1-flash

* 🐛 Enforce the share-link page scope on the remaining comment writes

Check the page scope of the share link before mutating in the four
comment writes that still skipped it: update-comment-thread-status,
update-comment, delete-comment-thread and delete-comment. The check
runs before the ownership checks, so an out-of-scope write fails with
not-found and the response does not reveal that the comment or thread
exists. Load the file in create-comment only after the permissions and
page scope checks, matching create-comment-thread, so a rejected reply
does not load the file data.

Cover the denial and the in-scope paths of the four commands, and the
fail-closed case of a link without pages on the comment reads. Move
the outsider thread creation after the profiles assertion so the
caller-union assertion fails when the union is removed. Document the
page scope invariant in the backend permissions memory.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:53:07 +01:00
Andrey Antukh
4874da450f
🐛 Escape SVG values and sanitize every SVG upload (#12887)
* 🐛 Escape SVG values and sanitize every SVG upload

The SVG sanitizer removed dangerous elements and attributes by name and
then wrote the tree back with clojure.xml/emit, which escapes nothing.
A payload hidden in an attribute value or in a text node became live
markup again: an id of "x' onload='alert(1)'" was stored as a real
onload handler, and an entity encoded script tag was stored as a real
script element.

Write the filtered tree with a serializer that escapes text and
attribute values, and route every path that stores an uploaded SVG
through it. The thumbnail writers (create-file-object-thumbnail and
create-file-thumbnail) and the organization logo stored the upload
untouched, so they could plant live markup in a public bucket.

The filter only knows about SVG elements, so it also accepted
documents that are not SVG at all: a page with an iframe or a meta
refresh was stored as an image. The sanitizer now rejects any document
whose root is not an svg element in the SVG namespace, and it compares
the scheme of an url the way the browser does, so a scheme split with
a tab or a line break no longer slips through the javascript: filter.
The attributes are written sorted by name, because the parser hands
them over in the reverse order of the document and the output changed
on every sanitization.

The profile picture of an identity provider is not stored either when
it is an SVG: the avatar is always rendered as an image, so an SVG
would only add the risk of keeping live markup in the storage. The
dimensions of an SVG are now read from its local name, so an SVG with
a namespace prefix can be uploaded like any other.

The logo had no size limit and the sanitizer reads the whole file, so
it now honors the media max file size.

The output shape changes with it: no line breaks, double quotes, and
the single quote of an attribute value escaped as an entity. The hash
of every stored SVG object changes with it.

The media tests use cuerdas.core instead of clojure.string, as the rest
of the codebase does.

Relates to GHSA-qwhj-rchj-58mg

AI-assisted-by: minimax-m3.1-flash

* 🐛 Drop non-SVG namespace subtrees in the SVG sanitizer

The sanitizer only checked the namespace of the root, so a child could
rebind xmlns to XHTML and smuggle live HTML (iframe/srcdoc/meta) past the
blocklist. The serializer escapes attribute values, but the browser decodes
them again, so srcdoc stays live.

Track the namespace bindings through the recursion and drop any element
that is not in the SVG namespace, with its subtree. The root is validated
up front with the same model, so a document that is not an svg document is
rejected with an honest hint instead of a misleading one.

The default namespace is stored under a namespaced key, because a prefix
literally named `default` collided with the internal key and re-opened the
same hole through a prefixed binding. The attribute filter now keeps
`xmlns`/`xmlns:*` declarations, which are not event handlers, so a prefix
named like one does not end up undeclared in the output.

The blocklist also drops HTML-only elements written in the svg namespace
(iframe, meta, embed, link, base).

Add regression tests for the child rebinding, for the prefixed variants,
for the `default` prefix, for the root local name, for the
create-file-media-object-from-url path (mocked download), and for the
HTML-only tags. Assert double-quoted attributes in the clean content test.

Relates to GHSA-qwhj-rchj-58mg

AI-assisted-by: longcat-2.5-free
2026-11-10 15:52:28 +01:00
Andrey Antukh
c39e3e8301
✨ Strictly validate uploads against file bytes on disk (#12801)
* ✨ Strictly validate uploads against file bytes on disk

Sniff the real content type of every upload with simplemagic magic
bytes instead of trusting the client-supplied mtype header. Declared
types outside the allowlist keep the not-allowed code; allowed
declarations whose bytes differ are rejected as mismatch. Covers SVG
by probing its root tag over leading bytes (magic never matches XML
text), TTF/OTF by magic name and WOFF/WOFF2 by four-cc.

Read the real file size from disk instead of trusting the declared
:size. Files over the limit are rejected even when the declared size
lies downwards, and small files pass even when it lies upwards.

Run the path check first everywhere so forged server paths keep the
params-validation code. Recheck from-url downloads after the write,
with a declared-only fast fail at header time. Thread the assembled
path through font validation.

Refresh the affected tests around real and sparse fixture files,
including a PNG-as-font chunked-upload rejection case.

Closes #12796

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Accept BOM-prefixed SVG uploads and cover WOFF/WOFF2/OTF/WEBP

Address review comments on #12801:

- SVG files saved with a UTF-8 BOM were rejected as
  :media-type-mismatch: read-head kept U+FEFF and the <svg probe
  never ran. A leading BOM is now stripped before the probe, and a
  generic XML sniff (application/xml, text/xml) is treated as
  "no magic evidence" for declared image/svg+xml, so the root-tag
  probe decides whether the file opens with an <svg root.
- sniff-match?/svg-file? docstrings claimed the magic database never
  matches SVG; simplemagic 1.17 does match an SVG with an XML prolog
  as image/svg+xml. Reworded to match reality.
- Added matching-pass tests for WOFF, WOFF2, OTF and WEBP (they hit
  the read-four-cc, magic-name and direct magic branches) and three
  BOM SVG regression cases.

AI-assisted-by: inkling-2-small-free
2026-11-10 15:51:48 +01:00
Elena Torró
7f0b62b994
🐛 Keep drop shadows out from under translucent shapes (#12904)
* 🐛 Keep drop shadows out from under translucent shapes at every zoom

Cut the shape out of its drop shadow, as CSS box-shadow does, and stop
large shadows from being cut off at high zoom.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep drop shadows out from under translucent shapes in SVG and PDF

Match the canvas: clip the shape out of its drop shadow on export.

AI-assisted-by: claude-opus-5-5
2026-11-10 15:05:08 +01:00
Juanfran
e1f924cf00 🐛 Send organization invitations to unregistered emails (#12937)
send-template drops nil values before enqueuing the mail job, but the
invite-to-organization schema required the :user-name key. Inviting an
email with no Penpot account left :user-name nil, the key was dropped
and the job failed its schema check, so the invitation email was never
sent. :user-name is now optional in invite-to-organization and in
renewal-notice, which has the same shape.

New test renders both templates through the queue with a nil
:user-name.

AI-assisted-by: claude-opus-5-5
2026-11-10 14:12:27 +01:00
penpot-renovate[bot]
c9d3cbb383
⬆️ update GitHub Actions (#12898)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker-devenv.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker-devenv.yml)
- zizmor 1.32.0 (.github/workflows/lint-workflows.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-10 13:48:10 +01:00
Miguel de Benito Delgado
cbd4c9634c
🔧 Use correct :cache-root in shadow-cljs.edn (#12883) 2026-11-10 13:47:50 +01:00
Marina López
4e8f05171e
🐛 Forward attribute deletion events to admin console (#12932) 2026-11-10 13:47:13 +01:00
Alonso Torres
79d4936cae
⬆️ Release plugins and mcp (#12934)
* ⬆️ Updated plugins release to 1.6.0

* ⬆️ Updated mcp release to 2.19.1
2026-11-10 13:44:51 +01:00
Juanfran
ae24118caf
🐛 Send organization invitations to unregistered emails (#12937)
send-template drops nil values before enqueuing the mail job, but the
invite-to-organization schema required the :user-name key. Inviting an
email with no Penpot account left :user-name nil, the key was dropped
and the job failed its schema check, so the invitation email was never
sent. :user-name is now optional in invite-to-organization and in
renewal-notice, which has the same shape.

New test renders both templates through the queue with a nil
:user-name.

AI-assisted-by: claude-opus-5-5
2026-11-10 13:44:26 +01:00
Andrés Moya
10b08d5224 🐛 Improve sync of layout item attributes in components (#12851)
* 🐛 Preserve layout item touched groups when switching variant

* 🔧 Add systematic tests to check layout items component sync

* 🔧 Add small updates after comments

* 🔧 Add tests for partial overrides
2026-11-10 13:41:54 +01:00
penpot-renovate[bot]
2e6c799bba
⬆️ Update dhi.io/node image to f943257 (#12931)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-10 11:45:57 +01:00
Filip Sajdak
ee3e2a0e61
🔧 Keep the CI expect timeout in the default and ds e2e projects (#12922)
The Playwright config gives the retrying expects 40 seconds on CI and 5
seconds elsewhere, but the default and ds projects set their own expect
block for the screenshot tolerance, and a project block replaces the
top-level one instead of merging with it. On CI those projects have run
with Playwright's 5 second default.

That is too short for the first dashboard load on a busy runner, where the
dashboard module is fetched lazily: specs that wait for the dashboard
heading time out and pass on the retry (flaky), or fail after the
retries. In the shard of one recent run that was 7 flaky and 4 failed
specs out of 112, and develop itself failed the same families of specs on
several runs the same day.

Name the timeout once and repeat it in both projects. A probe config that
imports this one and polls a condition that never holds reports 40000 ms
for both projects with this change, and 5000 ms without it. The
render-wasm project already repeats its own value.

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 11:33:42 +01:00
Filip Sajdak
90d98b1989
🐛 Call the team hook on every render of the team container (#12920)
team-container* called mf/deref only after returning early for a team id
that is not a uuid. When the same component rendered once without a team
id and then with one, the number of hooks changed between the two renders
and React threw "Rendered more hooks than during the previous render"
(error #310), which sent the user to the internal error page. The reverse
order throws "Rendered fewer hooks than expected".

Call the hook on every render and keep the uuid check in the condition
that builds the providers.

Two tests mount the component in jsdom and render it with a team id that
arrives and with one that leaves; both fail without this change.

Closes #12728

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 11:32:58 +01:00
Alonso Torres
4fb0ea9eee
🐛 Give unique values to same-name components combined as variants (#12881)
When components whose name matches the variant container are moved into
it, removing the container name leaves an empty value. Combining several
components named the same produced "Value 1" for the first variant and
empty values for the rest, so variants shared the same combination.

Assign the first unused "Value N" to the last property when all computed
values are blank, tracking values given within the same batch.

Combine moves the remaining mains into the container one at a time, in
the combine order, so the generated values follow that order instead of
the layer order.

Closes #12878
2026-11-10 09:16:08 +01:00
Eva Marco
72575e5c05
🐛 Make Add to an organization a keyboard button (#12722)
"Add to an organization" in the team settings opens a modal but was an
<a> without href: the keyboard could not reach it. It is now a
<button type="button"> that keeps its text-link look (accent color,
medium weight, underline on hover, the same box), so Tab reaches it and
Enter or Space opens the modal.

New visual test "Shows the Add to an organization link", with fixtures
for a team in no organization and an organization the user belongs to:
0 px of difference at rest and on hover, and 0 px on the whole settings
page.

AI-assisted-by: claude-opus-5-5
2026-11-10 08:33:04 +01:00
Luke Lau
cd1ece4beb
⚡ Keep the translation flag on undo and redo of a move (#12870)
* ⚡ Keep the translation flag on undo and redo of a move

Previously undos and redos always forced a component sync + thumbnail
rerender for every component touched even if the underlying commits were just
translations. This propagates the translation flag to undos and redos and
propagates it to commit-changes. An entry merged from several undos/redos is
a translation only if all of its undos/redos are translations.

This reduces a UI freeze when undoing a move of a board with 453 components
from 2.42s -> 0.22s.

AI-assisted-by: claude-sonnet-5-5

* :translation? -> :translation

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 20:47:42 +01:00
penpot-renovate[bot]
bdf16bcfda
⬆️ Update DHI base images (#12897)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@5c9d8c2 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@e8db455 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@7f8db65 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-09 17:39:54 +01:00
Luke Lau
6846afd08c
⚡ Send objects-map pages to the worker without a second encode (#12811)
* ⚡ Send objects-map pages to the worker without a second encode

When switching between pages on a large file e.g Material Design Icons, there's
some UI freeze from :index/initialize encoding object map shapes to send to the
worker. An objects map already stores the encoded JSON for each shape, and
`encode-str` was wrapping those strings in a second layer of JSON on the main
thread.

We can just transmit the already encoded JSON in a js/Map alongisde the
payload, similar to :blob, so structured clone just copies the strings verbaitm
without re-encoding. To do this this PR adds two functions to ObjectsMap that
converts to and from a js/Map.

The "Main components" page of the Material Design Icons has ~30k objects, so
this reduces the freeze when switching pages from 306ms -> 189ms (1.63x) on a
release build.

AI-assisted-by: claude-sonnet-5-5

* Split property tests out into new deftest

---------

Signed-off-by: Luke Lau <luke_lau@icloud.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:35:20 +01:00
Luke Lau
55360c52f5
✨ Cache the transit reader when decoding strings (#12704)
* ✨ Cache the transit reader when decoding strings

On the cljs side, we create a new transit reader on every call to decode-str.
This turns out to have a fair bit overhead in both transit-cljs and transit-js,
since it has to merge the handlers, call clj->js, allocate a new ReadCache,
etc.

By reusing it the measured decode time goes from ~58us -> ~41us, so about 1.4x
faster. This particularly helps for some operations like whenever a component
is edited and every object needs to be decoded, see
https://github.com/penpot/penpot/issues/9344, although they could probably be
made more efficient in other ways.

The reader usually clears its read cache after a successful read, but whenever
it throws it hangs around. So in this case we need to reset the cache
ourselves, otherwise the next read will be from a clobbered cache.

The tests were written by Claude Sonnet 5.5, and should cover the clobbering
behaviour above.

Signed-off-by: Luke Lau <luke@igalia.com>

* Keep one transit reader per type and rebuild it on add-handlers!

---------

Signed-off-by: Luke Lau <luke@igalia.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:34:24 +01:00
Luke Lau
3de7e62aa9
⚡ Don't send file data contents back from duplicate-file (#12911)
The client calls the duplicate-file command in two places, the dashboard and in
the plugin API. In both places the :data is dropped via `(dissoc file :data)`,
so the client never uses that field.

However the file data takes a while to send over the wire and decode. When
duplicating penpot-app there's a roughly 1s freeze whenever the response is
returned from the server with the file data. We can avoid it by dropping the
data from the response.

I also did a quick check and neither the MCP server nor any public client of
the API on GitHub seem to use the :data field.

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:23:39 +01:00
Belén Albeza
fb7468c3f5
🔧 Make all CI jobs use Rust 1.99 (#12903)
* 🔧 Bump Rust version of the devenv image to 1.99

* 🔧 Run the pinned Rust setup from the lint and format scripts

./build and ./test switch to the Rust version in render-wasm/_build_env
through setup, but ./lint did not, and formatting called cargo fmt
directly. In CI both ran on the devenv image's default toolchain.

Make ./lint run setup and add a ./fmt script that does the same, so all
render-wasm entry points use the pinned version. CI and scripts/ci now
call ./fmt instead of cargo fmt.

AI-assisted-by: claude-opus-5-5
2026-11-09 16:23:47 +01:00
Miguel de Benito Delgado
01191051c7
🐛 Replace stale atlas pixels when zooming (#12884)
Copy finished tiles with BlendMode::Src so transparent pixels replace what the atlas slot held before
2026-11-09 14:28:13 +01:00
Miguel de Benito Delgado
33602b9114
🔧 Allow skipping renderer build setup via env. var (#12882)
Set PENPOT_WASM_PREPARED=1 to speed up repeated builds
2026-11-09 12:53:18 +01:00
Andrey Antukh
4b5dd51b39 📚 Add 2.20.0 changelog section
Populate the 2.20.0 release notes from the milestone: 19 bugs fixed
and 31 new features & enhancements, classified by issue type with the
community contribution attributions and the merged PR references. No
💥 or 🚀 sections: no issue carries the breaking change or
release highlight labels. No GHSA entries: no published advisory lists
2.20.0 as patched version.

AI-assisted-by: inkling-2-small-free
2026-11-09 11:19:50 +00:00
Luke Lau
943e63a99e
⚡ Group color updates in selections (#12875) 2026-11-09 12:08:36 +01:00
752 changed files with 63527 additions and 10240 deletions

View File

@ -337,28 +337,32 @@ small. To delegate without switching models, delegate to `general`.
report with your intent: "investigate this and find the possible cause", report with your intent: "investigate this and find the possible cause",
"investigate and tell me where this points", or "does this still apply?". "investigate and tell me where this points", or "does this still apply?".
Explore until you and the agent roughly agree on the problem. Explore until you and the agent roughly agree on the problem.
2. **Write the plan.** Run `/make-a-plan` — it executes in Build mode. 2. **Write the plan.** Invoke `@make-a-plan`. Skills use the current agent;
If you need to step in and answer something yourself, press TAB to select Build mode first when the skill needs to write files. Use Plan mode
leave Build mode. Use Plan mode only when you want a hard guarantee only when you want a hard guarantee that the agent modifies no file under
that the agent modifies no file under any circumstance. If you any circumstance. If you
explored with a weaker model but want a stronger one to write the explored with a weaker model but want a stronger one to write the
plan, switch models first or delegate: plan, switch models first or delegate:
`/make-a-plan delegate to @engineer-glm`. `@make-a-plan delegate to @engineer-glm`.
3. **Iterate on the plan.** The plan is saved to `.agents/plans/`, so you 3. **Iterate on the plan.** The plan is saved to `.agents/plans/`, so you
never depend on LLM memory: read the file directly, or run `/review-plan` never depend on LLM memory: read the file directly, or invoke `@review-plan`
for a second opinion (delegation works here too). Complex plans deserve a for a second opinion (delegation works here too). It applies clear plan
review; simple ones can skip it. improvements and discusses consequential decisions one by one before
4. **Execute.** Run `/implement-plan`. It first prints the full picture — applying them. Complex plans deserve a review; simple ones can skip it.
4. **Execute.** Invoke `@implement-plan`. It first prints the full picture —
whether it will create an issue and a branch, the execution style, and a whether it will create an issue and a branch, the execution style, and a
task checklist — and waits for your go-ahead. Say "step by step" to stop task checklist — and waits for your go-ahead. Say "step by step" to stop
after each task (one commit per task) so you can verify as it goes; after each task (one commit per task) so you can verify as it goes;
the default runs all tasks with one final commit. the default runs all tasks with one final commit. For a plan based on a
5. **Land the work.** When it finishes, either push yourself and run code review, it then asks whether to have the original reviewer verify the
`/create-pr`, or loop `/review-code` → `/make-a-plan` → corrections. If that session is unavailable, it tells you and uses a new
`/implement-plan` until the findings are addressed, then push and reviewer with the review and implementation context.
`/create-pr`. Nothing pushes for you — you always push from your shell. 5. **Land the work.** When it finishes, either push yourself and invoke
`@create-pr`, or loop `@review-code` → `@make-a-plan` →
`@implement-plan` until the findings are addressed, then push and invoke
`@create-pr`. Nothing pushes for you — you always push from your shell.
> Note: `/implement-plan` checks the current branch. On a base branch > Note: `@implement-plan` checks the current branch. On a base branch
> (`main`, `develop`, `staging`) it creates a GitHub issue and a branch > (`main`, `develop`, `staging`) it creates a GitHub issue and a branch
> `issue-NNNN`; on an existing feature branch it continues there and > `issue-NNNN`; on an existing feature branch it continues there and
> creates nothing. The pre-run summary tells you which applies. Read the > creates nothing. The pre-run summary tells you which applies. Read the
@ -367,14 +371,14 @@ small. To delegate without switching models, delegate to `general`.
### Big feature with multiple plans ### Big feature with multiple plans
When the work is too large for a single plan, tell `/make-a-plan` When the work is too large for a single plan, tell `@make-a-plan`
up front: produce a high-level roadmap where each task will get its up front: produce a high-level roadmap where each task will get its
own execution plan, and the roadmap doubles as the progress tracker. own execution plan, and the roadmap doubles as the progress tracker.
From there the flow mirrors the issue flow above, one level down: From there the flow mirrors the issue flow above, one level down:
take each roadmap task in turn, write its own plan (`/make-a-plan`, take each roadmap task in turn, write its own plan (`@make-a-plan`,
delegating when it helps), review it when the task is complex delegating when it helps), review it when the task is complex
(`/review-plan`), implement it (`/implement-plan`), and mark progress (`@review-plan`), implement it (`@implement-plan`), and mark progress
on the roadmap as you land each piece. on the roadmap as you land each piece.
### Plan file naming (base + derivatives) ### Plan file naming (base + derivatives)
@ -392,7 +396,7 @@ keeps everything adjacent in `ls`:
`2026-09-20-upload-pipeline-roadmap--task-01-chunk-upload.md` `2026-09-20-upload-pipeline-roadmap--task-01-chunk-upload.md`
- Chained: `...--task-02-gc--review-01.md` - Chained: `...--task-02-gc--review-01.md`
While a plan is still unimplemented, `/make-a-plan` revises it in While a plan is still unimplemented, `@make-a-plan` revises it in
place. Once implemented and reviewed, it writes a new followup file. place. Once implemented and reviewed, it writes a new followup file.
Full rules live in the `planner` skill. Full rules live in the `planner` skill.
@ -401,11 +405,10 @@ Full rules live in the `planner` skill.
Every plan carries `Status: draft | reviewed | done` plus an Every plan carries `Status: draft | reviewed | done` plus an
append-only `Review Log` (UTC ISO 8601, one line per entry): append-only `Review Log` (UTC ISO 8601, one line per entry):
- `draft` — fresh from `/make-a-plan`. - `draft` — fresh from `@make-a-plan`.
- `reviewed` — review feedback incorporated. `/review-plan` stays - `reviewed` — review feedback incorporated. `@review-plan` applies clear
read-only; only your explicit "apply" makes `/make-a-plan` apply improvements and agreed decisions, then updates the status and log.
the changes, flip the status, and log one line. - `done` — `@implement-plan` closes it on completion, logging the
- `done` — `/implement-plan` closes it on completion, logging the
issue URL when one exists (never commit hashes). issue URL when one exists (never commit hashes).
A plan approved with no changes goes `draft` → `done` directly. A plan approved with no changes goes `draft` → `done` directly.
@ -505,7 +508,6 @@ JSON, REPL access, and so on.
| [`ripgrep`](skills/ripgrep/SKILL.md) | Fast content search with regular expressions. | | [`ripgrep`](skills/ripgrep/SKILL.md) | Fast content search with regular expressions. |
| [`jq-json-processor`](skills/jq-json-processor/SKILL.md) | Slice, filter, and reshape JSON output. | | [`jq-json-processor`](skills/jq-json-processor/SKILL.md) | Slice, filter, and reshape JSON output. |
| [`nrepl-eval`](skills/nrepl-eval/SKILL.md) | Run Clojure or ClojureScript code in the live REPL sessions (backend and frontend). | | [`nrepl-eval`](skills/nrepl-eval/SKILL.md) | Run Clojure or ClojureScript code in the live REPL sessions (backend and frontend). |
| [`taiga`](skills/taiga/SKILL.md) | Look up Penpot issues, user stories, and tasks in Taiga. |
| [`testing`](skills/testing/SKILL.md) | The repo's testing rules and TDD workflow, loaded before writing tests. | | [`testing`](skills/testing/SKILL.md) | The repo's testing rules and TDD workflow, loaded before writing tests. |
| [`local-ci`](skills/local-ci/SKILL.md) | Run CI-style lint, test, and format checks for the modules you touched with `scripts/ci`, and read the logs when they fail. | | [`local-ci`](skills/local-ci/SKILL.md) | Run CI-style lint, test, and format checks for the modules you touched with `scripts/ci`, and read the logs when they fail. |
| [`security-and-hardening`](skills/security-and-hardening/SKILL.md) | Security checks for code that handles user input, auth, or external services. | | [`security-and-hardening`](skills/security-and-hardening/SKILL.md) | Security checks for code that handles user input, auth, or external services. |
@ -516,11 +518,11 @@ JSON, REPL access, and so on.
### A typical round ### A typical round
1. `/make-a-plan` — you get a plan and a saved file in `.agents/plans/`. 1. `@make-a-plan` — you get a plan and a saved file in `.agents/plans/`.
2. `/review-plan` — a second opinion; approve or request changes. 2. `@review-plan` — a second opinion; approve or request changes.
3. `/implement-plan` — the code gets written and committed. Starting from a base branch, it also opens the GitHub issue and the `issue-NNNN` branch; the plans that follow continue on that same branch. 3. `@implement-plan` — the code gets written and committed. Starting from a base branch, it also opens the GitHub issue and the `issue-NNNN` branch; the plans that follow continue on that same branch.
4. `/review-code` — a reviewer checks the commit. 4. `@review-code` — a reviewer checks the commit.
5. `/create-pr` — the branch goes up as a pull request. 5. `@create-pr` — the branch goes up as a pull request.
Every step also works on its own, and you can always say what you want Every step also works on its own, and you can always say what you want
in plain words — the agents pick the right skill from what you say. in plain words — the agents pick the right skill from what you say.

View File

@ -1,246 +1,95 @@
--- ---
name: code-review-criteria name: code-review-criteria
description: Code review criteria — the five review axes, core principles, severity format, and verdict for reviewing code changes. Loaded by the reviewer subagent of the review-code flow. Not a user-facing flow — to review code, use the review-code flow. description: Code review criteria for correctness, clarity, architecture, security and performance. Loaded by the review-code subagent.
--- ---
# Code Review Criteria and Quality # Code Review Criteria
## Overview Assess whether the change improves the codebase and is safe to merge. Review
the diff in context, its intent and relevant tests. Follow project conventions
Multi-dimensional code review with quality gates. Every change gets reviewed before merge — no exceptions. Review covers five axes: correctness, readability, architecture, security, and performance. and module memories. Do not block on personal preferences or invent problems.
Apply each axis in proportion to the change and its risks.
**The approval standard:** Approve a change when it definitely improves overall code health, even if it isn't perfect. Perfect code doesn't exist — the goal is continuous improvement. Don't block a change because it isn't exactly how you would have written it. If it improves the codebase and follows the project's conventions, approve it.
Follow `review-code` for delegation and its read-only contract. Never run tests,
## When to Use builds, linters or other state-changing checks during a review or follow-up.
Inspect prior test results or CI evidence when available. Identify their source
- The reviewer subagent of the `review-code` flow loads this skill to perform and scope; if evidence is missing, stale or inaccessible, say what is
the review of a code change. unverified. Never claim a suite passed without evidence.
- To review code, always go through the `review-code` flow — never load this
skill directly for that. This is the criteria reference, not the flow. ## Review axes
## Core Principles ### Correctness and tests
These principles underpin every axis. When in doubt, default to them. Check requirements, relevant boundaries, error paths, concurrency and state.
Use the module's contracts to decide where validation belongs; do not demand
- **DRY (Don't Repeat Yourself):** Every piece of knowledge has one authoritative representation. If the same logic appears in two places, extract it into a shared helper, model, or type. Reviewers: flag duplicated logic as a required change — it's not "just similar," it's drift that will diverge. redundant guards inside functions with established preconditions. Check that
- **KISS (Keep It Simple, Stupid):** The simplest solution that works is the best solution. Complexity must earn its place. Reviewers: if you need more than one sentence to explain what a piece of code does, it's too complex — push for simplification before merge. new functionality has tests covering its behavior and important failure
- **YAGNI (You Aren't Gonna Need It):** Don't add abstractions, hooks, or generalizations for hypothetical future use cases. Generalize on the third occurrence, not the first. Reviewers: delete speculative generality. cases, using the appropriate layer and project testing guidance. Missing
- **Don't invent problems:** Do not manufacture issues to produce more feedback. Every finding must be a real risk, a real readability barrier, or a real architectural concern — not a hypothetical or a stylistic preference disguised as a problem. required tests are findings; documentation-only changes need not add tests.
Do not infer that a test failed before a fix without evidence.
## The Five-Axis Review
### Readability and simplicity
Every review evaluates code across these dimensions.
Check names, control flow and maintainability against existing conventions.
### 1. Correctness Flag duplication when it creates a concrete risk of drift or maintenance
cost, not merely because code appears more than once. In tests, prefer
Does the code do what it claims to do? self-contained, readable cases even when they repeat setup. Suggest the
simplest remedy that preserves behavior.
- Does it match the spec or task requirements?
- Are edge cases handled (null, empty, boundary values)? ### Architecture and compatibility
- Are error paths handled (not just the happy path)?
- Does it pass all tests? Are the tests actually testing the right things? Check module boundaries, existing abstractions, contracts and unnecessary
- Are there off-by-one errors, race conditions, or state inconsistencies? complexity. Review surrounding code when needed to understand the change.
Do not label a compatibility shim dead code without checking its purpose.
### 2. Readability & Simplicity For file formats or data model changes, consult the relevant compatibility,
migration and feature memories; for Clojure, logging and translations, read
Can another engineer (or agent) understand this code without the author explaining it? the corresponding guidance when relevant.
- Are names descriptive and consistent with project conventions? (No `temp`, `data`, `result` without context) ### Security
- Is the control flow straightforward (avoid nested ternaries, deep callbacks)?
- Are there any "clever" tricks that should be simplified? Evaluate relevant input boundaries, authorization, secrets, data handling and
- **KISS check:** Is this the simplest approach that solves the problem? A 20-line straightforward function beats a 5-line clever one that requires a comment to explain. external integrations. Consult project security guidance for the affected
- Could this be done in fewer lines? (1000 lines where 100 suffice is a failure) technology; do not assume a generic framework or tool applies to this module.
- Are abstractions earning their complexity? (Don't generalize until the third use case)
- Is a new conditional bolted onto an unrelated flow? Push the logic into its own helper, state, or policy. ### Performance
- Do repeated conditionals on the same shape appear? They signal a missing model or dispatcher.
- Are there dead code artifacts: no-op variables, backwards-compat shims, or `// removed` comments? Assess relevant hot paths, resource bounds, data access and scaling. Explain
the workload or conditions in which a problem matters; do not flag speculative
### 3. Architecture optimizations without a concrete risk.
Does the change fit the system's design? ## Scope and dependencies
- Does it follow existing patterns or introduce a new one? If new, is it justified? Review relevant generated files or lockfile changes when they affect the
- Does it maintain clean module boundaries? contract, dependencies or security; do not skip a lockfile-only dependency
- **DRY check:** Is there existing code that does the same thing? Reuse the canonical helper instead of writing a near-duplicate. If two branches do nearly the same thing, collapse them. change. Check dependency changes against available changelogs, project tools
- Are dependencies flowing in the right direction (no circular dependencies)? and relevant lockfile diffs. Treat size as a signal to assess reviewability,
- Is the abstraction level appropriate (not over-engineered, not too coupled)? not an automatic finding. Identify unrelated work in the target diff when it
- Does this refactor reduce complexity or just relocate it? Count the concepts a reader must hold. Prefer the restructuring that makes whole branches disappear over one that re-centralizes the same logic. Prefer deleting an abstraction to polishing it. materially impedes review; do not review unrelated working-tree changes.
- Is feature-specific logic leaking into a shared or general-purpose module?
- Are type boundaries explicit? Question gratuitous `any`/`unknown`/optional/casts and silent fallbacks. ## Findings and verdict
- **Structural remedies:** When you flag a problem, propose the move — not just the problem. Replace conditionals with dispatchers, collapse duplicate branches, separate orchestration from business logic, extract helpers, split large files. Prefer the remedy that removes moving pieces over one that spreads the same complexity around.
Assign each finding a stable ID (`F1`, `F2`, …), independent of severity.
### 4. Security Never renumber it within the review or follow-up; severity may change as
evidence changes. New findings receive unused IDs.
For detailed security guidance, see `security-and-hardening`.
For each finding state severity, file and location, concrete circumstances,
- Is user input validated and sanitized? impact, remedy and the tests needed to verify it; explain when no new test is
- Are secrets kept out of code, logs, and version control? needed. If a consequential decision requires user input, give the real options,
- Is authentication/authorization checked where needed? consequences and recommendation within that finding, without pausing to ask.
- Are SQL queries parameterized (no string concatenation)?
- Are outputs encoded to prevent XSS? - **Critical:** security incident, data loss or comparably severe breakage;
- Are dependencies from trusted sources with no known vulnerabilities? blocks merge.
- Is data from external sources (APIs, logs, user content, config files) treated as untrusted? - **High:** a significant defect or missing required test; must be addressed.
- **Medium:** a substantive issue worth fixing but not a merge blocker.
### 5. Performance - **Low / Suggestion:** an optional improvement with a concrete benefit.
- Any N+1 query patterns? Base severity on impact and project requirements. Missing tests required for
- Any unbounded loops or unconstrained data fetching? new functionality must be addressed; do not invent a test requirement for
- Any synchronous operations that should be async? changes where none applies.
- Any unnecessary re-renders in UI components?
- Any missing pagination on list endpoints? Return a short summary, findings ordered by impact and a verdict with the
- Any large objects created in hot paths? relevant IDs: **Approve**, **Approve with minor changes**, or **Request
changes**. State verification evidence and limitations. Do not duplicate
## Review Process remedies or tests in separate sections. If no findings exist, say so. In
follow-up, retain original IDs and
1. **Understand the intent** — What is this change trying to accomplish? What spec or task does it implement? report resolved, pending and not-addressed findings as `review-code` requires.
2. **Review tests first** — Tests reveal intent and coverage. Do they test behavior, not implementation details? Are edge cases covered?
3. **Review the implementation** — Walk through each file with the five axes in mind.
4. **Categorize findings** — Label every comment with its severity:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| **Critical:** | Blocks merge | Security vulnerability, data loss, broken functionality |
| **High:** | Required change | Must address before merge |
| **Medium:** | Should fix | Strongly recommended, not a blocker |
| **Low:** | Minor, optional | Author may ignore — formatting, style preferences |
| **Suggestion:** | Worth considering | Not required, but improves the code |
**Finding IDs.** Assign every finding a stable identifier by severity class: `R1`, `R2`, … for required findings (Critical, High), `M1`, `M2`, … for minor findings (Medium, Low), and `S1`, `S2`, … for suggestions. Number within each class in review order. Use the ID everywhere the finding is mentioned — in section headers, in the verdict, in follow-up discussion. Never renumber within a review. Example: `**R2** — `app/validate.cljs:42` — duplicate branch logic…`.
For each finding, describe the circumstances under which it could fail: specific inputs, load conditions, timing, or user actions that trigger the problem. "This crashes when input is null" is actionable; "this might crash" is not.
Lead with what matters: correctness and security first, then structural issues, then everything else. A few high-conviction comments beat a long list.
5. **Verify the verification** — What tests were run? Did the build pass? Was the change tested manually? Screenshots for UI changes?
## Review Output
Structure every review using this format:
### Summary
Briefly explain what the code does and give an overall assessment.
### Critical and High-Priority Issues
List problems that could cause security incidents, data loss, crashes, incorrect behavior, or major performance degradation. Each finding gets its unique ID (`R1`, `R2`, …). For each: state the severity, identify the file/function/code section, explain why it's a problem, describe failure circumstances, and provide a concrete improvement with corrected code when useful.
### Other Findings
List medium- and low-priority issues, including maintainability and design concerns. Use `M1`, `M2`, … for Medium/Low and `S1`, `S2`, … for Suggestions.
### Suggested Refactoring
Provide focused code changes or revised snippets. Preserve existing behavior unless a behavior change is explicitly justified.
### Testing Recommendations
Identify missing tests and describe specific test cases, including edge cases and failure scenarios.
### Positive Observations
Mention implementation choices that are clear, safe, efficient, or well designed. This is not fluff — it reinforces good patterns and tells the author what to keep doing.
### Final Verdict
Choose one:
- **Approve** — Ready to merge
- **Approve with minor changes** — Good to merge after addressing low/medium issues
- **Request changes** — Critical or high issues must be resolved before merge
List the finding IDs the verdict depends on (e.g. "Request changes: R1, R2").
## Change Sizing
Small, focused changes are easier to review, faster to merge, and safer to deploy.
```
~100 lines changed → Good. Reviewable in one sitting.
~300 lines changed → Acceptable if it's a single logical change.
~1000 lines changed → Too large. Split it.
```
**Watch file size, not just diff size.** Around 1000 *total* lines in a single file is a common inspection signal. When a change materially grows an already-large file, decompose first.
**Splitting strategies:**
| Strategy | How | When |
|----------|-----|------|
| **Stack** | Submit a small change, start the next one based on it | Sequential dependencies |
| **By file group** | Separate changes for groups needing different reviewers | Cross-cutting concerns |
| **Horizontal** | Create shared code/stubs first, then consumers | Layered architecture |
| **Vertical** | Break into smaller full-stack slices of the feature | Feature work |
**Separate refactoring from feature work.** A change that refactors and adds new behavior is two changes — submit them separately.
## Change Descriptions
- **First line:** Short, imperative, standalone. "Delete the FizzBuzz RPC" not "Deleting the FizzBuzz RPC."
- **Body:** What is changing and why. Include context and reasoning not visible in the code itself.
- **Anti-patterns:** "Fix bug," "Fix build," "Add patch," "Phase 1."
## Dependencies
Before adding any dependency:
1. Does the existing stack solve this? (Often it does.)
2. How large is the dependency? (Check bundle impact.)
3. Is it actively maintained? (Check last commit, open issues.)
4. Does it have known vulnerabilities? (`npm audit`)
5. What's the license? (Must be compatible with the project.)
**Rule:** Prefer standard library and existing utilities over new dependencies. Every dependency is a liability.
**Upgrading dependencies:**
- Read the changelog, not just the version number. Semver is a promise the maintainer may not have kept.
- One dependency per change. When a bulk bump breaks the build, you've lost which package did it.
- Let the tests decide — a green suite before *and* after, not just "it installed."
- Review the lockfile diff, not just `package.json`. Commit it and never hand-edit it.
For supply-chain risk triage, follow the `security-and-hardening` skill.
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "It works, that's good enough" | Working code that's unreadable, insecure, or architecturally wrong creates debt that compounds. |
| "I wrote it, so I know it's correct" | Authors are blind to their own assumptions. Every change benefits from another set of eyes. |
| "We'll clean it up later" | Later never comes. The review is the quality gate — use it. |
| "AI-generated code is probably fine" | AI code needs more scrutiny, not less. It's confident and plausible, even when wrong. |
| "The tests pass, so it's good" | Tests are necessary but not sufficient. They don't catch architecture, security, or readability problems. |
| "The refactor makes it cleaner" | Relocating complexity isn't reducing it. If the reader still holds the same number of concepts, the structure didn't improve. |
| "It's only a small addition to this file" | Small diffs still push files past healthy size and bolt branches onto unrelated flows. |
| "It's just a version bump" | A bump is a behavior change you didn't write. Read the changelog. |
| "I'll upgrade everything in one PR" | A bulk bump hides which package broke the build. One per change. |
| "It's duplicated but it's only two places" | Two becomes three becomes five. Extract now, before the copies diverge. |
| "The abstraction is future-proof" | YAGNI. Delete speculative generality — generalize on the third occurrence, not the first. |
| "It's clever but efficient" | Cleverness is a readability tax. If it needs a comment to understand, simplify it. |
## Red Flags
- PRs merged without any review
- Review that only checks if tests pass (ignoring other axes)
- "LGTM" without evidence of actual review
- Security-sensitive changes without security-focused review
- Large PRs that are "too big to review properly" (split them)
- No regression tests with bug fix PRs
- Accepting "I'll fix it later" — it never happens
- A refactor that moves code around without reducing the number of concepts a reader must hold
- New conditionals scattered into unrelated code paths (a missing abstraction)
- A bespoke helper that duplicates an existing canonical one
- A bulk "bump dependencies" PR with no changelog review
## Verification
Before emitting the verdict, verify the change as it stands. This is the reviewer's own due diligence — it covers the state of the code at review time, not the later resolution of findings (fixing findings is the author's job; confirming them is a new review):
- [ ] Tests pass — run them yourself, don't trust the claim
- [ ] Build succeeds
- [ ] The verification story is documented (what changed, how it was verified)
- [ ] Dependency upgrades reviewed against changelog, isolated per package, verified by green suite
## See Also
- For detailed security review guidance, see `security-and-hardening`

View File

@ -7,7 +7,7 @@ slash: true
# Skill: create-commit # Skill: create-commit
Produce a git commit that follows Penpot's commit message conventions. This Produce a git commit that follows Penpot's commit message conventions. This
skill owns the commit format, staging review, and safety checks — it does not skill handles staging review and safety checks — it does not
implement features or push. implement features or push.
## When to Use ## When to Use
@ -23,15 +23,9 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
## Iron Rules (non-negotiable) ## Iron Rules (non-negotiable)
1. **Wrap every body line at 80 characters or fewer.** Count characters, do Follow the message rules and exceptions in `mem:workflow/creating-commits`.
not eyeball. Exceptions: `Signed-off-by:` / `AI-assisted-by:` trailers and **Run `./scripts/check-commit` and require exit code 0.** A non-zero exit is a
lines carrying a URL. This is the rule agents skip most often. hard blocker: fix the message and re-check it before reporting success.
2. **Subject ≤80 chars**, imperative, capitalized, no trailing period.
`Merge`/`Revert`/`Reapply` commits skip the length check.
3. **Blank line between subject and body.**
4. **Run `./scripts/check-commit` and require exit code 0.** It mechanically
checks rules 1–3. A non-zero exit is a hard blocker: fix the message and
re-commit. Never report the commit as done with a failing checker.
## Workflow ## Workflow
@ -63,6 +57,8 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
- Do not pass `--author`. Author identity comes from the local git config. - Do not pass `--author`. Author identity comes from the local git config.
- Do not amend a commit you did not create in this session, unless explicitly asked. - Do not amend a commit you did not create in this session, unless explicitly asked.
- Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked. - Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
- Never add a `Co-authored-by:` trailer. It is forbidden and fails `./scripts/check-commit`. - Add `Co-authored-by:` only for a human co-author. AI providers, agents and
bots are rejected by `./scripts/check-commit`; for AI participation use the
`AI-assisted-by:` trailer.
- Do not add untracked files that were not created in this session. - Do not add untracked files that were not created in this session.
- Do not skip the `scripts/check-commit` verification step (Iron Rule 4). - Do not skip the `scripts/check-commit` verification step.

View File

@ -18,7 +18,7 @@ push — the user pushes from their own shell.
- The user asks to open or create a NEW PR for the current task branch, in - The user asks to open or create a NEW PR for the current task branch, in
any phrasing ("open a PR", "create the pull request", "put this up for any phrasing ("open a PR", "create the pull request", "put this up for
review") — or runs `/create-pr`. → **Open mode**. review") — or invokes `@create-pr`. → **Open mode**.
- The user asks to fix or update an EXISTING PR's title or description to - The user asks to fix or update an EXISTING PR's title or description to
match conventions. → **Update mode**. match conventions. → **Update mode**.
@ -59,7 +59,7 @@ Run all checks before reporting, then report every failure together:
5. **Branch not pushed.** The remote-tracking ref `origin/<branch>` is 5. **Branch not pushed.** The remote-tracking ref `origin/<branch>` is
missing, or `git rev-parse origin/<branch>` differs from HEAD — the missing, or `git rev-parse origin/<branch>` differs from HEAD — the
branch was never pushed, or has commits the remote does not have. Never branch was never pushed, or has commits the remote does not have. Never
push yourself; ask the user to push and to run `/create-pr` again push yourself; ask the user to push and to invoke `@create-pr` again
afterwards, then stop. afterwards, then stop.
### 3. Already-open PR ### 3. Already-open PR

View File

@ -26,12 +26,17 @@ in the local working tree.
- The user asks to implement or execute a plan, in any phrasing: - The user asks to implement or execute a plan, in any phrasing:
"implement the plan", "execute it", "go build it" — or runs "implement the plan", "execute it", "go build it" — or runs
`/implement-plan`. `@implement-plan`.
- A ready, reviewed plan is in the session context or a plan file path - A ready, reviewed plan is in the session context or a plan file path
was given (typically after `/make-a-plan` or `/review-plan`). was given (typically after `@make-a-plan` or `@review-plan`).
Do not use it to produce plans — that is the `make-a-plan` flow. Do not use it to produce plans — that is the `make-a-plan` flow.
For a plan derived from a code review, retain its review target, reviewer
session ID, original findings and decisions. Record the starting revision and
the exact commits or diff produced by this implementation, plus test evidence,
in session context for follow-up. Do not put commit hashes in the plan file.
## 1. Read the plan first ## 1. Read the plan first
Identify the plan to execute — from the file path the user gave, the Identify the plan to execute — from the file path the user gave, the
@ -111,19 +116,20 @@ Implement every task in one go. Work methodically, keeping changes
focused on what the issue requires. Respect the plan's proposed focused on what the issue requires. Respect the plan's proposed
parallelization when it applies. parallelization when it applies.
When the implementation is complete, close the plan file first: flip its When the implementation is complete, load the **`create-commit`** skill and
`Status` to `done` and append one UTC ISO 8601 line to its `Review Log` follow its workflow to commit
with the issue URL when one exists (standalone mode, e.g.
`https://github.com/penpot/penpot/issues/NNNN`); in continue/direct
mode with no issue, just `done` with no invented identifier. Never
record commit hashes. Leave the closed plan file local and uncommitted.
Then load the **`create-commit`** skill and follow its workflow to commit
only the implementation, tests, memory, and documentation changes. Before only the implementation, tests, memory, and documentation changes. Before
committing, verify that the staged file list does not contain the plan committing, verify that the staged file list does not contain the plan
file. Provide a brief summary of what was implemented and why, the issue file. Provide a brief summary of what was implemented and why, the issue
reference (`issue-NNNN`) when there is one, and the model name you are reference (`issue-NNNN`) when there is one, and the model name you are
running as so the `AI-assisted-by` trailer is set correctly. running as so the `AI-assisted-by` trailer is set correctly.
Only after the commit exists and passes its checks, close the plan: flip
`Status` to `done` and append one UTC ISO 8601 line to `Review Log` with the
issue URL when one exists; otherwise record `done` without inventing an
identifier. Never record commit hashes. Leave the plan local and uncommitted.
If the commit or its checks fail, do not close the plan.
### Step-by-step mode (on request) ### Step-by-step mode (on request)
When the user asks for it — "step by step", "task by task", "one commit When the user asks for it — "step by step", "task by task", "one commit
@ -146,12 +152,23 @@ stop and ask the user before splitting it.
## When you are done ## When you are done
After the entire plan is implemented and verified, including the final task in
step-by-step mode, if it addresses code review findings, ask whether the user
wants the original reviewer to verify the corrections. Use the `question` tool
if available; otherwise ask in chat. Wait for the answer. Do not start the
follow-up without consent, and do not ask after each intermediate task.
If the user accepts, follow `@review-code`'s follow-up procedure with the
review context, decisions, exact correction range or diff and test evidence.
If the original reviewer cannot be resumed, say so and use a new reviewer with
the same context. If the user declines, stop without starting a review. The
follow-up is read-only; further fixes require the user's direction.
End by suggesting the next steps (suggestions, not a required pipeline — any End by suggesting the next steps (suggestions, not a required pipeline — any
instruction from me overrides them): instruction from me overrides them):
- `/review-code` — to review the changes just committed; it routes to - `@review-code` — for a new independent review of the changes.
`/make-a-plan` by itself if the findings need one. - `@create-pr` — when the task is done and the branch is ready to merge.
- `/create-pr` — when the task is done and the branch is ready to merge.
## User input, overrides and additional context ## User input, overrides and additional context

View File

@ -86,7 +86,7 @@ log path.
./scripts/ci --lint frontend ./scripts/ci --lint frontend
# Changed common/ — validate its consumers too # Changed common/ — validate its consumers too
./scripts/ci frontend backend exporter ./scripts/ci common frontend backend exporter
# Preview what would run, without running it # Preview what would run, without running it
./scripts/ci --dry-run --all ./scripts/ci --dry-run --all

View File

@ -16,19 +16,18 @@ stop — this skill needs the build agent to save the plan.
- The user asks to plan, design, or break down a task, in any phrasing: - The user asks to plan, design, or break down a task, in any phrasing:
"make a plan", "how would we build X", "design an approach for Y" — "make a plan", "how would we build X", "design an approach for Y" —
or runs `/make-a-plan`. or invokes `@make-a-plan`.
- The user asks to rework or extend an existing plan. While the plan is - The user asks to rework or extend an existing plan. While the plan is
still unimplemented (pre-`implement-plan` iteration, e.g. after still unimplemented (pre-`implement-plan` iteration, e.g. after
`/review-plan` findings or user feedback), revise the saved plan file `@review-plan` findings or user feedback), revise the saved plan file
in place. Once the plan has been implemented and reviewed (post in place. Once the plan has been implemented and reviewed (post
`/review-code` findings on committed work), write a new derived plan `@review-code` findings on committed work), write a new derived plan
file instead — never rewrite the executed plan. The `planner` skill file instead — never rewrite the executed plan. The `planner` skill
defines the derived naming (`--review-NN`, `--task-NN`). defines the derived naming (`--review-NN`, `--task-NN`).
- The user says to apply `review-plan` findings to a still-unimplemented - The user says to apply additional review feedback to a still-unimplemented
plan: apply the agreed changes, flip `Status` to `reviewed`, and plan: apply the agreed changes, flip `Status` to `reviewed`, and append one
append one UTC ISO 8601 line to `Review Log` describing what changed. UTC ISO 8601 line to `Review Log`. `review-plan` can also update the plan
This apply step is the only pre-implementation write to those fields — and its metadata as part of its own review flow.
`review-plan` itself stays read-only and never writes.
Do not use it to execute a plan — that is the `implement-plan` flow. Do not use it to execute a plan — that is the `implement-plan` flow.
@ -39,6 +38,11 @@ Do not use it to execute a plan — that is the `implement-plan` flow.
example, `delegated` in the user context), delegate to the `general` subagent example, `delegated` in the user context), delegate to the `general` subagent
instead — the delegate must also follow the `planner` skill and receive all instead — the delegate must also follow the `planner` skill and receive all
the relevant session context (a review, user feedback, and so on). the relevant session context (a review, user feedback, and so on).
When the plan addresses a code review, preserve the review target, reviewer
session ID if available, original findings and IDs, and decisions including
deferred or rejected findings and their reasons in the local plan. Map
accepted findings to tasks. If context is unavailable, mark it unknown; do
not invent it. Pass this context to a delegated planner too.
2. Before asking me to decide anything, explain the plan and every open question in 2. Before asking me to decide anything, explain the plan and every open question in
plain language. Assume I know only the high-level project goal, not the codebase, plain language. Assume I know only the high-level project goal, not the codebase,
architecture, implementation terms, or the problem this task solves. architecture, implementation terms, or the problem this task solves.
@ -101,8 +105,8 @@ constraints and risks, what is deliberately out of scope, and the path where the
is saved. Never assume that a short list of task names is enough context. End is saved. Never assume that a short list of task names is enough context. End
the final response by suggesting the next steps, in this order: the final response by suggesting the next steps, in this order:
1. `/review-plan` — to get a second opinion on the plan before executing it. 1. `@review-plan` — to get a second opinion on the plan before executing it.
2. `/implement-plan` — to execute the plan from the current session context. 2. `@implement-plan` — to execute the plan from the current session context.
These are suggestions, not a required pipeline — any instruction from me These are suggestions, not a required pipeline — any instruction from me
overrides them (for example, asking you to implement the plan directly). overrides them (for example, asking you to implement the plan directly).

View File

@ -1,315 +1,105 @@
--- ---
name: plan-review-criteria name: plan-review-criteria
description: Plan review criteria — the six review axes, severity rubric, approval standard, and output format for reviewing implementation plans. Loaded by the reviewer subagent of the review-plan flow. Not a user-facing flow — to review a plan, use the review-plan flow. description: Criteria for reviewing implementation plans, including evidence, severity and verdicts. Loaded by the review-plan subagent.
--- ---
# Plan Review Criteria # Plan Review Criteria
## Overview Evaluate whether the plan can deliver its stated goal safely and whether a
skilled implementer can execute and verify it without guessing. Perfect plans
do not exist: do not block on personal preferences or missing sections that
add no useful information. Apply each criterion in proportion to the plan's
scope and risks; explain when a criterion is not applicable.
Multi-dimensional plan review with quality gates. Every plan gets reviewed before implementation starts — no exceptions. Review covers six axes: completeness, task quality, architecture & sequencing, risk coverage, actionability, and proposed code quality. Follow `review-plan` for delegation, permitted edits, decisions and handover.
Do not run tests, builds or other state-changing checks during the review.
**The approval standard:** Approve a plan when it is specific enough that a skilled implementer could execute it without guessing, the task ordering is sound, and risks are acknowledged. Perfect plans don't exist — the goal is confidence that implementation won't derail. Don't block a plan because it isn't exactly how you would have structured it. If it's executable and well-organized, approve it. ## Evidence and scope
## When to Use - Read the plan, relevant code, project guidance and module memories. Check
that cited paths, symbols and proposed commands exist and are appropriate;
distinguish verified facts from assumptions and unknowns. Do not invent
findings or claim checks were run when they were not.
- Check that tasks serve the stated goal and respect existing requirements and
decisions. Identify scope creep, contradictions and dependencies on external
work. A consequential change in scope, behavior or tradeoffs needs the user's
decision.
- Evaluate the proposed approach even without code snippets: does it solve the
problem, fit existing conventions and avoid unnecessary complexity? If the
plan includes code, also check its types, signatures and examples.
- The reviewer subagent of the `review-plan` flow loads this skill to perform ## Six review axes
the review of a plan.
- To review a plan, always go through the `review-plan` flow — never load this
skill directly for that. This is the criteria reference, not the flow.
**Do NOT use for:** Single-file changes with obvious scope, or when the task is trivial enough to just do.
## The Six-Axis Review
Every plan gets evaluated across these dimensions:
### 1. Completeness ### 1. Completeness
Does the plan cover everything needed to implement successfully? Are the problem, intended outcome, affected areas and relevant constraints
clear? Does the plan explain important decisions and include an appropriate
- Is the **context** clear? (What problem, why now, what's the goal?) testing strategy? Identify missing information that would change the outcome
- Are **affected modules** identified with paths? or force an implementer to guess. Require risk, rollback, migration or
- Are **architecture decisions** documented with rationale? parallelization details only when relevant.
- Is there a **testing strategy**?
- Are **verification commands** explicit (not "run the tests")? ### 2. Task quality
- Are **open questions** listed (not buried in someone's head)?
- Is there a **parallelization** assessment for multi-task plans? Are tasks scoped, ordered and specific enough to execute? Check likely files,
dependencies, rationale, acceptance criteria and verification where relevant.
**Missing any of these is a gap, not a nit.** Acceptance criteria should describe observable behavior, including important
failure cases. Split work when it is too large to understand or verify safely,
### 2. Task Quality not merely because it crosses an arbitrary task count.
Are the tasks well-defined and independently executable? ### 3. Architecture and sequencing
- Does every task have **acceptance criteria**? (Testable, not vague) Does the approach fit the existing architecture and reuse appropriate
- Does every task have **verification steps**? facilities? Do dependencies precede their consumers, and can the work be
- Are tasks **sized appropriately**? (XS–M is ideal, L is acceptable, XL must be split) verified as it progresses? Prefer independently testable slices where useful;
- Are **dependencies** between tasks explicitly stated? use checkpoints where risk or coordination warrants them, not at a fixed
- Are **files likely touched** listed? interval. Flag unrelated work or abstractions that add complexity without
- Is each task a **single, self-contained change**? (Not "implement the whole feature") helping the goal.
- Could a skilled implementer pick up any task and execute it without asking clarifying questions?
### 4. Risk coverage
### 3. Architecture & Sequencing
Consider relevant edge cases, security, performance, compatibility, data
Is the plan structured so implementation flows correctly? integrity, migrations and external dependencies. Where failure has meaningful
consequences, check detection, mitigation and rollback or recovery. Do not
- Does implementation order follow the **dependency graph** (foundations first)? invent risks or demand a rollback section for changes that do not need one.
- Are tasks **vertically sliced** (feature paths) rather than horizontally layered?
- Does each task leave the system in a **working state**? ### 5. Actionability and verification
- Are there **checkpoints** between major phases?
- Are **high-risk tasks early** (fail fast)? Can the implementer identify the affected code, follow project conventions and
- Is the total plan a reasonable number of tasks? (More than ~15 tasks suggests the scope should be split into multiple plans) run the stated project-specific checks? Are commands and prerequisites correct?
Do tests and other checks demonstrate the intended behavior and important
### 4. Risk Coverage failure cases? Surface unknowns honestly rather than presenting an unverified
command or assumption as fact.
Are the hard parts acknowledged and mitigated?
### 6. Proposed solution quality
- Are **edge cases** identified?
- Are **breaking changes** or **migration concerns** noted? Assess correctness, clarity, architecture, security and performance of the
- Are **security implications** considered? proposed solution, even when the plan contains no code. For any code sketches,
- Are **performance implications** considered? also check boundary cases, names, types and contracts. Preserve intended
- Are **external dependencies** or integration risks flagged? behavior unless a change is explicitly agreed; prefer the simplest solution
- Is there a plan for **rollback** if something goes wrong? that meets the requirements.
- Are **data integrity** risks addressed (what happens if a migration fails mid-way)?
## Findings and verdict
### 5. Actionability
Give each reported item a stable ID (`P1`, `P2`, …); retain it throughout the
Can an implementer actually execute this? review and any decision walkthrough. For each item state the evidence, impact,
concrete remedy and verification needed. For a consequential user decision,
- Are **file paths** specific (not "update the relevant files")? include real options, consequences and a recommendation.
- Are **function/method names** mentioned where applicable?
- Are **verification commands** copy-pasteable (not "run the linter")? - **Blocking:** implementation cannot proceed safely or an essential user
- Are **test commands** project-specific (not generic)? decision remains open. Explain what blocks it.
- Is the **code shape** described where the implementation isn't obvious? - **Improvement:** a clear correction that preserves the agreed goal, scope
- Are **conventions** referenced (naming, patterns, existing utilities to reuse)? and behavior. Apply it under `review-plan`'s permitted-edit rules.
- Does the plan reference **existing code** the implementer should read first? - **Optional:** a suggestion that does not prevent implementation. Do not
silently expand scope or override the user's decisions.
### 6. Proposed Code Quality *(when the plan includes implementation details)*
Report applied improvements and any outstanding items with their IDs. Give a
If the plan proposes code shapes, function signatures, data structures, or API designs, evaluate those proposals against `code-review-criteria`: provisional verdict before any walkthrough and a final verdict after applying
the decisions:
- **Correctness:** Do the proposed types/signatures handle edge cases (null, empty, boundaries)?
- **Readability:** Are proposed names descriptive and consistent with project conventions? - **Ready:** no blocking issue or essential decision remains.
- **Architecture:** Do proposed abstractions follow existing patterns? Are they justified (not over-engineered)? - **Awaiting decisions:** user decisions are needed before the plan is ready.
- **Security:** Do proposed APIs validate input at boundaries? Any injection/XSS vectors in the design? - **Blocked:** required work or evidence is missing and cannot be resolved by
- **Performance:** Do proposed data structures avoid N+1 patterns? Any unbounded operations in the design? applying clear improvements or obtaining the current decisions alone.
**When to apply:** Only when the plan includes specific code snippets, type definitions, API contracts, or function signatures. Plans that only describe "what" without showing "how" skip this axis. Do not approve an unresolved blocker. A plan may be ready with optional
suggestions; identify them without treating them as required.
## Structural Remedies
When you flag a structural problem in a plan, propose the fix — not just the problem:
- **A task is too large (XL):** Split it into vertical slices. Each slice should be independently testable.
- **Missing acceptance criteria:** Draft 2–3 specific, testable conditions for the task.
- **Wrong sequencing:** Identify the dependency and propose the correct order.
- **No checkpoints:** Suggest where checkpoints should go (typically after every 2–3 tasks).
- **Vague verification:** Replace "run tests" with the actual project command.
- **Horizontal slicing:** Restructure into vertical feature paths.
- **Missing risk section:** Draft the risks you can identify from the plan content.
Prefer the remedy that makes the plan immediately actionable over one that just flags the gap.
## Plan Sizing
Plans should be scoped to a single deliverable:
```
1–5 tasks → Good. A focused feature or bug fix.
6–10 tasks → Acceptable for a moderate feature.
11–15 tasks → Large. Consider splitting into phases.
15+ tasks → Too large. Split into multiple plans.
```
**What counts as "one plan":** A self-contained set of changes that delivers a single coherent capability. If you can describe the goal in one sentence, it's one plan.
## Categorize Findings
Label every comment with its severity so the author knows what's required vs optional:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| *(no prefix)* | Required change | Must address before implementation starts |
| **Critical:** | Blocks implementation | Missing security consideration, data integrity risk, fundamentally wrong approach |
| **Nit:** | Minor, optional | Author may ignore — wording, formatting |
| **Optional:** / **Consider:** | Suggestion | Worth considering but not required |
| **FYI** | Informational only | No action needed — context for future reference |
**Lead with what matters.** Order findings by leverage: missing risks and wrong sequencing first, then task quality gaps, then completeness, then nits. If you have one critical sequencing problem and ten nits, the sequencing problem *is* the review.
## Review Process
### Step 1: Understand the Goal
Before evaluating structure, understand intent:
```
- What is this plan trying to accomplish?
- What problem does it solve?
- What does "done" look like?
```
### Step 2: Check Completeness First
Scan for missing sections before diving into content:
```
- Context present?
- Affected modules listed?
- Architecture decisions documented?
- Risks acknowledged?
- Testing strategy defined?
- Verification commands explicit?
```
### Step 3: Review Task Quality
Walk through each task:
```
For each task:
1. Can I tell exactly what to build?
2. Are acceptance criteria specific and testable?
3. Is the size reasonable (not XL)?
4. Are dependencies clear?
5. Would I know which files to touch?
```
### Step 4: Validate Sequencing
Check the dependency graph:
```
- Are foundations built first?
- Does each task leave the system working?
- Are checkpoints placed correctly?
- Are high-risk items early?
- Is it vertically sliced?
```
### Step 5: Assess Actionability
Put yourself in the implementer's shoes:
```
- Could I pick up task 1 and start coding without asking any questions?
- Are the verification commands copy-pasteable?
- Are file paths and function names specific?
- Is existing code referenced where I'd need to read it?
```
### Step 6: Verify the Verification Story
Check that the plan can actually confirm it worked:
```
- What tests should pass after implementation?
- What build/compile commands are relevant?
- What manual checks are needed?
- How do we know the feature works end-to-end?
```
### Step 7: Evaluate Proposed Code Quality *(if applicable)*
If the plan includes code snippets, types, or API designs:
```
- Load code-review-criteria skill for criteria
- Check proposed signatures for edge cases
- Verify naming follows project conventions
- Confirm abstractions follow existing patterns
- Scan for security vectors in proposed APIs
- Check for performance issues in proposed data structures
```
## Review Checklist
```markdown
## Review: [Plan title]
### Completeness
- [ ] Context explains the problem and goal
- [ ] Affected modules are listed with paths
- [ ] Architecture decisions have rationale
- [ ] Testing strategy is defined
- [ ] Verification commands are explicit and project-specific
- [ ] Open questions are listed
### Task Quality
- [ ] Every task has acceptance criteria
- [ ] Every task has verification steps
- [ ] Tasks are sized XS–M (L acceptable, XL must be split)
- [ ] Task dependencies are stated
- [ ] Files likely touched are listed
### Architecture & Sequencing
- [ ] Order follows dependency graph (foundations first)
- [ ] Vertically sliced (not horizontal layers)
- [ ] Each task leaves system working
- [ ] Checkpoints exist between phases
- [ ] High-risk tasks are early
### Risk Coverage
- [ ] Edge cases identified
- [ ] Breaking changes / migrations noted
- [ ] Security implications considered
- [ ] Performance implications considered
- [ ] Rollback strategy exists (if applicable)
### Actionability
- [ ] File paths are specific
- [ ] Verification commands are copy-pasteable
- [ ] Existing code to read is referenced
- [ ] Conventions and patterns are noted
### Proposed Code Quality *(if plan includes implementation details)*
- [ ] Proposed types/signatures handle edge cases
- [ ] Proposed names follow project conventions
- [ ] Proposed abstractions follow existing patterns
- [ ] No security vectors in proposed APIs
- [ ] No performance issues in proposed structures
### Verdict
- [ ] **Approve** — Ready to implement
- [ ] **Request changes** — Gaps must be addressed
```
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "I'll figure out the details during implementation" | That's how you discover blocking dependencies mid-task. Surface them now. |
| "The tasks are obvious, no need for criteria" | Write them anyway. Explicit criteria surface hidden assumptions. |
| "It's just a small feature, it doesn't need a plan" | Small features have edge cases too. 3 tasks with criteria takes 5 minutes. |
| "The plan is good enough" | "Good enough" without acceptance criteria means the implementer defines "done" — and they might define it differently. |
| "I'll add verification steps later" | Later never comes. The plan is the contract — define verification now. |
| "Risks are minimal" | Every change has risks. If you can't name them, you haven't thought about them. |
| "The file paths are obvious" | They're obvious to the author. The implementer might not know the codebase. |
| "The code in the plan is fine, it'll get reviewed later" | Plan-level code review catches design problems before implementation — fixing them after coding is more expensive. |
## Red Flags
- No acceptance criteria on any task
- Tasks that say "implement the feature" without specifics
- No verification steps anywhere in the plan
- All tasks are XL-sized
- No checkpoints between phases
- Dependency order isn't considered (e.g., API handler before domain model)
- No testing strategy
- Verification commands are generic ("run tests") instead of project-specific
- Plan has 20+ tasks (scope too large for one plan)
- No risk section on a plan with migrations, breaking changes, or security implications
- Horizontal slicing (all domain, then all services, then all API)
- File paths are vague ("update the relevant files")
- Missing open questions section despite stated unknowns
- Proposed code ignores project conventions or existing patterns
- Proposed types use gratuitous `any`/`unknown`/optional without justification
- Proposed APIs don't validate input at boundaries
## See Also
- For producing plans, use the `planner` skill
- For reviewing implemented code, use `code-review-criteria` — also the criteria source for axis 6
- For security-specific concerns, see `security-and-hardening`
- For testing strategy guidance, see `testing`

View File

@ -42,9 +42,10 @@ Skipping this step is the #1 cause of incorrect or incomplete plans.
only file you may write is the plan itself, and only when the command or only file you may write is the plan itself, and only when the command or
user explicitly instructs you to save it. user explicitly instructs you to save it.
- You do **not** run builds, tests, linters, or any commands that modify state. - You do **not** run builds, tests, linters, or any commands that modify state.
- You do **not** create git commits or interact with version control. - You do **not** create commits or run commands that modify version control.
- You do **not** execute shell commands beyond read-only searches (`rg`, `ls`, Read-only inspection such as `git status`, `git log`, `git show` and
`find`, `cat`, `bat`). `git diff` is allowed.
- Shell commands must be read-only.
- Your output is a structured plan or analysis, ready for handoff to an - Your output is a structured plan or analysis, ready for handoff to an
engineer agent or developer. engineer agent or developer.
@ -192,8 +193,8 @@ Source: review-code over `<commit>` (branch `<branch>`) | task `NN` of roadmap `
Created: YYYY-MM-DD Created: YYYY-MM-DD
``` ```
End the response by suggesting the next steps: `/review-plan` to get a second End the response by suggesting the next steps: `@review-plan` to get a second
opinion on the plan and `/implement-plan` to execute it. opinion on the plan and `@implement-plan` to execute it.
### Plan Structure ### Plan Structure
@ -223,16 +224,16 @@ Review Log:
log. `reviewed` never means "a review was emitted" — it means log. `reviewed` never means "a review was emitted" — it means
"review feedback was incorporated". A plan approved with no changes "review feedback was incorporated". A plan approved with no changes
goes `draft` → `done` without passing through `reviewed`. goes `draft` → `done` without passing through `reviewed`.
- Only a user-triggered apply step writes them before implementation: - Before implementation, `review-plan` applies clear review feedback and
when the user says to apply `review-plan` findings, `make-a-plan` agreed decisions, flips to `reviewed` and appends one UTC ISO 8601 entry
applies the changes, flips to `reviewed`, and appends one UTC when the plan changed. `make-a-plan` can also do this for additional
ISO 8601 line describing what changed. feedback when the user asks.
- `implement-plan` flips to `done` on completion and appends one line - `implement-plan` flips to `done` on completion and appends one line
with the issue URL when one exists (standalone mode); otherwise with the issue URL when one exists (standalone mode); otherwise
just `done`. Never record commit hashes — they rot on amend/rebase just `done`. Never record commit hashes — they rot on amend/rebase
and git already links the commit. and git already links the commit.
- The log is append-only: never rewrite or delete lines. - The log is append-only: never rewrite or delete lines.
- `review-plan` and `review-code` never write these fields. - `review-code` never writes these fields.
Omit empty sections only when they do not apply. Every implementation task Omit empty sections only when they do not apply. Every implementation task
still requires acceptance criteria, verification, dependencies, likely files, still requires acceptance criteria, verification, dependencies, likely files,
@ -261,5 +262,5 @@ Before delivering the plan, confirm:
- [ ] No task is XL or larger — break it down instead - [ ] No task is XL or larger — break it down instead
- [ ] Checkpoints exist after every 2-3 tasks - [ ] Checkpoints exist after every 2-3 tasks
- [ ] The response states the plan's path (saved or suggested) and suggests - [ ] The response states the plan's path (saved or suggested) and suggests
`/review-plan` and `/implement-plan` `@review-plan` and `@implement-plan`
- [ ] The plan is ready for human review - [ ] The plan is ready for human review

View File

@ -12,7 +12,7 @@ Act as a senior software engineer and perform a thorough code review.
- The user asks to review code, in any phrasing: "review this diff", - The user asks to review code, in any phrasing: "review this diff",
"review the PR", "check my changes", "code review" — or runs "review the PR", "check my changes", "code review" — or runs
`/review-code`. `@review-code`.
- A commit, branch, PR, or diff is ready and the user wants it assessed - A commit, branch, PR, or diff is ready and the user wants it assessed
before merge. before merge.
@ -22,17 +22,61 @@ Act as a senior software engineer and perform a thorough code review.
diff, a commit range, a branch, a PR (number or URL), or specific files. If diff, a commit range, a branch, a PR (number or URL), or specific files. If
the target is ambiguous, ask before reviewing. the target is ambiguous, ask before reviewing.
2. Delegate the review to the `general` subagent (via the task tool), unless the 2. Delegate the review to the `general` subagent (via the task tool), unless the
user specifies another agent. Include in the prompt the user specifies another agent. The prompt carries the **`code-review-criteria`**
**`code-review-criteria`** skill name and all user context. skill name, all user context, and these requirements:
- Load `code-review-criteria` and follow its process and output format.
- Read `AGENTS.md` (if present) and the module memories it points to for the
code being reviewed.
- Return the COMPLETE review, verbatim, exactly as the skill instructs it to
be produced. Never summarize it.
- Read-only: modify no file, create no commit, and run no tests, builds,
linters or other state-changing checks. Use prior results or CI evidence;
state explicitly when evidence is unavailable.
- Do not invent problems: every finding must be real and actionable.
- Missing tests are an issue, not a suggestion — a severity-tagged finding.
- For each finding, give a concrete remedy and the tests needed to verify it.
If a user decision is needed, include the real options and a recommendation
in that finding. Do not pause to ask during the review.
- Prioritize by impact: one structural issue outweighs ten nits, and "this
could be better" is not a finding — say why and how.
- Inspect generated files and lockfiles when relevant to the change, its
dependencies or security; exclude unrelated working-tree modifications.
3. When the subagent returns, output the review to the user verbatim. Do not 3. When the subagent returns, output the review to the user verbatim. Do not
summarize it and do not act on its findings. summarize it and do not act on its findings. Retain the subagent session ID,
review target and complete findings in the session context for follow-up;
do not invent an ID if the tool does not provide one.
4. Right after the review, suggest how to proceed based on the findings. These 4. Right after the review, suggest how to proceed based on the findings. These
are suggestions — the user decides: are suggestions — the user decides:
- **Approve (no required changes):** say so — there is nothing to address. - **Approve (no required changes):** say so — there is nothing to address.
- **Minor findings (nits):** applying them directly as-is is fine once the - **Minor findings:** suggest direct fixes after the user ends the review if
review is done — no plan needed. appropriate; no plan is required.
- **Substantive findings:** suggest `/make-a-plan` to make a plan to address - **Substantive findings:** suggest `@make-a-plan` to make a plan to address
them. them.
- If the user wants to discuss findings one by one, use `@issue-walkthrough`
on request; do not start it automatically. The walkthrough returns its
decisions to `@make-a-plan`, not to automatic code editing.
### Follow-up after implementation
When the user agrees to verify corrections, resume the original reviewer using
its session ID if the tool supports it. Give it the original review target and
findings, the decisions and plan, the exact correction commit range or diff,
and the test evidence. Require it to inspect the actual corrections and
evidence, not rely on its earlier context alone. If the original session is
unavailable or cannot be resumed, tell the user and delegate to a new reviewer
with the same context; never claim it is the original reviewer. If the review,
decisions or correction range cannot be established, ask for the missing
information rather than inventing it.
The reviewer follows `code-review-criteria` and remains read-only. It reports
each original finding with its unchanged ID as **resolved**, **pending**, or
**not addressed** (including the agreed reason); report any new problems with
new, non-conflicting IDs. For pending and new problems, give evidence, a remedy,
tests and any user decision as in the original review. Summarize separately
whether all agreed corrections are resolved and whether any findings remain,
including deferred findings. Do not equate deferred with resolved. Present the
result without implementing fixes; the user chooses whether to iterate, request
an independent review, or stop.
### Hard rule — read-only while reviewing ### Hard rule — read-only while reviewing
@ -40,50 +84,26 @@ This flow is read-only **for the duration of the review**: from the moment it
starts until the user considers the review finished (including any feedback, starts until the user considers the review finished (including any feedback,
questions, or clarifications about it). During that period, never fix, questions, or clarifications about it). During that period, never fix,
implement, edit files or create commits — not even "obvious" fixes derived from implement, edit files or create commits — not even "obvious" fixes derived from
the findings. Once the user explicitly states the review is done (or moves on to the findings. Never run tests, builds, linters or state-changing checks during
a different task), this rule no longer applies and you act as a normal build the review; inspect existing evidence instead. Once the user explicitly states
agent again. the review is done (or moves on to a different task), this rule no longer
applies and you act as a normal build agent again.
## Instructions for the subagent
1. Load the **`code-review-criteria`** skill and follow its process and output
format.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and
reading all related testing documentation from memories before reviewing.
3. Return in your final message the COMPLETE review, verbatim, exactly as the
skill instructs it to be produced. Do not summarize it — include the full
structured review.
### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing
never writes.
3. Be specific and constructive. "This could be better" is not helpful — explain
why and how.
4. Prioritize by impact. One structural issue outweighs ten nits.
5. Missing tests are an issue, not a suggestion. Report as a severity-tagged
finding — never as a recommendation.
6. Skip generated files, lockfile-only changes, and unrelated modifications
unless they introduce security risks.
## Posting as a GitHub comment ## Posting as a GitHub comment
Only post the review as a comment when the user asks for it explicitly. Keep Post only when the user asks for it explicitly; otherwise keep the review in the
it in the session otherwise. session. PR comments go in English, with this minimal structure:
When posting: PR comments go in English, with this minimal structure:
1. **Intro line** — one sentence: what was found and the overall tone. No 1. **Intro line** — one sentence: what was found and the overall tone. No
greetings. greetings.
2. **`## Main issues`** — one subsection per required finding: what breaks, 2. **`## Main issues`** — one subsection per required finding: what breaks, the
the failure circumstances, and the minimal fix (code snippet when useful). failure circumstances, and the minimal fix (code snippet when useful). A
A missing test belongs here, not under minor suggestions. missing test belongs here, not under minor suggestions.
3. **`## Minor suggestions`** — one bullet per minor finding or suggestion: 3. **`## Minor suggestions`** — one bullet per minor finding: file/function,
file/function, problem, concrete remedy. problem, concrete remedy.
4. **Closing** — optional and short. Cut it when it adds nothing. 4. **Closing** — optional and short. Cut it when it adds nothing.
Reference every finding by its ID (`R1`, `M1`, `S1`, … from Reference every finding by its ID (`F1`, `F2`, … from
`code-review-criteria`) so follow-up replies can point to findings `code-review-criteria`) so follow-up replies can point to findings
unambiguously. unambiguously.

View File

@ -1,46 +1,60 @@
--- ---
name: review-plan name: review-plan
description: Plan review flow — evaluate an implementation plan before it is executed, delegating the review to a subagent that follows the plan-review-criteria skill. Use it when the user asks to review a plan, in any phrasing. description: Review an implementation plan with a subagent, apply clear improvements and resolve consequential decisions with the user.
slash: true slash: true
--- ---
# Review Plan # Review Plan
Act as a senior software engineer and perform a thorough review of an implementation plan. Review an unimplemented plan with a subagent and improve the plan without
implementing it. Use this flow when the user asks for a plan review or invokes
`@review-plan`.
## When to use ## Flow
- The user asks to review a plan, in any phrasing: "review this plan", "does this plan look right?", "second opinion on the plan" — or runs `/review-plan`. 1. Identify and read the complete plan and its file. If the target or path is
- A plan was just produced (typically by `/make-a-plan`) and the user wants it evaluated before executing it. ambiguous, ask. If no saved file exists, ask for a path or have the user save
it first. Do not edit a completed plan: propose a derived plan instead.
2. Delegate to `general`, unless the user specifies a different subagent or
model. Supply the plan path, relevant session context and these instructions:
- Load `plan-review-criteria`, read `AGENTS.md` and the relevant module and
testing memories, and review the plan against the criteria.
- Apply clear, evidence-based improvements directly to that plan file only.
Preserve the goal, scope, agreed decisions and intended behavior. Examples:
clarify tasks, add missing tests, correct dependencies and specify actual
verification commands. Do not guess commands or invent findings.
- A change that affects the goal, user-visible behavior, scope, tradeoffs or
an existing decision needs the user's decision. Leave it unresolved; report
concrete options, consequences and a recommendation. Do not ask questions
from the subagent.
- Return a concise summary of applied changes, remaining decisions with
stable IDs, evidence and recommendations, and an honest provisional
verdict. Retain the subagent session ID for follow-up if available.
- Do not change source files, other plans or documents, run state-changing
checks, commit, or implement the plan. If the plan is not writable, report
this and return the proposed edits instead.
3. Show the summary. If decisions remain, start `@issue-walkthrough` immediately
in the parent session with those questions and their IDs. Follow its one
question per turn and read-only rules. Do not ask questions that the code or
existing decisions settle. Wait for every answer.
4. Resume the same subagent with the decision table and plan path. It applies
the decisions to the plan, rechecks consistency and reports changes and any
newly discovered consequential decisions. Repeat the walkthrough for new
decisions, then apply those answers. If the original session cannot be
resumed, tell the user and apply the changes in the parent or delegate a new
subagent with the full context. If the reviewer cannot write, apply its
proposed edits in the parent only if the plan file is writable; otherwise
explain the blocker and stop.
5. Once all decisions are resolved and the plan is coherent, update its
metadata: when review feedback changed the plan, set `Status: reviewed` and
append one UTC ISO 8601 `Review Log` entry for this review, summarizing the
changes. Preserve prior log entries. If no feedback changed the plan, leave
status and log unchanged. Never record commit hashes. If decisions or
required changes remain unresolved, do not claim the plan is ready or mark
it reviewed.
6. Show the final summary, plan path and verdict. Suggest `@implement-plan` only
when the plan is ready. Never execute the plan automatically.
## Instructions The only permitted write in this flow is the specified local plan file,
including its status and log. Do not stage or commit it. If a user instruction
1. **Determine the plan under review** from the session context (for example, a plan just produced by `/make-a-plan`) or from a plan file path given by the user (typically under `.agents/plans/`). If a file path is given, read the file first so the complete plan is in context. overrides automatic editing, respect it and report findings without editing.
2. Delegate the review to the `general` subagent (via the task tool), unless the user specifies another agent. Include in the prompt the **`plan-review-criteria`** skill name and all user context.
3. When the subagent returns, output the review to the user verbatim. Do not summarize it and do not act on its findings.
4. Right after the review, suggest the next step based on the verdict. These are suggestions — the user decides, and any instruction overrides them:
- **Approve** → suggest `/implement-plan` to execute it.
- **Request changes** → suggest `/make-a-plan` to make a plan to address the findings.
### Hard rule — read-only while reviewing
This flow is read-only **for the duration of the review**: from the moment it starts until the user considers the review finished (including any feedback, questions, or clarifications about it). During that period, never fix, implement, edit files or create commits — not even "obvious" fixes derived from the findings. Once the user explicitly states the review is done (or moves on to a different task), this rule no longer applies and you act as a normal build agent again.
## Instructions for the subagent
1. Load the **`plan-review-criteria`** skill and follow its process and output format.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and reading all related documentation and testing memories before reviewing.
3. Cada hallazgo debe tener un identificador único y estable, con el formato `F1`, `F2`, etc. El identificador debe aparecer en el título del hallazgo y no debe reutilizarse dentro de la misma revisión. Esto permite que el usuario pueda responder sobre un hallazgo concreto.
4. Return in your final message the COMPLETE review, verbatim, exactly as the skill instructs it to be produced. Do not summarize it — include the full structured review.
### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing never writes.
3. Be specific and constructive. "This could be better" is not helpful — explain why and how.
4. Prioritize by impact. One structural issue outweighs ten nits.
5. Judge the plan as the implementer would: every task executable without guessing, ordering follows the dependency graph, risks named.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -6,9 +6,8 @@ slash: true
# Skill: rewrite-commit-message # Skill: rewrite-commit-message
HEAD's message is a concatenation of older messages (typical after a Rewrite HEAD's title and description from its own diff. This also handles a
squash): several subjects, bodies and repeated trailers stacked in a squash message with several subjects, bodies and repeated trailers.
single message. Replace it with one proper message and amend HEAD.
## When to Use ## When to Use
@ -17,9 +16,11 @@ single message. Replace it with one proper message and amend HEAD.
## Steps ## Steps
1. Read the full current message (`git log -1 --format=%B HEAD`) and 1. Check `git diff --cached` first. If any changes are staged, stop and ask
the branch diff (`git diff develop..HEAD --stat`, plus the code the user to handle them; an amend must never include those changes. Read
when the summary needs grounding). the full current message (`git log -1 --format=%B HEAD`) and HEAD's own
diff (`git show --format= --stat HEAD` and the patch). Respect the project
rule on amending pushed commits.
2. Collect every DISTINCT `AI-assisted-by:` value in the current 2. Collect every DISTINCT `AI-assisted-by:` value in the current
message. All of them survive into the new message, one line each: message. All of them survive into the new message, one line each:
attributions are never dropped in a rewrite, even when most of attributions are never dropped in a rewrite, even when most of
@ -27,7 +28,7 @@ single message. Replace it with one proper message and amend HEAD.
3. Draft one message per `mem:workflow/creating-commits` (read it 3. Draft one message per `mem:workflow/creating-commits` (read it
end-to-end first): single subject (imperative, capitalized, no end-to-end first): single subject (imperative, capitalized, no
trailing period, ≤80 chars), blank line, body wrapped at ≤80 trailing period, ≤80 chars), blank line, body wrapped at ≤80
chars covering the whole diff (not the old per-commit slices), chars covering HEAD's diff (not the old per-commit slices),
then the preserved trailers. then the preserved trailers.
4. Write it to a file (a heredoc keeps real newlines; inline `-m` 4. Write it to a file (a heredoc keeps real newlines; inline `-m`
with `\n` does not) and `git commit --amend -F <file>`. with `\n` does not) and `git commit --amend -F <file>`.

View File

@ -67,7 +67,7 @@ rg "user_\d+"
rg -F "function()" rg -F "function()"
# Multiline search # Multiline search
rg -U "start.*end" rg -U '(?s)start.*end'
# Only show matches, not lines # Only show matches, not lines
rg -o "https?://[^\s]+" rg -o "https?://[^\s]+"

View File

@ -1,6 +1,6 @@
--- ---
name: ste name: ste
description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "/ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it. description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "@ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it.
--- ---
# ASD-STE100 Simplified Technical English # ASD-STE100 Simplified Technical English
@ -11,7 +11,7 @@ Compliance note (for you, not for output): the official specification and its di
## When to use ## When to use
Only when the user explicitly invokes it: they type `/ste`, or say "use Only when the user explicitly invokes it: they type `@ste`, or say "use
the ste skill" / "apply ASD-STE100". Requests like "simplify this", the ste skill" / "apply ASD-STE100". Requests like "simplify this",
"make it clearer", or "shorter sentences" do NOT invoke it — respond "make it clearer", or "shorter sentences" do NOT invoke it — respond
normally unless it is named. normally unless it is named.

View File

@ -1,116 +0,0 @@
---
name: taiga
description: Fetch information from Taiga public API for the Penpot project (id 345963) — issues, user stories, and tasks, without authentication.
metadata: {"clawdbot":{"requires":{"bins":["python3"]}}}
---
# Taiga API Skill
Fetch information from Taiga public API for the **Penpot** project
(project id: `345963`, slug: `penpot`).
**No authentication required** — only public project data is accessed.
## When to use
- The user asks about Penpot issues, user stories, or tasks tracked in
Taiga — fetch them via the public API (project id 345963), no
authentication needed.
## Prerequisites
- `python3` — the `scripts/taiga.py` CLI script is self-contained (stdlib only)
## Quick Start
The easiest way is to use the bundled Python script:
```bash
# Pass a Taiga URL directly
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# Or use "<type> <ref>" syntax
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Add --json for raw output
python3 scripts/taiga.py --json issue 13714
# See full usage
python3 scripts/taiga.py --help
```
## URL Pattern Reference
Taiga web URLs follow these patterns:
| Type | Web URL Pattern |
|------|----------------|
| Issue | `https://tree.taiga.io/project/penpot/issue/<REF>` |
| User Story | `https://tree.taiga.io/project/penpot/us/<REF>` |
| Task | `https://tree.taiga.io/project/penpot/task/<REF>` |
To extract the **type** and **ref** from a URL:
- `issue/13714` → type=`issue`, ref=`13714`
- `us/14128` → type=`us`, ref=`14128`
- `task/13648` → type=`task`, ref=`13648`
## Python Script Reference
The `scripts/taiga.py` script wraps the Taiga API into a single convenient CLI
with sensible defaults.
### Usage
```
python3 scripts/taiga.py <taiga-url>
python3 scripts/taiga.py <type> <ref>
python3 scripts/taiga.py [--json] <taiga-url>
python3 scripts/taiga.py [--json] <type> <ref>
```
### Examples
```bash
# By URL (recommended — no need to think about type/ref)
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# By type and ref
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Raw JSON output
python3 scripts/taiga.py --json issue 13714
```
### Output
The script prints a clean, structured summary:
```text
User Story #11964 — 🔴 [DESIGN TOKENS] Typography Composite Input
================================
Status: Defining
Milestone: design-systems-sprint-26
Points: 3 role(s)
Assignee: Natacha Menjibar
Author: Natacha Menjibar
Created: 2025-09-01
Tags: iop-design-tokens
URL: https://tree.taiga.io/project/penpot/us/11964
================================
<full description text, unmodified>
```
The fields section includes type-specific information:
- **Issues:** Status, Type ID, Severity ID, Priority ID
- **User Stories:** Status, Milestone, Points
- **Tasks:** Status, Milestone, Parent US
## Reference
- API docs: https://docs.taiga.io/api.html
- Taiga instance: https://tree.taiga.io
- API base: https://api.taiga.io/api/v1
- Penpot project id: `345963`
- Penpot project slug: `penpot`

View File

@ -6,7 +6,7 @@ description: Update the project CHANGES.md with issues from a given GitHub miles
# Skill: update-changelog # Skill: update-changelog
Entry point for all changelog work. All rules — sections, exclusions, entry Entry point for all changelog work. All rules — sections, exclusions, entry
format, the `:boom:` and `:rocket:` label contracts, pre-flight, and the format, the `:boom:`, `:rocket:` and `:lock:` contracts, pre-flight, and the
anomaly report — live in `mem:workflow/updating-changelog` (file: anomaly report — live in `mem:workflow/updating-changelog` (file:
`.serena/memories/workflow/updating-changelog.md`). This skill routes to that `.serena/memories/workflow/updating-changelog.md`). This skill routes to that
memory; it does not repeat it. memory; it does not repeat it.
@ -28,18 +28,25 @@ memory; it does not repeat it.
## What to Read ## What to Read
1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth, 1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth,
layout, entry rules, the `## Breaking change note` comment contract, layout, entry rules, the `## Breaking change note` comment contract, the
pre-flight checks, anomaly types, and pitfalls. advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
2. `mem:scripts/changelog` — `boom`, `check-merged`, `cross-ref`, `report`. 2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`. `dev-only-bugs`, `report`.
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
## Commands ## Commands
```bash ```bash
python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog" python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"
python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog" # closed non-bug residue to review by hand
python3 scripts/gh.py advisories --patched-version "<VERSION>"
python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
python3 scripts/changelog.py check-merged <PR numbers...> python3 scripts/changelog.py check-merged <PR numbers...>
python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py security "<VERSION>" # print the :lock: section
python3 scripts/changelog.py security "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py cross-ref "<VERSION>" python3 scripts/changelog.py cross-ref "<VERSION>"
python3 scripts/changelog.py report "<VERSION>" python3 scripts/changelog.py report "<VERSION>"
``` ```
@ -48,5 +55,7 @@ python3 scripts/changelog.py report "<VERSION>"
- `CHANGES.md` is edited only on explicit request or during the release process. - `CHANGES.md` is edited only on explicit request or during the release process.
- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline. - Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
- `:boom:` and `:rocket:` are label-generated: never hand-edit them; edit the label or the `## Breaking change note` comment and re-run the command. - `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
- Verify PR merge status before writing; only merged PRs are referenced. - Verify PR merge status before writing; only merged PRs are referenced.
- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.

View File

@ -9,7 +9,8 @@
app.common.data.macros/get-in clojure.core/get-in app.common.data.macros/get-in clojure.core/get-in
app.common.data.macros/with-open clojure.core/with-open app.common.data.macros/with-open clojure.core/with-open
app.common.data.macros/select-keys clojure.core/select-keys app.common.data.macros/select-keys clojure.core/select-keys
app.common.logging/with-context clojure.core/do} app.common.logging/with-context clojure.core/do
benches.render-wasm.errors/attempt clojure.core/do}
:hooks :hooks
{:analyze-call {:analyze-call
@ -24,7 +25,11 @@
shadow.lazy/loadable hooks.export/rumext-lazycomponent shadow.lazy/loadable hooks.export/rumext-lazycomponent
app.util.i18n/tr hooks.i18n/tr-dynamic app.util.i18n/tr hooks.i18n/tr-dynamic
app.common.i18n/tr hooks.i18n/tr-dynamic app.common.i18n/tr hooks.i18n/tr-dynamic
}} benches.render-wasm.declarations/defscene hooks.benches/defscene
benches.render-wasm.declarations/defcase hooks.benches/defcase
benches.render-wasm.runtime.commands/defcommand hooks.benches/defcommand
benches.render-wasm.runtime.protocol/resolve-or-reject hooks.benches/resolve-or-reject
benches.render-wasm.runtime.protocol/resolve-or-reject-with hooks.benches/resolve-or-reject-with}}
:output :output
{:exclude-files {:exclude-files
@ -36,7 +41,7 @@
"^(?:backend|frontend|exporter|common)/deps.edn" "^(?:backend|frontend|exporter|common)/deps.edn"
"^(?:backend|frontend|exporter|common)/scripts/" "^(?:backend|frontend|exporter|common)/scripts/"
"^(?:backend|frontend|exporter|common)/dev/" "^(?:backend|frontend|exporter|common)/dev/"
"^(?:backend|frontend|exporter|common)/test/"] "^(?:backend|frontend|exporter|common)/test/(?!performance_tests/)"]
:linter-name true} :linter-name true}
@ -101,6 +106,4 @@
:unused-binding :unused-binding
{:exclude-destructured-as true {:exclude-destructured-as true
:exclude-destructured-keys-in-fn-args false :exclude-destructured-keys-in-fn-args false}}}
}
}}

View File

@ -0,0 +1,77 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.benches
(:require [clj-kondo.hooks-api :as api]))
(defn defscene
"Analyzes scene options and its compiled builder body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcase
"Analyzes case options and its compiled operation body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and (seq body) argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcommand
"Analyzes command options and both generated functions."
[{:keys [node]}]
(let [[_ command & forms] (:children node)
[doc forms] (if (string? (api/sexpr (first forms)))
[(first forms) (next forms)]
[nil forms])
[options forms] (if (api/map-node? (first forms))
[(first forms) (next forms)]
[nil forms])
[argv & body] forms
[conditions body] (if (and (api/map-node? (first body)) (next body))
[(first body) (next body)]
[nil body])
handlers (:async (some-> options api/sexpr))
core (api/list-node
(into [(api/token-node 'fn)
(api/vector-node (mapv api/token-node handlers))]
body))
starred (api/token-node (symbol (str (api/sexpr command) "*")))
measured (get (some-> options api/sexpr) :measured-variant true)
definition (api/list-node
(into [(api/token-node 'defn) command]
(concat (when doc [doc]) [argv]
(when conditions [conditions]) [core])))]
{:node (api/list-node
(into [(api/token-node 'do)]
(concat (when options [(api/coerce (dissoc (api/sexpr options) :async))])
[definition]
(when measured
[(api/list-node [(api/token-node 'def)
starred command])]))))}))
(defn resolve-or-reject
"Analyzes the body of a Promise-returning form."
[{:keys [node]}]
(let [[_ & body] (:children node)]
{:node (api/list-node (into [(api/token-node (quote do))] body))}))
(defn resolve-or-reject-with
"Analyzes the cause handler and body. Binds `cause` in the handler only."
[{:keys [node]}]
(let [[_ cause on-error & body] (:children node)]
{:node (api/list-node
(into [(api/token-node (quote do))
(api/list-node
[(api/token-node (quote let))
(api/vector-node [cause (api/token-node nil)])
on-error])]
body))}))

View File

@ -1,3 +1,9 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.export (ns hooks.export
(:require [clj-kondo.hooks-api :as api])) (:require [clj-kondo.hooks-api :as api]))

View File

@ -10,5 +10,7 @@
:extra-indents {rumext.v2/fnc [[:inner 0]] :extra-indents {rumext.v2/fnc [[:inner 0]]
cljs.test/async [[:inner 0]] cljs.test/async [[:inner 0]]
promesa.exec/thread [[:inner 0]] promesa.exec/thread [[:inner 0]]
specify! [[:inner 0] [:inner 1]]} specify! [[:inner 0] [:inner 1]]
benches.render-wasm.runtime.commands/defcommand [[:inner 0]]
benches.render-wasm.runtime.protocol/resolve-or-reject-with [[:inner 0]]}
} }

View File

@ -1,7 +1,7 @@
description: Create a report to help us improve description: An unexpected problem or behavior
name: Bug report
type: Bug
labels: ["needs triage"] labels: ["needs triage"]
name: Bug
type: Bug
body: body:
- type: markdown - type: markdown
@ -15,7 +15,7 @@ body:
- type: textarea - type: textarea
attributes: attributes:
label: Steps To Reproduce label: Steps To Reproduce
description: Steps to reproduce the behavior. description: Steps to reproduce the behavior
placeholder: | placeholder: |
Steps to reproduce the behavior: Steps to reproduce the behavior:
1. Go to '...' 1. Go to '...'
@ -26,22 +26,22 @@ body:
- type: textarea - type: textarea
id: expected id: expected
attributes: attributes:
description: A clear and concise description of what you expected to happen.
label: Expected behavior label: Expected behavior
description: A clear and concise description of what you expected to happen
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: actual id: actual
attributes: attributes:
description: A clear and concise description of what happens instead; what the bug is.
label: Actual behavior label: Actual behavior
description: A clear and concise description of what happens instead; what the bug is
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: screenshots id: screenshots
attributes: attributes:
description: If applicable, add screenshots to help explain your problem.
label: Screenshots or video label: Screenshots or video
description: If applicable, add screenshots to help explain your problem
- type: textarea - type: textarea
id: desktop id: desktop
attributes: attributes:
@ -87,4 +87,4 @@ body:
id: additional-context id: additional-context
attributes: attributes:
label: Additional context label: Additional context
description: Any other context about the problem. description: Add any other context or screenshots about the bug here

View File

@ -1 +1 @@
blank_issues_enabled: true blank_issues_enabled: false

38
.github/ISSUE_TEMPLATE/enhancement.yml vendored Normal file
View File

@ -0,0 +1,38 @@
description: Improvement to existing behavior
labels: ["needs triage"]
name: "Enhancement"
type: Enhancement
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: problem
attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
validations:
required: true
- type: textarea
id: solution
attributes:
label: Describe the solution you'd like
description: A clear and concise description of what you want to happen
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the enhancement here

26
.github/ISSUE_TEMPLATE/epic.yml vendored Normal file
View File

@ -0,0 +1,26 @@
description: An epic is a large user story that cannot be delivered as defined within a single iteration
labels: ["needs triage"]
name: "EPIC"
type: EPIC
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: description
attributes:
label: Epic description
description: Tell us which epic feature do you want to discuss
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the feature request here

View File

@ -1,7 +1,7 @@
description: Suggest an idea for this project. description: Suggest an idea for this project
labels: ["needs triage"] labels: ["needs triage"]
name: "Feature request" name: "Feature request"
type: Enhancement type: Feature
body: body:
- type: markdown - type: markdown
@ -15,24 +15,24 @@ body:
- type: textarea - type: textarea
id: problem id: problem
attributes: attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...) description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
label: Is your feature request related to a problem? Please describe.
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: solution id: solution
attributes: attributes:
description: A clear and concise description of what you want to happen. label: Describe the solution you'd like
label: Describe the solution you'd like. description: A clear and concise description of what you want to happen
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: alternatives id: alternatives
attributes: attributes:
label: Describe alternatives you've considered. label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered. description: A clear and concise description of any alternative solutions or features you've considered
- type: textarea - type: textarea
id: additional-context id: additional-context
attributes: attributes:
label: Additional context label: Additional context
description: Add any other context or screenshots about the feature request here. description: Add any other context or screenshots about the feature request here

26
.github/ISSUE_TEMPLATE/task.yml vendored Normal file
View File

@ -0,0 +1,26 @@
description: A specific piece of work
labels: []
name: "Task"
type: Task
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: task
attributes:
description: A clear and concise description of the task
label: Please describe
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the task here

View File

@ -5,7 +5,7 @@ name: Auto Label and Add to Project
# code, it only reads ids from the event payload. # code, it only reads ids from the event payload.
on: on:
issues: issues:
types: [opened] types: [opened, closed]
pull_request_target: pull_request_target:
types: [opened] types: [opened]
@ -43,6 +43,9 @@ jobs:
// === 1. CONFIGURATION === // === 1. CONFIGURATION ===
const PROJECT_NUMBER = 8; // <--- Replace with your project board number const PROJECT_NUMBER = 8; // <--- Replace with your project board number
const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization
const STATUS_FIELD_ID = 'PVTSSF_lADOAcyBPM4BV2pqzhRPY48';
const STATUS_BACKLOG = 'f75ad846';
const STATUS_TECH_REVIEW = 'b0f45db9';
const OWNER = context.repo.owner; const OWNER = context.repo.owner;
const REPO = context.repo.repo; const REPO = context.repo.repo;
@ -50,6 +53,100 @@ jobs:
const isPR = !!context.payload.pull_request; const isPR = !!context.payload.pull_request;
const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id; const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id;
// A completed issue is already closed by GitHub. Route its Main
// project status according to whether QA needs to verify it.
if (context.eventName === 'issues' && context.payload.action === 'closed') {
const issue = context.payload.issue;
if (issue.state_reason !== 'completed') {
console.log(`Issue #${issue.number} was closed as ${issue.state_reason}; leaving project status unchanged.`);
return;
}
const targetStatus = issue.labels.some(({ name }) => name === 'team-qa')
? 'Ready for QA'
: 'Product Review';
const projectQuery = `
query($owner: String!, $number: Int!) {
organization(login: $owner) {
projectV2(number: $number) {
id
fields(first: 100) {
nodes {
... on ProjectV2SingleSelectField {
id
name
options { id name }
}
}
}
}
}
}
`;
const projectRes = await github.graphql(projectQuery, {
owner: OWNER,
number: PROJECT_NUMBER
});
const project = projectRes.organization.projectV2;
if (!project) {
throw new Error(`Could not find Main project #${PROJECT_NUMBER}.`);
}
const statusField = project.fields.nodes.find(({ name }) => name === 'Status');
const statusOption = statusField?.options.find(({ name }) => name === targetStatus);
if (!statusField || !statusOption) {
throw new Error(`Could not find Status option "${targetStatus}" in Main.`);
}
const issueProjectItemsQuery = `
query($issueId: ID!) {
node(id: $issueId) {
... on Issue {
projectItems(first: 100) {
nodes {
id
project { id }
}
}
}
}
}
`;
const issueProjectItemsRes = await github.graphql(issueProjectItemsQuery, {
issueId: contentId
});
const projectItems = issueProjectItemsRes.node?.projectItems?.nodes || [];
const projectItem = projectItems.find(
({ project: itemProject }) => itemProject.id === project.id
);
if (!projectItem) {
console.log(`Issue #${issue.number} is not in Main project #${PROJECT_NUMBER}; skipping status update.`);
return;
}
const updateStatusMutation = `
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId
itemId: $itemId
fieldId: $fieldId
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item { id }
}
}
`;
await github.graphql(updateStatusMutation, {
projectId: project.id,
itemId: projectItem.id,
fieldId: statusField.id,
optionId: statusOption.id
});
console.log(`Issue #${issue.number} closed; Main status set to "${targetStatus}".`);
return;
}
// Define your labels here // Define your labels here
const labelToApply = 'needs triage'; const labelToApply = 'needs triage';
@ -88,6 +185,65 @@ jobs:
} }
`; `;
await github.graphql(addToProjectMutation, { projectId, contentId }); let itemId;
try {
const addRes = await github.graphql(addToProjectMutation, { projectId, contentId });
itemId = addRes.addProjectV2ItemById.item.id;
} catch (error) {
const messages = [
error && error.message,
...(Array.isArray(error && error.errors)
? error.errors.map(e => e && e.message)
: [])
].filter(Boolean).join('\n');
if (!messages.includes('Content already exists in this project')) {
throw error;
}
console.log('Item already in project, fetching existing item...');
const existing = await github.graphql(
`query($contentId: ID!) {
node(id: $contentId) {
... on Issue {
projectItems(first: 50) {
nodes { id project { id } }
}
}
... on PullRequest {
projectItems(first: 50) {
nodes { id project { id } }
}
}
}
}`,
{ contentId }
);
const nodes = (existing.node && existing.node.projectItems && existing.node.projectItems.nodes) || [];
const match = nodes.find(n => n.project.id === projectId);
if (!match) {
throw new Error(`Item already in a project, but not found in project ${projectId}`);
}
itemId = match.id;
}
// === 4. SET PROJECT STATUS ===
const statusOptionId = isPR ? STATUS_TECH_REVIEW : STATUS_BACKLOG;
const statusName = isPR ? 'Tech Review' : 'Backlog';
console.log(`Setting status to "${statusName}" for ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
await github.graphql(
`mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId,
itemId: $itemId,
fieldId: $fieldId,
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item {
id
}
}
}`,
{ projectId, itemId, fieldId: STATUS_FIELD_ID, optionId: statusOptionId }
);
console.log("Automation successfully completed!"); console.log("Automation successfully completed!");

View File

@ -38,13 +38,13 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry (push destination) - name: Login to Docker Registry (push destination)
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry (base image pull) - name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -200,7 +200,7 @@ jobs:
ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }} ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }}
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
@ -210,7 +210,7 @@ jobs:
# images from DockerHub for unregistered users. # images from DockerHub for unregistered users.
# https://docs.docker.com/docker-hub/usage/ # https://docs.docker.com/docker-hub/usage/
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
@ -222,7 +222,7 @@ jobs:
# reuses the same PUB_DOCKER_* credentials as the DockerHub # reuses the same PUB_DOCKER_* credentials as the DockerHub
# login above. # login above.
- name: Login to Docker Hardened Images registry (base image pull) - name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -301,7 +301,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
@ -371,7 +371,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}

View File

@ -114,20 +114,20 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -191,13 +191,13 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -242,7 +242,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -34,7 +34,7 @@ permissions: {}
env: env:
ACTIONLINT_VERSION: '1.7.12' ACTIONLINT_VERSION: '1.7.12'
ZIZMOR_VERSION: '1.31.1' ZIZMOR_VERSION: '1.32.0'
jobs: jobs:
lint: lint:

View File

@ -48,10 +48,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that # No dependency cache here: a deploy must not restore a cache that
# other workflows can write to. # other workflows can write to.
- name: Setup pnpm + Node.js - name: Setup pnpm + Node.js
uses: pnpm/setup@fa36a5ef7dcdefc3a7cb25655c122e00cf6e636e # v3.1.2 uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
with: with:
working-directory: plugins working-directory: plugins
runtime: node@26.13.0 runtime: node@24.22.3
install: false install: false
- name: Install deps - name: Install deps

View File

@ -46,10 +46,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that # No dependency cache here: a deploy must not restore a cache that
# other workflows can write to. # other workflows can write to.
- name: Setup pnpm + Node.js - name: Setup pnpm + Node.js
uses: pnpm/setup@fa36a5ef7dcdefc3a7cb25655c122e00cf6e636e # v3.1.2 uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
with: with:
working-directory: plugins working-directory: plugins
runtime: node@26.13.0 runtime: node@24.22.3
install: false install: false
- name: Install deps - name: Install deps

View File

@ -0,0 +1,90 @@
name: Require Linked Issue
# A pull request must have at least one closing-issue reference, unless it
# carries the `no issue required` label.
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- labeled
- unlabeled
- ready_for_review
- converted_to_draft
permissions:
contents: read
pull-requests: read
issues: read
jobs:
check-linked-issue:
name: Require Linked Issue
if: ${{ !github.event.pull_request.draft }}
runs-on: ubuntu-latest
steps:
- name: Check linked issue
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
OWNER: ${{ github.repository_owner }}
REPO: ${{ github.event.repository.name }}
run: |
set -euo pipefail
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
result=$(gh api graphql \
-F owner="$OWNER" \
-F repo="$REPO" \
-F number="$PR_NUMBER" \
-f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
isDraft
labels(first: 100) {
nodes {
name
}
}
closingIssuesReferences(first: 100) {
nodes {
number
title
url
}
}
}
}
}
')
pull_request=$(echo "$result" | jq '.data.repository.pullRequest')
if [ "$(echo "$pull_request" | jq -r '.isDraft')" = "true" ]; then
echo "Draft PR: linked issue check skipped."
exit 0
fi
has_exemption=$(echo "$pull_request" | jq -r '
.labels.nodes | map(.name) | index("no issue required") != null
')
if [ "$has_exemption" = "true" ]; then
echo 'Label "no issue required" found: linked issue check skipped.'
exit 0
fi
issues=$(echo "$pull_request" | jq -r '
.closingIssuesReferences.nodes[]
| "- #\(.number): \(.title) (\(.url))"
')
if [ -z "$issues" ]; then
echo '::error::This pull request must have a linked issue, or the "no issue required" label.'
exit 1
fi
echo "Linked issue(s):"
echo "$issues"

View File

@ -0,0 +1,76 @@
name: Require Milestone on Linked Issues
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- ready_for_review
- converted_to_draft
workflow_dispatch:
permissions:
contents: read
pull-requests: read
issues: read
jobs:
check-linked-issues:
name: Require Milestones on Linked Issues
runs-on: ubuntu-latest
steps:
- name: Check linked issues
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
IS_DRAFT: ${{ github.event.pull_request.draft }}
REPOSITORY: ${{ github.repository }}
run: |
if [ "$IS_DRAFT" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
OWNER="${REPOSITORY%%/*}"
REPO="${REPOSITORY#*/}"
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
ISSUES=$(gh api graphql \
-F owner="$OWNER" \
-F repo="$REPO" \
-F number="$PR_NUMBER" \
-f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
closingIssuesReferences(first: 100) {
nodes {
number
title
url
milestone {
title
}
}
}
}
}
}
')
MISSING=$(echo "$ISSUES" | jq -r '
.data.repository.pullRequest.closingIssuesReferences.nodes[]
| select(.milestone == null)
| "- #\(.number): \(.title) (\(.url))"
')
if [ -n "$MISSING" ]; then
echo "::error::The following linked issues have no milestone:"
echo "$MISSING"
exit 1
fi
echo "All linked issues have a milestone."

45
.github/workflows/require-milestone.yml vendored Normal file
View File

@ -0,0 +1,45 @@
name: Require Milestone
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- milestoned
- demilestoned
- ready_for_review
- converted_to_draft
permissions:
contents: read
pull-requests: read
jobs:
check-milestone:
name: Require Milestone
runs-on: ubuntu-latest
steps:
- name: Check milestone
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
pr_api="repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}"
is_draft=$(gh api "$pr_api" --jq '.draft')
milestone=$(gh api "$pr_api" --jq '.milestone.title // ""')
if [ "$is_draft" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
if [ -z "$milestone" ]; then
echo "::error::This pull request must have a milestone."
exit 1
fi
echo "Milestone assigned: $milestone"

View File

@ -34,6 +34,9 @@ permissions:
jobs: jobs:
test-backend: test-backend:
if: ${{ !github.event.pull_request.draft }} if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Backend Tests" name: "Backend Tests"
runs-on: penpot-extended-runner runs-on: penpot-extended-runner
container: container:
@ -91,4 +94,39 @@ jobs:
run: | run: |
mkdir -p /tmp/penpot; mkdir -p /tmp/penpot;
clojure -M:dev:test clojure -M:dev:test:coverage
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('backend/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Backend test coverage" \
--note "Namespaces under \`backend/src/app\`." \
--lcov backend/target/coverage/lcov.info \
> backend/target/coverage/summary.md
cat backend/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('backend/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Backend \
--lcov backend/target/coverage/lcov.info \
--emma-xml backend/target/coverage/coverage.xml \
--prefix backend/
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: backend-coverage
path: backend/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7

View File

@ -32,6 +32,9 @@ permissions:
jobs: jobs:
test-common: test-common:
if: ${{ !github.event.pull_request.draft }} if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Common Tests" name: "Common Tests"
runs-on: penpot-extended-runner runs-on: penpot-extended-runner
container: container:
@ -58,3 +61,38 @@ jobs:
working-directory: ./common working-directory: ./common
run: | run: |
./scripts/test ./scripts/test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('common/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Common JVM test coverage" \
--note "Namespaces under \`common/src/app\`, measured by the JVM tests." \
--lcov common/target/coverage/lcov.info \
> common/target/coverage/summary.md
cat common/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('common/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Common \
--lcov common/target/coverage/lcov.info \
--emma-xml common/target/coverage/coverage.xml \
--prefix common/
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: common-coverage
path: common/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7

View File

@ -36,6 +36,9 @@ permissions:
jobs: jobs:
test-frontend: test-frontend:
if: ${{ !github.event.pull_request.draft }} if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Frontend Tests" name: "Frontend Tests"
runs-on: penpot-extended-runner runs-on: penpot-extended-runner
container: container:
@ -66,6 +69,42 @@ jobs:
run: | run: |
./scripts/test ./scripts/test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('frontend/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Frontend unit test coverage" \
--note "Logic namespaces under \`frontend/src/app\`, without \`app/main/ui\`." \
--lcov frontend/coverage/lcov.info \
--unloaded frontend/coverage/unloaded.txt \
> frontend/coverage/summary.md
cat frontend/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('frontend/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Frontend \
--lcov frontend/coverage/lcov.info \
--json-summary frontend/coverage/coverage-summary.json \
--unloaded frontend/coverage/unloaded.txt
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: frontend-coverage
path: frontend/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7
- name: Component Tests - name: Component Tests
working-directory: ./frontend working-directory: ./frontend
env: env:

View File

@ -32,6 +32,9 @@ permissions:
jobs: jobs:
test-render-wasm: test-render-wasm:
if: ${{ !github.event.pull_request.draft }} if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Render WASM Tests" name: "Render WASM Tests"
runs-on: penpot-extended-runner runs-on: penpot-extended-runner
container: container:
@ -49,14 +52,71 @@ jobs:
- name: Format - name: Format
working-directory: ./render-wasm working-directory: ./render-wasm
run: | run: |
cargo fmt --check ./fmt --check
- name: Lint - name: Lint
working-directory: ./render-wasm working-directory: ./render-wasm
run: | run: |
./lint ./lint
- name: Install cargo-llvm-cov
env:
CARGO_LLVM_COV_VERSION: "0.9.1"
run: |
case "$(uname -m)" in
aarch64|arm64)
arch="aarch64"
sha256="abf5f13c1520f8756d2192bfaaeadb0208f5b7eaa8cadc15bf847befa42b7360"
;;
*)
arch="x86_64"
sha256="b3f68e625481fed9b16444174f3fa5ebcdbde4a1878803a35eabe2dcefcdc41a"
;;
esac
archive="cargo-llvm-cov-${arch}-unknown-linux-gnu.tar.gz"
curl -fsSL --retry 3 -o "$RUNNER_TEMP/$archive" \
"https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${CARGO_LLVM_COV_VERSION}/${archive}"
echo "${sha256} $RUNNER_TEMP/$archive" | sha256sum -c -
tar -xzf "$RUNNER_TEMP/$archive" -C /usr/local/bin cargo-llvm-cov
cargo llvm-cov --version
- name: Test - name: Test
working-directory: ./render-wasm working-directory: ./render-wasm
env:
RENDER_WASM_COVERAGE: "true"
run: | run: |
./test ./test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Render WASM test coverage" \
--note "Rust sources under \`render-wasm/src\`, measured by cargo-llvm-cov on the native test build." \
--lcov render-wasm/target/coverage/lcov.info \
> render-wasm/target/coverage/summary.md
cat render-wasm/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project "Render WASM" \
--lcov render-wasm/target/coverage/lcov.info \
--llvm-json render-wasm/target/coverage/summary.json
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: render-wasm-coverage
path: render-wasm/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7

3
.gitignore vendored
View File

@ -68,6 +68,7 @@ opencode.json
/frontend/playwright-report/ /frontend/playwright-report/
/frontend/playwright/ui/visual-specs/ /frontend/playwright/ui/visual-specs/
/frontend/text-editor/src/wasm/ /frontend/text-editor/src/wasm/
/frontend/coverage/
/frontend/dist/ /frontend/dist/
/frontend/npm-debug.log /frontend/npm-debug.log
/frontend/out/ /frontend/out/
@ -121,7 +122,7 @@ opencode.json
/.opencode/reports /.opencode/reports
/.opencode/prompts /.opencode/prompts
/.ci-logs /.ci-logs
/tmp/ /tmp
/.codex/ /.codex/
/scripts/__pycache__/ /scripts/__pycache__/
/tools/__pycache__ /tools/__pycache__

2
.nvmrc
View File

@ -1 +1 @@
v24.22.2 v24.22.3

View File

@ -1,10 +0,0 @@
---
description: Create a PR for the current task branch or update an existing one — loads and follows the create-pr skill
agent: build
---
Load the **`create-pr`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -1,10 +0,0 @@
---
description: Execute a ready plan — task checklist, your confirmation, then all tasks with one commit (default) or step by step with a commit and a pause per task; creates issue + branch when on a base branch, or commits on the current branch with "direct" — loads and follows the implement-plan skill
agent: build
---
Load the **`implement-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -1,10 +0,0 @@
---
description: Investigate the chosen task, produce an implementation plan, and save it — loads and follows the make-a-plan skill
agent: build
---
Load the **`make-a-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -1,6 +0,0 @@
---
description: Resolve local git conflicts and stage the resolved files; never continues the rebase — loads and follows the resolve-git-conflicts skill
agent: build
---
Load the **`resolve-git-conflicts`** skill and follow it as your only instruction.

View File

@ -1,10 +0,0 @@
---
description: Code review — review a diff, PR, or code change — loads and follows the review-code skill
agent: build
---
Load the **`review-code`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -1,10 +0,0 @@
---
description: Plan review — evaluate an implementation plan before executing it — loads and follows the review-plan skill
agent: build
---
Load the **`review-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -11,8 +11,10 @@
- Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in. - Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in.
- Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`. - Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`.
- Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI. - Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
- Organization SSO (Nitrate): the RPC `check-nitrate-sso` takes the route's **query string proper** (`:query-string`, e.g. `screen=workspace&team-id=…`), never a URL and never with the `?` separator: the client strips it (`rt/token->query-string`) and the server puts it back. The server builds the destination with `app.auth.oidc/build-dest-url` (`public-uri` + `?` + the params, parsed and rendered back), so the redirect can never leave the public URI; a value whose parsed keys are not param names is not a route: a URL parses into a single key holding the whole URL, and a key carrying `:`, `/`, `?` or `#` is rejected with a `:warn` and the user lands on the public URI (the RPC never fails because of it). Do not reason from `u/map->query-string`: it keeps a valueless key as `key=`, so rendering is not what filters a URL out — the key shape is. A leading `?` is still trimmed server-side, so a client bundled before the trim keeps its destination. Never require `screen`: the root of the app carries its params instead of a screen (`?template=…`), and `check-nitrate-sso` is called from that branch too (`dtm/resolve-login-team-id`, which today reads only `:authorized` and never follows the redirect, so a wrong destination there is a silent `:warn` and not a visible break); the callback re-checks the destination before redirecting, which covers state tokens minted before that. The state token also carries `:profile-id`: only that session is granted the `props :sso` entry (`session-mismatch` otherwise, and the redirect still happens).
- Organization SSO needs `app.nitrate/organization-access?` whenever the RPC receives an `:organization-id` (that id always comes from the client): the profile must be a member of the organization or its owner; otherwise the RPC answers `{:authorized true :reason :no-organization-access}`, the same short-circuit shape as `:no-team-access`.
- Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation. - Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. - When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. SSO registration (OIDC and compatible providers) follows the OIDC callback gate: it requires `:registration` or `:oidc-registration`, or a team invitation token issued for the registering email (an invitation substitutes the flags only when `member-email` matches the OIDC/`prepared-register` email via `clean-email`, as in the password flow, which raises `:email-does-not-match-invitation` otherwise). The OIDC callback verifies the invitation JWT and the email before redirecting to register, and `register-profile` revalidates the invitation under a row lock when no SSO flag is set; it never requires password login. Only OIDC-typed backends (`oidc-provider-types` in `app.auth.oidc`) take the SSO registration gate; any other `:backend` in a prepared-register token is rejected as `:invalid-token`.
- Access tokens (`app.http.access-token`): personal tokens have JWT issuer `access-token` and DB type other than `mcp`; MCP tokens have issuer `urn:penpot:mcp-token` (`actoken/mcp-token-issuer`) and DB type `mcp`. Authentication requires issuer and type to agree, and the claim `uid` to match the row's profile; other issuers and undecryptable tokens are treated as anonymous. - Access tokens (`app.http.access-token`): personal tokens have JWT issuer `access-token` and DB type other than `mcp`; MCP tokens have issuer `urn:penpot:mcp-token` (`actoken/mcp-token-issuer`) and DB type `mcp`. Authentication requires issuer and type to agree, and the claim `uid` to match the row's profile; other issuers and undecryptable tokens are treated as anonymous.
- Management RPC (`/api/management/methods/:name`) is shared-key only. The middleware authenticates `<id> <secret>`; the RPC wrapper `wrap-shared-key-authorization` authorizes the key id against the method's `::rpc/allowed-shared-keys` set. Absent metadata accepts any configured key; all current methods declare one (`exporter`/`media-processor` for `upload-tempfile`, `nexus` for file analysis, `exporter` for jobs, `admin-console` for Nitrate). A valid key outside the set answers 403 `:shared-key-not-allowed`; denied calls do not reach the audit wrapper. - Management RPC (`/api/management/methods/:name`) is shared-key only. The middleware authenticates `<id> <secret>`; the RPC wrapper `wrap-shared-key-authorization` authorizes the key id against the method's `::rpc/allowed-shared-keys` set. Absent metadata accepts any configured key; all current methods declare one (`exporter`/`media-processor` for `upload-tempfile`, `nexus` for file analysis, `exporter` for jobs, `admin-console` for Nitrate). A valid key outside the set answers 403 `:shared-key-not-allowed`; denied calls do not reach the audit wrapper.
- Personal tokens need flag `:access-tokens`. MCP tokens need `:mcp` or `:access-tokens`, and only on routes whose `actoken/authz` gets `::actoken/allow-mcp true` (the RPC routes); elsewhere (e.g. assets) they are anonymous. - Personal tokens need flag `:access-tokens`. MCP tokens need `:mcp` or `:access-tokens`, and only on routes whose `actoken/authz` gets `::actoken/allow-mcp true` (the RPC routes); elsewhere (e.g. assets) they are anonymous.
@ -24,13 +26,16 @@
- `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence. - `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence.
- Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read. - Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read.
- File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2). - File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2).
- The organization-owner fallback (`get-organization-owner-permissions` via Nitrate over HTTP) grants read-only access for a non-member organization owner. It resolves the owning team through `resolve-team-id`, which refuses soft-deleted file/project/team: the file and project lookups filter `deleted_at`, and a final `db/get*` check covers the team row itself — `nil` means no access. `db/get*` only filters soft-deleted rows when `deleted_at` is in the projection. The re-checks of open websocket subscriptions rely on this (`mem:backend/websocket-subscriptions`).
- File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics. - File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics.
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. - Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. RPCs that accept `share-id` enforce the link's page scope before mutating (`share-link-pages` + `check-page-scope`): a thread or page outside the link raises `:not-found`, and that check runs before the ownership checks. `get-profiles-for-file-comments` filters its query by the same pages. `mark-all-threads-as-read` does not take `share-id`.
## Teams, projects, and invitations ## Teams, projects, and invitations
- Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so. - Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so.
- Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token. - Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token.
- The whole invitation family is team-admin only: creating, updating and deleting invitations, minting the invitation token (`get-team-invitation-token`) and listing the pending ones (`get-team-invitations`, which carries the invited email and the offered role). The token of an invitation that grants the `owner` role is restricted to the team owner, because an admin cannot create nor promote an owner invitation (`cant-promote-to-owner`). This family is the exception to the rule above: it answers `:validation` / `:insufficient-permissions` instead of `:not-found`, so the five commands speak one language.
- With the `:admin-console` flag, the organization `send-invitations` policy (role value plus Nitrate `restrictions-met`, through `cto/allowed?`) gates every invitation command that hands out or manages access: create (and resend, which reuses it), `update-team-invitation-role`, `delete-team-invitation` and `get-team-invitation-token`. Only `get-team-invitations` stays on team admin alone, so a blocked admin still sees the list. The shared helper `check-send-invitations-policy` in `teams_invitations.clj` does a strict Nitrate lookup (fails closed; 404 means no organization) and runs outside a DB transaction, which is why the role and delete commands carry no `::db/transaction`.
- Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup. - Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup.
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing. - Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing.

View File

@ -9,12 +9,13 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment` - Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
- Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`. - Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`.
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains` - Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
- WebSocket subscriptions, the connection registry (local per backend, unlike the bus), and how a lost permission gets revoked: `mem:backend/websocket-subscriptions`.
- Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí) - Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí)
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`. - Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
## Stable namespace map ## Stable namespace map
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/rpc/command/<cmd-name>`. - `app.rpc.commands.*`: RPC command implementations exposed under `/api/main/methods/<method-name>`; `/api/rpc/command/<method-name>` remains as a backward-compatible route.
- `app.rpc.permissions`: permission predicate/check helper factories. - `app.rpc.permissions`: permission predicate/check helper factories.
- `app.http.*`: HTTP routes and middleware. - `app.http.*`: HTTP routes and middleware.
- `app.auth.*`: provider-specific authentication helpers such as LDAP/OIDC. - `app.auth.*`: provider-specific authentication helpers such as LDAP/OIDC.
@ -67,7 +68,8 @@ Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` i
- Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object. - Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids. - The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read). - Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
- The v3 import only materializes a library relation when both ids belong to the package and differ; a relation that points outside the package or a self link is ignored with a warning. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first. - The v3 and v1 imports only materialize a library relation when both ids belong to the package and the link would neither be a self link nor close a dependency cycle (the same `bfc/library-link-status` check that `link-file-to-library` uses to raise); relations are processed in sorted `(file_id, library_file_id)` order, which `get-files-rels` also returns, so the first edge wins deterministically, and every edge it ignores is logged with a warning. The v2 import stays out of that guard on purpose: it is only reachable from the developer REPL (`app.srepl.binfile`) and keeps inserting the relations as they come. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
- A version 1 package lists its files in the metadata and the importer creates one file per entry. The v1 export (gone from the product; only tests and the developer REPL use it) unioned the requested files with their libraries without deduplicating, so a package exported from a file inside a library cycle lists that file twice. The importer reads the pair of objects of every entry to keep the stream in sync, imports the first occurrence only, warns about the repeats, and answers with one id per imported file.
- The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default. - The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
- Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner. - Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
- The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote. - The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.

View File

@ -19,6 +19,12 @@
- Renewal fires when `modified-at` is older than 6h (`default-renewal-max-age`, not configurable). It `UPDATE`s the same row (`modified-at` only) and issues a new token that keeps the original `:exp`. - Renewal fires when `modified-at` is older than 6h (`default-renewal-max-age`, not configurable). It `UPDATE`s the same row (`modified-at` only) and issues a new token that keeps the original `:exp`.
- `read-session` does not check age. Idle expiration is enforced on every request by `wrap-authz` (sessions past the idle window authenticate like missing ones and are never renewed); the GC only deletes the rows afterwards. - `read-session` does not check age. Idle expiration is enforced on every request by `wrap-authz` (sessions past the idle window authenticate like missing ones and are never renewed); the GC only deletes the rows afterwards.
## Revocation paths
- Profile deletion → `session/invalidate-all` (all sessions of the profile die).
- Authenticated password change → `session/invalidate-others` (every session except the caller's dies).
- Password recovery (`recover-profile`, GHSA-3j2p-qr45-q365) → `session/invalidate-all` inside the same transaction as the password `UPDATE`; there is no current session to preserve, and a captured session must not survive recovery. Sibling of GHSA-mj9f-5cwq-7p3q (logout deletes the current session row).
## GC (`::tasks/gc`, cron `session-gc`, daily) ## GC (`::tasks/gc`, cron `session-gc`, daily)
- Deletes from `http_session_v2` where `modified_at < now - :auth-token-cookie-max-age` **or** `created_at < now - :auth-token-cookie-max-age-absolute`. - Deletes from `http_session_v2` where `modified_at < now - :auth-token-cookie-max-age` **or** `created_at < now - :auth-token-cookie-max-age-absolute`.

View File

@ -93,7 +93,7 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
| `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. | | `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
| `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. | | `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. | | `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. |
| `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. | | `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference: the `file_data.storage_object_id` column (FK `NO ACTION DEFERRABLE INITIALLY IMMEDIATE`); the object metadata keeps `file-id` and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
| `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. | | `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
| `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. | | `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
| `job-resource` | Storage objects owned by job rows (`job.resource_id`), written by `app.jobs.storage`. | No | Authentication required, owner-scoped | jobs-GC touch → reference scan. | | `job-resource` | Storage objects owned by job rows (`job.resource_id`), written by `app.jobs.storage`. | No | Authentication required, owner-scoped | jobs-GC touch → reference scan. |
@ -118,18 +118,40 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
## File Data ## File Data
- `file-data-backend` accepts `legacy-db`, `db`, or `storage`. - `file-data-backend` accepts `legacy-db`, `db`, or `storage`; `legacy-db` is read-only and the default is `db`. The configuration reads `legacy-db` as `db` at startup, with a warning (`normalize-file-data-backend`), so a deployment that kept the variable saves instead of failing on every write.
- `legacy-db` stores main data in `file.data` and snapshots in `file_change.data`. - `legacy-db` reads main data from `file.data` and snapshots from `file_change.data`; it no longer writes them (`fdata/upsert!` rejects it).
- `db` stores encoded data in `file_data.data`. - `db` stores encoded data in `file_data.data`.
- `storage` stores encoded data in storage subsystem with `file-data` bucket and keeps `data` nil in `file_data` table. - `storage` stores encoded data in storage subsystem with `file-data` bucket, keeps `data` nil in `file_data` table, and points at the object with `file_data.storage_object_id`.
- The `file_data.metadata.storage-ref-id` value points to the storage object. - `file_data.storage_object_id` is a `NO ACTION DEFERRABLE INITIALLY IMMEDIATE` foreign key to `storage_object(id)`: the database refuses to physically delete a referenced object. The `IMMEDIATE` is load-bearing: the RI violation fires inside the `storage-gc-deleted` chunk transaction, so the chunk rolls back, is logged at error level with a generic hint plus the `constraint` and `table` the database reports (any `NO ACTION` foreign key on `storage_object` can block the delete, not only `file_data`'s) and is retried on the next run; with `INITIALLY DEFERRED` the check would fire at commit, outside the catch, and the sweep would never move past the blocked chunk.
- `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row. - The column is the only source of truth. The release is a coordinated restart (no rolling deploy) when the instance uses the storage backend (`PENPOT_FILE_DATA_BACKEND=storage`) or still has rows carrying `metadata.storage-ref-id`; with the default `db` backend and no such rows the previous and the new code read the same `file_data.data` column, so a rolling deploy is safe. `metadata.storage-ref-id` is no longer written or read, and the `file-data` metadata schema does not declare it. The migration backfills the rows it can resolve (valid uuid with an existing `storage_object`) and removes the key from those rows; a corrupt or orphan pointer keeps the key, which nothing reads. A large instance runs `operations/2.20.0-file-data-storage-object-id.md`: column, index and FK online, then the backfill (only if a row still carries `metadata.storage-ref-id`) with every instance stopped, verifies, and marks the migration as applied (`INSERT INTO migrations`). The backfill must never run while the previous code serves when the storage backend is in use (or rows with the pointer remain), because that code writes and scans the metadata pointer. A rollback to the previous code cannot read the rows written by this release when the storage backend is in use.
- `fdata/upsert!` touches the `storage_object_id` it receives in the params (the pointer the persisted row had) before replacing it, so the replaced blob is reclaimed. Callers pass the column from the file map (main/restore), `nil` for a new row (fragments, snapshots, new files) or the row itself (offload); there is no query in the upsert path, which is why every caller reads the file under the row lock first (the migration-on-read path included, see `mem:backend/subtleties`). It returns `{:written <bool> :storage-object-id <uuid or nil>}`, the pointer of the object the write created (nil for `db`/`legacy-db`), and `binfile.common/update-file!` propagates it so a migrating read returns the map with the pointer of the row it persisted instead of the pre-write one. The write also compares the expected pointer with `IS NOT DISTINCT FROM` (the caller's value, not the new one) and raises `:stale-file-data-pointer` when the row changed under it, so a stale map fails instead of leaking the replaced blob; the check lives in `upsert-in-database` because the `legacy-db` branches do not use the upsert SQL.
- `binfile.common/update-file!` takes the file map read from the database plus the fields to change (`[cfg file & params]`), merges them and builds the `file_data` params from the merged map, so the pointer travels in the read map and a caller cannot lose it by building a partial map; the params cannot override `:storage-object-id`. `insert-file!` always drops it (an INSERT never replaces a persisted pointer; `duplicate-file` hands the source map there). `bfc/get-file` keeps the pointer for the write paths, and every response goes through `bfc/file->response`, which strips it and any residual `storage-ref-id` left in the metadata of the rows the migration does not backfill: each `file-data` object is authentication-only and any logged-in profile could otherwise fetch it from `/assets/by-id`.
- The in-place binfile import was removed: `import-files!` raises `:not-supported` when the cfg carries `::bfc/file-id`, so no caller can overwrite a storage-backed row without carrying its previous pointer.
- File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata. - File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
- Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer holds at least `FOR SHARE` on the `file` row. A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`. - Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil and `storage_object_id` points at the new object) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer locks the `file` row first (the update RPC and the snapshot paths with `FOR UPDATE` through `lock-for-update?`, the migration-on-read path with `FOR UPDATE`). A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`.
- The offload normalizes each `SELECT fd.*` row before `upsert!`: the row builder omits NULL columns, jsonb metadata arrives as a `PGobject`, and a JSON `null` decodes to nil and must disappear instead of travelling as `:metadata nil`, which the update schema rejects.
- Offload pacing: none by deliberate decision. Sleeps only make sense when batching multiple transactions; the per-file transaction is short by construction and the scheduler run takes minutes per production metrics, so neither loop sleeps. `jobs/heartbeat` (pool-based liveness, ~1 write/s) stays on both paths. - Offload pacing: none by deliberate decision. Sleeps only make sense when batching multiple transactions; the per-file transaction is short by construction and the scheduler run takes minutes per production metrics, so neither loop sleeps. `jobs/heartbeat` (pool-based liveness, ~1 write/s) stays on both paths.
- Offload metrics (bounded labels only, never ids or error text): `penpot_file_data_offload_rows_total{type}` and `penpot_file_data_offload_bytes_total{type}` (`type`: `main`/`snapshot`/`fragment`, successful runs only) plus `penpot_file_data_offload_timing_ms{result}` (`result`: `ok`/`skip`/`error`). Helpers live in `app.jobs.metrics`; the job-def requires `::mtx/metrics`. - Offload metrics (bounded labels only, never ids or error text): `penpot_file_data_offload_rows_total{type}` and `penpot_file_data_offload_bytes_total{type}` (`type`: `main`/`snapshot`/`fragment`, successful runs only) plus `penpot_file_data_offload_timing_ms{result}` (`result`: `ok`/`skip`/`error`). Helpers live in `app.jobs.metrics`; the job-def requires `::mtx/metrics`.
- Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`. - Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`.
- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/process-metadata`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects. - Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/touch-storage-object` with the row's `storage_object_id`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
- `storage-gc-touched` resolves the `file-data` reference with `SELECT EXISTS (SELECT 1 FROM file_data WHERE storage_object_id = ?)`, an indexed lookup that no longer parses the object metadata.
- A row with `backend='storage'` and no readable object makes `resolve-file-data` raise `:missing-storage-object` with the file and row ids instead of failing later in the storage backend.
- The `file-id` and row id kept in the storage object metadata are informational for diagnostics; nothing parses them to resolve the reference (the reference is the column).
## File Change
- `file_change` is partitioned by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition (`enable_partitionwise_join` is set on every pooled connection).
- The automatic migration (`0158-partition-file-change`) refuses to run when `file_change` is above 10 GiB total size or 2M rows (`pg_total_relation_size` + `pg_class.reltuples`, catalog reads). It analyzes the table first, because `reltuples` is -1 without statistics. It raises with the path of `operations/2.20.0-file-change-partition.md`, because the copy happens inside the migration transaction and a big instance would restart in a loop. Below the guard it raises `statement_timeout` to 30 minutes for the copy and the index builds, and analyzes the new partitions at the end, after the copy and the indexes, so the planner does not start with empty statistics.
- Its primary key is `(file_id, id)`: a partitioned table needs the partition key in the primary key, so every access filters by `file_id`. From 2.20 the four snapshot commands (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot`, `unlock-file-snapshot`) require the `file-id` param, because they use it to prune the partition; without it they answer `:params-validation`.
- It has a `kind` column: `1` is a snapshot (it has a label and its data lives in `file_data`; the read-only `legacy-db` backend keeps old payloads in `data`) and `2` is a revision (it has no label and it holds the changes). The values are in `app.features.file-changes`.
- `kind` is a deliberate reserve, not a leftover: today no query filters by it, the queries still tell a snapshot from a revision by its label (`label IS NOT NULL`), and `kind` takes over when they are rewritten. No constraint ties `kind` and `label` together on purpose: the day `kind` is the source of truth, that constraint would have to go. It stays nullable on purpose: the previous version inserts without it, and a rollback to it has to keep writing; the `NOT NULL` comes in a later migration.
- Every write of the application to a row of `file_change` moves `updated_at` to the moment of the write (`lock-by!`, `unlock!`, the snapshot rename and delete, `delete-object`, the restore of a file). The actions of the foreign keys (`ON DELETE SET NULL` on `locked_by` and `profile_id`) change the row without touching it; during the partition both tables carry the same keys, so both receive them. The runbook of the partition uses it as the watermark of its delta passes: a write of the application that leaves it behind is lost at the cutover.
- `file_change_legacy` is a frozen copy of the previous layout of the partition: it keeps the rows and the indexes of the old table. A new instance (empty table) does not keep it. On an upgraded instance the table stays until the operator drops it by hand days later (`DROP TABLE file_change_legacy;`, see `operations/2.20.0-file-change-partition.md`); nothing drops it automatically.
- A revision carries only the delta of a save (the changes). The features of the file are not repeated on it: nobody read them from there.
- A snapshot carries the features and the migrations of the file.
- It keeps the `data` column for reading only: migration `0157-migrate-legacy-file-data` moved the existing blobs to `file_data`, and `legacy-db` remains as a read-only backend for the payloads still in `data`. `fdata/upsert!` rejects `legacy-db` and the `file-data-backend` default is `db`. It has no `data_ref_id` or `data_backend` columns: nothing read them since the file storage refactor.
- It has no index for the system snapshots. The old one filtered them by `data IS NOT NULL`, and with the payload living in `file_data` the column no longer tells snapshots apart. The new one could only filter them by `deleted_at IS NULL`, but the system snapshots always carry a deletion date in the future, so it would never match a row. The query that would use it filters that date with an "or", which no partial index can serve.
- The `objects-gc` purge groups each chunk by `file_id` and locks it with `file_id = ?` and `id = ANY(?::uuid[])`: a tuple `IN ((file_id, id), ...)` gives the planner no constant partition key, so it scans all 16 partitions in every chunk. Each group is one `FOR UPDATE SKIP LOCKED` query inside the chunk transaction.
## Metrics ## Metrics

View File

@ -45,12 +45,15 @@
- `job` is deletion-protected at the SQL level (`deletion_protection__tgr`): a stray `DELETE FROM job` raises instead of dropping the row and its `job_event` cascade without touching the referenced `storage_object`. The only intended deleters, the jobs GC and the submit dedupe, disable the guard with `SET LOCAL rules.deletion_protection TO off` inside their transaction; tests use `th/db-force-delete`. - `job` is deletion-protected at the SQL level (`deletion_protection__tgr`): a stray `DELETE FROM job` raises instead of dropping the row and its `job_event` cascade without touching the referenced `storage_object`. The only intended deleters, the jobs GC and the submit dedupe, disable the guard with `SET LOCAL rules.deletion_protection TO off` inside their transaction; tests use `th/db-force-delete`.
- A missing job-def raises (`:no-job-definition`) instead of completing. Throwing with `ex-data :type ::retry` still controls retry behavior; `:strategy ::noop` retries without incrementing retry count. - A missing job-def raises (`:no-job-definition`) instead of completing. Throwing with `ex-data :type ::retry` still controls retry behavior; `:strategy ::noop` retries without incrementing retry count.
- Cron entries claim their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, submit one `job` row per entry when no active instance exists (no-overlap), and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only. - Cron entries claim their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, submit one `job` row per entry when no active instance exists (no-overlap), and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only.
- Integrant params schemas name injected components as bare `::ns/key` entries (pool, metrics, msgbus, job defs), and a bare entry is a malli registry lookup, not `any?`: the owning namespace must `sm/register!` it at load, or the backend crashes at boot with `:malli.core/invalid-schema` instead of asserting. Unit tests never build those keys, so `backend-tests.worker-test` compiles the three worker schemas to catch it. - Integrant params schemas name injected components as bare `::ns/key` entries (pool, metrics, msgbus, job defs), and a bare entry is a malli registry lookup, not `any?`: the owning namespace must `sm/register!` it at load, or the backend crashes at boot with `:malli.core/invalid-schema` instead of asserting. Unit tests never build those keys, so `backend-tests.worker-test` compiles the three worker schemas and `backend-tests.http-websocket-test` compiles the revocation-watcher one to catch it. The assertion only runs with `PENPOT_FLAGS=enable-backend-asserts`, so without it a broken schema reaches production and only crashes at boot.
- Whether a bare entry works has nothing to do with its position or with how many of them there are: it is a registry lookup, so it works only if that exact keyword was `sm/register!`'d. `[:map ::db/pool ::metrics/metrics]` is fine, `[:map :app.aaa/unregistered :app.ccc/unregistered]` throws. When a component is **not** registered as a malli type, spell it as an explicit pair instead of registering a type just to satisfy the lookup — `[:app.nitrate/client {:optional true} [:maybe :map]]` is how `auth.oidc/schema:routes-params` and `loggers.audit.archive-task/schema:audit-log-archive-params` declare it. `{:optional true}` matters because `init-key` yields `nil` when the owning feature flag (`:admin-console`) is off, and a required key would then fail the assert on a perfectly normal boot.
- `sm/valid?` catches every `Throwable` and returns `false`, so an uncompilable schema and a genuinely invalid value both surface as a bare `Assert failed:` with no cause. When an `assert-key` fires with no message, compile the schema by hand to see the real exception.
## Config and HTTP/session middleware ## Config and HTTP/session middleware
- `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them. - `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them.
- Duration config values are parsed as ISO-8601 with the `PT` prefix, so a day is written in hours (`:jobs-retention` and `:jobs-user-ttl` default to a week, and an operator writes `168h`): `7d` does not parse, and the value fails validation at boot instead of being silently ignored. - Duration config values are parsed as ISO-8601 with the `PT` prefix, so a day is written in hours (`:jobs-retention` and `:jobs-user-ttl` default to a week, and an operator writes `168h`): `7d` does not parse, and the value fails validation at boot instead of being silently ignored.
- `app.main` wires `PENPOT_DATABASE_*` into `::db/pool`, and `app.db/create-datasource-config` forwards pgjdbc's own knobs as `dataSourceProperties`, which `Properties.getProperty` silently drops unless the value is a String (hence the `str`). A query parameter of the same name in `PENPOT_DATABASE_URI` (for example `?prepareThreshold=0`) wins over the property: pgjdbc ignores the Properties value when the URL carries it, so the URI is the place to look when a variable seems to do nothing. The prepared statement cache is three optional `PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_*` keys: the threshold is the executions before a statement is server-prepared and accepts `-1` or non-negatives (`{:min -1}`; the driver clamps any other negative to `-1`, so rejecting it only turns a typo into a boot error), its `0` stops server-side prepared statements while the driver's client-side cache of parsed queries stays in use, `..._QUERIES` and `..._SIZE` reject negatives and `0` in either disables that cache, and the size is bytes rounded up to the whole MiB the driver accepts. Do not describe the threshold `0` as disabling the cache: it only stops the server-side prepares.
- `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag. - `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
- The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags. - The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413. - Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413.
@ -84,27 +87,29 @@
- The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation. - The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation.
- Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`. - Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
- SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing. - SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
- The SVG sanitizer (`app.media.svg`) is a **blocklist**, not an allowlist: it drops dangerous elements and attributes by name and escapes every text and attribute value when it writes the tree back, so a payload hidden in a value cannot become live markup again. The allowlist rework is [#12190](https://github.com/penpot/penpot/issues/12190) and lands in media-processor, not here. `sanitize-svg` also tracks the namespace bindings through the recursion and drops any element that is not in the SVG namespace, so a document whose root is not an `svg` element in the SVG namespace is rejected and a child that rebinds `xmlns` to XHTML (`iframe`/`srcdoc`/`meta`) is dropped with its subtree; it compares the scheme of an URL the way the browser does (dropping tab/CR/LF/spaces), so `java&#x09;script:` does not slip through the `javascript:` filter. The paths that store an uploaded SVG as its final object go through `app.media.svg/sanitize-svg-upload` (media upload, the URL download, both thumbnail writers and the organization logo); the three binfile imports go through `app.binfile.common/sanitize-imported-svg`, which calls `sanitize-svg` directly. The profile picture is not an SVG: `import-profile-picture` ignores an `image/svg+xml` download, because the avatar is always rendered as an image. The caller checks the size before calling it: the helper reads the whole file, and the size limit applies to the upload, not to the sanitized copy, which can be up to six times bigger.
- Upload handlers run the `media.v` checks in a fixed order: `check-upload-path`, then `check-media-type`, then `check-media-size` (or `check-font-size` for fonts). The path check gates every later read: `check-media-type` sniffs the bytes at `:path` and `check-media-size` reads `fs/size`, so a path the client named outside `allowed-upload-prefixes` must be rejected before either touches the disk. `check-media-size` leans on that and answers a file it cannot read with the path code (`:params-validation`), not `:media-max-file-size-reached`. Keep the order when touching or adding a handler; two branches that reorder these lines conflict.
- Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8. - Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
- Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error. - Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error.
- Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts. - Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts.
## File data persistence ## File data persistence
- File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data. - File data backends for writing are `db` and `storage`; `legacy-db` is a read-only alias for old payloads. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores nil data and points at the object with the `storage_object_id` column (FK `NO ACTION DEFERRABLE`).
- `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob. - `fdata/upsert!` touches the storage object the persisted row pointed at before replacing it; the pointer travels in the params (`file->file-data-params` from the read file map, `nil` for a new row, the row for the offload) and the params of `update-file!` cannot override it. `binfile.common/update-file!` merges the read file map with the fields to change, so the pointer survives; `insert-file!` drops it because an INSERT never replaces a persisted object. The column is the only source of truth and the metadata no longer carries `storage-ref-id` (the migration removes it from the backfilled rows and the write path never stores it); the release is a coordinated restart when the storage backend is in use (the previous code cannot read the new rows), while the default `db` backend reads the same data column and tolerates a rolling deploy. Responses built from a file map go through `bfc/file->response`, which strips the pointer and a residual `storage-ref-id` from the metadata.
- Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written. A pointer keeps its id while nothing changes; content (`assoc`/`without`) and metadata (`with-meta`) changes produce a fresh id, so a fragment row is never rewritten after it is written. Re-assoc with an identical value returns the same map in array and hash maps alike, so a save keeps the fragment id of every page it does not change. - Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written. A pointer keeps its id while nothing changes; content (`assoc`/`without`) and metadata (`with-meta`) changes produce a fresh id, so a fragment row is never rewritten after it is written. Re-assoc with an identical value returns the same map in array and hash maps alike, so a save keeps the fragment id of every page it does not change.
- `enable-objects-map` skips unloaded pointer pages: their fragment was already wrapped when it was written, and loading it only to wrap it again costs one page read per untouched page on every save. A page with a plain fragment is wrapped when it is first loaded or edited; a data migration that enables `fdata/objects-map` must rewrite the data itself. - `enable-objects-map` skips unloaded pointer pages: their fragment was already wrapped when it was written, and loading it only to wrap it again costs one page read per untouched page on every save. A page with a plain fragment is wrapped when it is first loaded or edited; a data migration that enables `fdata/objects-map` must rewrite the data itself.
- Snapshots share fragment rows with the live file (they are stored with pointers, not materialized). `file-gc` marks every fragment referenced neither by the current main file nor by an unpurged snapshot; the file is read unrealized and unchanged pages keep their ids via reattach, so a pass over an unchanged file writes no new fragments. - Snapshots share fragment rows with the live file (they are stored with pointers, not materialized). `file-gc` marks every fragment referenced neither by the current main file nor by an unpurged snapshot; the file is read unrealized and unchanged pages keep their ids via reattach, so a pass over an unchanged file writes no new fragments.
- A save attaches its automatic snapshot before applying pending file migrations: the snapshot keeps the pre-migration pointers, data and migration set, and restoring it leaves the next read to migrate the file. This also holds for files without the feature. The preview (`get-file-snapshot`) applies the pending migrations in memory with the live file's libraries and never persists them: the stored version keeps its data, the payload carries the snapshot's `:migrations` (an ordered set) and `fdata/objects-map` is realized for clients that do not declare it. The historical feature set is deliberately not checked against the enabled one: a version may predate a feature change. The window is narrow: the `get-file` RPC reads with `migrate?` on and its ETag hashes `available-migrations`, so a post-deploy read cannot be a 304 from the pre-deploy state and migrates before any save in the normal flow; the pre-migration snapshot only appears when the save is the first request to touch the file after the deploy (an open tab that saves without re-reading, or a non-UI writer). `create-file-snapshot` and the restore backup read with `migrate?` on, so those versions are stored already migrated. - A save attaches its automatic snapshot before applying pending file migrations: the snapshot keeps the pre-migration pointers, data and migration set, and restoring it leaves the next read to migrate the file. This also holds for files without the feature. The preview (`get-file-snapshot`) applies the pending migrations in memory with the live file's libraries and never persists them: the stored version keeps its data, the payload carries the snapshot's `:migrations` (an ordered set) and `fdata/objects-map` is realized for clients that do not declare it. The historical feature set is deliberately not checked against the enabled one: a version may predate a feature change. The window is narrow: the `get-file` RPC reads with `migrate?` on and its ETag hashes `available-migrations`, so a post-deploy read cannot be a 304 from the pre-deploy state and migrates before any save in the normal flow; the pre-migration snapshot only appears when the save is the first request to touch the file after the deploy (an open tab that saves without re-reading, or a non-UI writer). `create-file-snapshot` and the restore backup read with `migrate?` on, so those versions are stored already migrated.
- `:migrations` is always a `d/ordered-set`: `file-migrations/resolve-applied-migrations` builds one, `fmg/migrate` re-wraps the union after applying, and `file-snapshots/decode-snapshot` converts the `file_change` pgarray (as a plain vector, `set/difference` and `contains?` would compare indexes instead of names). A snapshot with NULL `:migrations` and an old `:version` predates the `file_change.migrations` column (SQL migration 0140): the preview regenerates the set from the stored version and applies that full chain, while `restore!` uses a fixed 67 baseline, so preview and restore can disagree for very old snapshots. - `:migrations` is always a `d/ordered-set`: `file-migrations/resolve-applied-migrations` builds one, `fmg/migrate` re-wraps the union after applying, and `file-snapshots/decode-snapshot` converts the `file_change` pgarray (as a plain vector, `set/difference` and `contains?` would compare indexes instead of names). A snapshot with NULL `:migrations` and an old `:version` predates the `file_change.migrations` column (SQL migration 0140): the preview regenerates the set from the stored version and applies that full chain, while `restore!` uses a fixed 67 baseline, so preview and restore can disagree for very old snapshots.
- The migration read path (`binfile.common/migrate-file`) captures `fdata/original-pointers` before realizing, reattaches unchanged positions after applying the migrations, and persists the reattached file; it returns the realized file because reads expose plain data. A migration that adds `fdata/objects-map` skips the reattach so the pages get wrapped. - The migration read path (`binfile.common/get-file*`) reads the `file` row once (`sql:get-file`, with its data columns; with the storage backend the blob lives in the object, not in the column) plus an explicit `resolve-applied-migrations`, because a row read does not resolve them. A read with no pending migration is completed by `attach-file-data` on that map: one row read and one migrations resolution. Only when a migration is pending is the row re-read under `FOR UPDATE` on the `file` row, where `need-migration?` is re-checked on the fresh map and the migrated file persisted (`binfile.common/migrate-file`); that re-read resolves the migrations again, and `migrate-file` once more after persisting, to return the applied set. The migration path holds two raw copies of the blob (the first read and the locked re-read) but a single decoded one: the alternative, a row-only probe before the read, avoided the first raw copy at the cost of an extra row read on every open, and was removed. `get-file` runs the read inside `db/tx-run!` (joining the caller's transaction when the cfg carries a connection) so the persist commits with the read; a read-only read (`:read-only?`) takes no lock and writes nothing, so it runs on a plain connection (`db/run!`) instead of opening a transaction of its own, and with a cfg that already carries a connection it still joins the caller's. It accepts a cfg map only and throws an `IllegalArgumentException` otherwise (`assert` is elided without `:backend-asserts`); a data read rejects `:lock-for-share?` (only row-only reads accept it), because the migration branch re-reads with `FOR UPDATE`, and a lock option needs a read-write transaction from the caller because PostgreSQL rejects row locks in a read-only transaction. A row-only read (`load-data?` false) resolves the applied migrations by default too (`:load-migrations?`), because the row can be completed later and the caller may need them; a caller that does not want them passes `:load-migrations? false`. Persisting is an optimization of the read (the read already returns migrated data): a row that changed between the first read and the locked read is not overwritten, and a pending migration discovered on the unlocked read is applied in memory only (the next read persists it under the lock). This was the only writer of the file row outside the lock protocol (save, library absorption, restore, `file-gc` and offload already take it); without it, a save that committed in between was overwritten by the stale map (lost content, `revn` moving back so the next save fails with `:revn-conflict`, and the object the save left as a frozen orphan). `migrate-file` captures `fdata/original-pointers` before realizing, reattaches unchanged positions after applying the migrations, and persists the reattached file; it returns that reattached map when it persists (the pages the migration does not change keep their pointers and fragment ids) and the realized map when it does not, so a read is plain only when the caller asks with `:realize?`, and a migration that adds `fdata/objects-map` skips the reattach so the pages get wrapped.
- `fdata/load-pointer` reads rows that a previous `file-gc` pass marked on purpose (explicit `::db/remove-deleted false`): the fragment is only gone once `objects-gc` purges it. The `get-file-fragment` RPC keeps rejecting marked rows. - `fdata/load-pointer` reads rows that a previous `file-gc` pass marked on purpose (explicit `::db/remove-deleted false`): the fragment is only gone once `objects-gc` purges it. The `get-file-fragment` RPC keeps rejecting marked rows.
- Bind the fragment loader through `fdata/with-pointers` (reads), `fdata/with-tracked-pointers` (writes with a loader) or `fdata/with-tracked` (writes with nothing to load yet, e.g. file creation), and `fdata/with-untracked` (validation passes that must track nothing) instead of hand-rolled `binding` forms. Two direct bindings remain on purpose: the fine-grained tracked toggles of `app.rpc.commands.files-update` (validation/repair turn tracking off, restore the existing atom and read it before repairing), and `binfile.common/encode-file` (a fresh tracked set inheriting the current one). Converge them only if atom-scoped macros (`with-tracking`, `with-inherited-tracking`) are added; today the extra indirection buys nothing. - Bind the fragment loader through `fdata/with-pointers` (reads), `fdata/with-tracked-pointers` (writes with a loader) or `fdata/with-tracked` (writes with nothing to load yet, e.g. file creation), and `fdata/with-untracked` (validation passes that must track nothing) instead of hand-rolled `binding` forms. Two direct bindings remain on purpose: the fine-grained tracked toggles of `app.rpc.commands.files-update` (validation/repair turn tracking off, restore the existing atom and read it before repairing), and `binfile.common/encode-file` (a fresh tracked set inheriting the current one). Converge them only if atom-scoped macros (`with-tracking`, `with-inherited-tracking`) are added; today the extra indirection buys nothing.
- Snapshot creation takes `FOR SHARE` on the file row: both `create-file-snapshot` and the backup read of `restore-file-snapshot` hold it from the read to the insert, so they serialize with a `file-gc` pass (`FOR UPDATE ... SKIP LOCKED`) instead of pinning fragments the pass is about to mark. `get-file-snapshot` realizes the snapshot pointers with the live file id (fragment rows are keyed by file, not by snapshot). - Snapshot creation takes `FOR UPDATE` on the file row: both `create-file-snapshot` and the backup read of `restore-file-snapshot` hold it from the read to the insert, so they serialize with a `file-gc` pass (`FOR UPDATE ... SKIP LOCKED`) and with a concurrent save, and never pin fragments the pass is about to mark. The lock is exclusive and not `FOR SHARE` because the migration-on-read path may re-read the same row with `FOR UPDATE`: a shared lock that asks for the exclusive one deadlocks two concurrent readers (one creating a snapshot, the other saving, both seeing a pending migration). `get-file-snapshot` realizes the snapshot pointers with the live file id (fragment rows are keyed by file, not by snapshot).
- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders. - `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
- A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data. - A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data.
- Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision. - Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision.
- File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only written by the `legacy-db` backend, plus the rows written before the 2025-07 file storage refactor. - File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only read by the `legacy-db` backend (it is no longer written), plus the rows written before the 2025-07 file storage refactor.
- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. `0157-migrate-legacy-file-data` moved the labelled leftovers to `file_data` and deleted the unlabelled payload rows, so from 2.19 on the label is the only marker there is. - Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. `0157-migrate-legacy-file-data` moved the labelled leftovers to `file_data` and deleted the unlabelled payload rows, so from 2.19 on the label is the only marker there is.
- The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again. - The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again.
- Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter. - Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter.

View File

@ -3,14 +3,18 @@
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`. JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs. - READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
- Test DB/Redis come from the `PENPOT_TEST_*` env vars already set in the shell (`PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`, ...); per instance they point at `penpot_test` / `penpot_test_wsN` and valkey DB `6+N` (`mem:devenv/core`). Do not override them with values copied from CI logs or other sources unless the user asks (e.g. "use a temporary database"); the CI env (`redis://redis/1`, `postgresql://postgres/penpot_test`) only works inside the CI docker network.
- The test database may have migrations applied from newer versions (it is shared dev infra). Reinitializing it — `dropdb`/`createdb` of the `PENPOT_TEST_*` database only — is explicitly permitted when that blocks a test run; never drop the main `penpot` database or another instance's test database.
- All CLI commands must be executed from the `backend/` subdirectory. - All CLI commands must be executed from the `backend/` subdirectory.
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed. - Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
- **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand. - **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand.
- **Why the flag matters:** `clojure.core/assert` is elided at compile time when `*assert*` is false, and `app.config` only sets that var from the flags when it loads. The test fixture redefines the flags after that, so they never reach it: without the flag the suite runs with every assertion compiled out, and a test that depends on an assert only means something with it set. `scripts/ci` and `tests-backend.yml` export the flag (GitHub Actions sets it too). When running the suite directly — including a `--focus` run — the command is `PENPOT_FLAGS="enable-backend-asserts" clojure -M:dev:test`. - **Why the flag matters:** `clojure.core/assert` is elided at compile time when `*assert*` is false, and `app.config` only sets that var from the flags when it loads. The test fixture redefines the flags after that, so they never reach it: without the flag the suite runs with every assertion compiled out, and a test that depends on an assert only means something with it set. `scripts/ci` and `tests-backend.yml` export the flag (GitHub Actions sets it too). When running the suite directly — including a `--focus` run — the command is `PENPOT_FLAGS="enable-backend-asserts" clojure -M:dev:test`.
- Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated. - Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
- Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var. - Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var. Keep the `PENPOT_FLAGS="enable-backend-asserts"` prefix on these too; a focused run without it hides assert-dependent failures exactly like a full one.
- A **whole-suite** crash reported as `Execution error ... Assert failed: ...` with no `FAIL in` line is not a test failure: an `ig/assert-key` ran while the test system was being built, which happens outside `clojure.test`, so kaocha reports it as a load/build error and the summary has no counts. Fix the offending schema (see `mem:backend/subtleties`) rather than hunting for a test.
- Multiple focus symbols are supported by REPEATING the flag (kaocha accumulates them, last-wins does not apply): `clojure -M:dev:test --focus backend-tests.ns-one --focus backend-tests.ns-two` — preferred over a shell loop of isolated runs. - Multiple focus symbols are supported by REPEATING the flag (kaocha accumulates them, last-wins does not apply): `clojure -M:dev:test --focus backend-tests.ns-one --focus backend-tests.ns-two` — preferred over a shell loop of isolated runs.
- Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas. - Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas.
- Line coverage: append the `:coverage` alias (`clojure -M:dev:test:coverage`, kaocha-cloverage). It instruments every `app.*` namespace under `src/` at load, before any test runs, so even a `--focus` run instruments them all; options live under `:cloverage/opts` in `tests.edn`, output in `target/coverage/` (lcov, HTML, and EMMA `coverage.xml` whose form counts the PR report shows as statements; cloverage measures no functions or branches) (`lcov.info`, HTML). About 2x slower and noisy with reflection warnings (instrumented forms lose type hints): never use it for normal runs. `tests-backend.yml` runs it and renders the step summary with `scripts/coverage-summary.mjs`.
- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`. - If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
- RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request. - RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request.
- The test system is `main/system-config` + `main/worker-config` with the - The test system is `main/system-config` + `main/worker-config` with the
@ -30,4 +34,4 @@ JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
`worker-config` can talk to a different Redis key prefix than a cfg the test `worker-config` can talk to a different Redis key prefix than a cfg the test
builds itself. The symptom is a component that silently does nothing builds itself. The symptom is a component that silently does nothing
locally and races in CI (which sets no `PENPOT_TENANT`). locally and races in CI (which sets no `PENPOT_TENANT`).
- `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there). - `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there). The logger swaps the record on an executor thread, after the call that logged it returns: keep the watch installed until the record arrives (poll for it) and only then remove it, or the record lands on nobody under load.

View File

@ -0,0 +1,44 @@
# Backend WebSocket Subscriptions
## Registry
- `app.http.websocket/state` holds `{:connections {conn-id → wsp} :by-profile {profile-id → #{conn-id}}}` in **one** atom, so a single `swap!` keeps both consistent; `register-connection`/`unregister-connection` maintain them together and drop a profile from the index once its last connection leaves.
- Each connection's own `::ws/state` atom holds its subscriptions (`::file-subscription`, `::team-subscription`). `state` is shadowed by `::ws/state` in every `handle-message` destructuring, so only `on-connect`, `on-disconnect` and the `repl-*` helpers touch the registry — reading the registry from a handler is a bug.
- `::profile-id` and `::session-id` on a connection live in the `app.http.websocket` namespace, while `::ws/id`, `::ws/state`, `::ws/output-ch`, `::ws/session-id` live in `app.util.websocket`. Mixing them up yields silently `nil` values (a `nil` `session-id` makes the channel's `:xf` drop **every** message).
- The registry is local to one backend instance. The message bus is shared (`mem:prod-infra/core`). Anything that must reach connections owned by another instance has to go through the bus — closing a connection from an RPC handler only ever works in a single-backend deployment.
## Revocation model
A subscription is authorized once, at `:subscribe-file`/`:subscribe-team`, and that decision used to be trusted for the whole connection life (GHSA-m53j-2766-6jqw, medium). Two independent mechanisms now bound it:
- **Announcement (primary).** A mutation that can change access publishes on `app.rpc.notifications/internal-revocation-topic` (`notify-permissions-changed` for one profile, `notify-team-permissions-changed` for one team id — the watcher resolves the members itself, so a large team is one event, not N). The `::revocation-watcher` Integrant component in `app.http.websocket` subscribes and calls `revalidate-profile-subscriptions`, which looks the profile up in `:by-profile` and re-checks every subscription with `file-readable?`/`team-readable?`: existence (the file gate is the same `sql:get-minimal-file` as the `:subscribe-file` gate, made non-throwing via `{:db/throw-if-not-exists false}`; the team gate is `db/get*` with `::sql/columns [:id :deleted-at]` — `deleted_at` must be in the projection or the soft-delete filter is a no-op) AND the `has-read-permissions?` predicates — and `sql:file-permissions` additionally filters `deleted_at` on file, project and team, as do `sql:get-team-id-for-file`/`-project` plus a final team-row check in `resolve-team-id`, so the organization-owner fallback grants nothing through soft-deleted file/project/team. The existence gate closes that fallback (expelled org owner, deleted resource). The event is only the trigger; the fresh check is the decision, which is what keeps a downgrade to `viewer` (still readable) and a non-member organization owner on a live resource from being mistaken for a revocation. Both publishers defer through `db/after-commit`, because the watcher checks on another connection and would otherwise see the pre-commit state. `:subscribe-team` deliberately has no existence gate of its own; only the re-checks add one, matching the file side (user decision, PR #12189 review).
- **Interval (safety net).** `start-relay` runs the relay as a `go-loop` that waits on the subscription channel and a one-shot tick channel together via `sp/alts!` with `:priority true` on the tick: when both hold something, the tick wins, so a steady stream of messages can never push the re-check away. The cadence is `subscription-revalidation-interval` (default `default-revalidation-interval`, 5 min, a code constant in `app.http.websocket` and **not** in the `config.clj` default map, same rule as session lifetimes). The deadline is fixed per cadence: a single tick channel covers a whole cadence and message turns reuse it (minting a fresh timeout per turn would both churn timers and starve under load exactly like an in-loop timer does); a new one is minted only after a tick fires. It covers what no announcement can: Nitrate transferring organization ownership (an HTTP call to another service) and rows deleted straight over SQL. Closing the channel ends the loop either way, because a take on a closed channel is always ready with nil — every way a subscription can end closes it, so a finished relay never keeps polling. Cost is three or four queries per subscription per interval (existence row + predicates), plus one Nitrate call per team per interval when the `:admin-console` flag is on (cached 30 s); one `timeout-chan` per subscription per cadence on promesa's default scheduler (one or two threads shared with every other scheduled task) — do not shorten the interval without sizing that. A tick abandoned at teardown (the subscription closed while it was pending) still fires its scheduled close harmlessly. Two places fail open, deliberately: `still-authorized?` (a transient DB error logs and keeps the subscription; `InterruptedException` is rethrown) and the `catch Throwable` in `watch-revocations`.
Which mutations announce: `delete-team-member`, `leave-team`, `update-team-member-role` (member only), `delete-team`, `delete-file`, `delete-project`, `move-files`, `move-project` (one team event each). Anything not listed here is covered only by the interval. `delete-share-link` deliberately does not: a share-link visitor never passes the websocket subscribe check, because `check-read-permissions!` is called without a `share-id` and that arity only considers team membership.
`close-file-subscription`/`close-team-subscription` are idempotent: no-op on a mismatched id, and they `dissoc` the subscription so a repeat call and `:pointer-update` do not publish to a dead topic. Closing the channel is what ends the relay (`take!` on a closed channel returns `nil`).
## Topic and layering
- `internal-revocation-topic` is a fixed uuid that is never a real resource id, so it cannot collide with a file, team or profile topic — and deliberately not the `uuid/zero` system topic, which is `sp/pipe`d straight into the client output and anything published there reaches every client. No client can subscribe to the internal topic: `:subscribe-file`/`:subscribe-team` only accept ids that already passed a permission check, and `:broadcast`/`:pointer-update` only republish to ids the server already holds.
- The topic and the publish helpers live in `app.rpc.notifications`, not in `app.http.websocket`: the command namespaces cannot require the watcher, because `app.http.websocket` already requires `app.rpc.commands.files`/`teams`. The watcher keeps the registry, which is the part it owns.
## Time and channel traps in this namespace
- The relay waits with `sp/alts!` over `[tick-ch channel]` with `:priority true`, following the `msgbus.clj` pattern (`start-io-loop` destructures the same `[val port]` shape and compares ports with `identical?`). A take on a closed channel is always ready with nil, which is what ends the loop — and what makes closing the channel from any teardown path prompt even while parked in `alts!`. `sp/timeout-chan` is one-shot: it closes once after the delay, and one abandoned at teardown still fires its scheduled close harmlessly.
- `ct/diff a b` returns **b − a** (`Duration/between`), so "time until a future instant" is `(inst-ms (ct/diff (ct/now) future))`.
- `ct/duration` unit keywords are singular: `{:millis 150}`, `{:minutes 5}`, never `:milliseconds`/`:days`.
- `still-authorized?` takes the check itself: `(boolean (check-access))`. Calling `(boolean pred)` on the closure is always true and silently disables the interval — this exact bug shipped once and no test caught it until the resubscribe-counting test existed. Naming rule here: no `?` for the action that queries and can fail (`check-access`), `?` only for the safe yes/no answer (`still-authorized?`).
## Tests
`backend/test/backend_tests/http_websocket_test.clj` drives the real Redis-backed msgbus, so the revocation tests are end-to-end: register the connection in the real registry, run the real command, wait for the subscription to drop, then assert no content arrives. A connection built without `register-connection` is invisible to the watcher, so the watcher-based tests must go through `open-registered-connection`. Interval tests shorten the config via `with-redefs` on `cf/get` rather than waiting minutes. `with-clean-registry` runs as an `:each` fixture, so no test can leak a live relay loop into the next one. `captured-events` forwards the keyword args through to the real `mbus/pub!` — rebuilding them positionally publishes to the topic `:message` with a `nil` payload and silently untests every other notice. Interval tests take access away with a direct row delete, never with a command: a command announces, so the test would pass even with the interval removed. The same rule covers the two counting/fan-out tests: `revocation-closes-subscriptions-that-are-no-longer-authorized` takes access away with `delete-team-membership!` (raw row delete, publishes nothing) because a command announcement lets the live watcher close both subscriptions before the test's own `revalidate-profile-subscriptions` call, and its `(= 2 …)` count then reads 0; `team-event-closes-every-member-subscription` soft-deletes the team row (`sql:team-members` joins `team_profile_rel` and `profile` only, so the watcher still resolves every member) and asserts nothing closed before one synthetic `notify-team-permissions-changed`, leaving the command path to `deleting-a-team-announces-once` and `deleting-a-team-closes-open-subscriptions`. The resubscribe-counting test is the one that proves finished relays stop polling. The soft-delete + org-owner re-check tests pair a direct row soft-delete with `with-redefs` on `nitrate/organization-owner-of-team?`: before the existence gate, the org-owner fallback kept the subscription open on a deleted resource. The soft-deleted-project/team-closes-org-owner-file-subscription tests pin the root fix in `app.rpc.permissions` (`resolve-team-id` refuses dead file/project/team for every caller of the fallback). `watcher-consumes-revocation-events-from-the-bus` takes access away with a direct row delete and drives the watcher with a synthetic bus event — never a command and never a second `notify-*` — so the "still subscribed" assertion cannot race an after-commit hook and a broken command announcement cannot mask a broken watcher. `moving-a-project-closes-source-team-subscriptions` pins the move-project asymmetry: the file subscription drops, the source-team subscription survives (the member is still a member).
`revocation-watcher-schema-resolves-every-ref` covers the `::revocation-watcher` params schema, which no other test compiles. It existed broken for a while: the three injected keys were written as bare keywords, which is a valid malli *property* position and not a key, so the schema raised `:malli.core/invalid-schema` — invisible unless `PENPOT_FLAGS=enable-backend-asserts` was set, and a crash of the whole system build rather than one test (see `mem:backend/testing` on the flag).
## See also
- `mem:backend/auth-permissions-product-domains` for the permission predicates and the not-found semantics that the re-checks reuse.
- `mem:prod-infra/core` for why the bus crosses backends and the registry does not.
- `mem:frontend/routing-app-shell-subtleties` for the client side: the frontend already reacts to `team-membership-change` by navigating away, which is why revocation needs no new client message.
- `.agents/plans/2026-10-07-websocket-subscription-revocation.md` for the original plan, and the sibling `.agents/plans/2026-10-05-leave-team-file-access-cleanup.md` whose hardened permission queries this re-verification depends on (without live team membership required in `sql:file-permissions`, the re-check would keep reporting access for a member who already left).

View File

@ -1,4 +1,4 @@
# Clojure Design Rules # Clojure Core
How to shape a function in this codebase. Each rule came from a review round; do not re-derive from taste. How to shape a function in this codebase. Each rule came from a review round; do not re-derive from taste.
@ -8,7 +8,8 @@ How to shape a function in this codebase. Each rule came from a review round; do
- **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is. - **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is.
- **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one. - **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one.
- **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them. - **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them.
- **All code-facing text MUST be in English** — identifiers, comments, docstrings, test descriptions. No exceptions; see the hard rule in `AGENTS.md`.
Reference implementation: the http metrics samplers in `app.http` (`sample-worker-metrics` / `sample-connector-metrics` are guard-free; `sample-http-metrics` guards at the boundary). Reference implementation: the http metrics samplers in `app.http` (`sample-worker-metrics` / `sample-connector-metrics` are guard-free; `sample-http-metrics` guards at the boundary).
See also: `mem:clojure/idioms` (language behaviors), `mem:testing` (TDD and test conventions). See also: `mem:clojure/idioms` (language behaviors and naming exceptions), `mem:clojure/logging` (structured logging), `mem:testing` (TDD and test conventions).

View File

@ -9,6 +9,7 @@ Behaviors confirmed against the language/stdlib — do not re-derive from assump
- Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself). - Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself).
- `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO). - `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO).
- Inside an `^:async` fn, a nested call that needs a temp binding (e.g. calling a local fn `(g (f x))`, whose arg gets a `G__` binding) compiles to `(await (async function(){…})())`: `await` unwraps a returned promise, so `(settled (upload (make-ctx)))` blocks until the upload promise settles instead of passing the promise on (a test that pushes the answer later hangs, idle). Bind each step in its own `let` binding. - Inside an `^:async` fn, a nested call that needs a temp binding (e.g. calling a local fn `(g (f x))`, whose arg gets a `G__` binding) compiles to `(await (async function(){…})())`: `await` unwraps a returned promise, so `(settled (upload (make-ctx)))` blocks until the upload promise settles instead of passing the promise on (a test that pushes the answer later hangs, idle). Bind each step in its own `let` binding.
- `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use. - `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/core`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
- Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant). - Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant).
- The threading macros insert the threaded value at DIFFERENT positions: `->` and `some->` put it FIRST (`(-> x (f a))` → `(f x a)`), `->>` and `some->>` put it LAST (`(->> x (f a))` → `(f a x)`). `some->`/`some->>` also short-circuit on nil. With unary calls both nest the same way (`(-> x (f) (g))` and `(->> x (f) (g))` are both `(g (f x))`): what changes between them is the position inside each call, not the order of the steps. A pipeline that calls a fn whose first argument is a config/context (`app.features.fdata/resolve-file-data` is `(cfg, file)`) must use `->>`/`some->>`; with `->`/`some->` the arguments are swapped and the call silently returns the wrong value (it cost a broken `get-snapshot` in `app.features.file-snapshots`).
- `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`). - `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`).

View File

@ -0,0 +1,7 @@
# Clojure Logging Conventions
Structured logging through `app.common.logging` (`l/trc`/`l/dbg`/`l/info`/`l/wrn`/`l/error`/`l/fatal`): `:hint` carries the human-readable message, every other datum goes in its own keyword field (`:id`, `:container`, `:removed`, ...).
- `:hint` values start lowercase: `"fix dangling children"`, not `"Fix dangling children"`. Uppercase-first hints exist as legacy (~57 occurrences, the densest in `backend/src`; also `l/trc` in migration `legacy-28` and `log/error` in `app.common.files.repair`) — do not copy them, and do not mass-rewrite them unless the surrounding code needs the change.
- A name that requires uppercase (an env var like `PENPOT_MEDIA_PROCESSING_SERVICE_URI`, an acronym like `MCP` or `LDAP`) stays uppercase inside an otherwise lowercase sentence.
- Repair work done by a file migration logs at `l/trc`, only when it changes something, and with the ids it touched: the house style of `migrations.cljc` (`legacy-28` precedes; the file has no `warn`/`info` lines). The shipped `log4j2.xml` pins the `app` logger to `info` (static file, `monitorInterval="60"`), so those trace lines appear only after the operator raises the level; the durable record of a repaired file is the migration id in its `:migrations`.

View File

@ -8,6 +8,7 @@ Each change is a map such as `{:type ... :id ... :page-id ...}`. Common families
- `:add-obj`, `:mod-obj`, `:del-obj`: shape lifecycle. `:mod-obj` contains `:operations`, commonly `{:type :set :attr ... :val ... :ignore-geometry ... :ignore-touched ...}` or `{:type :set-touched ...}`. - `:add-obj`, `:mod-obj`, `:del-obj`: shape lifecycle. `:mod-obj` contains `:operations`, commonly `{:type :set :attr ... :val ... :ignore-geometry ... :ignore-touched ...}` or `{:type :set-touched ...}`.
- `:add-component`, `:mod-component`, `:del-component`: component/library metadata. - `:add-component`, `:mod-component`, `:del-component`: component/library metadata.
- Shapes stored inside the component record (deleted components keep them in `:objects`) are never written as a whole map: `mod-component` carries only component metadata by default, and shapes travel as `:mod-obj`/`:add-obj`/`:del-obj` changes scoped with `:component-id`. `pcb/update-component` has an `:objects? true` option for the two repairs that move or clear that key, and `ctkl/mod-component` reads an absent `:objects` as "leave the stored shapes alone" (only an explicit `nil` drops them). Before this, one component swap resent the full object map of every deleted component it touched (~6 MB in the request); do not reintroduce it.
- `:add-children`, `:remove-children`, `:reg-objects`: tree and object-map edits. - `:add-children`, `:remove-children`, `:reg-objects`: tree and object-map edits.
- `:set-option`, `:add-page`, `:mov-page`, and related file/page metadata changes. - `:set-option`, `:add-page`, `:mov-page`, and related file/page metadata changes.
@ -21,7 +22,7 @@ High-value builder operations:
- `pcb/with-page-id`, `pcb/with-objects`, `pcb/with-library-data`: set context for following operations. - `pcb/with-page-id`, `pcb/with-objects`, `pcb/with-library-data`: set context for following operations.
- `pcb/update-shapes ids update-fn`: emits `:mod-obj` with diff-derived `:set` ops. Options include `{:with-objects? true}`, `{:ignore-touched true}`, and `{:attrs #{...}}`. - `pcb/update-shapes ids update-fn`: emits `:mod-obj` with diff-derived `:set` ops. Options include `{:with-objects? true}`, `{:ignore-touched true}`, and `{:attrs #{...}}`.
- `pcb/add-objects`, `pcb/change-parent`, `pcb/remove-objects`, `pcb/resize-parents`: shape/tree edits. - `pcb/add-objects`, `pcb/change-parent`, `pcb/remove-objects`, `pcb/resize-parents`: shape/tree edits.
- `pcb/add-component`, `pcb/update-component`, `pcb/mod-component`: component/library edits. - `pcb/add-component`, `pcb/update-component`, `pcb/mod-component`: component/library edits. Repairs that fix shapes inside a component use `pcb/with-container` + `pcb/with-objects` + `pcb/update-shapes` so the changes come out scoped with `:component-id`.
- `pcb/set-translation? true`: marks the whole change set as translation-only, which lets component sync skip expensive work. - `pcb/set-translation? true`: marks the whole change set as translation-only, which lets component sync skip expensive work.
## Applying changes in tests ## Applying changes in tests

View File

@ -26,6 +26,12 @@ Respect the intended abstraction direction in new/refactored code:
Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical. Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical.
## Build-mode dependent laziness
`app.common.data.macros/lazy-delay` returns a delay whose expression is evaluated eagerly by default. Only the builds that set the `app.common.data.macros/lazy-delay?` closure define to `true` defer it to the first deref; every production build does (frontend `main` and `worker`, exporter, library), so the Closure compiler replaces the constant and drops the eager branch. On the JVM it is always eager.
`app.common.schema/check-fn` resolves its schema through it: an invalid schema definition fails when the namespace loads in development and in tests, and only on the first check call in production. The other lazily built parts of `check-fn` (validator and explainer) are always deferred.
## Focused memory routing ## Focused memory routing
Model, schema, and persistence shape: Model, schema, and persistence shape:

View File

@ -52,3 +52,8 @@ its own drift. Comparing `:selrect.width` from shape A to
`:selrect.width` from shape B is comparing two values that `:selrect.width` from shape B is comparing two values that
"semantically" should be equal but were computed via different "semantically" should be equal but were computed via different
operation chains — exact equality will often be false. operation chains — exact equality will often be false.
## Library sync comparison
- `update-attrs` (`app.common.logic.libraries`) compares main vs copy attrs exactly first, then with tolerance for: `drifting-geometry-attrs` (`:x :y :width :height :selrect :points`, via `gsh/close-attrs?`); path and bool `:content` (`segment/content-close?`; bool shapes keep their path in `:content`); and text `:position-data` (`position-data-close?` over `mth/close-map?`: every numeric field of every entry within `mth/close?`, other fields such as fills, fonts and text exact). All other attrs (`:opacity`, radii, `:transform`, `:rotation`, text `:content`) stay exact: tolerance there hides real changes.
- `close-attrs?` pairs lists with `d/zip`, which truncates to the shorter list and treats nil vs vector as equal for `:points`/`:position-data`. Do not use it for variable-length data. That is why `:position-data` has its own helper, and why `attr-equal?` checks the `:points` count before calling it.

View File

@ -34,3 +34,7 @@
- When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data. - When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data.
- Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it. - Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it.
- Rebase collision on a migration number: upstream keeps the id. Renumber the branch migration to the next free one, keep it last in `available-migrations`, and rename everything that spells the number — its test file and namespace (`files_migrations_00NN_test.cljc`), the test names, and comments in production code or memories that name it. - Rebase collision on a migration number: upstream keeps the id. Renumber the branch migration to the next free one, keep it last in `available-migrations`, and rename everything that spells the number — its test file and namespace (`files_migrations_00NN_test.cljc`), the test names, and comments in production code or memories that name it.
- Withdrawing a migration id: remove both the `defmethod` and the id from `available-migrations`; the replacement migration subsumes it and files that recorded the old id re-migrate through the replacement (persisted `:migrations` keep the old id via union — do not rewrite them, including playwright fixture JSONs). Idempotent/pure-cleanup migrations only; never withdraw one whose transform must not run again unless the replacement is safe for those files. Precedents: "0021-fix-shape-svg-attrs" -> "0035-fix-shape-svg-attrs-and-defs"; "legacy-36" -> "0036-clear-nil-keys-in-objects".
- A shipped migration whose behavior must change AND reach every file (most files already recorded its id, so `migrate` skips it): rename it to `<id>-v2` and swap the old id for the new one in `available-migrations` at the SAME position. Precedents: `0002-normalize-bool-content-v2`, `0003-convert-path-content-v2`, `0008-fix-library-colors-v4`, and `0035-fix-shape-svg-attrs-and-defs` withdrawing `0021-fix-shape-svg-attrs`. Files keep the old id in `:migrations`, which is harmless because `migrate` diffs against the available set. Cost: one full pass over every file on its next read (the `get-file` ETag hashes `available-migrations`, so clients refetch).
- The alternative is patch the old id plus add a new migration id for the files that already applied it. Then the position in the list decides which repair runs first: a coerce-then-drop repair must come before a drop-only one, or the strings are gone before they can be parsed.
- `0033-normalize-main-instance-names-v2` repairs the container before its naming walk: child ids stored as strings are parsed with `uuid/coerce`, ids missing from `:objects` are dropped, duplicates are removed (the one `0032` adds when a child is stored as a string). The walk only re-assocs a shape when the normalization changed it (`identical?` on the normalized shape), so a healthy container keeps its `ObjectsMap` instance: no `modified` flag, no page rewrite. `0035-fix-dangling-children` prunes whatever is left.

View File

@ -15,6 +15,7 @@ From `common/`:
- Full JS test run (always builds, build output visible): `pnpm run test` - Full JS test run (always builds, build output visible): `pnpm run test`
- Focus a JVM test namespace: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test` - Focus a JVM test namespace: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test`
- Focus a JVM test var: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test/test-basic-switch` - Focus a JVM test var: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test/test-basic-switch`
- JVM line coverage: `clojure -M:dev:test:coverage` (kaocha-cloverage, about 2x slower; `:cloverage/opts` in `tests.edn`, which excludes `app.common.test-helpers.*`; output in `target/coverage/` (lcov, HTML, and EMMA `coverage.xml` whose form counts the PR report shows as statements; cloverage measures no functions or branches)). `scripts/test` (CI) uses it; the JS side has no coverage.
- Focus a JS test namespace: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test` - Focus a JS test namespace: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test`
- Focus a JS test var: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute` - Focus a JS test var: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute`
- Quiet logging during a JS run: append `--log-level warn` (or `trace|debug|info|warn|error`) - Quiet logging during a JS run: append `--log-level warn` (or `trace|debug|info|warn|error`)

View File

@ -6,8 +6,9 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories. - A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories.
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`. - Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`. - Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
- Structured logging conventions (`:hint` casing and fields, log levels): `mem:clojure/logging` — read before adding or changing any log call.
- Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics. - Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API. - Read `mem:clojure/core` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
- Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables. - Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables.
# Development workflow # Development workflow
@ -23,7 +24,6 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked. - **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked.
- **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or on explicit user request. - **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or on explicit user request.
- You have access to the GitHub CLI `gh` or corresponding MCP tools. - You have access to the GitHub CLI `gh` or corresponding MCP tools.
- Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool.
- Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`). - Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`).
- Align `let` binding values: when a `let` form has multiple bindings spanning several lines, align the value forms to the same column with spaces. - Align `let` binding values: when a `let` form has multiple bindings spanning several lines, align the value forms to the same column with spaces.
- If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files, fix them with `scripts/paren-repair` BEFORE running lint/format checks. See `mem:scripts/paren-repair` for usage. - If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files, fix them with `scripts/paren-repair` BEFORE running lint/format checks. See `mem:scripts/paren-repair` for usage.
@ -60,6 +60,8 @@ The memory is structured in a way that you can get the critical information abou
- `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`. - `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`.
- `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`. - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`.
- `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`. - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`.
- `scripts/coverage-summary.mjs` — Renders an lcov file as a markdown table grouped by source area, for `$GITHUB_STEP_SUMMARY` (`--title`, `--lcov`, optional `--note`, `--unloaded`). Used by the backend, common and frontend test workflows; module coverage commands are in `mem:backend/testing`, `mem:common/testing`, `mem:frontend/testing`.
- `scripts/coverage-comment.mjs` — Keeps one PR comment (`<!-- coverage-report -->`) with the coverage table of Backend, Common, Frontend and Render WASM, in the format of the Vitest Coverage Report Action (status, metric, %, covered/total, changed-file coverage, run/commit footer). Each test job updates its own project: the data of all three lives base64-encoded in a hidden `coverage-data` block, and the job re-reads after `SETTLE_MS` and rewrites if a concurrent job dropped its data; extra report comments from simultaneous creates are merged and deleted. Targets (`THRESHOLDS`, per project and metric, status colour only, never fail CI) are on lines only: 70 Backend, 70 Common, 50 Frontend, 40 Render WASM; other metrics get the blue dot. Statements are Clojure forms for Backend/Common (`--emma-xml`) and llvm regions for Render WASM (`--llvm-json`). Shared lcov parsing: `scripts/coverage-lcov.mjs`. The test workflows run it only for PRs from branches of this repo (job-level `pull-requests: write`, `continue-on-error`): fork PRs get a read-only token, so they only get the run-page summary. A fork-safe `workflow_run` commenter would run only from the default branch's copy of the workflow.
- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`. - `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`.
- `scripts/changelog.py` — Changelog checks for `mem:workflow/updating-changelog`: `boom` (generate the `:boom:` section from `breaking change` issues), `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. - `scripts/changelog.py` — Changelog checks for `mem:workflow/updating-changelog`: `boom` (generate the `:boom:` section from `breaking change` issues), `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`.
- `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`. - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`.

View File

@ -15,6 +15,7 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is. - For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is.
- Per-instance test databases are created by `ensure-instance-test-database` (called from `start-instance`): `CREATE DATABASE` guarded by a `pg_database` existence check, so it is idempotent. Needed because `postgresql_init.sql` only runs on first volume creation. The backend applies migrations itself on first use, so an empty database is enough. - Per-instance test databases are created by `ensure-instance-test-database` (called from `start-instance`): `CREATE DATABASE` guarded by a `pg_database` existence check, so it is idempotent. Needed because `postgresql_init.sql` only runs on first volume creation. The backend applies migrations itself on first use, so an empty database is enough.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_s3_bucket()`. Never duplicates `defaults.env`. - `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_s3_bucket()`. Never duplicates `defaults.env`.
- Frontend admin-console is off by default in devenv: `docker-compose.main.yml` sets `PENPOT_DEVENV=true`, which makes `app.config` (`goog-define devenv-mode`) prepend `disable-admin-console` ahead of the user `config.js` flags; flags parse in order and user flags win, so `enable-admin-console` in `frontend/resources/public/js/config.js` re-enables it. Outside devenv (prod builds) the default is unset and the admin-console stays on, matching `flags/default`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails. - Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants ## Invariants

View File

@ -12,7 +12,7 @@ Frontend: CLJS SPA; React/Rumext; Potok; RxJS; okulary refs; SCSS modules; share
- `app.render_wasm.*`: frontend bridge to Rust/WASM renderer. - `app.render_wasm.*`: frontend bridge to Rust/WASM renderer.
- `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities. - `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities.
- `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app. - `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app.
- Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared permission rules are in `common/src/app/common/types/nitrate_permissions.cljc`. - Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared organization permission rules (`allowed?`, `can-send-invitations?`) are in `common/src/app/common/types/organization.cljc`.
- `app.config` external bridges (`external-feature-flag`, `external-session-id`, `external-context-info`, `external-notify-register-success`, `initialize-external-context-info`) delegate to `globalThis` hooks injected by the SaaS host. Never delete `external-feature-flag`, even with zero call sites: it is the seam for future A/B tests. - `app.config` external bridges (`external-feature-flag`, `external-session-id`, `external-context-info`, `external-notify-register-success`, `initialize-external-context-info`) delegate to `globalThis` hooks injected by the SaaS host. Never delete `external-feature-flag`, even with zero call sites: it is the seam for future A/B tests.

View File

@ -5,8 +5,11 @@
- Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id. - Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id.
- Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly. - Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly.
- Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications. - Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications.
- The import wizard runs the package as a job of the backend: it reads the manifest of the file in the main thread (`app.main.data.imports`, no web worker) and follows the job, so its header paints the milestone of the job (the stage and its counter) instead of the messages the old SSE stream sent; every file of a container reports the same milestone, so `jp/current-milestone-text` writes it once and the rows never repeat it. Before the job exists, the header paints a client-side `:upload` milestone built from the chunked upload (`uploads/upload-blob-chunked` `:on-progress`, chunks done/total), which never comes from the backend; the chunks go one at a time and it writes them as a percentage of the file, not as a counter. Once created and still waiting for a worker, no row says anything: the header says the import waits for a slot while no job of it has started. The bar of the header measures the whole import, never a file: indeterminate while checking, waiting for a slot or rebuilding, because the client cannot know a global index and total of operations. The upload is the exception — the chunked upload knows its chunks, so the bar is determinate while the containers are sent, weighted by their size (`imp/upload-pct` over `:container-size`, the only percentage the client really knows). The export modal follows the same one-job-per-file model, but every file is queued up front (even the ones whose job is created only after the previous one is over) and its queue ends only with a milestone: `:start`/`:retry` do not end it there. Neither modal paints a generic "…ing file…" line on the rows any more: the rows of the import wizard carry only what is not progress — the name, the size, `Ready` once their container finished uploading, and the error text —, and the button that drops a file only shows while the wizard still offers its import step (the confirmation screen). Above the rows, the import wizard paints a stepper (Check/Upload/Import) plus one phase line: `imp/import-step` derives the step from the dialog status and the entries (upload holds while any file is pending, either `:import-ready` or carrying its own `:upload` milestone), so Check covers analyze, Upload covers the chunked client-side upload, and Import covers queue plus job. A file of a version 3 package shows what it occupies inside its container, computed while analyzing (`imp/file-sizes`): its own `files/<id>…` entries plus the storage objects its media and thumbnails reference, whose metadata is the only link to `objects/<id>…`. The container's own size stays in `:container-size` (what the dropped file weighs) and is what the upload bar weighs, never what a row shows. A file the manifest lists but the container holds no entry of has no size at all, and its row then shows no subtitle: what the package weighs never stands in for it. The subtitle is the size alone, with no extension. While analyzing with no failures nothing renders below the phase header — no rows, no spinner (`imp/awaiting-analysis?`); the header text is the only "Checking…" message. Rows appear once the analysis resolves (one per file inside the package), and a failed analysis takes the error-summary path instead. The ready step and its Continue wait for the end of the analysis (`imp/ready-to-import?`, fed with the completion of `imp/analyze`): a ready file is not enough, because the files still to arrive would never be imported. While rebuilding, the phase title counts the files and `imp/imported-count` counts them done with the job's own `:files` counter, so the number moves while a container rebuilds its files one by one. The `:error` of an import entry is user-facing text (the backend hint, or a translated label), never an i18n key: the wizard renders it as-is, it does not translate it again. The link-later step takes the file id from the `:id` of its resolution entry, never from the key of the resolution map: the map comes from the job result, so what a key holds depends on the result schema of the job-def that wrote it, while the wizard indexes its selection by the id of the entry. The legacy SSE import stays in the backend for other clients. - The v3 format carries no size per file: the size of one is the sum of its own entries plus the `objects/` its media and thumbnail metadata reference, the only link between a file and that shared storage.
- Those metadata entries are read one at a time on purpose: a container holds thousands and every read opens its own decompression stream, which makes Chromium fail the analysis — zip.js then masks the cause behind a `Cannot close a ERRORED writable stream` — while Firefox tolerates it. Do not turn this back into a parallel read. The read is best-effort: a reference that cannot be read leaves that file without size, never the whole package.
- The `:error` of an import entry is user-facing text, never an i18n key. The legacy SSE import stays in the backend for other clients.
- File creation/duplication strips file `:data` before putting file summaries into dashboard state. - File creation/duplication strips file `:data` before putting file summaries into dashboard state.
- The pending invitations are a team-admin only list (backend `:is-admin` already includes the owner). The dashboard hides the invitations entry from the header tabs and the team-options dropdown for non-admins, `invitations-section-content` keys the section on `:is-admin`, and `fetch-invitations` answers `insufficient-permissions` by emitting `invitations-fetched` with `[]`, so a demoted admin does not keep the cached emails and the header does not paint the invite button from a stale list. The organization `send-invitations` policy (`can-invite?`) does not gate the list, but it gates every action on it: inviting (header and empty-state buttons render disabled with the `dashboard.invite-profile-disabled` tooltip), the role selector (read-only label), row selection (so resend and delete never show) and copy link (`invitation-link-copyable?`). The backend enforces the same rule.
## Viewer ## Viewer

View File

@ -15,6 +15,9 @@
## Runtime initialization and permissions ## Runtime initialization and permissions
- The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling. - The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling.
- `app.plugins/init-plugins-runtime` returns whether `ɵloadPlugin` was installed. It calls `preg/signal-runtime-ready` even on failure: `workspace.cljs` waits on that promise before `mcp/init`, so skipping it hangs MCP initialization forever.
- When SES bootstrap fails, `app.plugins` emits one `plugins-runtime-unavailable` telemetry event (with the SES error message), and `app.main.data.plugins/load-plugin!` shows the lazy `plugins.runtime-unavailable` toast when the user opens a plugin. `start-plugin!` (MCP path) stays silent by design.
- Runtime availability is detected by the existing `(fn? (unchecked-get global "ɵloadPlugin"))` check; do not add a second mechanism.
- Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id. - Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id.
- Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host. - Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host.
- The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code. - The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code.
@ -34,5 +37,7 @@
- Plugin listeners are watches on the global store and callbacks are debounced about 10ms. Callback exceptions are caught and logged so plugin code does not crash the app. - Plugin listeners are watches on the global store and callbacks are debounced about 10ms. Callback exceptions are caught and logged so plugin code does not crash the app.
- `selectionchange` callbacks receive arrays of shape id strings, while `filechange`, `pagechange`, and `shapechange` return proxies. - `selectionchange` callbacks receive arrays of shape id strings, while `filechange`, `pagechange`, and `shapechange` return proxies.
- `contentsave` fires only when persistence status transitions to `:saved`; it calls the callback with no value. - `contentsave` fires only when persistence status transitions to `:saved`; it calls the callback with no value.
- `penpot.waitForSave(timeout?)` (`app.plugins.persist/wait-for-save`; runtime forwarder checks `content:read`) waits on the **shared** persistence queue, not only the plugin's own commits. Flow: ~100ms intake race (buffer/queue non-empty vs deadline, capped to the timeout) → empty resolves; pending emits `::dps/force-persist` and races `dps/wait-persisted-or-error` against `::dps/rollback-rejected` / `::dps/rolled-back` (rollback rejects even if status later reads `:saved`). Default 60s. `:error` status at intake rejects at once (`earlier-failure`): `save-retrying` while `dps/save-retrying?` (slow retry cycle, recovers by itself), otherwise the stored `[:error :code]` with reopen advice (a terminal failure stays until the file is reopened, so it fails every call). Rejections are `Error`s with `code`: `save-timeout`, `rolled-back`, `save-retrying`, or the failure code (`save-failed` unless the backend gave one).
- Plugin timeouts validate through `u/valid-timeout?` (positive finite number ≤ `u/max-timeout`, signed 32-bit) and reject through `u/reject-not-valid`, which rejects an `Error` (never a bare string) so callers can read `.message`.
- `contentrollback` is not a state watch: it subscribes to `st/stream` for `::dps/rolled-back` (kept in `subscriptions`, disposed by `remove-listener`), not debounced, delivered async so the reverted state is in place. Payload `{fileIds pageIds shapeIds}` from `pcm/rollback-targets` (page shapes only, from the reverted commits' redo changes). When the rollback falls back to reloading the file, `::dps/rolled-back` waits for the reloaded data, so the callback never reads the rejected data. The runtime's `validEvents` must list every event name, or `penpot.on` rejects it; a stale runtime build rejects new events. The MCP plugin forwards these rollbacks with the next task response and the server puts a warning first in that tool response (`RollbackNotices`). Unsent ones survive a tab or plugin reload in `penpot.localStorage` (key `rollbacks/<file-id>`, `LocalStorageRollbackStore`; the MCP manifest in `app.main.data.workspace.mcp` and `mcp/packages/plugin/public/manifest.json` grant `allow:localstorage`, which the runtime checks on access); the entry goes once a response carried them. - `contentrollback` is not a state watch: it subscribes to `st/stream` for `::dps/rolled-back` (kept in `subscriptions`, disposed by `remove-listener`), not debounced, delivered async so the reverted state is in place. Payload `{fileIds pageIds shapeIds}` from `pcm/rollback-targets` (page shapes only, from the reverted commits' redo changes). When the rollback falls back to reloading the file, `::dps/rolled-back` waits for the reloaded data, so the callback never reads the rejected data. The runtime's `validEvents` must list every event name, or `penpot.on` rejects it; a stale runtime build rejects new events. The MCP plugin forwards these rollbacks with the next task response and the server puts a warning first in that tool response (`RollbackNotices`). Unsent ones survive a tab or plugin reload in `penpot.localStorage` (key `rollbacks/<file-id>`, `LocalStorageRollbackStore`; the MCP manifest in `app.main.data.workspace.mcp` and `mcp/packages/plugin/public/manifest.json` grant `allow:localstorage`, which the runtime checks on access); the entry goes once a response carried them.
- Plugin history `undoBlockBegin` creates a workspace undo transaction with a JS `Symbol`; `undoBlockFinish` commits that symbol. Missing finish eventually relies on the workspace transaction timeout. - Plugin history `undoBlockBegin` creates a workspace undo transaction with a JS `Symbol`; `undoBlockFinish` commits that symbol. Missing finish eventually relies on the workspace transaction timeout.

View File

@ -7,6 +7,8 @@
- `set-token!` and `replace-token!` are deliberately asymmetric. `set-token!` is a total no-op when the token already is the current one (no URL write, no report), so navigating to the screen you are on cannot grow the history stack. `replace-token!` always writes and always reports, because it is how the app corrects the URL in place (legacy `#/…` translation, cleaned share links) and that correction must reach the router even when the token did not change. This matches `Html5History`, which did the same; do not "fix" the asymmetry without deciding what navigates to the current screen. - `set-token!` and `replace-token!` are deliberately asymmetric. `set-token!` is a total no-op when the token already is the current one (no URL write, no report), so navigating to the screen you are on cannot grow the history stack. `replace-token!` always writes and always reports, because it is how the app corrects the URL in place (legacy `#/…` translation, cleaned share links) and that correction must reach the router even when the token did not change. This matches `Html5History`, which did the same; do not "fix" the asymmetry without deciding what navigates to the current screen.
- `app.util.dom/replace-history-state!` is the deliberate bypass used by `app.main.ui.dashboard`/`app.main.ui.static` to rewrite the URL **without** routing. Keep using it there instead of `replace-token!`, which would re-enter the router. - `app.util.dom/replace-history-state!` is the deliberate bypass used by `app.main.ui.dashboard`/`app.main.ui.static` to rewrite the URL **without** routing. Keep using it there instead of `replace-token!`, which would re-enter the router.
- Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required. - Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required.
- The organization SSO flow sends the **query string** (`rt/get-current-query-string`), not the URL and not the token: the backend builds the destination URL from `public-uri` + `?` + those params, so the client never chooses the origin of the redirect. The history token (`rt/get-current-token`, `location.search`) keeps its leading `?` because every other consumer concatenates it over the base path (`str cf/public-uri token`, `dom/append-query-param`, `dom/remove-query-param`); `rt/token->query-string` is the only bridge between the two shapes, so do not trim at a call site by hand; `rt/token->url` is the single client builder of an absolute destination from either shape (used by the SSO retry and error flow), so do not assemble `public-uri + "?" + params` at call sites. The gate is not the only caller: `dtm/resolve-login-team-id` runs from the root branch of `on-navigate` too, and there the URL carries its params instead of a screen (`?template=…`) — that call only reads `:authorized`, so it never follows the destination, which is exactly why a wrong destination there goes unnoticed. Do not add a rule that requires `screen`. See `mem:backend/auth-permissions-product-domains`.
- The router events expose their payload through `@event` (`IDeref`): `rt/navigate` returns `{:id :params :options}` and `rt/nav-raw` returns its `:href`/`:uri`. `rt/nav-raw` is the way out of the router (raw URL, outside the token scheme) and nothing in the app listens to `::rt/nav-raw`; the destination travels in the effect, so `@event` is the only way to tell where the user is being sent (tests included).
- Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete). - Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete).
- Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions. - Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions.
- The root app has two error layers, and they reset differently. `:exception` in the store renders `static/exception-page*` directly and is cleared by `rt/navigated`; a React render error is caught by `app.main.ui.error-boundary` (`react-error-boundary`), whose error lives in internal React state that only `reset-error-boundary` clears. Because every exit from the error page (the logo, the no-permission dialogs, the SSO dialog, the feedback link) only *navigates*, the boundary is given `:resetKeys [(eb/reset-key route)]` — the screen name — so a navigation counts as a fresh attempt at rendering. Three traps in that `resetKeys` value, all learned the hard way: it must be a real JS array (`#js [...]`), never a ClojureScript vector, because `react-error-boundary` reads `.length` and `.some` and a `PersistentVector` has neither, so `componentDidUpdate` throws `some is not a function`; it must hold the screen name as a *string*, because the library compares entries with `Object.is` and ClojureScript does not intern keywords, so two equal keywords are two objects and a keyword key would read as new on every route rebuilt, resetting the boundary on a param change inside the same screen; and it must be the screen rather than the params, so a param change does not replay the screen that just crashed. Navigating to the *same* screen therefore does not reset, and a reload is the only way out of that case. - The root app has two error layers, and they reset differently. `:exception` in the store renders `static/exception-page*` directly and is cleared by `rt/navigated`; a React render error is caught by `app.main.ui.error-boundary` (`react-error-boundary`), whose error lives in internal React state that only `reset-error-boundary` clears. Because every exit from the error page (the logo, the no-permission dialogs, the SSO dialog, the feedback link) only *navigates*, the boundary is given `:resetKeys [(eb/reset-key route)]` — the screen name — so a navigation counts as a fresh attempt at rendering. Three traps in that `resetKeys` value, all learned the hard way: it must be a real JS array (`#js [...]`), never a ClojureScript vector, because `react-error-boundary` reads `.length` and `.some` and a `PersistentVector` has neither, so `componentDidUpdate` throws `some is not a function`; it must hold the screen name as a *string*, because the library compares entries with `Object.is` and ClojureScript does not intern keywords, so two equal keywords are two objects and a keyword key would read as new on every route rebuilt, resetting the boundary on a param change inside the same screen; and it must be the screen rather than the params, so a param change does not replay the screen that just crashed. Navigating to the *same* screen therefore does not reset, and a reload is the only way out of that case.
@ -22,6 +24,9 @@
- `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler. - `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler.
- The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission. - The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission.
- Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason. - Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason.
- A team scoped navigation carries a team whenever any of the four tiers exists, and `app.main.data.common/resolve-team-id` decides it for every team scoped `go-to-*` helper of that namespace: the UUID the caller passes, `:current-team-id`, the team of the route already on screen (`?team-id`), then the profile `:default-team-id`. `go-to-feedback` and `go-to-viewer` are not team scoped and do not go through it, and `:team-id :default` keeps meaning the profile default, resolved before the rest. The route goes before the profile default because an emitter that runs from a team screen knows which team the user is looking at, and the personal team would open their file under another team. Two `rt/nav` emitters outside that namespace still decide their own id and can still emit nil, and they are the residual: `app.main.ui.nitrate.entry/nitrate-entry*` (the profile default) and the root branch of `app.main.ui.routes/on-query-navigate` (`dtm/resolve-login-team-id`, which answers nil when the profile has no default team). When none of the tiers exists the helper still fires the navigation with a nil id, so the not-found page appears instead of the click doing nothing.
- `app.main.ui/team-container*` raises `:not-found :invalid-team-id` for a dashboard or workspace route without a valid `team-id`; the raise is what keeps a broken URL from painting a blank shell, so it stays after the hook fix of #12728. `exception-section*` skips `errors/submit-report` for `:not-found`, so this failure never reaches PostHog and cannot be measured from the error dashboard. The exit from that page is the `error-container*` header (logo plus `not-found.no-permission.go-dashboard`), which emits `go-to-dashboard-recent` with the profile default and falls back to `rt/nav-root`; every not-found and authentication page already carries it, so there is no button to add there.
- A click cannot reach a team screen before its team is resolved: `team-container*` paints its children only when the route id matches the store team, and `refs/team` derives from `:current-team-id`, so with that key empty the dashboard and the workspace paint nothing clickable. The nil-id emissions come from events that run on their own (login, invitation acceptance, membership change, the sidebar logo, the workspace shortcuts) and from typed or bookmarked URLs.
## Store and websocket ## Store and websocket

View File

@ -17,7 +17,8 @@ Frontend testing is async-first: everything essentially asynchronous is modeled
- `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests. - `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests.
- `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`. - `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`.
- `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses). - `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses).
- Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for` (immediate check + bounded poll, fails instead of hanging), `observe` (stream → termination promise; asserts provided, timeout rejects). - Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for`, `wait-for-store`, `observe` (stream → termination promise; asserts provided, timeout rejects).
- `wait-for` waits on a **signal**, not on turns: `:signal` (an observable) re-checks the predicate on every emission and resolves on the event that changes the state; `:timeout-ms` (default 5000) only bounds failure then. Without a signal it polls against a real-time deadline — 50 turns were ~54 ms idle, and turns are not a unit of time. `wait-for-store` passes `ptk/input-stream` as the signal (the store notifies after applying the update, so the predicate sees the new state); the persistence fixture exposes `requests$` for `(= N (count @requests))`. A wait that resolves on an event consumes no turn. The waits with nothing to listen to stay on the poll: test atoms filled outside the store (`@cycles`, `@reports`, `@done`), `@failures`, and predicates over JS objects or closures.
- Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left). - Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left).
- Valid `^:async` placements: `mem:clojure/idioms`. - Valid `^:async` placements: `mem:clojure/idioms`.
@ -37,9 +38,15 @@ To assert over emitted event sequences, observe termination: subscribe through `
- `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`. - `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`.
- Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline. - Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline.
- Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis. - Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis.
- A wait that observes an effect the test asserts takes a signal. A wait that means "the side effect advanced before the next emit" has no event to listen to: keep the poll (or an explicit `settle`) and say so, because a signal would resolve inside the same `emit!` and the next step would overtake the pipeline (`persistence-rollback-test` drove its four emits this way).
- With `rp/cmd!` mocked, a store-state wait can resolve before the flow that changes it ends, and the mock scope then closes in flight: the next call hits the real HTTP layer (malformed URL from a nil `cf/public-uri`) and the error is reported on a later test. `unlink-file-from-library` dissocs the library before its async chain, so the only proof the chain ran inside the mock is its second RPC call; wait for that, through the mock.
- Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows). - Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows).
- A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop. - A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop.
- `app.main.errors` prints every handled cause with `ex/print-throwable` before notifying, and the app logger does not gate it. A test that runs a real handler end to end stubs that var (`main-errors-test/quiet-throwable`): without it, the 10 000-error governor test alone wrote 190 k of the 192 k lines of a run log (13,8 MB), and GitHub truncates a step log past ~4 MB, cutting the summary and any failing assertion. Same for a production `console.error` a test provokes on purpose: `code-gen-markup-svg-wasm-test` silences `(.-error js/console)` in a `try/finally` while two WASM panics go through `get-svg-markup` (2 blocks, 70 lines, and the exception dumps its own `data`/`arr`).
- The `:esm` test build cannot load the CommonJS `sax` module behind `tubax`, so SVG parsing fails in tests (`dwm/valid-svg-string?` answers false, `dwm/svg->clj` errors): mock both with the parsed data. - The `:esm` test build cannot load the CommonJS `sax` module behind `tubax`, so SVG parsing fails in tests (`dwm/valid-svg-string?` answers false, `dwm/svg->clj` errors): mock both with the parsed data.
- A hook-order violation in a mounted component reaches neither the root's `onUncaughtError`/`onCaughtError`/`onRecoverableError` nor the markup: React logs an internal error to `console.error` and recovers, so the callbacks stay empty and the tree repaints. Assert on a captured `console.error` too. It only fires when a render *adds* hooks (nil → uuid); a render that calls none at all slips through React's bookkeeping, so the regression test needs the "hooks arrive" direction. `frontend-tests.ui.team-container-test` does both, with a jsdom document.
- `beicon.v2.core` has no `never`. A stream that must **not emit** is `(rx/empty)`: it completes at once, which is enough for `merge`, `filter` and `take` where only the emission matters. A stream that must **stay open** is a `(rx/subject)` nobody pushes to; `rx/empty` would complete the flow behind it. An undeclared `rx/never` compiles to an undefined var and the test dies as a rejected body, which `cljs.test` reports as a test with zero assertions instead of a failure.
- `set!` mocks are global and outlive the fixture, so a flow another module left running (a store's pending timer) can still reach the mocked transport mid-test; `with-persistence` records every command for that reason. Filter to the commands your layer issues — persistence only sends `update-file` — before counting or asserting emptiness, and never let a stray call release a `wait-for` on one.
From `frontend/`: From `frontend/`:
- Full unit test run (always builds, suppressed output): `pnpm run test:quiet`. - Full unit test run (always builds, suppressed output): `pnpm run test:quiet`.
@ -50,12 +57,13 @@ From `frontend/`:
- Build test target only (no run): `pnpm run build:test`. - Build test target only (no run): `pnpm run build:test`.
- After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`. - After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
- Watch tests: `pnpm run watch:test`. - Watch tests: `pnpm run watch:test`.
- Coverage: `pnpm run test:coverage` (builds, runs the suite under `NODE_V8_COVERAGE`, keeps its exit code). `scripts/coverage.js` maps V8 data back to `.cljs`/`.cljc` through the test build's source maps (`ast-v8-to-istanbul` + acorn) and writes `coverage/` (`lcov.info`, `coverage-summary.json`, `html/`, `unloaded.txt`); `tests-frontend.yml` renders the step summary with `scripts/coverage-summary.mjs` and uploads `coverage/` as the `frontend-coverage` artifact. `scripts/test` (CI) runs `test:coverage`, not `test`. Scope: `frontend/src/app/**` minus `app/main/ui/` (`EXCLUDED_DIRS`); namespaces no test loads are listed apart, never counted. Never pass `ignoreSourceCode` to `convert`: CLJS maps point past source lines and it throws. Trust line % over branch % (branches are of the generated JS).
New frontend test namespaces must be required/listed in `frontend_tests/runner.cljs`; new vars in existing namespaces need no runner change. New frontend test namespaces must be required/listed in `frontend_tests/runner.cljs`; new vars in existing namespaces need no runner change.
## Playwright integration tests ## Playwright integration tests
Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, use `pnpm run test:e2e` or `pnpm run test:e2e --grep "pattern"` from `frontend/`; ensure dependencies are installed through `./scripts/setup` if the environment is not prepared. Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, run the whole suite with a worker count sized to the machine: `pnpm run test:e2e --workers=N` from `frontend/`, about one worker per two hardware threads, because each keeps two or three busy and holds about 2 GB of browser memory. The config's default of one worker leaves it slow; four take a 16-thread host from 48 minutes to 35 and fit a 64 GB machine beside a watching devenv, while a 32 GB machine takes two, or four with a slim devenv. `--grep "pattern"` narrows the run. Ensure dependencies are installed through `./scripts/setup` if the environment is not prepared.
Integration tests fake backend behavior by intercepting network/websocket traffic, so every RPC or websocket the page needs must be mocked. Use existing Page Object Models: Integration tests fake backend behavior by intercepting network/websocket traffic, so every RPC or websocket the page needs must be mocked. Use existing Page Object Models:
- `BasePage.mockRPC` intercepts RPC calls and already prefixes `/api/rpc/command/`; pass command names such as `get-profile`, not full URLs. - `BasePage.mockRPC` intercepts RPC calls and already prefixes `/api/rpc/command/`; pass command names such as `get-profile`, not full URLs.

View File

@ -46,6 +46,7 @@
- `start-undo-transaction` stamps the transaction with a fresh `:commit-group` (dropped from the history entry); `commit-changes` uses `(or undo-group (:undo-group tx) (:commit-group tx) (uuid/next))`, so all commits of one transaction are one rollback unit. Pre-setting `:undo-group` instead breaks entries whose group is set later (sync transactions). - `start-undo-transaction` stamps the transaction with a fresh `:commit-group` (dropped from the history entry); `commit-changes` uses `(or undo-group (:undo-group tx) (:commit-group tx) (uuid/next))`, so all commits of one transaction are one rollback unit. Pre-setting `:undo-group` instead breaks entries whose group is set later (sync transactions).
- Undo transactions can stay open only temporarily; timed-out pending transactions are force-committed after about 20s. Undo entries are capped at 50. - Undo transactions can stay open only temporarily; timed-out pending transactions are force-committed after about 20s. Undo entries are capped at 50.
- Undo/redo are ignored while a normal editor/drawing interaction is active, except grid-layout edition handles undo through this path. `undo`, `redo` and `undo-to-index` also do nothing while the save lock holds: `materialize-undo` would move the index while the commit is refused. - Undo/redo are ignored while a normal editor/drawing interaction is active, except grid-layout edition handles undo through this path. `undo`, `redo` and `undo-to-index` also do nothing while the save lock holds: `materialize-undo` would move the index while the commit is refused.
- Undo entries carry a required boolean `:translation` (`schema:undo-entry` rejects an entry without it; `initialize-workspace` sets it when it records a local commit) and `undo`, `redo` and `undo-to-index` pass it back to `commit-changes`, so replaying a pure move skips component sync and thumbnails like the original commit did. Merging ANDs the flags: `stack-undo-entry`, `accumulate-undo-entry` (`empty-tx` starts at true) and undo groups in `undo-to-index` (`every?`). Test harnesses that mirror the recorder (`tokens/helpers/state.cljs`, `composable_tests/interpreter.cljs`) must pass it too.
- After local commits and when render-wasm is active, text shapes get derived `:position-data` recomputed in a separate commit tagged `#{:position-data}`; that tag is excluded from the position-data watcher to avoid loops. - After local commits and when render-wasm is active, text shapes get derived `:position-data` recomputed in a separate commit tagged `#{:position-data}`; that tag is excluded from the position-data watcher to avoid loops.
## Refs ## Refs

View File

@ -57,7 +57,7 @@ mcp/
│ ├── src/ │ ├── src/
│ │ ├── main.ts # handles communication │ │ ├── main.ts # handles communication
│ │ └── plugin.ts # plugin implementation │ │ └── plugin.ts # plugin implementation
│ └── package.json # Includes @penpot-mcp/common dependency │ └── package.json # Plugin deps: published plugin-styles, linked plugin-types
└── prepare-api-docs # Python project for the generation of API docs └── prepare-api-docs # Python project for the generation of API docs
``` ```
@ -107,12 +107,27 @@ From the `mcp/` directory, run
(node reports `Promise resolution is still pending but the event loop has (node reports `Promise resolution is still pending but the event loop has
already resolved` for the file's root test). Inside tests the logger writes to already resolved` for the file's root test). Inside tests the logger writes to
stdout directly. `NODE_TEST_CONTEXT` is the env var `node --test` sets. stdout directly. `NODE_TEST_CONTEXT` is the env var `node --test` sets.
* `packages/plugin` takes `@penpot/plugin-styles` from the registry but links
`@penpot/plugin-types` to `plugins/libs/plugin-types` (`link:` in the manifest
and in `mcp/pnpm-lock.yaml`, with no registry entry, because a linked package
has no resolution): the published types do not declare the `penpotMgmt`
global the plugin needs. On a plugins libs version bump, only the styles
version moves. After editing the manifest, `pnpm install --lockfile-only` in
`mcp/` must leave the lock untouched.
* Cross-cutting testing principles and anti-patterns: `mem:testing`. * Cross-cutting testing principles and anti-patterns: `mem:testing`.
## OAuth for remote MCP clients ## OAuth for remote MCP clients
- Authorization/discovery URLs, self-contained client IDs, consent and PKCE/code binding, backend Redis state, deployment wiring, and compatibility limits: `mem:mcp/oauth`. - Authorization/discovery URLs, self-contained client IDs, consent and PKCE/code binding, backend Redis state, deployment wiring, and compatibility limits: `mem:mcp/oauth`.
## Host and Origin allowlist (`RequestGuard`)
- `RequestGuard` checks `Host` and `Origin` (hostname only, port ignored) on `/mcp`, the REPL (Express middleware, 403) and the plugin WebSocket (close `1008` before any message is read). It stops web pages on the user's machine from driving a local server (DNS rebinding, cross-site WebSocket).
- Allowlist: `PENPOT_MCP_ALLOWED_HOSTS` (`*` turns the check off); unset → loopback only in single-user mode (local or remote: no other barrier, `execute_code` still runs in the user's session), any host in multi-user mode (relies on user tokens; behind nginx `/mcp/ws` the server sees `Host: penpot-mcp:4402`). A missing `Origin` passes (non-browser clients).
- Entries must be `*`, a hostname or a bracketed IPv6 address; one with a scheme, port or path makes `fromEnvironment` throw, so the server does not start. Startup logs a `warn` when the check is off in single-user mode.
- The plugin iframe has `allow-same-origin`, so its WebSocket carries the real plugin origin (`http://localhost:4400` locally, the Penpot origin when integrated).
- Test mocks of `PenpotMcpServer` passed to `PluginBridge` must provide `requestGuard`.
## Backend tools (`submit_feedback`) ## Backend tools (`submit_feedback`)
- Tools that call the Penpot backend directly (not through the plugin) register only when `PENPOT_MCP_BACKEND_URI` is nonblank (`PenpotMcpServer.initTools`). Currently only `SubmitFeedbackTool`. - Tools that call the Penpot backend directly (not through the plugin) register only when `PENPOT_MCP_BACKEND_URI` is nonblank (`PenpotMcpServer.initTools`). Currently only `SubmitFeedbackTool`.
@ -133,10 +148,11 @@ For parallel devenvs, prefer same-origin MCP routing: each Penpot instance shoul
## Plugin reconnect policy ## Plugin reconnect policy
- The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS. - The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS.
- The MCP server emits `1008` for a missing `userToken` in multi-user mode, for missing/invalid connection metadata, and to close a connection displaced by a newer one with the same user and session ID. - The MCP server emits `1008` for a `Host`/`Origin` outside the `RequestGuard` allowlist, for a missing `userToken` in multi-user mode, for missing/invalid connection metadata, and to close a connection displaced by a newer one with the same user and session ID.
- Session IDs are deterministic per tab and file, so a reconnect reuses its ID while the old socket may be half-open: the newest connection wins (`PluginBridge.displaceConnection`). Across instances, each new connection publishes a session claim on a per-user Redis channel; an instance displaces its local connection only on a foreign claim that arrives after its own claim (channel order, no clocks). Discovery deduplicates by session ID. - Session IDs are deterministic per tab and file, so a reconnect reuses its ID while the old socket may be half-open: the newest connection wins (`PluginBridge.displaceConnection`). Across instances, each new connection publishes a session claim on a per-user Redis channel; an instance displaces its local connection only on a foreign claim that arrives after its own claim (channel order, no clocks). Discovery deduplicates by session ID.
- A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via the MCP menu. - A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via the MCP menu.
## Rollback warnings ## Rollback warnings
- `ExecuteCodeTaskHandler` awaits `penpot.waitForSave()` after the code and outside its swallow catch, so a save failure, save timeout or rollback during the wait fails that `execute_code` call itself. It hard-fails as "missing Plugin API support" when `penpot.waitForSave` is absent (old Penpot or stale runtime build, or broken `api.cljs` wiring). Both waits run only when `penpotMgmt.workspace.status` is `"ready"` (or there is no `penpotMgmt`), checked at each wait: dashboard calls skip them, because persistence `:error` outlives the workspace and would fail successful `penpotMgmt` calls. Budget: layout wait 30s before the code + save wait 60s after it, under `PENPOT_MCP_TOOL_TIMEOUT_S` (120s). `RollbackNotices` below only cover rollbacks that land after the call returned (concurrent edits, a response after a timeout).
- The plugin's `RollbackLog` (`packages/plugin/src/RollbackLog.ts`) holds the `contentrollback` events not yet sent with a task response; `Task.sendResponse` attaches them and acknowledges them once sent, and the server's `unclaimed-rollbacks` message (a response that came after its task timed out) puts them back. With a `RollbackStore` (`LocalStorageRollbackStore` over `penpot.localStorage`, one entry per file) every change is written through, so a reload of the tab or plugin keeps them; storage failures fall back to memory. Plugin tests run with `node --experimental-strip-types`, which rejects TypeScript parameter properties: declare fields explicitly. - The plugin's `RollbackLog` (`packages/plugin/src/RollbackLog.ts`) holds the `contentrollback` events not yet sent with a task response; `Task.sendResponse` attaches them and acknowledges them once sent, and the server's `unclaimed-rollbacks` message (a response that came after its task timed out) puts them back. With a `RollbackStore` (`LocalStorageRollbackStore` over `penpot.localStorage`, one entry per file) every change is written through, so a reload of the tab or plugin keeps them; storage failures fall back to memory. Plugin tests run with `node --experimental-strip-types`, which rejects TypeScript parameter properties: declare fields explicitly.

View File

@ -18,6 +18,7 @@
## Sandbox and global cleanup ## Sandbox and global cleanup
- `src/index.ts` calls `bootstrapSES()` (in `src/lib/bootstrap.ts`) at module load. If `repairIntrinsics` throws (e.g. SES 2.2+ rejects a `Temporal` wrapped by a browser extension), the error is caught, `getPluginsRuntimeError()` exposes it, and `initPluginsRuntime` installs no `ɵ*` globals: the frontend app still boots, plugins and MCP are unavailable in that environment. On failure, `Function.prototype.toString` has already been replaced but no intrinsics are hardened and no compartment/sandbox is created — never run plugins on that path.
- The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code. - The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code.
- Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json. - Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json.
- Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them. - Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them.

View File

@ -43,7 +43,13 @@ Production runs the deployed toolbox, not the devenv one:
- **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups. - **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups.
- **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them. - **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them.
- The read-only replica cuts long queries: see `mem:prod-infra/grafana`. - The read-only replica cuts long queries: see `mem:prod-infra/grafana`.
- A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table (file saves write to `file_change` on every save) build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block); the shipped migration uses `IF NOT EXISTS`, finds it and does nothing. A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_indexes` and `pg_index.indisvalid`). - A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block). A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_index` and `pg_index.indisvalid`).
- A partitioned parent cannot take `CREATE INDEX CONCURRENTLY` (`cannot create index on partitioned table ... concurrently`). Build one index per partition with `CONCURRENTLY`, then `CREATE INDEX ... ON ONLY` on the parent and `ALTER INDEX ... ATTACH PARTITION` for each child; the parent index only becomes valid when every partition is attached.
- Write the migration as plain, idempotent SQL: `ADD COLUMN IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DROP CONSTRAINT IF EXISTS` followed by `ADD CONSTRAINT`. Do not encode shape checks as `pg_catalog` introspection: they are easy to get wrong (a missing `NOT condeferred` accepts an `INITIALLY DEFERRED` foreign key) and they duplicate what the runbook verifies by hand. `IF NOT EXISTS` matches the name, not the definition, so the out-of-band procedure verifies the index shape and completeness.
- The runbook is the contract for a large instance: it builds and verifies the out-of-band work and then marks the migration step as applied (`INSERT INTO migrations (module, step) ...`), so the body does not run at startup. Verify the column, the pending rows, the index (valid, no predicate or expression, every partition attached) and the constraint shape before inserting the row; the marker is the only thing that skips the migration.
- A reference that a GC task resolves must be `DEFERRABLE INITIALLY IMMEDIATE`, never `INITIALLY DEFERRED`: the immediate check fires inside the transaction that catches the violation and retries the chunk; the deferred one fires at commit, outside the catch. A validated foreign key belongs out of band: `ADD FOREIGN KEY` runs as an `ALTER TABLE` and it scans the whole referencing table; `NOT VALID` on a partitioned table only exists since PostgreSQL 18.
- A backfill that removes a key an old reader uses (for example `metadata.storage-ref-id`, which the previous storage GC scans) must run with the old code stopped. Otherwise the old reader loses the reference and can delete a live object, and the rows it writes in the meantime miss the backfill.
- `ADD COLUMN` takes a brief `ACCESS EXCLUSIVE` even when `IF NOT EXISTS` skips it, and `ADD FOREIGN KEY` takes `ACCESS EXCLUSIVE` on the referencing table and `SHARE ROW EXCLUSIVE` on the referenced one. A migration accepts those locks; a deployment that cannot take them runs the runbook and marks the step as applied.
## See also ## See also

View File

@ -34,8 +34,10 @@ From `render-wasm/`:
- Clean one target: `./clean [frontend|export]`; never run root `cargo clean` on the shared `target/`. - Clean one target: `./clean [frontend|export]`; never run root `cargo clean` on the shared `target/`.
- Watch rebuild: `./watch [frontend|export]`; its initial and change-triggered builds use `./build`. - Watch rebuild: `./watch [frontend|export]`; its initial and change-triggered builds use `./build`.
- Rust tests: `./test` or `cargo test <name>`. - Rust tests: `./test` or `cargo test <name>`.
- Rust coverage: `RENDER_WASM_COVERAGE=1 ./test` runs the same tests under `cargo llvm-cov` (needs `cargo-llvm-cov` on PATH; the script adds `llvm-tools-preview`) and writes `target/coverage/` (`lcov.info` with absolute paths, `summary.json` llvm export totals, `html/`). Native build only: code behind wasm-only `cfg` is not measured. `tests-wasm.yml` downloads a pinned, checksum-verified `cargo-llvm-cov` release because the devenv image lacks it.
- Cross-cutting testing principles and anti-patterns: `mem:testing`. - Cross-cutting testing principles and anti-patterns: `mem:testing`.
- Lint: `./lint`. - Lint: `./lint`.
- Format check: `cargo fmt --check`. - Format: `./fmt`; check only: `./fmt --check`.
- `./build`, `./test`, `./lint` and `./fmt` all run `setup`, which selects the Rust version pinned in `_build_env` (skip with `PENPOT_WASM_PREPARED=1`).
Do not change exported WASM function signatures without updating the corresponding frontend bridge and verifying the frontend renderer path. Do not change exported WASM function signatures without updating the corresponding frontend bridge and verifying the frontend renderer path.

View File

@ -6,8 +6,10 @@
- `check-merged <PR...>` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries. - `check-merged <PR...>` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries.
- `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`. - `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`.
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings; GHSA entries never appear here by design. - `security <MILESTONE> [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix <description> (<advisory URL>) (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place. - `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design.
- `dev-only-bugs <MILESTONE> [--apply]` — list closed milestone bugs that never reached production, without `no changelog` (bugs carrying `release blocker` are analyzed too). Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or the milestone version itself (`only_unreleased_envs`; any production hint or other version keeps the bug in — versions compare as full semver with missing patch = 0, so for milestone 2.19.2 the released 2.19.0 keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
## Conventions ## Conventions
@ -17,4 +19,4 @@
- PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms. - PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms.
- Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes. - Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes.
- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry), G `breaking change` issue missing from `:boom:` (counted, same contract as F); C/D are `:rocket:` gaps (warnings, never counted). - Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry), G `breaking change` issue missing from `:boom:` (counted, same contract as F); C/D are `:rocket:` gaps (warnings, never counted).
- `issue_excluded` (closed, non-Task, non-Rejected, no `release blocker`/`no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift. - `issue_excluded` (closed, non-Task, non-Rejected, no `no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift.

View File

@ -106,7 +106,7 @@ produce deployable artifacts.
## Notes ## Notes
- `mcp` has no lint task and `media-processor` has no lint task; both show - `mcp` has no lint task and `media-processor` has no lint task; both show
as skipped. `render-wasm` uses `./lint`, `./test`, and `cargo fmt`. as skipped. `render-wasm` uses `./lint`, `./test`, and `./fmt`.
- `scripts/test_ci.py` (unittest, `python3 scripts/test_ci.py`) covers the - `scripts/test_ci.py` (unittest, `python3 scripts/test_ci.py`) covers the
flag wiring, one build command per module and the help text through flag wiring, one build command per module and the help text through
`--dry-run`; it executes no real build. `--dry-run`; it executes no real build.

View File

@ -40,7 +40,7 @@ python3 scripts/gh.py issues "2.16.0" --label "bug"
python3 scripts/gh.py issues "2.16.0" --label "bug,regression" python3 scripts/gh.py issues "2.16.0" --label "bug,regression"
# Exclude by label # Exclude by label
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" python3 scripts/gh.py issues "2.16.0" --exclude "no changelog"
# Show only issues NOT yet in CHANGES.md # Show only issues NOT yet in CHANGES.md
python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md
@ -81,7 +81,7 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts. Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts.
**Output**: JSON array to stdout; progress to stderr. **Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both.
### `comments` ### `comments`
@ -118,6 +118,9 @@ python3 scripts/gh.py advisories --severity critical
# Filter by state # Filter by state
python3 scripts/gh.py advisories --state triage python3 scripts/gh.py advisories --state triage
# Filter by patched version declared in the advisory
python3 scripts/gh.py advisories --patched-version 2.19.1
# Get full detail for a single advisory # Get full detail for a single advisory
python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7 python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
``` ```
@ -126,6 +129,8 @@ python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
**Detail output** (single advisory) adds: description, vulnerabilities (package, version ranges), credits, timestamps. **Detail output** (single advisory) adds: description, vulnerabilities (package, version ranges), credits, timestamps.
`--patched-version` filters the raw advisories before formatting: the match is an exact token of `vulnerabilities[*].patched_versions` (split on `,`/`;`), so `2.19.1` never matches `2.19.10`. Every state is included (`triage`, `draft`, `published`, `closed`, `withdrawn`). The raw `vulnerabilities` stay internal to the filter: the list output keeps the summary schema.
**Output**: JSON to stdout; progress to stderr. **Output**: JSON to stdout; progress to stderr.
## Key principles ## Key principles

View File

@ -73,6 +73,8 @@ a test that reproduces it:
- **One Assertion Per Concept** — each test verifies one behavior; split - **One Assertion Per Concept** — each test verifies one behavior; split
compound assertions compound assertions
- **Descriptive Test Names** — names read like specifications - **Descriptive Test Names** — names read like specifications
- **English Only** — test names, `testing` strings, comments, and every
code-facing text MUST be in English; see the hard rule in `AGENTS.md`.
## Prefer Real Implementations Over Mocks ## Prefer Real Implementations Over Mocks
@ -142,10 +144,10 @@ E2E tests should not be added unless explicitly requested.
When running ANY test command (CLJS/JS or JVM): When running ANY test command (CLJS/JS or JVM):
1. **NEVER pipe test output directly to `| head`, `| tail`, `| grep`, or similar filters** — this can hide failures and cause you to miss critical errors. 1. **NEVER pipe test output directly to `| head`, `| tail`, `| grep`, or similar filters** — this can hide failures and cause you to miss critical errors.
2. **ALWAYS pipe to a file first, then read the file:** 2. **ALWAYS redirect to a file first, then read the file:**
```bash ```bash
# CORRECT: # CORRECT:
pnpm run test 2>&1 > /tmp/test-output.txt pnpm run test > /tmp/test-output.txt 2>&1
grep -A 5 "failures" /tmp/test-output.txt grep -A 5 "failures" /tmp/test-output.txt
# WRONG: # WRONG:

View File

@ -43,10 +43,9 @@ that breaks them is wrong, even if the code is right.
- For another commit: `./scripts/check-commit -c <ref>` - For another commit: `./scripts/check-commit -c <ref>`
- **NEVER** hand-wave the body as "one long line". If a line exceeds 80, - **NEVER** hand-wave the body as "one long line". If a line exceeds 80,
break it at a space. break it at a space.
- **NEVER add a `Co-authored-by:` trailer.** It is forbidden; `scripts/check-commit` fails the commit when any line matches it (case-insensitive). - **`Co-authored-by:` is for human co-authors only** (`Co-authored-by: Name <email>`). AI providers, agents and bots must not be credited that way; use `AI-assisted-by:` instead. `scripts/check-commit` rejects a trailer value that names an AI provider or agent (`Claude`, `Copilot`, `Cursor`, `Codex`, `Gemini`, `Aider`, `Cline`, `Windsurf`, `DeepSeek`, `OpenCode`, `devin-ai`, `amazon q`, and similar) or carries a bot identity (`[bot]`, `bot@`). `NON_HUMAN_COAUTHOR_PATTERN` is the list; extend it when a new tool starts adding trailers. A person whose own name or email matches one of those tokens (a surname like `Cline`, an address at a provider) is rejected too; amend the trailer or credit them with `Reviewed-by:`.
- CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build. - CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build.
- Exceptions inside the body (do not wrap these): `Signed-off-by:`, - Exceptions inside the body (do not wrap these): `Signed-off-by:`, `Co-authored-by:`, `AI-assisted-by:` trailers, and lines carrying a URL.
`AI-assisted-by:` trailers, and lines carrying a URL.
**AI-assisted-by trailer rules:** **AI-assisted-by trailer rules:**
- Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash` - Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash`
@ -56,6 +55,14 @@ that breaks them is wrong, even if the code is right.
`:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight `:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight
## Atomic Commits
A PR squash-merges into one commit by default, so the split inside the branch
does not reach `main` and nothing depends on it. When the PR asks for a
regular merge (`mem:workflow/creating-prs` → Merge Strategy), every commit
must stand alone: it builds, passes the linter, and needs no later commit on
the same branch to make sense.
## Referencing Issues ## Referencing Issues
Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link. Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link.

View File

@ -60,6 +60,7 @@ the issue (`scripts/changelog.py boom`), so the issue must carry both:
Post the note as a new comment. Do not add a breaking-change section to the Post the note as a new comment. Do not add a breaking-change section to the
issue body and do not hand-write the changelog block. When the issue comes issue body and do not hand-write the changelog block. When the issue comes
from a PR, reuse the text of the PR's breaking-change section. from a PR, reuse the text of the PR's breaking-change section.
Do not hard-wrap the note: one line per paragraph or bullet.
The full changelog rules (sections, exclusions, `:rocket:`, pre-flight and The full changelog rules (sections, exclusions, `:rocket:`, pre-flight and
anomaly report) live in `mem:workflow/updating-changelog`. anomaly report) live in `mem:workflow/updating-changelog`.

View File

@ -32,6 +32,19 @@ PR titles follow commit title conventions:
See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters. See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters.
## Merge Strategy
Policy: `CONTRIBUTING.md` → "Merging", under "Pull Requests".
- Squash merge is the default, so the PR's own commits never reach `main`
and the PR title becomes the landed subject.
- A regular merge is possible on request only. The request is the **first
line** of the PR body, above the `**Note:**` line:
`> **Merge:** do not squash; each commit is self-contained.`
- Ask for it only when every commit on the branch is self-contained (builds
and passes lint on its own). Default to squash unless the user asks for
the opposite.
## Description Body ## Description Body
Include concise sections covering: Include concise sections covering:
@ -60,6 +73,8 @@ PR descriptions follow this structure:
``` ```
The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR. The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
When the PR requests a regular merge instead of a squash merge, its
`**Merge:**` line goes above the `**Note:**` line (see Merge Strategy).
## Writing Principles ## Writing Principles

View File

@ -2,5 +2,6 @@
Node and pnpm pins: `mem:workflow/updating-node` and `mem:workflow/updating-pnpm`. Only the parts the Dockerfiles cannot show: Node and pnpm pins: `mem:workflow/updating-node` and `mem:workflow/updating-pnpm`. Only the parts the Dockerfiles cannot show:
- JDK: the image tracks a non-LTS line, so there is no minor to pick — the last build of the line is the ceiling; moving line is a major and needs a real build to validate the `jlink` module list. - JDK: two pins that move together — `docker/devenv/Dockerfile` (`setup-jvm` stage, full JDK) and `docker/images/Dockerfile.backend` (JDK + `jlink` runtime). Same build, same sums in both. They compile and run the same backend: `manage.sh` builds the bundle inside the devenv image (`build-backend-bundle` → `build "backend"`) and `backend/scripts/run.template.sh` starts the runtime with `--enable-preview`, which requires the runtime major to match the compiler major. A runtime line behind the compiler line is a silent trap.
- JDK line: tracks a non-LTS line, so there is no minor to pick — the last build of the line is the ceiling; moving line is a major and needs a real build to validate the `jlink` module list (`--add-modules` in `Dockerfile.backend`). JDK 27 keeps all of them; check with `bin/jlink --add-modules <list>` against the downloaded JDK before bumping. Source of the URL and both sums: the Azul metadata API (`api.azul.com/metadata/v1/zulu/packages/?java_version=<n>&os=linux&arch=<x64|aarch64>&java_package_type=jdk&archive_type=tar.gz&latest=true&release_status=ga&availability_types=CA`), then confirm the sums by downloading the tarball and running `sha256sum`.
- ImageMagick: publish `penpotapp/imagemagick:<ver>` **before** bumping the `COPY --from` refs in the backend, exporter and devenv Dockerfiles; otherwise both builds break on the missing tag. - ImageMagick: publish `penpotapp/imagemagick:<ver>` **before** bumping the `COPY --from` refs in the backend, exporter and devenv Dockerfiles; otherwise both builds break on the missing tag.

View File

@ -2,7 +2,7 @@
> Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human release managers who need full context. That is why it uses plain language and examples. It is the canonical home of the changelog flow; the `update-changelog` skill is only an entry point that routes here. > Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human release managers who need full context. That is why it uses plain language and examples. It is the canonical home of the changelog flow; the `update-changelog` skill is only an entry point that routes here.
`CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory. It is never written from scratch by hand; the only manual parts are the security advisory entries and the wording of a description when the title is not enough. `CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory whose `patched_versions` include the version. It is never written from scratch by hand; the only manual parts are the wording of a description when the title is not enough and the advisory summaries (polished on GitHub, the source of the generated security entries).
The commands live in `scripts/gh.py` (GitHub queries) and `scripts/changelog.py` (changelog checks). Their flags and internals are documented in `mem:scripts/gh` and `mem:scripts/changelog`. The commands live in `scripts/gh.py` (GitHub queries) and `scripts/changelog.py` (changelog checks). Their flags and internals are documented in `mem:scripts/gh` and `mem:scripts/changelog`.
@ -19,14 +19,17 @@ Editing `CHANGES.md` happens only on explicit request or as part of the release
| Question | Source | | Question | Source |
|---|---| |---|---|
| Which issues belong to the version? | GitHub milestone | | Which issues belong to the version? | GitHub milestone |
| Which security fixes? | Advisories whose `patched_versions` include the version |
| Which section? | Issue Type | | Which section? | Issue Type |
| Is it a breaking change? | `breaking change` label | | Is it a breaking change? | `breaking change` label |
| Is it a highlight? | `release highlight` label | | Is it a highlight? | `release highlight` label |
- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them. - **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
- **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug).
- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry. - **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry.
- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge. - **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge.
- **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status. - **Excluded issues**: `no changelog`, `Task` issue type, `Rejected` project status. `release blocker` is a workflow label (the issue blocks the release), not a changelog exclusion: bugs carrying it are analyzed by `dev-only-bugs` (section 8, step 3), and closed non-bug issues carrying it are reviewed by hand (section 8, step 3).
- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3).
- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own. - **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own.
- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly. - **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly.
@ -37,7 +40,7 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
```markdown ```markdown
# CHANGELOG # CHANGELOG
## 2.19.0 (Unreleased) ## 2.20.0 (Unreleased)
### :boom: Breaking changes & Deprecations ### :boom: Breaking changes & Deprecations
@ -51,21 +54,31 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237)) - Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237))
### :bug: Bugs fixed ### :lock: Security fixes
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) - Fix <user-facing description of the vulnerability> (<advisory URL>) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :sparkles: New features & Enhancements ### :tada: Features
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) - Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :sparkles: Enhancements
- Improve <user-facing description of the existing behaviour> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :bug: Bugs
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
``` ```
- `:boom:` first, then `:rocket:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions. - `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update.
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. - **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it.
- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories.
- One blank line between the last entry of a section and the next heading. - One blank line between the last entry of a section and the next heading.
- No manual line wrapping: one line per bullet or paragraph, however long. - No manual line wrapping: one line per bullet or paragraph, however long.
## 4. Regular entries (`:bug:`, `:sparkles:`) ## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`)
The entry shape is: The entry shape is:
@ -116,11 +129,12 @@ Rules for the note:
- **Write it for admins**: what changes, what they must do, and what breaks if they do nothing. Do not paste the full runbook; explain the implications and link the runbook or docs. - **Write it for admins**: what changes, what they must do, and what breaks if they do nothing. Do not paste the full runbook; explain the implications and link the runbook or docs.
- **Link a long-lived branch**, `staging` or `main`, never a feature branch (`issue-1234`). The link may 404 until the PR merges; that is expected. - **Link a long-lived branch**, `staging` or `main`, never a feature branch (`issue-1234`). The link may 404 until the PR merges; that is expected.
- **Do not add issue or PR links**: the generator appends them. - **Do not add issue or PR links**: the generator appends them.
- **Do not hard-wrap the text**: one line per paragraph or bullet. The generator copies lines verbatim, so wrapped lines end up wrapped in `CHANGES.md`.
- **One note per issue**. If several comments carry the marker, the generator uses the last one and warns. - **One note per issue**. If several comments carry the marker, the generator uses the last one and warns.
- **No note**: the generator warns on stderr and emits only the heading and references; fix the issue, not `CHANGES.md`. - **No note**: the generator warns on stderr and emits only the heading and references; fix the issue, not `CHANGES.md`.
- **`####`, not `###`**: `###` is the level of the changelog sections. An entry at `###` would render as a sibling section and the scripts that read and rewrite `:boom:` would cut the section there. The note author never writes the `####`: the generator adds it. - **`####`, not `###`**: `###` is the level of the changelog sections. An entry at `###` would render as a sibling section and the scripts that read and rewrite `:boom:` would cut the section there. The note author never writes the `####`: the generator adds it.
- **Only merged closing PRs** are referenced; unmerged or superseded PRs are skipped. - **Only merged closing PRs** are referenced; unmerged or superseded PRs are skipped.
- **Never hand-edit the section**. Generate it with `boom` (section 7). The note lives on GitHub; edit it there. - **Never hand-edit the section**. Generate it with `boom` (section 8). The note lives on GitHub; edit it there.
## 6. `:rocket:`: epics and highlights ## 6. `:rocket:`: epics and highlights
@ -129,35 +143,52 @@ Rules for the note:
- If no issue is labelled, leave the section out; do not pick highlights yourself. - If no issue is labelled, leave the section out; do not pick highlights yourself.
- Never edit `:rocket:` by hand: add or remove the label and regenerate. - Never edit `:rocket:` by hand: add or remove the label and regenerate.
## 7. Flow ## 7. `:lock:`: security fixes
Generated from the advisories that declare the version in `patched_versions`; never written by hand.
- **Source and filter**: `python3 scripts/gh.py advisories --patched-version <VERSION>` keeps the advisories whose `vulnerabilities[*].patched_versions` contain the version as an exact token (so `2.19.1` never matches `2.19.10`). Any state is valid — `triage`, `draft`, `published` — because the advisory may still be unpublished when the changelog is written.
- **Fill the advisories first**: `patched_versions` must be filled before generating the changelog. The release process publishes the advisories with (or after) the release; the changelog does not wait for publication.
- **PR reference**: a merged PR in the milestone labelled `pr pt` whose body contains the GHSA id contributes its number inline; several PRs go comma-separated. An advisory with no PR is legal: the entry links only the advisory.
- **Missing `patched_versions`**: when a `pr pt` PR references an advisory that declares no matching `patched_versions`, `security` prints a warning on stderr and leaves it out. Fix the advisory, not `CHANGES.md`.
- **Entry format**: `- Fix <description> (<advisory URL>) (PR: [#PR](...))`. The description is `Fix` plus the advisory summary, with the first word decapitalized unless it starts with an acronym (`SVG`, `CVE-...`).
- **Wording**: the flow may polish the description. Re-running `security --write` rewrites it from the summary, so the durable place for wording is the advisory summary on GitHub; keep it a problem statement, because the generator adds `Fix`.
- **Empty version**: no matching advisories → the subsection is not inserted (and is removed by `--write`).
- **Never hand-edit the section**: generate it with `security` (section 8).
## 8. Flow
1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious. 1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious.
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`. 2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
3. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state. 3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs "<VERSION>"` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration. Bugs carrying `release blocker` are analyzed like any other bug.
4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4). - **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these.
5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it. - **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or the milestone version itself. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10).
6. **Reconcile the existing text** (section 8). - **Closed non-bug issues carrying `release blocker`** have no automatic check: `python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog"` lists them (closed, non-Task). Show them to the user: the internal ones get the `no changelog` label, the rest keep their entry.
7. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent. 4. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
8. **Generate `:rocket:`** from the `release highlight` label (section 6). 5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
9. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. GHSA entries never appear; that absence is expected. 6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
10. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 9). 7. **Reconcile the existing text** (section 9).
11. **Security advisories (GHSA)**: add them by hand under `:bug:` as `- Fix <user-facing description> (<advisory URL>)`, with no issue or PR link. Do not fetch or verify the URL: it may be draft and return 404. Write the description in imperative mood from the advisory title. These entries are invisible to the automation; apply only the duplicate/backport check to them. 8. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
9. **Generate `:rocket:`** from the `release highlight` label (section 6).
10. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
12. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
## 8. Pre-flight: reconcile before writing ## 9. Pre-flight: reconcile before writing
Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` before adding the new section: Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` before adding the new section:
1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports. 1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports. The same holds after merging a patch branch into `staging`: the fix keeps its released patch entry and leaves the newer, unreleased section (the merge conflict in `CHANGES.md` is resolved by keeping both sections, newest first, minus the duplicate).
2. **Stale milestone assignment** (the issue moved out of this milestone) → remove the entry. 2. **Stale milestone assignment** (the issue moved out of this milestone) → remove the entry.
3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry. 3. **Newly applied exclusion labels** (`no changelog`) → remove the entry. (`release blocker` is not an exclusion label; it never removes an entry by itself.)
4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry. 4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry.
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 9); do not remove it silently. 5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently.
6. **Issue Type changed to `Task`** → remove the entry. 6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section.
7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry. 7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry.
8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it. 8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it.
9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section. 9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section.
## 9. Anomaly report ## 10. Anomaly report
`report` classifies problems it cannot decide alone. The total counts A, B, E, F and G; C and D are warnings. `report` classifies problems it cannot decide alone. The total counts A, B, E, F and G; C and D are warnings.
@ -169,19 +200,22 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights). - **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights).
- **D — `:rocket:` entry without issue AND PR references** (gap). - **D — `:rocket:` entry without issue AND PR references** (gap).
**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total.
Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run. Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run.
## 10. Pitfalls ## 11. Pitfalls
- **Open issues with provisional entries**: a PR author may add the entry before the merge. `boom` will not generate it until the issue closes; keep the hand-written block identical to what the generator would produce. - **Open issues with provisional entries**: a PR author may add the entry before the merge. `boom` will not generate it until the issue closes; keep the hand-written block identical to what the generator would produce.
- **Labels, not emoji**: title prefixes are cosmetic and often missing; the section comes from Issue Type and the two labels. - **Labels, not emoji**: title prefixes are cosmetic and often missing; the section comes from Issue Type and the two labels.
- **Taiga references**: old entries link to Taiga. Resolve them to the GitHub issue via the description text or the PRs that mention the Taiga URL, then link issue and PR. - **Taiga references**: old entries link to Taiga. Resolve them to the GitHub issue via the description text or the PRs that mention the Taiga URL, then link issue and PR.
- **Long-lived links only**: runbooks and docs referenced from a note must point to `staging` or `main`, never to a feature branch that will be deleted. - **Long-lived links only**: runbooks and docs referenced from a note must point to `staging` or `main`, never to a feature branch that will be deleted.
- **Generated sections**: `:boom:` and `:rocket:` are the only label-generated parts. Never patch them by hand; edit the label or the note comment and re-run. - **Generated sections**: `:boom:`, `:rocket:` and `:lock:` are generated. Never patch them by hand; edit the label, the note comment or the advisory and re-run.
- **Deleted or transferred issues**: an issue referenced in the changelog that returns 410/404 (deleted or moved out of the repo) must lose its entry in the pre-flight. Its security fixes, if any, stay in `:lock:` through the advisory.
## 11. Where the code is ## 12. Where the code is
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories`. Details: `mem:scripts/gh`. - `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`.
- `scripts/changelog.py`: `check-merged`, `boom`, `cross-ref`, `report`. Details: `mem:scripts/changelog`. - `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`.
- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`. - Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`.
- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**. - The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**.

View File

@ -19,6 +19,7 @@ No single source of truth: nothing reads `.nvmrc` at build time, and the version
- Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites. - Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites.
- A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift. - A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift.
- The newer version wins in both directions. A maintenance→main merge (`main-2.18.x` into `main`) can carry the newer Node while `.nvmrc` and the `packageManager` fields stay on the older pair; then adopt the newer set and finish the bump on all six sites plus the 35 `packageManager` fields. - The newer version wins in both directions. A maintenance→main merge (`main-2.18.x` into `main`) can carry the newer Node while `.nvmrc` and the `packageManager` fields stay on the older pair; then adopt the newer set and finish the bump on all six sites plus the 35 `packageManager` fields.
- Node stays on the 24.x line even when the rest of `.github/` moves on: `99db795966` (`update GitHub Actions`) took the two `plugins-deploy-*` `runtime:` pins to `node@26.13.1` on develop while `.nvmrc` stayed on `v24.22.3`. The project keeps every site on the `.nvmrc` value; resolve that conflict back to the 24.x pin instead of taking develop's.
- Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import. - Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import.
## Verification ## Verification

View File

@ -28,6 +28,12 @@
- Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, AI note) - Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, AI note)
Don't infer format from the title of a previous commit/issue/PR — the memory Don't infer format from the title of a previous commit/issue/PR — the memory
is the source of truth. is the source of truth.
- **Everything in code MUST be in English.** Identifiers (functions,
variables, types, namespaces), code comments, docstrings, test
descriptions, log hints — always English, regardless of the language the
author works in. Tests are not an exception. The only place another
language may appear in the codebase is product copy rendered through the
i18n system.
## CRITICAL: Read module memories BEFORE writing any code ## CRITICAL: Read module memories BEFORE writing any code
@ -39,6 +45,8 @@ Do this **before planning, before coding, before touching any file**:
3. Read each affected module's **core memory** — the name is `<module>/core` 3. Read each affected module's **core memory** — the name is `<module>/core`
(e.g. `frontend/core`, `backend/core`, `common/core`). (e.g. `frontend/core`, `backend/core`, `common/core`).
4. If the core memory references deeper `mem:` memories relevant to your task, read those too. 4. If the core memory references deeper `mem:` memories relevant to your task, read those too.
5. Before writing or modifying Clojure or ClojureScript, read `mem:clojure/core`
and the relevant focused Clojure memories it references.
**STOP: Do not proceed until you have read the core memory of every affected module.** **STOP: Do not proceed until you have read the core memory of every affected module.**
Skipping this step is the #1 cause of incorrect or incomplete work. Skipping this step is the #1 cause of incorrect or incomplete work.
@ -80,11 +88,6 @@ Review every prose output against these rules before delivering.
Memories are the **primary project guidance** — not docs or readme files. Memories are the **primary project guidance** — not docs or readme files.
They are dense, agent-oriented notes: terse bullets, invariants, no prose. They are dense, agent-oriented notes: terse bullets, invariants, no prose.
## Entry point
Start at `critical-info` (the graph root). It describes the project structure,
module dependency graph, and references section-level core memories.
## Progressive discovery model ## Progressive discovery model
Memories form a **reference graph**, not a flat list: Memories form a **reference graph**, not a flat list:
@ -96,11 +99,8 @@ critical-info ← read first (graph root)
└─ ... ← deeper memories as needed └─ ... ← deeper memories as needed
``` ```
When working on a task: Follow the mandatory reading sequence above. Cross-cutting topics can also be
1. Read `critical-info` to identify which sections are affected. referenced directly from `critical-info` or other memories.
2. Read the affected section's `core` memory for an overview.
3. Follow `mem:` references in the core memory to focused memories relevant to your task.
4. Continue following references deeper as needed.
## Accessing memories ## Accessing memories
@ -120,9 +120,8 @@ When you encounter a `mem:` reference relevant to your task, read that memory ne
## Topic/folder organization ## Topic/folder organization
Memories are grouped into folders that mirror project modules or topics: Memories are grouped into folders for modules or topics. Project modules have
`backend/`, `common/`, `frontend/`, `render-wasm/`, `exporter/`, `workflow/`, etc. a `<module>/core` memory; other topic folders may contain focused memories only.
Each folder's top-level memory is `<folder>/core`.
--- ---
@ -158,4 +157,3 @@ precision while maintaining a strong focus on maintainability and performance.
- `scripts/check-fmt-clj` — Check Clojure formatting without modifying files. - `scripts/check-fmt-clj` — Check Clojure formatting without modifying files.
- `scripts/ci` — CI orchestration script for running lint, tests, format checks and, on request (`--build`), production builds across modules. See `mem:scripts/ci`. - `scripts/ci` — CI orchestration script for running lint, tests, format checks and, on request (`--build`), production builds across modules. See `mem:scripts/ci`.
- `scripts/gh.py` — Multi-purpose GitHub CLI helper. Subcommands: `issues` (list issues in a milestone), `prs` (fetch PR details), `advisories` (list/inspect security advisories). See `python3 scripts/gh.py --help`. - `scripts/gh.py` — Multi-purpose GitHub CLI helper. Subcommands: `issues` (list issues in a milestone), `prs` (fetch PR details), `advisories` (list/inspect security advisories). See `python3 scripts/gh.py --help`.

View File

@ -1,5 +1,167 @@
# CHANGELOG # CHANGELOG
## 2.21.0
### :boom: Breaking changes & Deprecations
### :lock: Security fixes
- Fix real-time collaboration WebSocket never re-validates file access after `:subscribe-file`, letting revoked users keep receiving live edit content (https://github.com/penpot/penpot/security/advisories/GHSA-m53j-2766-6jqw) (PR: [#12189](https://github.com/penpot/penpot/pull/12189))
### :tada: Features
- Auto-link libraries during import based on slugified name [#9263](https://github.com/penpot/penpot/issues/9263) (PR: [#9958](https://github.com/penpot/penpot/pull/9958))
- Implement basic bullet lists [#12630](https://github.com/penpot/penpot/issues/12630) (PR: [#12720](https://github.com/penpot/penpot/pull/12720))
### :sparkles: Enhancements
- Improve performance of payload decoding in the workspace (by @lukel97) [#12916](https://github.com/penpot/penpot/issues/12916) (PR: [#12704](https://github.com/penpot/penpot/pull/12704))
- Keep the search when changing library in the Swap component menu (by @desty) [#12335](https://github.com/penpot/penpot/issues/12335) (PR: [#12873](https://github.com/penpot/penpot/pull/12873))
### :bug: Bugs
- Fix workspace stuttering when switching pages in a large file (by @lukel97) [#12917](https://github.com/penpot/penpot/issues/12917) (PR: [#12811](https://github.com/penpot/penpot/pull/12811))
- Fix workspace freezes when undoing or redoing a move of a board with many components (by @lukel97) [#12913](https://github.com/penpot/penpot/issues/12913) (PR: [#12870](https://github.com/penpot/penpot/pull/12870))
- Fix dashboard freezes for about a second after duplicating a large file (by @lukel97) [#12914](https://github.com/penpot/penpot/issues/12914) (PR: [#12911](https://github.com/penpot/penpot/pull/12911))
- Fix not being able to collapse a token set group after reordering it [#12625](https://github.com/penpot/penpot/issues/12625) (PR: [#12671](https://github.com/penpot/penpot/pull/12671))
- Fix slow variant switch with the WebGL renderer [#12968](https://github.com/penpot/penpot/issues/12968) (PR: [#12985](https://github.com/penpot/penpot/pull/12985))
- Fix login page GitHub logo showing with low contrast in the light theme [#12938](https://github.com/penpot/penpot/issues/12938) (PR: [#12943](https://github.com/penpot/penpot/pull/12943))
- Fix leak and black screen on wasm-critical errors [#12977](https://github.com/penpot/penpot/issues/12977) (PR: [#12978](https://github.com/penpot/penpot/pull/12978))
## 2.20.0
### :boom: Breaking changes & Deprecations
#### Add table partitioning for file-change
The file history table (`file_change`) is now partitioned by file, with the same 16 partitions as the file payload table (`file_data`). Queries that filter by file, and the join between the two tables, resolve against a single partition, so they stay fast as the table grows. The rewrite also removes the dead columns and stores the table in its final layout; the space the old table held comes back when it is dropped. A save no longer writes the file features on every revision, and the team snapshot quota is a single lookup.
The four snapshot RPC methods (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot` and `unlock-file-snapshot`) now require `file-id`. The frontend already sends it; API clients that call them must add it.
On small and medium instances the table is rewritten automatically on boot; there is nothing to do. On large instances (hundreds of GB in `file_change`) it does not fit in the startup transaction: run it by hand **before upgrading to 2.20.0** following the [large-instance runbook](https://github.com/penpot/penpot/blob/staging/operations/2.20.0-file-change-partition.md). The copy runs while the instance is online, followed by a cutover of a few minutes with the backend and the workers stopped. The old table is kept for rollback; check the disk headroom before starting, and drop the old table days later to get the space back.
[#12416](https://github.com/penpot/penpot/issues/12416) (PR: [#12417](https://github.com/penpot/penpot/pull/12417))
#### Protect the file data storage objects with a foreign key in the backend
This release moves the file data storage pointer out of `file_data.metadata->>'storage-ref-id'` and into a new `file_data.storage_object_id` column with a foreign key to `storage_object(id)`. The metadata key is gone: nothing writes or reads it.
What it means for self-hosted instances:
- The release is a coordinated restart. The new code reads the column; the migration drops `storage-ref-id` from the rows it backfills. Rolling back to the previous binary is not supported after this release runs.
- A fresh database or a small instance needs nothing special: the startup migration does the whole job.
- A large instance should not let the startup migration run. Build the column, the index and the foreign key online beforehand, stop the instances, run the backfill only if a row still carries `metadata.storage-ref-id`, verify the end state, and mark the migration as applied. The full procedure is documented in [`operations/2.20.0-file-data-storage-object-id.md`](https://github.com/penpot/penpot/blob/main/operations/2.20.0-file-data-storage-object-id.md).
- If the storage backend was never enabled, there is nothing to backfill and the stopped window is just the restart.
If you upgrade blindly: on fresh databases and small instances the startup migration completes the change. On a large instance, the foreign key validation scan can block writes for a long time; follow the runbook instead.
[#12768](https://github.com/penpot/penpot/issues/12768) (PR: [#12769](https://github.com/penpot/penpot/pull/12769))
### :rocket: Epics and highlights
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
### :tada: Features
- Add a feedback channel to the MCP server [#12490](https://github.com/penpot/penpot/issues/12490) (PR: [#12551](https://github.com/penpot/penpot/pull/12551))
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Implement full text format paste within Penpot [#12047](https://github.com/penpot/penpot/issues/12047) (PR: [#12137](https://github.com/penpot/penpot/pull/12137))
- Add team, project and file management to the Plugin API for global plugins [#12170](https://github.com/penpot/penpot/issues/12170) (PR: [#12169](https://github.com/penpot/penpot/pull/12169))
- Serve Penpot under a subpath [#3194](https://github.com/penpot/penpot/issues/3194) (PR: [#12358](https://github.com/penpot/penpot/pull/12358))
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
- Add OAuth support for MCP client connections [#12034](https://github.com/penpot/penpot/issues/12034) (PR: [#12480](https://github.com/penpot/penpot/pull/12480))
### :sparkles: Enhancements
- Auto-scroll sortable lists when dragging an item near their top or bottom edge (by @ShreyashAgare26) [#10478](https://github.com/penpot/penpot/issues/10478) (PR: [#11595](https://github.com/penpot/penpot/pull/11595))
- Make WebGL the default renderer for new accounts [#11855](https://github.com/penpot/penpot/issues/11855) (PR: [#12880](https://github.com/penpot/penpot/pull/12880))
- Prevent the hosted MCP server from running execute_code twice when the plugin is open in two tabs [#12008](https://github.com/penpot/penpot/issues/12008) (PR: [#11927](https://github.com/penpot/penpot/pull/11927))
- Allow detaching typography tokens [#11887](https://github.com/penpot/penpot/issues/11887) (PR: [#11969](https://github.com/penpot/penpot/pull/11969))
- Complete the Russian translations [#11955](https://github.com/penpot/penpot/issues/11955) (PR: [#11956](https://github.com/penpot/penpot/pull/11956))
- Run .penpot file import and export as durable background jobs [#12194](https://github.com/penpot/penpot/issues/12194) (PR: [#12056](https://github.com/penpot/penpot/pull/12056), [#12192](https://github.com/penpot/penpot/pull/12192))
- Open files and projects in a new tab with Ctrl/Cmd+click in the dashboard [#12228](https://github.com/penpot/penpot/issues/12228) (PR: [#12230](https://github.com/penpot/penpot/pull/12230))
- Add an RPC method to read file-level plugin data [#12258](https://github.com/penpot/penpot/issues/12258) (PR: [#10171](https://github.com/penpot/penpot/pull/10171))
- Add a get-file-tokens RPC method to read a file's design tokens [#12260](https://github.com/penpot/penpot/issues/12260) (PR: [#12172](https://github.com/penpot/penpot/pull/12172))
- Show the progress and the size of each file in the import and export dialogs [#12340](https://github.com/penpot/penpot/issues/12340) (PR: [#12192](https://github.com/penpot/penpot/pull/12192))
- Focus the search field when opening the color tokens tab of the color picker [#12344](https://github.com/penpot/penpot/issues/12344) (PR: [#12345](https://github.com/penpot/penpot/pull/12345))
- Harden the file data offload job with a row lock, per-file transactions, throttle and metrics [#12398](https://github.com/penpot/penpot/issues/12398) (PR: [#12403](https://github.com/penpot/penpot/pull/12403))
- Render email templates inside the mail job in the backend [#12445](https://github.com/penpot/penpot/issues/12445) (PR: [#12448](https://github.com/penpot/penpot/pull/12448))
- Run local WASM exports for shapes and boards without freezing the UI [#12512](https://github.com/penpot/penpot/issues/12512) (PR: [#12515](https://github.com/penpot/penpot/pull/12515))
- Use WASM to generate SVG markup in the code inspector when render-wasm is active [#12567](https://github.com/penpot/penpot/issues/12567) (PR: [#12608](https://github.com/penpot/penpot/pull/12608))
- Close the workspace search with the Escape key (by @filipsajdak) [#9540](https://github.com/penpot/penpot/issues/9540) (PR: [#12374](https://github.com/penpot/penpot/pull/12374))
- Restrict management RPC methods to their declared shared keys [#12730](https://github.com/penpot/penpot/issues/12730) (PR: [#12736](https://github.com/penpot/penpot/pull/12736))
- Reduce memory used by duplicate shape ids in the workspace [#12791](https://github.com/penpot/penpot/issues/12791) (PR: [#12792](https://github.com/penpot/penpot/pull/12792))
- Reduce memory consumption on shape decoding [#12789](https://github.com/penpot/penpot/issues/12789) (PR: [#12790](https://github.com/penpot/penpot/pull/12790))
- Notify other users when a snapshot is restored [#12425](https://github.com/penpot/penpot/issues/12425) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Normalize the configured tenant to lowercase [#12842](https://github.com/penpot/penpot/issues/12842) (PR: [#12843](https://github.com/penpot/penpot/pull/12843))
- Speed up saves on large files by checking only the pages a save changes [#12602](https://github.com/penpot/penpot/issues/12602) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Retry saves after short server failures and keep the workspace open when a save fails [#12601](https://github.com/penpot/penpot/issues/12601) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Switch the workspace to read-only mode when the file can't be saved [#12507](https://github.com/penpot/penpot/issues/12507) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Validate upload content against real file bytes in media uploads [#12796](https://github.com/penpot/penpot/issues/12796) (PR: [#12801](https://github.com/penpot/penpot/pull/12801))
- Add the file and MCP session to audit events sent from plugins and the MCP [#12902](https://github.com/penpot/penpot/issues/12902) (PR: [#12972](https://github.com/penpot/penpot/pull/12972))
- Add skip-if-active option to job submission in the backend jobs substrate [#12265](https://github.com/penpot/penpot/issues/12265) (PR: [#12266](https://github.com/penpot/penpot/pull/12266))
- Add HTML paste from clipboard in text editor v3 [#10465](https://github.com/penpot/penpot/issues/10465)
- Skip loading the thumbnail rasterizer in WebGL renderer sessions to cut workspace memory [#13029](https://github.com/penpot/penpot/issues/13029) (PR: [#12835](https://github.com/penpot/penpot/pull/12835))
- Speed up uploading page shapes to WASM [#13030](https://github.com/penpot/penpot/issues/13030) (PR: [#12854](https://github.com/penpot/penpot/pull/12854))
### :bug: Bugs
- Fix font upload failing with a regex error [#10436](https://github.com/penpot/penpot/issues/10436) (PR: [#11951](https://github.com/penpot/penpot/pull/11951))
- Fix dashboard file thumbnails loading slow for large multi-page files [#12053](https://github.com/penpot/penpot/issues/12053) (PR: [#12054](https://github.com/penpot/penpot/pull/12054))
- Fix Plugin API resolving a font family token to its separate words when the family name has several words [#11748](https://github.com/penpot/penpot/issues/11748) (PR: [#11750](https://github.com/penpot/penpot/pull/11750))
- Fix file import recreating circular library references when the exported libraries use each other [#12052](https://github.com/penpot/penpot/issues/12052) (PR: [#12062](https://github.com/penpot/penpot/pull/12062))
- Fix asset redirect responses advertising a cache lifetime of about 1000 days instead of 24 hours (by @ners) [#12761](https://github.com/penpot/penpot/issues/12761) (PR: [#12758](https://github.com/penpot/penpot/pull/12758))
- Fix an internal error on typography tokens due to circular dependencies on other token types [#9580](https://github.com/penpot/penpot/issues/9580)
- Fix the MCP server rejecting request bodies over 100kb (express.json default) with PayloadTooLargeError (by @GitNimay) [#12080](https://github.com/penpot/penpot/issues/12080) (PR: [#12450](https://github.com/penpot/penpot/pull/12450))
- Fix library component, color and typography counters showing zero when opening files from the dashboard (by @lukel97) [#12826](https://github.com/penpot/penpot/issues/12826) (PR: [#12787](https://github.com/penpot/penpot/pull/12787))
- Fix workspace crash with Maximum call stack size exceeded when applying a library update [#12576](https://github.com/penpot/penpot/issues/12576) (PR: [#12661](https://github.com/penpot/penpot/pull/12661))
- Fix canvas rendering an imported shape without its hole when the SVG fill rule is evenodd [#11947](https://github.com/penpot/penpot/issues/11947) (PR: [#12006](https://github.com/penpot/penpot/pull/12006))
- Fix Plugin API component.instance() and createShapeFromSvg() nesting the new shape into whatever board is at (0, 0) of the current page (by @makesomethingshit) [#12086](https://github.com/penpot/penpot/issues/12086) (PR: [#12499](https://github.com/penpot/penpot/pull/12499))
- Fix view mode breaking zoom, board navigation, and comments when the URL has no page id [#12684](https://github.com/penpot/penpot/issues/12684) (PR: [#12687](https://github.com/penpot/penpot/pull/12687))
- Fix autosave stopping and valid edits being lost when part of a save fails validation [#12553](https://github.com/penpot/penpot/issues/12553) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix dashboard duplicate copying a file without the edits that are still saving [#12603](https://github.com/penpot/penpot/issues/12603) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Alt measurements still showing a shape that was deleted [#12605](https://github.com/penpot/penpot/issues/12605) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Plugin API ignoring changes to a deleted shape without an error [#12606](https://github.com/penpot/penpot/issues/12606) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix typography sample overflowing its row in locales with a wider sample (ko, ar) (by @desty) [#11486](https://github.com/penpot/penpot/issues/11486) (PR: [#11487](https://github.com/penpot/penpot/pull/11487))
- Fix token not showing when selecting two text layers (by @ShreyashAgare26) [#11924](https://github.com/penpot/penpot/issues/11924) (PR: [#11944](https://github.com/penpot/penpot/pull/11944))
- Fix inner shadow fill not working in the SVG render while it works in WebGL [#11967](https://github.com/penpot/penpot/issues/11967) (PR: [#12141](https://github.com/penpot/penpot/pull/12141))
- Fix @penpot/plugin-styles loading Work Sans from a third-party CDN (by @ump45nose) [#12023](https://github.com/penpot/penpot/issues/12023) (PR: [#12087](https://github.com/penpot/penpot/pull/12087))
- Fix schema error on swatch component for image fills [#12135](https://github.com/penpot/penpot/issues/12135) (PR: [#12136](https://github.com/penpot/penpot/pull/12136))
- Fix worker storing more shape data than needed [#12740](https://github.com/penpot/penpot/issues/12740) (PR: [#12840](https://github.com/penpot/penpot/pull/12840))
- Fix worker not freeing pages [#12738](https://github.com/penpot/penpot/issues/12738) (PR: [#12751](https://github.com/penpot/penpot/pull/12751))
- Fix editing the L value in HSL mode resetting H and S to 0 (by @makesomethingshit) [#12615](https://github.com/penpot/penpot/issues/12615) (PR: [#12629](https://github.com/penpot/penpot/pull/12629))
- Restrict dashboard plugins to the MCP plugin and localhost plugins [#12948](https://github.com/penpot/penpot/issues/12948) (PR: [#12949](https://github.com/penpot/penpot/pull/12949))
- Fix workspace memory growing on every page switch with the WebGL renderer [#13028](https://github.com/penpot/penpot/issues/13028) (PR: [#12877](https://github.com/penpot/penpot/pull/12877))
- Fix workspace showing React error #310 when the team container renders without a team id (by @filipsajdak) [#12728](https://github.com/penpot/penpot/issues/12728) (PR: [#12920](https://github.com/penpot/penpot/pull/12920))
## 2.19.2
### :bug: Bugs fixed
- Fix referential integrity error when copying component variants with nested components across libraries and teams [#10704](https://github.com/penpot/penpot/issues/10704) (PR: [#12928](https://github.com/penpot/penpot/pull/12928))
- Fix migration 0033 inserting nil shape entries for dangling child references that blocked file opening [#12936](https://github.com/penpot/penpot/issues/12936) (PR: [#12979](https://github.com/penpot/penpot/pull/12979))
- Fix switching a variant and adding a component instance taking over 10 seconds in files with many token-styled shapes [#13019](https://github.com/penpot/penpot/issues/13019) (PR: [#13021](https://github.com/penpot/penpot/pull/13021))
- Fix autosave failing after uploading an SVG as a component in the assets panel, and on other mains named unlike their component [#12773](https://github.com/penpot/penpot/issues/12773) (PR: [#12794](https://github.com/penpot/penpot/pull/12794))
- Fix `enable-oidc-registration` not working when password login is disabled [#12964](https://github.com/penpot/penpot/issues/12964) (PR: [#12974](https://github.com/penpot/penpot/pull/12974))
- Fix self-hosted backend opening 60 database connections at startup and keeping them open while idle (by @UsamaSadiq) [#12622](https://github.com/penpot/penpot/issues/12622) (PR: [#13052](https://github.com/penpot/penpot/pull/13052))
## 2.19.1
### :lock: Security fixes
- Fix editor-role team members retrieving the signed invitation token of any pending invitation (https://github.com/penpot/penpot/security/advisories/GHSA-4mmc-j8rh-74pf) (PR: [#12892](https://github.com/penpot/penpot/pull/12892))
- Fix share-link page-scope escape in `get-profiles-for-file-comments` leaking the identity of comment authors outside the link's scope (https://github.com/penpot/penpot/security/advisories/GHSA-89cq-3wv7-987r) (PR: [#12888](https://github.com/penpot/penpot/pull/12888))
- Fix missing authorization and unbounded array in mark-all-threads-as-read that let any authenticated user write the status of inaccessible threads (https://github.com/penpot/penpot/security/advisories/GHSA-j4vc-cmf6-jhr7) (PR: [#12894](https://github.com/penpot/penpot/pull/12894))
- Fix incomplete SVG sanitizer escaping that let a "sanitized" SVG carry live script (CVE-2026-105694) (https://github.com/penpot/penpot/security/advisories/GHSA-qwhj-rchj-58mg) (PR: [#12887](https://github.com/penpot/penpot/pull/12887))
### :bug: Bugs fixed
- Fix component instance (variant) inside a layout resetting to its original size after a shared library update of an unrelated component [#12670](https://github.com/penpot/penpot/issues/12670) (PR: [#12851](https://github.com/penpot/penpot/pull/12851))
## 2.19.0 ## 2.19.0
### :boom: Breaking changes & Deprecations ### :boom: Breaking changes & Deprecations

View File

@ -20,6 +20,7 @@ Center](https://help.penpot.app/).
- [Title format](#title) - [Title format](#title)
- [Description](#description) - [Description](#description)
- [Review process](#review-process) - [Review process](#review-process)
- [Merging](#merging)
- [What we won't accept](#what-we-wont-accept) - [What we won't accept](#what-we-wont-accept)
- [Good first issues](#good-first-issues) - [Good first issues](#good-first-issues)
- [Commit Guidelines](#commit-guidelines) - [Commit Guidelines](#commit-guidelines)
@ -111,8 +112,8 @@ Branch names are not enforced, but we recommend the following:
- Otherwise, use a short, descriptive name with words separated by hyphens - Otherwise, use a short, descriptive name with words separated by hyphens
and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`). and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`).
Since PRs are squash-merged, the branch name does not survive into the Since PRs are squash-merged by default, the branch name does not survive
commit history — what matters is the [PR title](#title). into the commit history — what matters is the [PR title](#title).
### Format ### Format
@ -156,9 +157,27 @@ for more concrete information.
- Address review feedback by **pushing new commits** — do not - Address review feedback by **pushing new commits** — do not
force-push during review, as it breaks comment threads. force-push during review, as it breaks comment threads.
- PRs require at least **one approval** before merge. - PRs require at least **one approval** before merge.
- We use **squash-merge** by default. The PR title becomes the final - We **squash-merge** by default; see [Merging](#merging) for the reason
and for how to ask for a regular merge. The PR title becomes the final
commit message, so follow the [title format](#title) above. commit message, so follow the [title format](#title) above.
### Merging
PRs are **squash-merged** by default. One PR lands as one commit, which keeps
the change atomic: the history stays readable, and `git bisect` never stops
on an intermediate commit that fails the tests or the linter.
This is the default, not a hard rule. If you keep your commits atomic and
self-contained — each one builds, passes the linter and makes sense on its
own — you may ask for a regular merge instead. Put the request in the first
line of the PR description, above everything else:
> **Merge:** do not squash; each commit is self-contained.
Maintainers honor the request when the commits hold up, and squash-merge
otherwise. After a squash merge the PR title becomes the commit subject on
the main branch; see [Title format](#title).
### What we won't accept ### What we won't accept
To save time on both sides, please avoid submitting PRs that: To save time on both sides, please avoid submitting PRs that:
@ -225,8 +244,11 @@ Commit messages must follow this format:
those prefixes to an existing subject) those prefixes to an existing subject)
- **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted) - **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted)
- Separate the subject from the body with a **blank line** - Separate the subject from the body with a **blank line**
- Never add a `Co-authored-by` trailer — it is forbidden and fails - Use `Co-authored-by:` only for a human co-author
`./scripts/check-commit` (`Co-authored-by: Name <email>`). AI providers, agents and bots are not
allowed there; for AI participation add an `AI-assisted-by: <model-name>`
trailer instead. `./scripts/check-commit` rejects the trailer when its value
names one of them.
You can check a commit against these rules with `./scripts/check-commit`. You can check a commit against these rules with `./scripts/check-commit`.
CI runs the same script on every new commit in CI runs the same script on every new commit in

View File

@ -13,9 +13,6 @@
<a href="https://tree.taiga.io/project/penpot/" rel="nofollow"> <a href="https://tree.taiga.io/project/penpot/" rel="nofollow">
<img alt="Managed with Taiga.io" src="https://img.shields.io/badge/managed%20with-TAIGA.io-709f14.svg"> <img alt="Managed with Taiga.io" src="https://img.shields.io/badge/managed%20with-TAIGA.io-709f14.svg">
</a> </a>
<a href="https://gitpod.io/#https://github.com/penpot/penpot" rel="nofollow">
<img alt="Gitpod ready-to-code" src="https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod">
</a>
</p> </p>
<p align="center"> <p align="center">

View File

@ -40,7 +40,6 @@
nrepl/nrepl {:mvn/version "1.8.0"} nrepl/nrepl {:mvn/version "1.8.0"}
org.postgresql/postgresql {:mvn/version "42.7.13"} org.postgresql/postgresql {:mvn/version "42.7.13"}
org.xerial/sqlite-jdbc {:mvn/version "3.53.4.0"}
com.zaxxer/HikariCP {:mvn/version "7.1.0"} com.zaxxer/HikariCP {:mvn/version "7.1.0"}
@ -54,6 +53,8 @@
org.jsoup/jsoup {:mvn/version "1.23.2"} org.jsoup/jsoup {:mvn/version "1.23.2"}
com.j256.simplemagic/simplemagic {:mvn/version "1.17"}
at.yawk.lz4/lz4-java at.yawk.lz4/lz4-java
{:mvn/version "1.12.0"} {:mvn/version "1.12.0"}
@ -96,6 +97,11 @@
:jvm-opts ["-Dlog4j2.configurationFile=log4j2-devenv-repl.xml"] :jvm-opts ["-Dlog4j2.configurationFile=log4j2-devenv-repl.xml"]
:extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}}} :extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}}}
;; Use after :test (`-M:dev:test:coverage`) to run the suite with cloverage.
:coverage
{:main-opts ["-m" "kaocha.runner" "--plugin" "cloverage"]
:extra-deps {lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}}}
:outdated :outdated
{:extra-deps {com.github.liquidz/antq {:mvn/version "2.11.1276"}} {:extra-deps {com.github.liquidz/antq {:mvn/version "2.11.1276"}}
:main-opts ["-m" "antq.core"]} :main-opts ["-m" "antq.core"]}

View File

@ -4,7 +4,7 @@
"license": "MPL-2.0", "license": "MPL-2.0",
"author": "Kaleidos INC Sucursal en España SL", "author": "Kaleidos INC Sucursal en España SL",
"private": true, "private": true,
"packageManager": "pnpm@12.18.0+sha512.9d7882d5dc82147ecfc4e062aa34834e0ad0296710a1837a1115ae012127d36b7109e321ac1598a1dd0a65597681b1e99917e12ed0d749eae91187943363beb3", "packageManager": "pnpm@12.21.0+sha512.66fd8278ee28935ce951792d34367ff3af5c432a2756445823029533835b8da529bbe0be5130afd56ed0713fb8789e47bad82d93bd5ae2fe54b15562d6276ef0",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "https://github.com/penpot/penpot" "url": "https://github.com/penpot/penpot"

131
backend/pnpm-lock.yaml generated
View File

@ -7,153 +7,162 @@ importers:
configDependencies: {} configDependencies: {}
packageManagerDependencies: packageManagerDependencies:
pnpm: pnpm:
specifier: 12.18.0 specifier: 12.21.0
version: 12.18.0 version: 12.21.0
packages: packages:
'@pnpm/exe.android-arm64@12.18.0': '@pnpm/exe.android-arm64@12.21.0':
resolution: {integrity: sha512-fOF6IwgvnUKEmVedGqiAsNPfN2Ry8GvaqzSQOiF3RJFbpBfPKO/6YT54s1GjToyYinRxw9preHHPe4y2N3SzhA==} resolution: {integrity: sha512-iaWidxR5B9NUBpwMznCQu1yUpKB6BjTY5BYX2lORj76VsfINdj8Z8XWSzSNrGI8qlxa28Jzdp5fiaspzOaJZlA==}
cpu: [arm64] cpu: [arm64]
os: [android] os: [android]
'@pnpm/exe.android-x64@12.18.0': '@pnpm/exe.android-x64@12.21.0':
resolution: {integrity: sha512-6OiolnaCsB3/GcJ9XNrenbUH3zFHRJ+5QynrMUUEyThbgnnCzWxa199hpQlOmxN4rAXuYQ0IYgWFNUZxyzg31Q==} resolution: {integrity: sha512-35e5v3wlwjIYxDywq+R6/z8CE4gLOEgRxt18yqz1tMRLmx2mnW+CFJI2Q8+srTNjn7gykXUC0x1k195JNikC5A==}
cpu: [x64] cpu: [x64]
os: [android] os: [android]
'@pnpm/exe.darwin-arm64@12.18.0': '@pnpm/exe.darwin-arm64@12.21.0':
resolution: {integrity: sha512-zkq+4YzVIX39ETYLyC3W7LY7MpHEU/J3nRhHl4i2yPld2Cf2+rEKRh3ejJ4AmaQ77HHJLCktY4Z3jq0fpcxFXg==} resolution: {integrity: sha512-F5OyMX/6zLSowpTZEQRojbCmPS1Pq+fyRaIinslsIe6Glh3qbyQ52u+Fe0eEzSaRuIbWZkXmAAXSWh6Q5Wrlgg==}
cpu: [arm64] cpu: [arm64]
os: [darwin] os: [darwin]
'@pnpm/exe.darwin-x64@12.18.0': '@pnpm/exe.darwin-x64@12.21.0':
resolution: {integrity: sha512-qMsL9qx92unOTa6hgkfiTJv8ceZaTx0IQg7InhYDVX2ZrTECujaVEEVypnJK5m3eGUsP1KD7Y4qIgleCwfRsYw==} resolution: {integrity: sha512-EzpvcsaE6hrrgYeEiAFYd2rSgFybYyxXhkAY7K3y6Hg3loVfcyJ81hHczpFzi8lW5RXtUlFECRPZ8ZAMRFQYYg==}
cpu: [x64] cpu: [x64]
os: [darwin] os: [darwin]
'@pnpm/exe.freebsd-x64@12.18.0': '@pnpm/exe.freebsd-x64@12.21.0':
resolution: {integrity: sha512-/9vy/4ms52cU4MjZ5NhHd6N2d9L9W0kARf5/ff2tweAHu0VeUzs/0qp/7ZHFuddm2Wa/m0AKQoPum9AVUCqKsw==} resolution: {integrity: sha512-wNjuEnmdLks4aayPu/QZWQ5tQM9Jn+odUp/vwQM8BXRYoulkTZhtZTHoD60N6mUdLWef1hgUeUh8llNGEUsimw==}
cpu: [x64] cpu: [x64]
os: [freebsd] os: [freebsd]
'@pnpm/exe.linux-arm64-musl@12.18.0': '@pnpm/exe.linux-arm64-musl@12.21.0':
resolution: {integrity: sha512-Vp+TIMPPyackVCXkOdEKH/RmAn/r07EcIyrgZYQK7VrkksqZOie0GnWIx+85mPqeFEYCRtXIZAFJ9L4ceTndDg==} resolution: {integrity: sha512-Ceit9zn4g5ZqCJtx1fV8eJE7p8aeXMrGZ+Dc01vRqZLIUNgNeI+Ur0ruyqxtsnJx6rGADyXzohumxPLHjaJ5AQ==}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
libc: [musl] libc: [musl]
'@pnpm/exe.linux-arm64@12.18.0': '@pnpm/exe.linux-arm64@12.21.0':
resolution: {integrity: sha512-2dpyVBCHT+FfpuQh9i+7MXAqNWYLCvD5vYr5p6Eimxoga8USLnjbUgyj72AOrO0hjc5ri3TEFP3jiyBHjkSZpQ==} resolution: {integrity: sha512-4Oj/iF2TDCe2lcH0n4TupPR5sSdrNLclJB6xBElLMPCUOAvBhdf2rRlSaO5M4erJ6g4Fyy7YTXTpcmfFsKyxEA==}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-ppc64@12.18.0': '@pnpm/exe.linux-ppc64@12.21.0':
resolution: {integrity: sha512-BBOOOfNJ2QwIIya3eKqOjiN4WNTszfRz7HCvw9XggcT7+ftMjNP8XJ/GfW+ltCBeUFR9d8WBVsr2m/NLjX2CZQ==} resolution: {integrity: sha512-s2o7rdSDeUTBFsNqxptpZXlTgT6cojmMVUOvTQvFNSv6Easian02ha4tR32fwwD97Au4YzIIjMCwywCF4D82Sw==}
cpu: [ppc64] cpu: [ppc64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-riscv64@12.18.0': '@pnpm/exe.linux-riscv64@12.21.0':
resolution: {integrity: sha512-7kMN77xSOYJX8GPSH5RbNAc5oM61LMmcSUukDDUwllnZPXm8GRPskCNsTWy8SCw01Kr1FB+5SwGNMiTdiziLNQ==} resolution: {integrity: sha512-EYv3GzxeBTITCuM3yGHHbpRFpUvyulkXpQ6LsjKAQNL3iaba1E6h9iDmlouLb3Io4AMMsfdk0/uG6poQ41uewA==}
cpu: [riscv64] cpu: [riscv64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-s390x@12.18.0': '@pnpm/exe.linux-s390x@12.21.0':
resolution: {integrity: sha512-htuSCpEcmDzhKoJqXbPkvfr176dBHU0QIF5mWcRiqxRXAB/1JoYiMYotfjWpk+ztgpB9qlx4XnWlLFAf7gzc6g==} resolution: {integrity: sha512-8VXGNGVNEfs7pL9GTZOzFxjFEnMt4WL3rLOADR5/kPQ7ROTV9GfpVI2DPO/zArXFSkvfc9Zad/WypHLbvzoaTQ==}
cpu: [s390x] cpu: [s390x]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-x64-musl@12.18.0': '@pnpm/exe.linux-x64-musl@12.21.0':
resolution: {integrity: sha512-Uj+vzfIQ62kDP10x36yHZIXVs5ohdNtiPYwn5FhCLlNScHPo0OAI+ShGDtLofiBXKE7umu71Jmjm7N8a8zVcDw==} resolution: {integrity: sha512-MnC5vpIXYubhDZCgCwlfJMg7Cg6HupN2IuouT9gD/tUmOfVdQYqeyYccWbthyo/Jeto8de1oQlxeRN1+1Hl2DQ==}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
libc: [musl] libc: [musl]
'@pnpm/exe.linux-x64@12.18.0': '@pnpm/exe.linux-x64@12.21.0':
resolution: {integrity: sha512-OJ+X2wcraldCwj75pCIerWARQSRpdfj0eBTs/2ODyhfeW46ylOnjTIE/GKra6Mn6E72/gBmXS8igrUioa+F9fQ==} resolution: {integrity: sha512-8eufCjHXspKbNS/kh8q4Q0gqJoOuvC4Q5pgos6XS8By/nAMtP+jf8F1Cw9er840kfAX+PJ/ts+A9qnLE91rvXQ==}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.win32-arm64@12.18.0': '@pnpm/exe.netbsd-x64@12.21.0':
resolution: {integrity: sha512-wEY6jiACGcbhhi5yD59doHqNTndS7AaujaPmtWAu1DKkHru2hy5fJ9yUGecW72Ypaw2/I3BEprfMOopap2NQtw==} resolution: {integrity: sha512-eyh1M8hQZCkLWxuva7S/fc1SbFbgqg0/5di0dgWmyiuc2Io3dbWVihDun8X+XM8zsR5uhIDvvZpbS9aZQbCovg==}
cpu: [x64]
os: [netbsd]
'@pnpm/exe.win32-arm64@12.21.0':
resolution: {integrity: sha512-7xTxxUXo/grRLivSeIl6B21gnsz7ooe5EMPcJmQqwxc8WLJVReN8MzrlHru8bqXfVr4X3+6ZGZscntKenlCBqQ==}
cpu: [arm64] cpu: [arm64]
os: [win32] os: [win32]
'@pnpm/exe.win32-x64@12.18.0': '@pnpm/exe.win32-x64@12.21.0':
resolution: {integrity: sha512-5FJkghO7RLRkQpWxeVEygW9XsmnNuxzPfXbgwFWCHsFyFhpDn/vOdL88ToQEM0Kt5igdW5s6U0/IDH8Dvkt3pg==} resolution: {integrity: sha512-9TpkJemVApxJ1Z1w0ZFGN+AuSx4nJOqXSJUqMqDvaXTYwGOzvBiiWdFP9GN0ketiWCQS1Q0/pza2E7I3pP0OBQ==}
cpu: [x64] cpu: [x64]
os: [win32] os: [win32]
pnpm@12.18.0: pnpm@12.21.0:
resolution: {integrity: sha512-nXiC1dyCFH7PxOBiqjSDTgrQKWcQoYN6ERWuASEn02txCeMhrBWYod0KZVl2gbHpmRfhLtDXSerpEYeUM2O+sw==} resolution: {integrity: sha512-Zv2CeO4ok1zpUXktNDZ/869cQyonVkRYIwKVM4NbjaUpu+C+UTCv1W7QcT+4eJ5Hutgtk71a4v5UsVVi1idu8A==}
engines: {node: '>=18'} engines: {node: '>=18'}
hasBin: true hasBin: true
snapshots: snapshots:
'@pnpm/exe.android-arm64@12.18.0': '@pnpm/exe.android-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.android-x64@12.18.0': '@pnpm/exe.android-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.darwin-arm64@12.18.0': '@pnpm/exe.darwin-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.darwin-x64@12.18.0': '@pnpm/exe.darwin-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.freebsd-x64@12.18.0': '@pnpm/exe.freebsd-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-arm64-musl@12.18.0': '@pnpm/exe.linux-arm64-musl@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-arm64@12.18.0': '@pnpm/exe.linux-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-ppc64@12.18.0': '@pnpm/exe.linux-ppc64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-riscv64@12.18.0': '@pnpm/exe.linux-riscv64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-s390x@12.18.0': '@pnpm/exe.linux-s390x@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-x64-musl@12.18.0': '@pnpm/exe.linux-x64-musl@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-x64@12.18.0': '@pnpm/exe.linux-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.win32-arm64@12.18.0': '@pnpm/exe.netbsd-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.win32-x64@12.18.0': '@pnpm/exe.win32-arm64@12.21.0':
optional: true optional: true
pnpm@12.18.0: '@pnpm/exe.win32-x64@12.21.0':
optional: true
pnpm@12.21.0:
optionalDependencies: optionalDependencies:
'@pnpm/exe.android-arm64': 12.18.0 '@pnpm/exe.android-arm64': 12.21.0
'@pnpm/exe.android-x64': 12.18.0 '@pnpm/exe.android-x64': 12.21.0
'@pnpm/exe.darwin-arm64': 12.18.0 '@pnpm/exe.darwin-arm64': 12.21.0
'@pnpm/exe.darwin-x64': 12.18.0 '@pnpm/exe.darwin-x64': 12.21.0
'@pnpm/exe.freebsd-x64': 12.18.0 '@pnpm/exe.freebsd-x64': 12.21.0
'@pnpm/exe.linux-arm64': 12.18.0 '@pnpm/exe.linux-arm64': 12.21.0
'@pnpm/exe.linux-arm64-musl': 12.18.0 '@pnpm/exe.linux-arm64-musl': 12.21.0
'@pnpm/exe.linux-ppc64': 12.18.0 '@pnpm/exe.linux-ppc64': 12.21.0
'@pnpm/exe.linux-riscv64': 12.18.0 '@pnpm/exe.linux-riscv64': 12.21.0
'@pnpm/exe.linux-s390x': 12.18.0 '@pnpm/exe.linux-s390x': 12.21.0
'@pnpm/exe.linux-x64': 12.18.0 '@pnpm/exe.linux-x64': 12.21.0
'@pnpm/exe.linux-x64-musl': 12.18.0 '@pnpm/exe.linux-x64-musl': 12.21.0
'@pnpm/exe.win32-arm64': 12.18.0 '@pnpm/exe.netbsd-x64': 12.21.0
'@pnpm/exe.win32-x64': 12.18.0 '@pnpm/exe.win32-arm64': 12.21.0
'@pnpm/exe.win32-x64': 12.21.0
--- ---
lockfileVersion: '9.0' lockfileVersion: '9.0'

View File

@ -690,15 +690,113 @@
(assoc :query (u/map->query-string params)))] (assoc :query (u/map->query-string params)))]
(redirect-response uri)))) (redirect-response uri))))
(defn- public-uri
"The configured public URI, parsed, with the app base path normalized with
a trailing slash."
[]
(u/ensure-path-slash (u/uri (cf/get :public-uri))))
(defn- public-uri-base
"The public URI as a string, with the app base path normalized with a
trailing slash. Every URL the server builds starts here, so the app base
path is never lost."
[]
(str (public-uri)))
(defn- redirect-with-organization-sso-error (defn- redirect-with-organization-sso-error
[{:keys [dest-url organization-id organization-name]}] [{:keys [dest-url organization-id organization-name]}]
(-> (str (or dest-url (cf/get :public-uri))) (-> (str (or dest-url (public-uri-base)))
(u/append-query-param :sso-error true) (u/append-query-param :sso-error true)
(u/append-query-param :organization-id organization-id) (u/append-query-param :organization-id organization-id)
(cond-> organization-name (cond-> organization-name
(u/append-query-param :organization-name organization-name)) (u/append-query-param :organization-name organization-name))
(redirect-response))) (redirect-response)))
(defn- routable-params?
"True when `params` are params of a route: at least one, and no key carries
a scheme, a path or a fragment. `screen=workspace&team-id=…` and the root
params of the application (`template=…`) qualify; `https://evil.example/`
does not."
[params]
(and (seq params)
(every? #(nil? (re-find #"[/:?#]" (name %))) (keys params))))
(defn build-dest-url
"Return the absolute URL for `query-string` under the current public URI.
The query string is the route of the client (`screen=…&params`, and the root
of the application may carry only its params, like `template=…`): it does not
carry the `?` that separates it from the path, which is put back here, so the
destination is always built by the server and the client only picks the route.
A value that still comes with the separator is trimmed, so a client bundled
before that change keeps its destination.
The params are parsed and rendered back, so a value that is not a route never
becomes a destination: it is ignored with a warning and the user lands on the
public URI, which is also where a blank one lands."
[query-string]
(try
(let [base (public-uri-base)
query (str query-string)
route (if (str/starts-with? query "?")
(subs query 1)
query)
params (u/query-string->map route)]
(cond
(str/blank? query)
base
(not (routable-params? params))
(do (l/warn :hint "ignoring a redirect target that is not a route"
:query-string (subs route 0 (min 200 (count route))))
base)
:else
(str base "?" (u/map->query-string params))))
(catch Exception e
;; The destination must never break the SSO flow: a query string the
;; server cannot parse lands on the public URI instead.
(l/warn :hint "could not build the redirect target" :cause e)
(public-uri-base))))
(defn- same-origin?
"True when `uri` has the same origin as `base`: scheme, host (case
insensitive) and port, with the default port made explicit."
[uri base]
(let [default-port (fn [scheme] (if (= "https" (str/lower (str scheme))) 443 80))
port (fn [u] (or (:port u) (default-port (:scheme u))))]
(and (= (str/lower (str (:scheme uri)))
(str/lower (str (:scheme base))))
(= (str/lower (str (:host uri)))
(str/lower (str (:host base))))
(= (port uri) (port base)))))
(defn- under-base-path?
"True when the path of `uri` is under the base path of `base`."
[uri base]
(str/starts-with? (str (:path uri)) (:path base)))
(defn- trusted-dest-url
"Return `dest-url` when it points inside the current public URI, nil
otherwise.
The destination is built by the server (see `build-dest-url`), but state
tokens minted before that was the case may carry any URL, so the callback
checks the destination before redirecting to it. The check compares the
parsed origin, so neither a userinfo prefix
(`https://penpot.example.com@evil.example/`) nor a look-alike host
(`https://penpot.example.com.evil.example/`) can pass, and it requires
the path to be under the base path of the public URI, so the destination
stays inside the application."
[dest-url]
(when (some? dest-url)
(let [dest (try (u/uri (str dest-url)) (catch Exception _ nil))
base (public-uri)]
(when (and (some? dest)
(same-origin? dest base)
(under-base-path? dest base))
(str dest-url)))))
(defn- redirect-to-register (defn- redirect-to-register
[cfg info provider] [cfg info provider]
(let [info (assoc info (let [info (assoc info
@ -915,11 +1013,18 @@
(defn build-organization-sso-auth-redirect-uri (defn build-organization-sso-auth-redirect-uri
"Build the OIDC authorization redirect URI for an organization SSO config. "Build the OIDC authorization redirect URI for an organization SSO config.
Raises if the config is incomplete or OIDC discovery fails." Raises if the config is incomplete or OIDC discovery fails.
[cfg sso & {:keys [dest-url organization-id provider]}]
`query-string` is the route of the client (`screen=…&params`, the query
string proper, without its separator): the destination is built here, so it
can never point outside the public URI.
`profile-id` is the profile that started the flow; it travels in the
state token so the callback only grants the organization SSO entry to that
very session."
[cfg sso & {:keys [query-string organization-id provider profile-id]}]
(let [organization-id (or organization-id (:organization-id sso)) (let [organization-id (or organization-id (:organization-id sso))
issuer (organization-sso-discovery-uri sso) issuer (organization-sso-discovery-uri sso)
dest-url (or dest-url (str (cf/get :public-uri)))] dest-url (build-dest-url query-string)]
(when-not issuer (when-not issuer
(raise-invalid-sso-config (raise-invalid-sso-config
:hint "missing issuer" :hint "missing issuer"
@ -929,6 +1034,7 @@
state-token (tokens/generate cfg {:iss "oidc" state-token (tokens/generate cfg {:iss "oidc"
:dest-url dest-url :dest-url dest-url
:organization-id organization-id :organization-id organization-id
:profile-id profile-id
:issuer issuer :issuer issuer
:exp (ct/in-future "4h")})] :exp (ct/in-future "4h")})]
(build-auth-redirect-uri oidc-provider state-token)) (build-auth-redirect-uri oidc-provider state-token))
@ -1024,16 +1130,30 @@
(defn- organization-sso-callback-handler (defn- organization-sso-callback-handler
"Handle the organization-SSO branch of the OIDC callback: state carries "Handle the organization-SSO branch of the OIDC callback: state carries
:dest-url — exchange the authorization code with the OIDC provider to :dest-url — exchange the authorization code with the OIDC provider to
verify authentication actually occurred, then redirect back to dest-url." verify authentication actually occurred, then redirect back to dest-url.
The state also carries the :profile-id that started the flow: only that
session is granted the organization SSO entry, so a state prepared by
someone else cannot mark the session of the one who completes it."
[cfg request state code] [cfg request state code]
(let [dest-url (:dest-url state)] (let [dest-url (trusted-dest-url (:dest-url state))]
(try (try
(let [organization-id (:organization-id state) (let [organization-id (:organization-id state)
sso (nitrate/call cfg :get-organization-sso {:organization-id organization-id}) sso (nitrate/call cfg :get-organization-sso {:organization-id organization-id})
provider (prepare-organization-sso-provider cfg sso) provider (prepare-organization-sso-provider cfg sso)
_info (get-info cfg provider state code) _info (get-info cfg provider state code)
session (session/get-session request) session (session/get-session request)
exp (ct/in-future {:hours 4})] exp (ct/in-future {:hours 4})
;; A state minted before the profile claim existed never
;; matches, and neither does the session of another profile.
same-profile? (and (some? session)
(some? (:profile-id state))
(= (:profile-id state) (:profile-id session)))]
(if (and (some? session) (not same-profile?))
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-failed"
:failure-reason "session-mismatch")
(do
(when (and session organization-id) (when (and session organization-id)
(let [props (-> (or (:props session) {}) (let [props (-> (or (:props session) {})
(update :sso assoc organization-id exp))] (update :sso assoc organization-id exp))]
@ -1041,8 +1161,8 @@
(submit-organization-sso-auth-event (submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-token-issued"))) cfg request (:profile-id session) organization-id "organization-sso-token-issued")))
(submit-organization-sso-auth-event (submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded") cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded")))
(redirect-response dest-url)) (redirect-response (or dest-url (public-uri-base))))
(catch Throwable cause (catch Throwable cause
(let [{:keys [code]} (ex-data cause)] (let [{:keys [code]} (ex-data cause)]
(binding [l/*context* (errors/request->context request)] (binding [l/*context* (errors/request->context request)]
@ -1060,10 +1180,36 @@
(let [organization-id (:organization-id state) (let [organization-id (:organization-id state)
organization-name (:name (nitrate/call cfg :get-organization-summary {:organization-id organization-id}))] organization-name (:name (nitrate/call cfg :get-organization-summary {:organization-id organization-id}))]
(redirect-with-organization-sso-error (redirect-with-organization-sso-error
{:dest-url dest-url {:dest-url (or dest-url (public-uri-base))
:organization-id organization-id :organization-id organization-id
:organization-name organization-name})))))) :organization-name organization-name}))))))
(def oidc-provider-types
"Backends authenticating through an OIDC-compatible provider.
Only these backends take the SSO registration gate."
#{"oidc" "google" "github" "gitlab"})
(defn sso-registration-allowed?
"SSO registration is allowed when public registration or explicit
OIDC registration is enabled. Shared by the callback gate and the
`register-profile` token gate — keep one definition."
[]
(or (contains? cf/flags :registration)
(contains? cf/flags :oidc-registration)))
(defn- valid-invitation-for?
"Returns true when `invitation-token` is a verifiable
`:team-invitation` JWT issued for `email`. Never throws: a garbage
token means there is no valid invitation."
[cfg invitation-token email]
(try
(boolean
(when (and (some? invitation-token) (some? email))
(let [invitation (tokens/verify cfg {:token invitation-token :iss :team-invitation})]
(= (profile/clean-email (:member-email invitation))
(profile/clean-email email)))))
(catch Throwable _ false)))
(defn- callback-handler (defn- callback-handler
[cfg {:keys [params] :as request}] [cfg {:keys [params] :as request}]
(if-let [error (get params :error)] (if-let [error (get params :error)]
@ -1096,8 +1242,8 @@
(redirect-with-error "email-domain-not-allowed") (redirect-with-error "email-domain-not-allowed")
:else :else
(if (or (contains? cf/flags :registration) (if (or (sso-registration-allowed?)
(contains? cf/flags :oidc-registration)) (valid-invitation-for? cfg (:invitation-token info) (:email info)))
(redirect-to-register cfg info provider) (redirect-to-register cfg info provider)
(redirect-with-error "registration-disabled"))) (redirect-with-error "registration-disabled")))

View File

@ -24,7 +24,6 @@
[app.common.weak :as weak] [app.common.weak :as weak]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.db.sql :as sql]
[app.features.fdata :as fdata] [app.features.fdata :as fdata]
[app.features.file-migrations :as fmigr] [app.features.file-migrations :as fmigr]
[app.loggers.audit :as-alias audit] [app.loggers.audit :as-alias audit]
@ -43,7 +42,6 @@
(def ^:dynamic *state* nil) (def ^:dynamic *state* nil)
(def ^:dynamic *options* nil) (def ^:dynamic *options* nil)
(def ^:dynamic *reference-file* nil)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; DEFAULTS ;; DEFAULTS
@ -130,18 +128,12 @@
(defn initial-state (defn initial-state
[] []
{:storage-objects #{} {:storage-objects #{}
:files #{} :files #{}})
:teams #{}
:projects #{}})
(defn collect-storage-objects (defn collect-storage-objects
[state items] [state items]
(update state :storage-objects into xf-map-media-id items)) (update state :storage-objects into xf-map-media-id items))
(defn collect-summary
[state key items]
(update state key into xf-map-media-id items))
(defn lookup-index (defn lookup-index
[id] [id]
(when id (when id
@ -159,7 +151,7 @@
[index obj & attrs] [index obj & attrs]
(reduce (fn [index attr-fn] (reduce (fn [index attr-fn]
(let [old-id (attr-fn obj) (let [old-id (attr-fn obj)
new-id (if (::overwrite *options*) old-id (uuid/next))] new-id (uuid/next)]
(assoc index old-id new-id))) (assoc index old-id new-id)))
index index
attrs)) attrs))
@ -209,6 +201,7 @@
p.team_id, p.team_id,
coalesce(fd.backend, 'legacy-db') AS backend, coalesce(fd.backend, 'legacy-db') AS backend,
fd.metadata AS metadata, fd.metadata AS metadata,
fd.storage_object_id AS storage_object_id,
fd.data AS data fd.data AS data
FROM file AS f FROM file AS f
LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id) LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id)
@ -218,7 +211,6 @@
(str sql:files-with-data " WHERE f.id = ?")) (str sql:files-with-data " WHERE f.id = ?"))
(def sql:get-file-without-data (def sql:get-file-without-data
(str "WITH files AS (" sql:files-with-data ")"
"SELECT f.id, "SELECT f.id,
f.project_id, f.project_id,
f.created_at, f.created_at,
@ -233,12 +225,23 @@
f.features, f.features,
f.version, f.version,
f.vern, f.vern,
f.team_id p.team_id
FROM files AS f FROM file AS f
WHERE f.id = ?")) INNER JOIN project AS p ON (p.id = f.project_id)
WHERE f.id = ?")
(defn- migrate-file (defn- migrate-file
[{:keys [::db/conn] :as cfg} {:keys [read-only?]} {:keys [id] :as file}] "Apply the pending migrations to the file.
With `persist-after-migration?` the migrated map is also persisted, which is
only safe from a map read under the `file` row lock (see `get-file*`). The
returned map is then the reattached one: decoded data where the pages the
migration did not change keep their pointers and their fragment ids, plus
the `:storage-object-id` of the row the write left, so a caller that
persists again touches the current object. Without `persist-after-migration?`
the realized map is returned, which is what a read-only context can do.
`get-file*` realizes the result when the caller asks for it."
[cfg {:keys [id] :as file} {:keys [persist-after-migration?]}]
(fdata/with-tracked-pointers cfg id (fdata/with-tracked-pointers cfg id
(let [libs (delay (get-resolved-file-libraries cfg file)) (let [libs (delay (get-resolved-file-libraries cfg file))
;; Capture the pointer layout before realizing: the migration ;; Capture the pointer layout before realizing: the migration
@ -252,51 +255,38 @@
objects-map-added? (and (not objects-map-before?) objects-map-added? (and (not objects-map-before?)
(contains? (:features realized) "fdata/objects-map"))] (contains? (:features realized) "fdata/objects-map"))]
(if (or read-only? (db/read-only? conn)) (if persist-after-migration?
realized
(let [file (if objects-map-added? (let [file (if objects-map-added?
realized realized
(fdata/reattach-unchanged original realized))] (fdata/reattach-unchanged original realized))
written (update-file! cfg file)]
;; When the file is migrated, we break the rule of not ;; When the file is migrated, we break the rule of not
;; performing mutations on get operations and update the file ;; performing mutations on get operations and update the file
;; with all migrations applied. The caller still receives the ;; with all migrations applied. The reattached map is also what
;; realized file: the read contract is plain data. ;; the read returns: the caller realizes it only when it asks.
(update-file! cfg file) (assoc (fmigr/resolve-applied-migrations cfg file)
(fmigr/resolve-applied-migrations cfg realized)))))) :storage-object-id (:storage-object-id written)))
realized))))
(defn- get-file* (defn- fetch-row
"Read the file row. With `load-data?` the file data columns come along, but
they are not resolved nor decoded (see `attach-file-data`). With
`load-migrations?` the applied migrations are resolved, which is what
`need-migration?` expects. A skip-locked read answers nil instead of raising
when the row is gone."
[{:keys [::db/conn] :as cfg} id [{:keys [::db/conn] :as cfg} id
{:keys [migrate? {:keys [include-deleted?
realize?
decode?
skip-locked? skip-locked?
include-deleted?
load-data?
throw-if-not-exists?
lock-for-update? lock-for-update?
lock-for-share?] lock-for-share?
:or {lock-for-update? false throw-if-not-exists?
lock-for-share? false load-migrations?
load-data? true load-data?]}]
migrate? true
decode? true
include-deleted? false
throw-if-not-exists? true
realize? false}
:as options}]
(assert (db/connection? conn) "expected cfg with valid connection") (let [sql (if load-data?
(when (and (not load-data?)
(or lock-for-share? lock-for-update? skip-locked?))
(throw (IllegalArgumentException. "locking is incompatible when `load-data?` is false")))
(let [sql
(if load-data?
sql:get-file sql:get-file
sql:get-file-without-data) sql:get-file-without-data)
sql (cond
sql
(cond
lock-for-update? lock-for-update?
(str sql " FOR UPDATE of f") (str sql " FOR UPDATE of f")
@ -306,42 +296,20 @@
:else :else
sql) sql)
sql sql (if (and skip-locked? (or lock-for-share? lock-for-update?))
(if skip-locked?
(str sql " SKIP LOCKED") (str sql " SKIP LOCKED")
sql) sql)
file file (db/get-with-sql conn [sql id] {::db/throw-if-not-exists false
(db/get-with-sql conn [sql id]
{::db/throw-if-not-exists false
::db/remove-deleted (not include-deleted?)}) ::db/remove-deleted (not include-deleted?)})
file (some-> file
file
(-> file
(d/update-when :features db/decode-pgarray #{}) (d/update-when :features db/decode-pgarray #{})
(d/update-when :metadata fdata/decode-metadata))] (d/update-when :metadata fdata/decode-metadata))
file (if (and file load-migrations?)
(if file (fmigr/resolve-applied-migrations cfg file)
(if load-data? file)]
(let [file
(->> file
(fmigr/resolve-applied-migrations cfg)
(fdata/resolve-file-data cfg))
will-migrate?
(and migrate? (fmg/need-migration? file))]
(if decode?
(cond->> (fdata/decode-file-data cfg file)
(and realize? (not will-migrate?))
(fdata/realize cfg)
will-migrate?
(migrate-file cfg options))
file))
file)
(or file
(when-not (or skip-locked? (not throw-if-not-exists?)) (when-not (or skip-locked? (not throw-if-not-exists?))
(ex/raise :type :not-found (ex/raise :type :not-found
:code :object-not-found :code :object-not-found
@ -349,14 +317,124 @@
:table :file :table :file
:file-id id))))) :file-id id)))))
(defn- attach-file-data
"Resolve the data of an already read row and decode it when `decode?`. It
completes the map it receives instead of reading the row again. It does not
realize: `get-file*` owns that step, once per path."
[cfg file {:keys [decode?]}]
(let [file (fdata/resolve-file-data cfg file)]
(if decode?
(fdata/decode-file-data cfg file)
file)))
(defn- fetch-file
"Read the row with its data resolved and decoded, under the requested lock."
[cfg id options]
(when-let [file (fetch-row cfg id options)]
(attach-file-data cfg file options)))
(def ^:private
default-get-file-options
{:load-migrations? true
:throw-if-not-exists? true
:persist-after-migration? true
:load-data? true
:migrate? true
:decode? true})
(defn- get-file*
[{:keys [::db/conn] :as cfg} id options]
(assert (db/connection? conn) "expected cfg with valid connection")
(let [{:keys [migrate?
realize?
decode?
load-data?
lock-for-share?
read-only?]
:as options}
(merge default-get-file-options options)
read-only?
(or read-only? (db/read-only? conn))]
(when (and realize? (or (not load-data?) (not decode?)))
(throw (IllegalArgumentException. "invalid options, you only can realize when loading data and decoding")))
(when (and lock-for-share? load-data?)
(throw (IllegalArgumentException. "invalid options, `:lock-for-share?` is only allowed on row-only reads")))
(cond
;; The caller only wants the row: that read is the answer.
(not load-data?)
(fetch-row cfg id options)
(and migrate? decode? (not read-only?))
(when-let [file (fetch-row cfg id options)]
(if (fmg/need-migration? file)
(when-let [file (fetch-file cfg id (assoc options :lock-for-update? true))]
(cond-> file
(fmg/need-migration? file)
(as-> $ (migrate-file cfg $ options))
realize?
(->> (fdata/realize cfg))))
(let [file (attach-file-data cfg file options)]
(cond-> file
realize?
(->> (fdata/realize cfg))))))
:else
(when-let [file (fetch-file cfg id options)]
(cond-> file
(and migrate? decode? (fmg/need-migration? file))
(as-> $ (migrate-file cfg $ (assoc options :persist-after-migration? false)))
realize?
(->> (fdata/realize cfg)))))))
(defn get-file (defn get-file
"Get file, resolve all features and apply migrations. "Get file, resolve all features and apply migrations.
Runs inside a transaction: it joins the caller's when the cfg already
carries a connection and opens one otherwise, so the migration write
commits with the read. A read-only read (`:read-only?`) takes no lock and
writes nothing, so it runs on a plain connection instead of opening a
transaction of its own; with a cfg that already carries a connection it
still joins the caller's. Accepts a cfg map only: a bare pool or a
connection raises `IllegalArgumentException`.
Useful when you have plan to apply massive or not surgical Useful when you have plan to apply massive or not surgical
operations on file, because it removes the overhead of lazy fetching operations on file, because it removes the overhead of lazy fetching
and decoding." and decoding.
Options:
- `:realize?` returns plain data instead of fragment pointers; it
requires `:load-data?` and `:decode?`.
- `:decode?` decodes the stored blob (default true).
- `:load-data?` reads the data columns (default true). A row-only read
(`false`) is the only mode that accepts `:lock-for-share?`.
- `:migrate?` applies the pending migrations on read (default true).
- `:load-migrations?` resolves the applied migrations on a row read
(default true).
- `:read-only?` applies the pending migrations in memory only and skips
the transaction. It does not take a lock.
- `:lock-for-update?`, `:skip-locked?`, `:include-deleted?` and
`:throw-if-not-exists?` are the usual read knobs. A lock needs a
read-write transaction from the caller: PostgreSQL rejects row locks in
a read-only transaction.
- `:persist-after-migration?` is internal: `get-file*` turns it off
for the in-memory migration of a read-only context. Do not pass it."
[cfg file-id & {:as opts}] [cfg file-id & {:as opts}]
(db/run! cfg get-file* file-id opts)) (when-not (map? cfg)
(throw (IllegalArgumentException.
"expected cfg map, not a pool or a connection")))
(if (:read-only? opts)
(db/run! cfg get-file* file-id opts)
(db/tx-run! cfg get-file* file-id opts)))
(defn clean-file-features (defn clean-file-features
[file] [file]
@ -384,19 +462,27 @@
inner join project as p on (p.id = f.project_id) inner join project as p on (p.id = f.project_id)
where fpr.file_id = ? where fpr.file_id = ?
and fpr.profile_id = ? and fpr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1 and exists (select 1
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id where tpr.team_id = p.team_id
and tpr.profile_id = fpr.profile_id) and tpr.profile_id = fpr.profile_id
and t.deleted_at is null)
union all union all
select tpr.is_owner, select tpr.is_owner,
tpr.is_admin, tpr.is_admin,
tpr.can_edit tpr.can_edit
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
inner join project as p on (p.team_id = tpr.team_id) inner join project as p on (p.team_id = tpr.team_id)
inner join file as f on (p.id = f.project_id) inner join file as f on (p.id = f.project_id)
where f.id = ? where f.id = ?
and tpr.profile_id = ? and tpr.profile_id = ?
and t.deleted_at is null
and p.deleted_at is null
and f.deleted_at is null
union all union all
select ppr.is_owner, select ppr.is_owner,
ppr.is_admin, ppr.is_admin,
@ -406,10 +492,14 @@
inner join project as p on (p.id = ppr.project_id) inner join project as p on (p.id = ppr.project_id)
where f.id = ? where f.id = ?
and ppr.profile_id = ? and ppr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1 and exists (select 1
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id where tpr.team_id = p.team_id
and tpr.profile_id = ppr.profile_id)") and tpr.profile_id = ppr.profile_id
and t.deleted_at is null)")
(defn- get-file-permissions* (defn- get-file-permissions*
[conn profile-id file-id] [conn profile-id file-id]
@ -457,27 +547,6 @@
[cfg project-id] [cfg project-id]
(db/get cfg :project {:id project-id})) (db/get cfg :project {:id project-id}))
(def ^:private sql:get-teams
"SELECT t.* FROM team WHERE id = ANY(?)")
(defn get-teams
[cfg ids]
(let [conn (db/get-connection cfg)
ids (db/create-array conn "uuid" ids)]
(->> (db/exec! conn [sql:get-teams ids])
(map decode-row-features))))
(defn get-team
[cfg team-id]
(-> (db/get cfg :team {:id team-id})
(decode-row-features)))
(defn get-fonts
[cfg team-id]
(db/query cfg :team-font-variant
{:team-id team-id
:deleted-at nil}))
(def ^:private sql:get-files-rels (def ^:private sql:get-files-rels
"SELECT flr.*, "SELECT flr.*,
fls.synced_at fls.synced_at
@ -488,10 +557,13 @@
ON fls.file_id = flr.file_id ON fls.file_id = flr.file_id
AND fls.library_file_id = flr.library_file_id AND fls.library_file_id = flr.library_file_id
WHERE flr.file_id = ANY(?) WHERE flr.file_id = ANY(?)
AND l.deleted_at IS NULL;") AND l.deleted_at IS NULL
ORDER BY flr.file_id, flr.library_file_id;")
(defn get-files-rels (defn get-files-rels
"Given a set of file-id's, return all matching relations with the libraries" "Given a set of file-id's, return all matching relations with the libraries.
Rows come back in stable `(file-id, library-file-id)` order so export
and import resolve cycles deterministically."
[cfg ids] [cfg ids]
(assert (set? ids) "expected a set of uuids") (assert (set? ids) "expected a set of uuids")
@ -534,6 +606,20 @@
(->> (db/exec! conn [sql:get-libraries ids']) (->> (db/exec! conn [sql:get-libraries ids'])
(into #{} xf-map-id)))))) (into #{} xf-map-id))))))
(defn library-link-status
"Check whether linking `file-id` to `library-id` is safe. Returns
`:ok` when the link can be created, `:self-link` when both ids are
the same, and `:circular` when the link would close a dependency
cycle (i.e. `file-id` is already among the transitive libraries of
`library-id`). Shared by the UI command (`link-file-to-library`,
which raises) and the v3 importer (`import-file-relations`, which
skips and logs), so both stay consistent."
[cfg file-id library-id]
(cond
(= file-id library-id) :self-link
(contains? (get-libraries cfg [library-id]) file-id) :circular
:else :ok))
(defn get-file-object-thumbnails (defn get-file-object-thumbnails
"Return all file object thumbnails for a given file." "Return all file object thumbnails for a given file."
[cfg file-id] [cfg file-id]
@ -542,15 +628,6 @@
:deleted-at nil}) :deleted-at nil})
(not-empty))) (not-empty)))
(defn get-file-thumbnail
"Return the thumbnail for the specified file-id"
[cfg {:keys [id revn]}]
(db/get* cfg :file-thumbnail
{:file-id id
:revn revn
:data nil}
{::sql/columns [:media-id :file-id :revn]}))
(def ^:private sql:get-missing-media-references (def ^:private sql:get-missing-media-references
"SELECT fmo.* "SELECT fmo.*
FROM file_media_object AS fmo FROM file_media_object AS fmo
@ -709,7 +786,12 @@
(let [library-ids (get-libraries cfg [file-id]) (let [library-ids (get-libraries cfg [file-id])
data (reduce (fn [data library-id] data (reduce (fn [data library-id]
(if-let [library (get-file cfg library-id :include-deleted? true)] (if-let [library (get-file cfg library-id :include-deleted? true)]
(cond-> (ctf/absorb-assets data (:data library)) ;; The library is read without `:realize?`, so its
;; `:components` may be a pointer map. Bind the
;; library's own loader: the absorb walks the
;; components and would raise without one.
(cond-> (fdata/with-pointers cfg library-id
(ctf/absorb-assets data (:data library)))
(= library-id (cfo/get-tokens-source data)) (= library-id (cfo/get-tokens-source data))
(cfo/absorb-tokens-lib (:data library))) (cfo/absorb-tokens-lib (:data library)))
data)) data))
@ -792,18 +874,33 @@
(dissoc :metadata) (dissoc :metadata)
(dissoc :migrations))) (dissoc :migrations)))
(defn file->response
"Strip the internal keys of a file map before it leaves the backend in a
response. The file data storage pointer is bookkeeping for the write path:
a `file-data` object is authentication-only, so a leaked id would let any
logged-in profile fetch the blob from `/assets/by-id`. The metadata key of
the old pointer is stripped too: a row the migration does not backfill keeps
it, and the metadata schema is open, so it would travel in the response."
[file]
(-> file
(dissoc :storage-object-id)
(d/update-when :metadata #(dissoc % :storage-ref-id))))
(defn- file->file-data-params (defn- file->file-data-params
[{:keys [id] :as file} & {:as opts}] [{:keys [id] :as file}]
(let [created-at (or (:created-at file) (ct/now)) (let [created-at (or (:created-at file) (ct/now))
modified-at (or (:modified-at file) created-at)] modified-at (or (:modified-at file) created-at)]
(d/without-nils (-> (d/without-nils
{:id id {:id id
:type "main" :type "main"
:file-id id :file-id id
:data (:data file) :data (:data file)
:metadata (:metadata file) :metadata (:metadata file)
:created-at created-at :created-at created-at
:modified-at modified-at}))) :modified-at modified-at})
;; The storage object the persisted row pointed at (nil for a new or
;; imported file). `upsert!` touches it before replacing the row.
(assoc :storage-object-id (:storage-object-id file)))))
(defn insert-file! (defn insert-file!
"Insert a new file into the database table. Expectes a not-encoded file. "Insert a new file into the database table. Expectes a not-encoded file.
@ -827,34 +924,63 @@
:cause cause) :cause cause)
(throw cause)))) (throw cause))))
(->> (file->file-data-params file) ;; An INSERT never replaces a persisted pointer: `duplicate-file` hands
;; the source file map here, and touching the source object would be a
;; write on a live object.
(->> (file->file-data-params (dissoc file :storage-object-id))
(fdata/upsert! cfg)) (fdata/upsert! cfg))
nil)) nil))
(defn update-file! (defn update-file!
"Update an existing file on the database. Expects not encoded file." "Update an existing file on the database. Expects a not-encoded file map,
[{:keys [::db/conn] :as cfg} {:keys [id] :as file} & {:as opts}] as returned by `get-file`, and optionally the fields to change as keyword
args; they are merged into the file map before persisting.
(if (::reset-migrations? opts false) The storage object the persisted row pointed at travels in the map
(`:storage-object-id`), and `upsert!` touches it before replacing the row.
That is why the caller must pass the map it read instead of building a
partial one: nothing is queried here. The params only change mutable fields
of the file: the pointer and the identity (`:id`, `:file-id`) always come
from the read map.
Returns the map `fdata/upsert!` answers with: `:written` and the
`:storage-object-id` of the object this write created, so a caller that
persists again touches the current object instead of the replaced one."
[{:keys [::db/conn] :as cfg} file & {:as params}]
;; A map read with `:load-data? false` has no `:data`; persisting it would
;; wipe the file content. `ex/raise` instead of `assert`: the guard must
;; exist in the deployed binary, where `clojure.core/assert` is compiled
;; out.
(when-not (contains? file :data)
(ex/raise :type :internal
:code :missing-file-data
:hint "expected a file map read with data"
:file-id (:id file)))
(let [reset? (::reset-migrations? params false)
;; The pointer and the identity are bookkeeping of the persisted row,
;; not editable fields: they can only come from the read map. A pointer
;; in the params would skip the touch and leak the replaced blob; an
;; `:id` would retarget the UPDATE.
file (merge file (dissoc params ::reset-migrations?
:storage-object-id
:id
:file-id))]
(if reset?
(fmigr/reset-migrations! conn file) (fmigr/reset-migrations! conn file)
(fmigr/upsert-migrations! conn file)) (fmigr/upsert-migrations! conn file))
(let [file (let [file (encode-file cfg file)
(encode-file cfg file) file-params (file->params (dissoc file :id))]
file-params
(file->params (dissoc file :id))
file-data-params
(file->file-data-params file)]
(db/update! conn :file file-params (db/update! conn :file file-params
{:id id} {:id (:id file)}
{::db/return-keys false}) {::db/return-keys false})
(fdata/upsert! cfg file-data-params) (fdata/upsert! cfg (file->file-data-params file)))))
nil))
(defn save-file! (defn save-file!
"Applies all the final validations and perist the file, binfile "Applies all the final validations and perist the file, binfile
@ -867,8 +993,7 @@
(let [file (-> file (let [file (-> file
(assoc :created-at timestamp) (assoc :created-at timestamp)
(assoc :modified-at timestamp) (assoc :modified-at timestamp)
(cond-> (not (::overwrite cfg)) (assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5})))
(assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5}))))
(update :features (update :features
(fn [features] (fn [features]
(-> (::features cfg #{}) (-> (::features cfg #{})
@ -885,9 +1010,7 @@
(when (ex/exception? result) (when (ex/exception? result)
(l/error :hint "file schema validation error" :cause result)))) (l/error :hint "file schema validation error" :cause result))))
(if (::overwrite cfg) (insert-file! cfg file opts)))
(update-file! cfg file (assoc opts ::reset-migrations? true))
(insert-file! cfg file opts))))
(def ^:private sql:get-file-libraries (def ^:private sql:get-file-libraries
"WITH RECURSIVE libs AS ( "WITH RECURSIVE libs AS (

View File

@ -482,9 +482,11 @@
(bfm/apply-pending-migrations! cfg) (bfm/apply-pending-migrations! cfg)
;; Knowing that the ids of the created files are in index, ;; Knowing that the ids of the created files are in index,
;; just lookup them and return them in the same shape as v3. ;; just lookup them and return them in the same shape as v3. The
;; metadata of an old package may repeat a file id, so the answer
;; carries one id per imported file.
(let [files (-> bfc/*state* deref :files)] (let [files (-> bfc/*state* deref :files)]
{:file-ids (into [] (keep #(get-in @bfc/*state* [:index %])) files)}))))) {:file-ids (into [] (comp (keep #(get-in @bfc/*state* [:index %])) (distinct)) files)})))))
(defmethod read-import :v1 (defmethod read-import :v1
[options] [options]
@ -510,15 +512,11 @@
(defmethod read-section :v1/files (defmethod read-section :v1/files
[{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}] [{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}]
(let [imported (volatile! #{})]
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))] (doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
(let [file (read-obj! input) (let [file (read-obj! input)
media (read-obj! input) media (read-obj! input)
file-id (:id file)]
file-id (:id file)
file-id' (bfc/lookup-index file-id)
file (bfc/clean-file-features file)
thumbnails (:thumbnails file)]
(when (not= file-id expected-file-id) (when (not= file-id expected-file-id)
(ex/raise :type :validation (ex/raise :type :validation
@ -527,6 +525,20 @@
:expected-id expected-file-id :expected-id expected-file-id
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)")) :hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
(if (contains? @imported file-id)
;; Old exports could repeat a file id in the metadata (the file was
;; also one of its own libraries). The entry is read above to keep
;; the stream in sync, but the file is imported only once.
(l/warn :hint "ignoring repeated file entry"
:file-id (str file-id)
::l/sync? true)
(let [file-id' (bfc/lookup-index file-id)
file (bfc/clean-file-features file)
thumbnails (:thumbnails file)]
(vswap! imported conj file-id)
(l/dbg :hint "processing file" (l/dbg :hint "processing file"
:id (str file-id) :id (str file-id)
:features (:features file) :features (:features file)
@ -559,8 +571,8 @@
(dissoc :thumbnails)) (dissoc :thumbnails))
file (bfc/process-file system file)] file (bfc/process-file system file)]
;; All features that are enabled and requires explicit migration are ;; All features that are enabled and requires explicit migration
;; added to the state for a posterior migration step. ;; are added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features system) (doseq [feature (-> (::bfc/features system)
(set/difference cfeat/no-migration-features) (set/difference cfeat/no-migration-features)
(set/difference (:features file)))] (set/difference (:features file)))]
@ -569,19 +581,34 @@
(l/dbg :hint "create file" :id (str file-id') ::l/sync? true) (l/dbg :hint "create file" :id (str file-id') ::l/sync? true)
(bfc/save-file! system file ::db/return-keys false) (bfc/save-file! system file ::db/return-keys false)
file-id')))) file-id')))))))
(defmethod read-section :v1/rels (defmethod read-section :v1/rels
[{:keys [::db/conn ::bfc/input ::bfc/timestamp]}] [{:keys [::db/conn ::bfc/input ::bfc/timestamp] :as cfg}]
(let [rels (read-obj! input) (let [rels (->> (read-obj! input)
(sort-by (fn [{:keys [file-id library-file-id]}]
[(str file-id) (str library-file-id)])))
ids (into #{} (-> bfc/*state* deref :files))] ids (into #{} (-> bfc/*state* deref :files))]
;; Insert all file relations
(doseq [{:keys [library-file-id] :as rel} rels] (doseq [{:keys [library-file-id] :as rel} rels]
(let [rel (-> rel (let [rel (-> rel
(update :file-id bfc/lookup-index) (update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))] (update :library-file-id bfc/lookup-index))]
(if (contains? ids library-file-id) (if (contains? ids library-file-id)
(case (bfc/library-link-status cfg (:file-id rel) (:library-file-id rel))
:self-link
(l/warn :hint "ignoring self-referencing library link"
:file-id (:file-id rel)
:lib-id (:library-file-id rel)
::l/sync? true)
:circular
(l/warn :hint "ignoring circular library link"
:file-id (:file-id rel)
:lib-id (:library-file-id rel)
::l/sync? true)
:ok
(do (do
(l/dbg :hint "create file library link" (l/dbg :hint "create file library link"
:file-id (:file-id rel) :file-id (:file-id rel)
@ -592,7 +619,7 @@
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params) (bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params)
:library-file-id (:library-file-id rel-params) :library-file-id (:library-file-id rel-params)
:synced-at (or (:synced-at rel) :synced-at (or (:synced-at rel)
timestamp)}))) timestamp)}))))
(l/warn :hint "ignoring file library link" (l/warn :hint "ignoring file library link"
:file-id (:file-id rel) :file-id (:file-id rel)

View File

@ -1,445 +0,0 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns app.binfile.v2
"A sqlite3 based binary file exportation with support for exportation
of entire team (or multiple teams) at once."
(:refer-clojure :exclude [read])
(:require
[app.binfile.common :as bfc]
[app.common.data :as d]
[app.common.features :as cfeat]
[app.common.logging :as l]
[app.common.time :as ct]
[app.common.transit :as t]
[app.common.uuid :as uuid]
[app.config :as cf]
[app.db :as db]
[app.db.sql :as sql]
[app.loggers.audit :as-alias audit]
[app.loggers.webhooks :as-alias webhooks]
[app.storage :as sto]
[app.storage.tmp :as tmp]
[app.util.events :as events]
[app.worker :as-alias wrk]
[clojure.set :as set]
[cuerdas.core :as str]
[datoteka.io :as io]
[promesa.util :as pu])
(:import
java.sql.DriverManager))
(set! *warn-on-reflection* true)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; LOW LEVEL API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- create-database
([cfg]
(let [path (tmp/tempfile :prefix "penpot.binfile." :suffix ".sqlite")]
(create-database cfg path)))
([cfg path]
(let [db (DriverManager/getConnection (str "jdbc:sqlite:" path))]
(assoc cfg ::db db ::path path))))
(def ^:private
sql:create-kvdata-table
"CREATE TABLE kvdata (
tag text NOT NULL,
key text NOT NULL,
val text NOT NULL,
dat blob NULL
)")
(def ^:private
sql:create-kvdata-index
"CREATE INDEX kvdata__tag_key__idx
ON kvdata (tag, key)")
(defn- setup-schema!
[{:keys [::db]}]
(db/exec-one! db [sql:create-kvdata-table])
(db/exec-one! db [sql:create-kvdata-index]))
(defn- write!
[{:keys [::db]} tag k v & [data]]
(db/insert! db :kvdata
{:tag (d/name tag)
:key (str k)
:val (t/encode-str v {:type :json-verbose})
:dat data}
{::db/return-keys false}))
(defn- read-blob
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:dat]})]
(:dat obj)))
(defn- read-seq
([{:keys [::db]} tag]
(->> (db/query db :kvdata
{:tag (d/name tag)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str)))
([{:keys [::db]} tag k]
(->> (db/query db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str))))
(defn- read-obj
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:val]})]
(-> obj :val t/decode-str)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; IMPORT/EXPORT IMPL
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(declare ^:private write-project!)
(declare ^:private write-file!)
(defn- write-team!
[cfg team-id]
(let [team (bfc/get-team cfg team-id)
fonts (bfc/get-fonts cfg team-id)]
(events/tap :progress
{:op :export
:section :write-team
:id team-id
:name (:name team)})
(l/trc :hint "write" :obj "team"
:id (str team-id)
:fonts (count fonts))
(when-let [photo-id (:photo-id team)]
(vswap! bfc/*state* update :storage-objects conj photo-id))
(vswap! bfc/*state* update :teams conj team-id)
(vswap! bfc/*state* bfc/collect-storage-objects fonts)
(write! cfg :team team-id team)
(doseq [{:keys [id] :as font} fonts]
(vswap! bfc/*state* update :team-font-variants conj id)
(write! cfg :team-font-variant id font))))
(defn- write-project!
[cfg project]
(events/tap :progress
{:op :export
:section :write-project
:id (:id project)
:name (:name project)})
(l/trc :hint "write" :obj "project" :id (str (:id project)))
(write! cfg :project (str (:id project)) project)
(vswap! bfc/*state* update :projects conj (:id project)))
(defn- write-file!
[cfg file-id]
(let [file (bfc/get-file cfg file-id :realize? true)
thumbs (bfc/get-file-object-thumbnails cfg file-id)
media (bfc/get-file-media cfg file)
rels (bfc/get-files-rels cfg #{file-id})]
(events/tap :progress
{:op :export
:section :write-file
:id file-id
:name (:name file)})
(vswap! bfc/*state* (fn [state]
(-> state
(update :files conj file-id)
(update :file-media-objects into bfc/xf-map-id media)
(bfc/collect-storage-objects thumbs)
(bfc/collect-storage-objects media))))
(write! cfg :file file-id file)
(write! cfg :file-rels file-id rels)
(run! (partial write! cfg :file-media-object file-id) media)
(run! (partial write! cfg :file-object-thumbnail file-id) thumbs)
(when-let [thumb (bfc/get-file-thumbnail cfg file)]
(vswap! bfc/*state* bfc/collect-storage-objects [thumb])
(write! cfg :file-thumbnail file-id thumb))
(l/trc :hint "write" :obj "file"
:thumbnails (count thumbs)
:rels (count rels)
:media (count media))))
(defn- write-storage-object!
[{:keys [::sto/storage] :as cfg} id]
(let [sobj (sto/get-object storage id)
data (with-open [input (sto/get-object-data storage sobj)]
(io/read input))]
(l/trc :hint "write" :obj "storage-object" :id (str id) :size (:size sobj))
(write! cfg :storage-object id (meta sobj) data)))
(defn- read-storage-object!
[{:keys [::sto/storage ::bfc/timestamp] :as cfg} id]
(let [mdata (d/update-when (read-obj cfg :storage-object id)
:content-type bfc/normalize-content-type)
data (read-blob cfg :storage-object id)
;; NOTE: v2 bundles are legacy and no longer produced; the
;; sanitizer call stays as defense in depth and is covered by
;; the shared helper unit tests, with no dedicated v2
;; import test.
clean (when (bfc/svg-object? mdata)
(bfc/sanitize-imported-svg mdata data))
data (or (:bytes clean) data)
hash (or (:hash clean) (sto/calculate-hash data))
content (-> (sto/content data)
(sto/wrap-with-hash hash))
params (-> mdata
(assoc ::sto/content content)
(assoc ::sto/deduplicate? true)
(assoc ::sto/touched-at timestamp))
sobject (sto/put-object! storage params)]
(vswap! bfc/*state* update :index assoc id (:id sobject))
(l/trc :hint "read" :obj "storage-object"
:id (str id)
:new-id (str (:id sobject))
:size (:size sobject))))
(defn read-team!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} team-id]
(l/trc :hint "read" :obj "team" :id (str team-id))
(let [team (read-obj cfg :team team-id)
team (-> team
(update :id bfc/lookup-index)
(update :photo-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-team
:id team-id
:name (:name team)})
(db/insert! conn :team
(update team :features db/encode-pgarray conn "text")
::db/return-keys false)
(doseq [font (->> (read-seq cfg :team-font-variant)
(filter #(= team-id (:team-id %))))]
(let [font (-> font
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(update :woff1-file-id bfc/lookup-index)
(update :woff2-file-id bfc/lookup-index)
(update :ttf-file-id bfc/lookup-index)
(update :otf-file-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(db/insert! conn :team-font-variant font
::db/return-keys false)))
team))
(defn read-project!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} project-id]
(l/trc :hint "read" :obj "project" :id (str project-id))
(let [project (read-obj cfg :project project-id)
project (-> project
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-project
:id project-id
:name (:name project)})
(db/insert! conn :project project
::db/return-keys false)))
(defn read-file!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} file-id]
(l/trc :hint "read" :obj "file" :id (str file-id))
(let [file (-> (read-obj cfg :file file-id)
(update :id bfc/lookup-index)
(update :project-id bfc/lookup-index))
file (bfc/process-file cfg file)]
(events/tap :progress
{:op :import
:section :read-file
:id file-id
:name (:name file)})
;; All features that are enabled and requires explicit migration are
;; added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features cfg)
(set/difference cfeat/no-migration-features)
(set/difference (:features file)))]
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature (:id file)]))
(bfc/save-file! cfg file ::db/return-keys false))
(doseq [thumbnail (read-seq cfg :file-object-thumbnail file-id)]
(let [thumbnail (-> thumbnail
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index))
file-id (:file-id thumbnail)
thumbnail (update thumbnail :object-id
#(str/replace-first % #"^(.*?)/" (str file-id "/")))]
(db/insert! conn :file-tagged-object-thumbnail thumbnail
{::db/return-keys false})))
(doseq [rel (read-obj cfg :file-rels file-id)]
(let [rel (-> rel
(update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))]
(db/insert! conn :file-library-rel rel
::db/return-keys false)
(bfc/upsert-file-library-sync! conn (assoc rel :synced-at timestamp))))
(doseq [media (read-seq cfg :file-media-object file-id)]
(let [media (-> media
(update :id bfc/lookup-index)
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index)
(update :thumbnail-id bfc/lookup-index))]
(db/insert! conn :file-media-object media
::db/return-keys false
::sql/on-conflict-do-nothing true))))
(def ^:private empty-summary
{:teams #{}
:files #{}
:projects #{}
:file-media-objects #{}
:team-font-variants #{}
:storage-objects #{}})
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PUBLIC API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn export-team!
[cfg team-id]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (create-database cfg)]
(l/inf :hint "start"
:operation "export"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [cfg]
(setup-schema! cfg)
(binding [bfc/*state* (volatile! empty-summary)]
(write-team! cfg team-id)
(run! (partial write-project! cfg)
(bfc/get-team-projects cfg team-id))
(run! (partial write-file! cfg)
(bfc/get-team-files-ids cfg team-id))
(run! (partial write-storage-object! cfg)
(-> bfc/*state* deref :storage-objects))
(write! cfg :manifest "team-id" team-id)
(write! cfg :manifest "objects" (deref bfc/*state*))
(::path cfg))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "export"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))
(defn import-team!
[cfg path]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (-> (create-database cfg path)
(assoc ::bfc/timestamp (ct/now)))]
(l/inf :hint "start"
:operation "import"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
(db/exec-one! conn ["SET idle_in_transaction_session_timeout = 0"])
(db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])
(binding [bfc/*state* (volatile! {:index {}})]
(let [objects (read-obj cfg :manifest "objects")]
;; We first process all storage objects, they have
;; deduplication so we can't rely on simple reindex. This
;; operation populates the index for all storage objects.
(run! (partial read-storage-object! cfg) (:storage-objects objects))
;; Populate index with all the incoming objects
(vswap! bfc/*state* update :index
(fn [index]
(-> index
(bfc/update-index (:teams objects))
(bfc/update-index (:projects objects))
(bfc/update-index (:files objects))
(bfc/update-index (:file-media-objects objects))
(bfc/update-index (:team-font-variants objects)))))
(let [team-id (read-obj cfg :manifest "team-id")
team (read-team! cfg team-id)
features (cfeat/get-team-enabled-features cf/flags team)
cfg (assoc cfg ::bfc/features features)]
(run! (partial read-project! cfg) (:projects objects))
(run! (partial read-file! cfg) (:files objects))
;; (run-pending-migrations! cfg)
team)))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "import"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))

View File

@ -988,8 +988,7 @@
:old-id (str (:id item)) :old-id (str (:id item))
::l/sync? true) ::l/sync? true)
(db/insert! conn :file-media-object params (db/insert! conn :file-media-object params)))
::db/on-conflict-do-nothing? (::bfc/overwrite cfg))))
(doseq [[index item] (d/enumerate thumbnails)] (doseq [[index item] (d/enumerate thumbnails)]
(let [media-id (bfc/lookup-index (:media-id item)) (let [media-id (bfc/lookup-index (:media-id item))
@ -1038,18 +1037,41 @@
file-id'))) file-id')))
(defn- import-file-relations (defn- import-file-relations
"Import library relations from the manifest, ignoring relations whose
endpoints are not both part of the package. Of the rest, skip edges that
would close a dependency cycle or link a file to itself (same check as
`link-file-to-library`, see `bfc/library-link-status`). Relations are
processed in sorted manifest order and the first edge wins: an edge
that would close a cycle with previously imported relations is skipped
and logged."
[{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}] [{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}]
(let [relations (:relations manifest) (let [relations (sort-by (fn [[file-id libr-id]] [(str file-id) (str libr-id)])
(:relations manifest))
file-ids (into #{} (map :id) (:files manifest))] file-ids (into #{} (map :id) (:files manifest))]
(doseq [[index [file-id libr-id]] (d/enumerate relations)] (doseq [[index [file-id libr-id]] (d/enumerate relations)]
(tap-progress cfg {:section :relations (tap-progress cfg {:section :relations
:current (inc index) :total (count relations)}) :current (inc index)
:total (count relations)})
(if (and (not= file-id libr-id) (if (and (contains? file-ids file-id)
(contains? file-ids file-id)
(contains? file-ids libr-id)) (contains? file-ids libr-id))
(let [file-id (bfc/lookup-index file-id) (let [file-id (bfc/lookup-index file-id)
libr-id (bfc/lookup-index libr-id)] libr-id (bfc/lookup-index libr-id)]
(case (bfc/library-link-status cfg file-id libr-id)
:self-link
(l/warn :hint "ignoring self-referencing library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
:circular
(l/warn :hint "ignoring circular library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
:ok
(do
(l/dbg :hint "create file library link" (l/dbg :hint "create file library link"
:file-id (str file-id) :file-id (str file-id)
:lib-id (str libr-id) :lib-id (str libr-id)
@ -1057,7 +1079,7 @@
(let [rel-params {:file-id file-id (let [rel-params {:file-id file-id
:library-file-id libr-id}] :library-file-id libr-id}]
(db/insert! conn :file-library-rel rel-params) (db/insert! conn :file-library-rel rel-params)
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp)))) (bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))
(l/warn :hint "ignoring invalid file library relation" (l/warn :hint "ignoring invalid file library relation"
:file-id (str file-id) :file-id (str file-id)
:lib-id (str libr-id)))))) :lib-id (str libr-id))))))
@ -1282,56 +1304,6 @@
{:file-ids file-ids {:file-ids file-ids
:resolution resolution}))) :resolution resolution})))
(defn- invalidate-thumbnails
[cfg file-id]
(let [storage (sto/resolve cfg ::db/reuse-conn true)
sql-1
(str "update file_tagged_object_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")
sql-2
(str "update file_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")]
(run! #(sto/touch-object! storage %)
(sequence
(keep :media-id)
(concat
(db/exec! cfg [sql-1 file-id])
(db/exec! cfg [sql-2 file-id]))))))
(defn- import-file-and-overwrite*
[{:keys [::manifest ::bfc/file-id] :as cfg}]
(when (not= 1 (count (:files manifest)))
(ex/raise :type :validation
:code :invalid-condition
:hint "unable to perform in-place update with binfile containing more than 1 file"
:manifest manifest))
(bfc/configure-import-transaction cfg)
(let [ref-file (bfc/get-minimal-file cfg file-id ::db/for-update true)
file (first (get manifest :files))
cfg (assoc cfg ::bfc/overwrite true)]
(vswap! bfc/*state* update :index assoc (:id file) file-id)
(binding [bfc/*options* cfg
bfc/*reference-file* ref-file]
(import-storage-objects cfg)
(import-file cfg file {:current 1 :total 1})
(invalidate-thumbnails cfg file-id)
(bfm/apply-pending-migrations! cfg)
{:file-ids [file-id]
:resolution {}})))
(defn- import-files (defn- import-files
[{:keys [::bfc/timestamp ::bfc/input] :or {timestamp (ct/now)} :as cfg}] [{:keys [::bfc/timestamp ::bfc/input] :or {timestamp (ct/now)} :as cfg}]
@ -1382,9 +1354,7 @@
(tap-progress cfg {:section :manifest}) (tap-progress cfg {:section :manifest})
(binding [bfc/*state* (volatile! {:media [] :index {}})] (binding [bfc/*state* (volatile! {:media [] :index {}})]
(if (::bfc/file-id cfg) (db/tx-run! cfg import-files*))))
(db/tx-run! cfg import-file-and-overwrite*)
(db/tx-run! cfg import-files*)))))
;; --- PUBLIC API ;; --- PUBLIC API
@ -1453,6 +1423,12 @@
(defn import-files! (defn import-files!
[{:keys [::bfc/input] :as cfg}] [{:keys [::bfc/input] :as cfg}]
(when (::bfc/file-id cfg)
(ex/raise :type :validation
:code :not-supported
:hint "in-place import is not supported"
:file-id (::bfc/file-id cfg)))
(assert (assert
(and (uuid? (::bfc/profile-id cfg)) (and (uuid? (::bfc/profile-id cfg))
(uuid? (::bfc/project-id cfg))) (uuid? (::bfc/project-id cfg)))

View File

@ -10,6 +10,7 @@
[app.common.data :as d] [app.common.data :as d]
[app.common.exceptions :as ex] [app.common.exceptions :as ex]
[app.common.flags :as flags] [app.common.flags :as flags]
[app.common.logging :as l]
[app.common.schema :as sm] [app.common.schema :as sm]
[app.common.time :as ct] [app.common.time :as ct]
[app.common.uri :as u] [app.common.uri :as u]
@ -158,6 +159,10 @@
[:http-server-io-threads {:optional true} ::sm/int] [:http-server-io-threads {:optional true} ::sm/int]
[:http-server-max-worker-threads {:optional true} ::sm/int] [:http-server-max-worker-threads {:optional true} ::sm/int]
;; How often a websocket subscription re-checks that its access still
;; holds. The default lives in `app.http.websocket`, not here.
[:subscription-revalidation-interval {:optional true} ::ct/duration]
;; Explicit CORS allowlist used when the :cors flag is enabled. ;; Explicit CORS allowlist used when the :cors flag is enabled.
;; Configured via PENPOT_ALLOWED_ORIGINS as a comma/whitespace ;; Configured via PENPOT_ALLOWED_ORIGINS as a comma/whitespace
;; separated list of origins (e.g. "https://plugins.example.com"). ;; separated list of origins (e.g. "https://plugins.example.com").
@ -240,6 +245,12 @@
[:database-readonly {:optional true} ::sm/boolean] [:database-readonly {:optional true} ::sm/boolean]
[:database-min-pool-size {:optional true} ::sm/int] [:database-min-pool-size {:optional true} ::sm/int]
[:database-max-pool-size {:optional true} ::sm/int] [:database-max-pool-size {:optional true} ::sm/int]
;; pgjdbc prepared statement cache (see app.db). Size is in bytes and is
;; converted to the whole MiB the driver accepts. Absent keys mean the
;; driver defaults; see docs/technical-guide/configuration.md.
[:database-prepared-statement-cache-threshold {:optional true} [::sm/int {:min -1}]]
[:database-prepared-statement-cache-queries {:optional true} [::sm/int {:min 0}]]
[:database-prepared-statement-cache-size {:optional true} [::sm/int {:min 0}]]
[:quotes-teams-per-profile {:optional true} ::sm/int] [:quotes-teams-per-profile {:optional true} ::sm/int]
[:quotes-access-tokens-per-profile {:optional true} ::sm/int] [:quotes-access-tokens-per-profile {:optional true} ::sm/int]
@ -408,6 +419,22 @@
(def explain-config (def explain-config
(sm/explainer schema:config)) (sm/explainer schema:config))
(defn- normalize-file-data-backend
"The file data moved to `file_data` (migration `0157-migrate-legacy-file-data`), so
`file.data` and `file_change.data` are read-only leftovers and the `legacy-db`
back end is read-only: a write against it fails on every save. Accepting it would
start a healthy instance that cannot save a single file, so it reads as `db`,
which serves the legacy rows too. Warn instead of aborting: the operator may
still be halfway through the migration."
[config]
(if (= "legacy-db" (c/get config :file-data-backend))
(do
(l/warn :hint "file-data-backend=legacy-db is read-only, using db"
:file-data-backend "legacy-db"
:replacement "db")
(assoc config :file-data-backend "db"))
config))
(defn read-config (defn read-config
"Reads the configuration from environment variables and decodes all "Reads the configuration from environment variables and decodes all
known values." known values."
@ -415,7 +442,8 @@
(->> (read-env prefix) (->> (read-env prefix)
(merge default) (merge default)
(normalize-tenant) (normalize-tenant)
(decode-config))) (decode-config)
(normalize-file-data-backend)))
(def version (def version
(v/parse (or (some-> (io/resource "version.txt") (v/parse (or (some-> (io/resource "version.txt")

View File

@ -57,7 +57,9 @@
(def ^:private schema:pool-options (def ^:private schema:pool-options
[:map {:title "pool-options"} [:map {:title "pool-options"}
[::connect-timeout {:optional true} ::sm/int] [::connection-timeout {:optional true} ::sm/int]
[::idle-timeout {:optional true} ::sm/int]
[::max-lifetime {:optional true} ::sm/int]
[::max-size {:optional true} ::sm/int] [::max-size {:optional true} ::sm/int]
[::min-size {:optional true} ::sm/int] [::min-size {:optional true} ::sm/int]
[::name {:optional true} :keyword] [::name {:optional true} :keyword]
@ -66,10 +68,22 @@
[::username {:optional true} :string] [::username {:optional true} :string]
[::validation-timeout {:optional true} ::sm/int] [::validation-timeout {:optional true} ::sm/int]
[::read-only {:optional true} ::sm/boolean] [::read-only {:optional true} ::sm/boolean]
;; pgjdbc prepared statement cache. Absent keys mean the driver
;; defaults; the size is in bytes and is converted to whole MiB for
;; the driver (see create-datasource-config). The threshold is the
;; executions before a statement is server-prepared: 0 stops
;; server-side prepared statements (the cache of parsed queries stays
;; in use) and -1 prepares from the first execution. Any other
;; negative is a typo the driver would clamp to -1, so the schema
;; rejects it.
[::prepared-statement-cache-threshold {:optional true} [::sm/int {:min -1}]]
[::prepared-statement-cache-queries {:optional true} [::sm/int {:min 0}]]
[::prepared-statement-cache-size {:optional true} [::sm/int {:min 0}]]
[::mtx/metrics ::mtx/metrics]]) [::mtx/metrics ::mtx/metrics]])
(def defaults (def defaults
{::name :main {::name :main
::min-size 0
::max-size 60 ::max-size 60
::connection-timeout 30000 ::connection-timeout 30000
::validation-timeout 10000 ::validation-timeout 10000
@ -92,9 +106,11 @@
:read-only read-only :read-only read-only
:credentials (and (contains? cfg ::username) :credentials (and (contains? cfg ::username)
(contains? cfg ::password)) (contains? cfg ::password))
:min-size (or (::min-size cfg) :min-size (::min-size cfg)
(::max-size cfg)) :max-size (::max-size cfg)
:max-size (::max-size cfg)) :prepared-statement-cache-threshold (::prepared-statement-cache-threshold cfg)
:prepared-statement-cache-queries (::prepared-statement-cache-queries cfg)
:prepared-statement-cache-size (::prepared-statement-cache-size cfg))
(create-pool cfg)))) (create-pool cfg))))
(defmethod ig/halt-key! ::pool (defmethod ig/halt-key! ::pool
@ -108,15 +124,27 @@
(def initsql (def initsql
(str "SET statement_timeout = 300000;\n" (str "SET statement_timeout = 300000;\n"
"SET idle_in_transaction_session_timeout = 300000;")) "SET idle_in_transaction_session_timeout = 300000;\n"
;; lets the planner resolve a join between two partitioned tables
;; pairwise, when both are partitioned with the same bounds
"SET enable_partitionwise_join = on;"))
;; pgjdbc reads dataSourceProperties through Properties.getProperty, which
;; drops non-String values silently, so everything is passed as a string.
;; preparedStatementCacheSizeMiB only accepts a whole-MiB integer: the
;; configured size (bytes) is converted up, so any positive value keeps the
;; cache enabled and 0 disables the cache. A query parameter of the same name
;; in the URI wins over these properties: pgjdbc ignores the Properties value
;; when the URL carries it.
(defn- bytes->mib-ceil
[bytes]
(Math/ceilDiv (long bytes) (* 1024 1024)))
(defn- create-datasource-config (defn- create-datasource-config
[{:keys [::uri] :as cfg}] [{:keys [::uri] :as cfg}]
;; (app.common.pprint/pprint cfg) ;; (app.common.pprint/pprint cfg)
(let [config (HikariConfig.) (let [config (HikariConfig.)]
max-size (::max-size cfg)
min-size (or (::min-size cfg) max-size)]
(doto config (doto config
(.setJdbcUrl (str "jdbc:" uri)) (.setJdbcUrl (str "jdbc:" uri))
(.setPoolName (d/name (::name cfg))) (.setPoolName (d/name (::name cfg)))
@ -126,11 +154,18 @@
(.setValidationTimeout (::validation-timeout cfg)) (.setValidationTimeout (::validation-timeout cfg))
(.setIdleTimeout (::idle-timeout cfg)) (.setIdleTimeout (::idle-timeout cfg))
(.setMaxLifetime (::max-lifetime cfg)) (.setMaxLifetime (::max-lifetime cfg))
(.setMinimumIdle min-size) (.setMinimumIdle (::min-size cfg))
(.setMaximumPoolSize max-size) (.setMaximumPoolSize (::max-size cfg))
(.setConnectionInitSql initsql) (.setConnectionInitSql initsql)
(.setInitializationFailTimeout -1)) (.setInitializationFailTimeout -1))
(some->> (::prepared-statement-cache-threshold cfg) (str)
(.addDataSourceProperty config "prepareThreshold"))
(some->> (::prepared-statement-cache-queries cfg) (str)
(.addDataSourceProperty config "preparedStatementCacheQueries"))
(some->> (::prepared-statement-cache-size cfg) (bytes->mib-ceil) (str)
(.addDataSourceProperty config "preparedStatementCacheSizeMiB"))
(->> (mtx/get-registry (::mtx/metrics cfg)) (->> (mtx/get-registry (::mtx/metrics cfg))
(PrometheusMetricsTrackerFactory.) (PrometheusMetricsTrackerFactory.)
(.setMetricsTrackerFactory config)) (.setMetricsTrackerFactory config))

View File

@ -475,7 +475,7 @@
(def ^:private schema:invite-to-organization (def ^:private schema:invite-to-organization
[:map [:map
[:invited-by ::sm/text] [:invited-by ::sm/text]
[:user-name [:maybe ::sm/text]] [:user-name {:optional true} [:maybe ::sm/text]]
[:token ::sm/text] [:token ::sm/text]
[:organization schema:organization-data]]) [:organization schema:organization-data]])
@ -497,7 +497,7 @@
(def ^:private schema:renewal-notice (def ^:private schema:renewal-notice
[:map [:map
[:user-name [:maybe :string]] [:user-name {:optional true} [:maybe :string]]
[:renewal-date ::sm/text] [:renewal-date ::sm/text]
[:estimated-amount ::sm/text] [:estimated-amount ::sm/text]
[:organizations [:vector schema:organization-data]]]) [:organizations [:vector schema:organization-data]]])

Some files were not shown because too many files have changed in this diff Show More