Compare commits

..

627 Commits

Author SHA1 Message Date
Francis Santiago
617e56b71e
Merge pull request #13153 from penpot/fc-issue-done-status-automation
✨ Route completed issues by QA requirement
2026-11-18 16:29:49 +01:00
Francis Santiago
b488f6b2df ✨ Route completed issues by QA requirement
Set issues with the team-qa label to Ready for QA.
Set all other completed issues to Product Review.
2026-11-18 16:15:13 +01:00
Luke Lau
5166b8d5ea
⚡ Avoid iterating over every object in frame-titles* (#13089)
* ⚡ Avoid iterating over every object in frame-titles*

Similarly to #13075, frame-titles* used to iterate over every object to work
out the frames at the root that need titles.

For files with lots of objects this can be slow, but we can just get the root
frames by looking at the root object's children.

We need to remove non-main-component instances to copy the behaviour of
`get-frame-ids :skip-copies? true`. This also now causes the title to be drawn
on a child frame whenever it's focused.

AI-assisted-by: claude-opus-5-5

* 🐛 Don't show frame title on focused frames

Restores the existing behaviour when focused
2026-11-18 15:17:44 +01:00
Yamila Moreno
dedd435db5
🔧 Add PRs to status tech-review (#13152)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 14:59:46 +01:00
Andrey Antukh
efd95dcf0c Merge remote-tracking branch 'origin/staging' into develop 2026-11-18 13:20:18 +01:00
Andrey Antukh
dad4479b9c Merge remote-tracking branch 'origin/staging' into staging 2026-11-18 13:15:27 +01:00
Andrey Antukh
2377ab0a16 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-18 13:15:10 +01:00
Eva Marco
28a4794d73
🔧 Test the direction of the move shortcuts (#13115)
#13068 fixed the move left and right shortcuts, which moved and showed
the opposite direction. Its test only checks the key binding. These
tests run each move shortcut, in the workspace and the path editor,
and check that it moves in its own direction, fast or by one unit, and
that its tooltip shows the matching arrow.

Without the fix of #13068, 8 assertions fail.

AI-assisted-by: claude-opus-5-5
2026-11-18 13:05:44 +01:00
Eva Marco
eb7860fcd7
🐛 Show the date of a restored autosaved version in the restore notice (#13120)
When a peer restored an autosaved version, the notice said "restored an
earlier version" instead of naming it by its date and time.

The modal wrapper passes the props map to the component without
camelCasing its keys, so the `created-at` prop reached the component as
"created-at" while it read `createdAt`, and the date was always nil.
Named versions were fine because `label` is one word. The prop is now
`date`, and a test renders the registered modal the way the wrapper
does.

Closes #13037

AI-assisted-by: claude-opus-5-5
2026-11-18 13:04:55 +01:00
Andrey Antukh
aa12e52081
⬆️ Update the system dependencies of the devenv and the images (#13144)
* ⬆️ Bump the JDK to 27.0.1.1 on devenv and the backend image

The devenv image compiled the backend bundle with JDK 27 while the
production image ran JDK 26. The runtime starts with --enable-preview,
so a runtime line behind the compiler line is a silent trap rather than
a working setup.

Both pins now use Zulu 27.0.1.1 (zulu27.30.201-ca), the current GA of
the line, with the same sums on both architectures. The jlink module
list in the backend image still resolves on 27, so it does not change.

Sums checked by downloading both tarballs; the jlink JRE built from
them starts and reports 27.0.1.1.

AI-assisted-by: deepseek-v4.1-flash

* 🔥 Remove the unused gitpod setup

No workflow built docker/gitpod and its AdoptOpenJDK 16 install no
longer matches the devenv. The README badge linked to a setup that no
longer exists, so it goes with it.

The historical changelog entry stays: CHANGES.md is generated from the
milestones.

AI-assisted-by: deepseek-v4.1-flash

* ⬆️ Bring the docker/ updates from develop

Move the devenv and the production images to what develop already
carries:

- Base images to the current dhi.io digests.
- Node 24.22.3 and pnpm 12.21.0 in the devenv image and in every
  production Dockerfile that ships pnpm.
- Rust 1.99.0 and Serena 2.1.0, with SERENA_UPDATE_VERSION raised to
  match and a note that it must never be lower than the image pin.
- tini as the exporter ENTRYPOINT.

Devenv additions: PENPOT_DEVENV=true so the frontend compiles with the
admin console disabled by default, xauth and libvulkan1 plus the GPU
compose overlay, the MCP OAuth metadata route in nginx, and clipboard
support in tmux.

The JDK stays at 27.0.1.1 on both pins.

AI-assisted-by: deepseek-v4.1-flash

* ⬆️ Finish the Node 24.22.3 bump outside docker/

The docker/ import brought Node 24.22.3 into the devenv and the image
tags while .nvmrc and the two plugins-deploy workflows stayed on
24.22.2.

Node stays on the 24.x line on every site: the plugins-deploy runtime
pins follow .nvmrc, even though develop moved them to node@26.13.1 in
99db795966.

AI-assisted-by: deepseek-v4.1-flash
2026-11-18 12:51:30 +01:00
Yamila Moreno
89d5a862b5
🔧 Require milestone in PRs (#13129)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 11:54:01 +01:00
Álvaro Tejero-Cantero
a84d76d397
🐛 Stop camelCase svg attributes from breaking files (#13132)
* 🐛 Validate the shapes of a page added by an update-file change

The backend validates every shape an `:add-obj` or a `:mod-obj` change
writes, but an `:add-page` change carries a whole page and its shapes
reached the file unchecked. Duplicate page and the undo of a page
deletion send that page as the client holds it, so a tab that loaded a
file before a migration ran could write the old format back into it.
A tab opened before #12006 deployed holds camelCase svg attr keys such
as `:fillRule`, and the closed `:svg-attrs` schema rejects them. The
file saves, and the next read that migrates it fails the file data
check, so the whole file stops opening (#13107).

`:add-page` now validates each shape of the page on the backend, as
`:add-obj` does for one.

AI-assisted-by: claude-opus-5-5

* 🐛 Rerun the svg attr normalization on files that recorded 0035

Files that recorded migration 0035 never run it again, so camelCase
svg attr keys written after it ran stay in the file. A tab that loaded
a file before 0035 ran could save such keys through an `:add-page`
change, and the closed `:svg-attrs` schema then rejects the whole file
on its next migrating read (#13107).

The svg attr normalization now runs under the id
`0037-fix-shape-svg-attrs-and-defs`, and the 0035 id leaves
`available-migrations`, as 0021 did when 0035 replaced it. Every file
that recorded 0035 migrates again. Kebab input passes through
unchanged, so the rerun touches only the keys written after 0035.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the 0037 migration tests to their own namespace

The svg-attr migration was renamed to 0037, but its tests stayed in
the 0035 test namespace under migration-0035-* names. Move the whole
section to files_migrations_0037_test.cljc, register the new namespace
in the test runner, and rename its tests to match the migration id.

Replace the doseq over the two withdrawn ids with two self-contained
deftests, one for 0021 and one for 0035. No production code changes.

AI-assisted-by: hy4-flash-free

* 🐛 Keep the log watch installed until the rejection record arrives

The logger swaps the record on an executor thread, after the call that
logged it returns. The test removed the watch in the `finally` around
`update-file` and only then polled for the record, so under load the
record landed on nobody and the test read a nil `:attempts`.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Count only update-file calls in the save-test transport log

The mocked transport is installed with `set!` and outlives the fixture: a
timer another module left pending can still reach it mid-test, as a
`:delete-file-object-thumbnails` call did in CI. Filter to the only
command persistence sends, `update-file`, so a stray call neither fails
the emptiness assertion nor releases a `wait-for` on a save.

AI-assisted-by: deepseek-v4.1-flash

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 11:39:32 +01:00
Luke Lau
d563444985
✨ Delay building schemas in check-fn in release builds (#12711)
* ✨ Delay building schemas in check-fn

check-fn eagerly builds its schemas which contributes to the startup time. But
we can delay this to when the check function is actually called, which seems to
be when an edit is made. On the Travel A2B - Locofy Sample project file this
reduced the time to the first viewport render by ~15%.

* ✨ Make schema resolution lazy only on production builds

Add a lazy-delay macro that defers the expression only on builds that
set the app.common.data.macros/lazy-delay? closure define, and use it
in check-fn to resolve the schema.

The previous change made the schema resolution unconditionally lazy,
which hid wrong schema definitions until the first check call. With
this change development builds, tests and storybook resolve the schema
when the namespace loads, so a bad definition fails early, and
production builds keep the deferred resolution and the startup win.

The define is enabled in the release builds of frontend (main and
worker), exporter and library. On the JVM the expression is always
evaluated immediately.

AI-assisted-by: deepseek-v4.1-flash

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 11:36:33 +01:00
David Barragán Merino
240e757586 🐛 Size frontend nginx workers from the container CPU limit
worker_processes auto starts one worker per host CPU, ignoring the
container CPU limit, and since worker_connections went from 2048 to
65535 each worker reserves about 30 MB at startup. A frontend limited
to a few CPUs on a 64-core node needs close to 2 GB before serving a
request and gets OOM killed.

Default the worker count to the CPUs the container can use, its cpuset
and its cgroup CPU quota rounded up, when they are fewer than the CPUs
of the host, and keep auto otherwise. Make both settings configurable
through PENPOT_HTTP_SERVER_WORKER_PROCESSES and
PENPOT_HTTP_SERVER_WORKER_CONNECTIONS, and refuse to start on values
that are not valid.

Refs #13103

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-18 10:51:18 +01:00
Yamila Moreno
dfc2ff6b56
🔧 Improve issue templates (#13126)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 10:00:39 +01:00
Andrey Antukh
3f6c69bb6a Merge remote-tracking branch 'origin/staging' into develop 2026-11-18 09:32:49 +01:00
Álvaro Tejero-Cantero
3b3669b63b
🐛 Drop map-valued svg attrs when migrating legacy files (#13130)
Files written by old Penpot versions hold maps where an svg
attribute value should be, such as a `:filter` that once read
`url(#filter_2)`. Since #12006 the `:svg-attrs` schema is closed and
admits scalars only, and migration 0035 kept those maps, so the file
data check at the end of every migration rejected the file. Such a
file can be neither imported nor opened. Two built-in templates,
Avataaars and Tutorial for beginners, are among them.

`attrs->kebab-props` now drops a value that is not a string, a number,
or a boolean, at the top level and inside `:style`. Migration 0035 and
every birth path go through it, so stored attrs match the schema.

Of the 143 files in penpot/penpot-files, 4 fail to import at
`:svg-attrs :filter` before this change, and all 4 import after it.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:31:39 +01:00
Andrey Antukh
930f74218c Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-18 09:29:54 +01:00
Andrey Antukh
09259623da 🔥 Remove the unused Signed-off-by validator
check_signed_off_by was never registered in the validator list, so it never
ran, and no test exercised it. The DCO sign-off applies to external
contributors through CONTRIBUTING.md, not to every commit, so the local
checker has nothing to enforce there.

Drop the function and the DCO mention in the script docstring.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:28:51 +00:00
Andrey Antukh
5c93ece9b9 📚 Slim the code review skills without losing checks
code-review-criteria and review-code carried weight that changed no
decision: a 16-row rationalizations table, red flags that repeated the axes,
and three sections written for the author of a change rather than for the
reviewer. 338 lines become 251.

Drop that layer, dedupe DRY/KISS/YAGNI between the core principles and the
axes, merge the two instruction lists of the flow, and compress the GitHub
comment format.

The reviewer no longer re-runs the suite or a build: the author's run and CI
already provide that evidence, and repeating it only made reviews longer. It
runs at most the single test file that settles a specific doubt, and states
which evidence it relied on.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:28:28 +00:00
Eva Marco
01ceba34ec
♻️ Give the team hero and recovery page links a real href (#12708)
* ♻️ Give the Team management link in the team hero a real href

"Team management" in the "Team up!" notice of a team's Projects page was
an <a> without href. It now links to the team Members page; a plain click
still runs the in-app navigation (prevent default), and Ctrl/Cmd-click
opens the link in a new tab.

New visual test "Shows the Team management link": 0 px of difference at
rest and on hover.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the Go to login link of the recovery page a real href

"Go to login" on the password recovery page was an <a> without href. It
now links to the login page; a plain click still runs the in-app
navigation (prevent default), and Ctrl/Cmd-click opens the link in a new
tab.

New visual test "Shows the Go to login link": 0 px of difference at rest
and on hover.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:22:07 +01:00
Eva Marco
1a355958a4
♻️ Give the team page section links a real href (#12705)
Members, Invitations, Webhooks and Settings in the team page header
were <a> elements without href: the keyboard could not reach them and
they could not open in a new tab. Each one now links to its section; a
plain click still runs the in-app navigation (prevent default), and
Ctrl/Cmd-click opens the link in a new tab. The current section gets
aria-current="page".

They keep their look: the "Team header menu" visual test gives 0 px of
difference at rest and on hover.

AI-assisted-by: claude-opus-5-5
2026-11-18 09:21:14 +01:00
Andrey Antukh
dce4d8fe56 ✨ Allow human co-authors in the commit checker
The checker rejected any Co-authored-by trailer, which also blocked a real
person. The trailer is meant to credit a human co-author, and git and
GitHub use it for that; only AI providers, agents and bots are unwanted.

Match the trailer value against a blocklist of providers and agents plus
bot identities, and point to the dedicated AI-assisted-by trailer for AI
participation. Co-authored-by joins the trailer exemption of the
80-character body rule.

Update the tests, CONTRIBUTING.md, the create-commit skill and the commit
memory to the new rule.

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-11-18 08:20:43 +00:00
Florian Schrödl
cdfbff7c53
🐛 Fix active themes in token export and the empty file import error (#13123)
* 🐛 Fix token export writing the wrong active themes

The export passed only the tokens library, so the active themes and
sets in `$metadata` were the ones of the imported file, kept in the
library, instead of the ones activated in the file. The multi-file
export now needs the file's status, so no caller falls back to them.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

* 🐛 Say no tokens were found when importing a file without tokens

Importing `{}`, or a JSON file without any token, said the token data
was invalid.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

---------

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-18 09:19:55 +01:00
Yamila Moreno
d403a66122
🔧 Require milestone to issues linked to a PR (#13133)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-18 09:07:22 +01:00
penpot-renovate[bot]
a9972d5b47
⬆️ Update dhi.io/nginx image to 5f75855 (#13136)
- dhi.io/nginx 1.31.8-debian13-dev@5f75855 (docker/images/Dockerfile.frontend)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-18 09:05:41 +01:00
Luke Lau
14abaf93f4
⚡ Cull tile shapes to the tile instead of its whole margin (#13112)
After background blur support was added to render-wasm in 5ba53f7296,
we needed to start rendering shapes in the 256px margins of the tile,
since they contribute to the blur result.

This means that a tile may end up drawing shapes from the tiles around
it, and in the worst case we end up drawing 4x the number of shapes.

We can limit this to only the tiles that have background blur, which
significantly reduces the number of shapes drawn.

Masked groups do something similar so this generalizes and reuses the
`group_reach` stack, which stores how far past the current tile is
painted.

Before we start drawing the shapes in `render_shape_tree_partial`, we
walk the shapes in the tile to check how far the background blur
"reaches" and extend `group_reach` by just that amount, rather than
a 256px radius. As part of the generalization we also need to update
`masked_group_filter_reach` so it works on non-masked groups.

On the mem_benchmark file a full render calls render_shape 77.1% less
often. On the admin dahsboard there's 62% less. This gain shows up
particularly in interactive transforms, e.g. dragging another board
across the heroicons board goes from about 21 to 48 fps.

AI-assisted-by: claude-opus-5-5
2026-11-18 07:07:53 +01:00
Pablo Alba
73e7815817
🐛 Make execute_code wait for save before success (#13027)
* 🐛 Make execute_code wait for save before success

* 🐛 Address review feedback on waitForSave

Reject with an Error carrying a `code`: save-timeout, rolled-back,
save-retrying, or the failure code (save-failed by default). Lower the
default timeout from 90s to 60s, leaving room for the 30s layout wait
and the code under the MCP tool timeout.

When persistence is already in error at the start of the wait, keep the
stored failure code, and tell a failure Penpot keeps retrying apart from
one that needs the file reopened.

Share timeout validation with waitForLayoutUpdate through
app.plugins.utils, and make reject-not-valid reject with an Error, which
also fixes the string rejections of connectLibrary, restore, remove,
pin, export, findVersions and saveVersion.

The MCP execute_code handler now waits for the save only when the
workspace is ready, like the layout wait, so dashboard calls no longer
fail on a save error from an earlier file.

Move the waitForSave tests to frontend-tests.plugins, add tests for the
error codes, the retrying state and a failure during the wait, and add
plugin API suite and MCP handler tests. Update the docs, CHANGELOG and
the generated API files.

AI-assisted-by: claude-opus-5-5
2026-11-17 22:35:37 +01:00
Yamila Moreno
f65242fb00
🔧 Manage renovate PRs (#13131)
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-17 19:00:05 +01:00
Elena Torró
14b9d77e9f
🐛 Fix duplicated suffix on single-file WASM exports (#13125) 2026-11-17 17:39:04 +01:00
David Barragán Merino
0915f89419 🐛 Size frontend nginx workers from the container CPU limit
worker_processes auto starts one worker per host CPU, ignoring the
container CPU limit, and since worker_connections went from 2048 to
65535 each worker reserves about 30 MB at startup. A frontend limited
to a few CPUs on a 64-core node needs close to 2 GB before serving a
request and gets OOM killed.

Default the worker count to the CPUs the container can use, its cpuset
and its cgroup CPU quota rounded up, when they are fewer than the CPUs
of the host, and keep auto otherwise. Make both settings configurable
through PENPOT_HTTP_SERVER_WORKER_PROCESSES and
PENPOT_HTTP_SERVER_WORKER_CONNECTIONS, and refuse to start on values
that are not valid.

Refs #13103

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-17 16:49:09 +01:00
Belén Albeza
dd83c668cc
🐛 Fix UX error handling (wasm) (#13100)
* 🐛 Show the error page for critical WASM errors

Since #8651 the :wasm-error handler showed the error page for :panic
only, so a :wasm-critical error (a WASM trap with no error code) fell
through to a toast and the user kept working on a broken renderer.

Only the codes in recoverable-wasm-codes now show a toast: :non-blocking
from Rust, plus :wasm-reload-context-failure and :viewer-canvas-failed,
raised by the frontend. Any other code shows the error page. This covers
errors from the render loop, event handlers, promises and store events;
errors thrown inside React effects still reach the error boundary.

The h/call docstring now describes what it does: it always throws.

Closes #12990

AI-assisted-by: claude-opus-5-5

* ✨ Add a guard for React effects that call WASM

React sends any error thrown in an effect, or in its cleanup, to the
error boundary at the app root, which replaces the whole workspace with
the error page. That holds even for recoverable WASM errors, which
should only show a toast.

errors/wrap-wasm-effect runs an effect fn and the cleanup fn it
returns. A recoverable WASM error goes to on-error (toast); any other
error is rethrown, so it still reaches the boundary. The macro
with-wasm-effect wraps mf/with-effect with it.

The macro lives in its own namespace, app.main.ui.hooks.wasm-effect:
app.main.ui.hooks cannot require app.main.errors, because the renderer
loads it and that makes a circular dependency.

AI-assisted-by: claude-opus-5-5

* 🐛 Show a toast for recoverable WASM errors in the viewport

A recoverable WASM error thrown inside a viewport effect reached the
React error boundary, which replaced the whole workspace with the
error page.

Every viewport effect that calls WASM in its body or cleanup now uses
with-wasm-effect, so a recoverable error shows a toast and the
workspace stays; panics and critical errors still show the error page.
Effects that call WASM only from a listener, ts/asap or ts/raf keep
mf/with-effect: those errors never reach React.

AI-assisted-by: claude-opus-5-5

* 🐛 Show a toast for recoverable WASM errors in the text editor

The WASM text editor calls WASM from two React effects: on mount (select
all, render) and in the first caret blink. A recoverable error there
reached the error boundary and replaced the workspace with the error
page.

Both effects now use with-wasm-effect. The mount effect also guards its
select-all and render step on its own: it opens an undo transaction
with no timeout first, and without that inner guard a recoverable error
would stop the effect before it returned the cleanup that commits the
transaction and disposes the WASM editor.

AI-assisted-by: claude-opus-5-5
2026-11-17 16:37:27 +01:00
Belén Albeza
2a9b9a57cc
🔧 Set as default the flag to enable html paste in v3 (#13114) 2026-11-17 16:15:26 +01:00
Alonso Torres
eb2c8cf359
🐛 Check Host and Origin headers in the MCP server (#13117)
* 🐛 Check Host and Origin headers in the MCP server

The MCP endpoint, the REPL server and the plugin WebSocket accepted
requests with any Host and Origin header, so a web page open in the
user's browser could drive a local MCP server: connect as the plugin
and call its tools.

RequestGuard checks both headers against an allowlist and rejects other
requests with HTTP 403, or WebSocket close code 1008. The list comes
from PENPOT_MCP_ALLOWED_HOSTS, where "*" turns the check off. When it
is unset, local mode allows only loopback names and remote mode allows
any host, since remote mode relies on user tokens and has no file
access.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep the Host and Origin check on in single-user remote mode

The default allowlist followed remote mode, so a single-user server with
PENPOT_MCP_REMOTE_MODE=true accepted any Host and Origin. It requires no
user token either, so a web page could still drive it and run code in the
user's Penpot session. The default now follows multi-user mode: single-user
servers allow only loopback names, and multi-user servers, which rely on
user tokens, allow any host. Startup warns when the check is off in
single-user mode.

PENPOT_MCP_ALLOWED_HOSTS entries with a scheme, port or path never matched,
so every request failed with 403. The server now refuses to start and names
the bad entry.

AI-assisted-by: claude-opus-5-5
2026-11-17 16:01:49 +01:00
Andrey Antukh
e6eb8bcd5a
✨ Add pgjdbc prepared statement cache settings (#13099)
* ✨ Add pgjdbc prepared statement cache settings

Expose pgjdbc's server-side prepared statement cache as three
optional backend variables: PENPOT_DATABASE_PREPARE_THRESHOLD,
PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_QUERIES and
PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_SIZE (bytes, converted
to whole MiB for the driver, rounding up). When unset, the
driver defaults apply (5 executions, 256 entries, 5 MiB), so the
behavior is unchanged. The values reach Hikari as
dataSourceProperties strings, because pgjdbc only reads String
values; flushCacheOnDdl stays at its default.

Closes #13092

AI-assisted-by: inkling-2-small-free

* ♻️ Sync the pool options schema with the code

The integrant pool options schema validated a ::connect-timeout
option that nothing reads, while the keys the code consumes
(::connection-timeout, ::idle-timeout, ::max-lifetime) were
missing from it. The schema now declares the three, matching the
code and the defaults map. The removed key was not referenced
anywhere else.

Follow-up to #13092.

AI-assisted-by: inkling-2-small-free

* ♻️ Rename the prepared statement cache threshold setting

Keep the three knobs under one name in the configuration:

  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_THRESHOLD
  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_QUERIES
  PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_SIZE

The old PENPOT_DATABASE_PREPARE_THRESHOLD read as a verb and was the only
one of the three that did not name the prepared statement cache.

Fix the documentation of the values: 0 in either cache value disables the
cache, not only the size, and -1 prepares a statement from its first
execution and forces binary transfer, which is what pgjdbc does.

Cover the -1 threshold end to end: it already decoded and validated, but
nothing proved that it reaches the driver. The tests also pin 0, so a
falsy check cannot silently drop it.

Closes #13092

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Fix the prepared statement cache contracts

The threshold schema accepted any integer, but the driver clamps every
negative to -1 and turns on binary transfer, so a typo like -10 changed
the wire protocol instead of failing at boot. Both schemas now declare
{:min -1}: -1 and the non-negatives stay valid.

The tests compared our own dataSourceProperties literals with our own
literals, which cannot catch the failure the feature fears (a property
the driver ignores). They now read the threshold back from a real
connection: 5 with no options, 1 and 0 when configured. The size ceiling
is covered one byte over a MiB, and queries=0 too.

The docs, the app.db comment and the memory described the threshold 0 as
disabling the cache. It does not: 0 stops the server-side prepared
statements while the driver's cache of parsed queries stays in use,
bounded by the other two values. The same texts lost the
pg_prepared_statements pointer (the view only lists the session that runs
the query) together with the flushCacheOnDdl and PgBouncer sentences, and
the section now carries its version marker.

Closes #13092

AI-assisted-by: deepseek-v4.1-flash

* 💄 Format the prepared statement cache pool test

The continuation arguments of the two with-connection calls were aligned
under the options map instead of two spaces past the form, which cljfmt
rejects.

Local verification missed it because `./scripts/ci backend --test` runs
the tests only: lint and format are separate tasks, so the pre-push check
is the whole `./scripts/ci backend` or an explicit --lint/--fmt.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Address the review of the prepared statement cache

The pool startup line shows the three values, so the configured cache is
visible without reading the environment back; a nil means the driver
default.

bytes->mib-ceil uses Math/ceilDiv: the same rounding in one expression,
and the size schema already rejects negatives.

The cache-limit tests count a closed statement in pg_prepared_statements
instead of comparing the property literals. A closed statement stays
prepared in the server only while the driver's cache holds it, which is
what tells an accepted property from an effective one; the positive case
keeps the cache enabled with a sub-MiB size. The config test drops the
groups that only exercised the decoder and keeps the ones that pin the
accepted range.

The manual and the memory note that a query parameter of the same name in
PENPOT_DATABASE_URI wins over the variable: pgjdbc ignores the Properties
value when the URL carries it.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: claude-opus-5-5
2026-11-17 14:06:04 +01:00
Luke Lau
0bfcaa4109
⚡ Avoid redundant copy of tile atlas (#13104)
61cd9fe88683a4068817f538a2ae54d80aca095e stored a reference to the
last snapshot of the tile atlas, presumably to avoid the call to
image_snapshot when not needed.

But it turns out image_snapshot doesn't actually cost anything since
it just returns a reference to the surface texture. Storing the
reference actually causes a 4096x4096 texture copy because skia has
copy-on-write behaviour. Skia sees the reference stored in Surfaces,
so when the next render tries to draw on the surface it needs to
allocate + copy over the tile atlas texture.

By removing the reference skia can reuse the texture on the next
render which avoids a 64MiB copy on the GPU.

AI-assisted-by: claude-opus-5-5
2026-11-17 13:46:27 +01:00
Miguel de Benito Delgado
6043edb95a
♻️ Share WASM modifier bindings (#13016)
- Extract modifier encoding and FFI operations into common bindings
  for renderer benchmarks and application callers.
- Keep readiness checks and render scheduling in the frontend API.
- Release propagation output after decoding failures.
- Cover payload ownership, heap growth, snapping and interactive
  lifecycle behaviour with shared fixtures and integration tests.

AI-assisted-by: gpt-6.1-sol, deepseek-flash
2026-11-17 13:28:33 +01:00
Florian Schrödl
94b13d08fe
✨ Read and write token files with the tokenscript package (#12481)
Token import and export move to tokenscript.document.read and
tokenscript.document.write in the tokenscript-interpreter package
(penpot/tokenscript-interpreter#4).
tokens-lib keeps an adapter that maps its documents to a TokensLib and
back: penpot's token types, the keys of composite values and the
activation of themes and sets. About 600 lines of format detection,
legacy rewriting and tree walking go.

- Imports read DTCG 2025.10 files: {value, unit} dimensions, color
  objects in their color space (oklch(l, c, h)), the fontFamily and
  fontWeight type names, $root, $extends and $ref aliases. Legacy and
  DTCG nodes can be mixed in one file.
- What penpot does not store is dropped and listed in one warning
  toast: $extensions, $deprecated, group descriptions, values the
  reader cannot use (such as a Rec. 2020 color without hex), groups
  whose $extends fails and tokens of types penpot does not have.
- Sets a theme marks disabled are no longer part of the theme, and
  activating a theme deactivates the other themes of its group.
- Exports are unchanged by default. export-dtcg-json and
  export-dtcg-multi-file take {:profile :dtcg} for 2025.10 values and
  types. Such a file reads back with the DTCG types: spacing and
  sizing tokens come back as dimensions.
- The package moves from frontend/deps.edn to common/deps.edn, since
  the backend and the SDK read and write token files too.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 13:04:45 +01:00
Pablo Alba
e0832335c7
🐛 Keep text mounted until its measurement settles (#13097)
The text measurement in update-text-shape used ->> with p/catch. But
p/catch takes the promise first, so the handler was read as the promise.
The call then returned a promise that resolved to the handler function at
once. Measurement errors were never logged, and a failed measurement was
left as an unhandled rejection.

The caller ran p/finally right away, so the text left the pending set and
its :text-measure reflow task closed before it was measured. The text
could then unmount: the measurement read a detached node, got a zero rect
and skipped resize-text. waitForLayoutUpdate could also resolve before
the text was measured. This bug came in with #9898.

Use p/merr instead, which takes the function first. It logs the error and
resolves to nil, so the promise settles only when the measurement does.

Tests check that the promise stays pending until the measurement settles,
that the measurement then completes, and that a failed measurement
resolves to nil.

AI-assisted-by: claude-opus-5-5
2026-11-17 12:48:19 +01:00
Juan de la Cruz
81c2408470
✨ Add scale options panel to the right sidebar (#12453)
* ✨ Extend combobox icons and input handling

Allow a combobox to show a fallback icon when no option provides one.
Clear delivered input values so unmounting does not emit them again.

AI-assisted-by: gpt-6-luna

* ✨ Add scale options panel to the right sidebar

Add scale controls for selection dimensions, presets, and anchor points.
Scale shape content around the selected anchor and preview canvas resizes.
Reset scale state when selection changes, and exit scale mode with Escape.
Cover scaling and resize previews with frontend tests.

AI-assisted-by: gpt-6-luna

---------

Co-authored-by: Luis de Dios <luis.dedios@kaleidos.net>
2026-11-17 12:33:51 +01:00
Florian Schrödl
8b8def3b6e
🐛 Fix clicking a set in the theme editor (#13067)
Clicking a set's row in the theme editor threw an assertion error and
left the set as it was: the row passed the set's name to the toggle,
which looks the set up by id. The row now uses the checkbox's handler,
so the two can't drift apart again.

The editor's set and group toggles move into pure functions next to
it, with tests. A new spec clicks a set's name to tick and untick it.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 12:11:05 +01:00
Rayen Khammar
fa1249c1a5 🐛 Reap zombie Chromium processes in the exporter container (#12969)
* 🐛 Reap zombie Chromium processes in the exporter container

The exporter runs node as PID 1 with no init process. Chromium helper
processes orphaned when Playwright closes a browser are re-parented to
node, which never reaps them, so every browser-based export left
defunct chrome-headless processes behind.

Enable init for the exporter service so Docker's built-in init runs as
PID 1, reaps the orphans and forwards signals to node.

Closes #10339

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

* 🐛 Move zombie reaping fix into the exporter image

Install tini and run it as the image entrypoint with -s, so orphaned
Chromium helpers are reaped in every deployment, including Kubernetes.
-s keeps tini working without a warning when it runs under docker-init
(init: true). Drop the compose init: true change to keep one mechanism.

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

---------

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>
2026-11-17 11:46:19 +01:00
Rayen Khammar
445d02e012
🐛 Reap zombie Chromium processes in the exporter container (#12969)
* 🐛 Reap zombie Chromium processes in the exporter container

The exporter runs node as PID 1 with no init process. Chromium helper
processes orphaned when Playwright closes a browser are re-parented to
node, which never reaps them, so every browser-based export left
defunct chrome-headless processes behind.

Enable init for the exporter service so Docker's built-in init runs as
PID 1, reaps the orphans and forwards signals to node.

Closes #10339

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

* 🐛 Move zombie reaping fix into the exporter image

Install tini and run it as the image entrypoint with -s, so orphaned
Chromium helpers are reaped in every deployment, including Kubernetes.
-s keeps tini working without a warning when it runs under docker-init
(init: true). Drop the compose init: true change to keep one mechanism.

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>

---------

Signed-off-by: Rayen Khammar <khammarrayen4@gmail.com>
2026-11-17 11:41:43 +01:00
Andrey Antukh
38a9878981
🔥 Remove the unused SQLite-based binfile v2 format (#12947)
The SQLite-based v2 binfile package was never released, and its only
caller was the developer REPL helper `app.srepl.binfile`, which exported
and imported a whole team. With the format gone, the helper, the
functions of `app.binfile.common` without a caller (`get-fonts`,
`get-file-thumbnail`, `get-team`, `get-teams`, `collect-summary`) and
the `sqlite-jdbc` dependency go with it.

The public surface does not change: the import RPC accepts only versions
1 and 3, and the test that pins the rejection of version 2 stays as it
is.

AI-assisted-by: deepseek-v4.1-flash
2026-11-17 10:59:10 +01:00
Andrey Antukh
78cfe7fa58 Merge remote-tracking branch 'origin/staging' into develop 2026-11-17 10:53:02 +01:00
Marina López
e422de62d8
🐛 Fix create team inside organization (#13106) 2026-11-17 10:44:54 +01:00
Andrey Antukh
94155508e5 Merge branch 'staging-2.19.x' into staging 2026-11-17 10:16:51 +01:00
Andrey Antukh
1d60ba65b5 Merge remote-tracking branch 'origin/main' into staging-2.19.x 2026-11-17 10:02:29 +01:00
Pablo Alba
ea0ac02900 🐛 Fix duplicate-file crash on nil keys in deleted component objects (#13045)
* 🐛 Fix duplicate-file crash on nil keys in deleted component objects

* ♻️ Withdraw the legacy-36 migration in favor of 0036

legacy-36 and 0036-clear-nil-keys-in-objects perform the same
cleanup of nil keys in `:objects`. Migrations only clean up
invalid data here, so they carry no order dependency between
them and running both is wasteful.

Remove the legacy-36 defmethod and its id from
`available-migrations`, the same way 0021-fix-shape-svg-attrs
was withdrawn in favor of 0035. Files that recorded legacy-36
re-migrate through 0036; their persisted `:migrations` lists
keep the old id, and the playwright fixtures recording it stay
untouched.

Add a migrate-file test proving a file that recorded legacy-36
is re-armed for 0036 and passes schema validation after.
Document the withdrawal pattern in the migration subtleties
memory.

AI-assisted-by: inkling-2-small-free

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-17 09:45:48 +01:00
Roshan Ramani
32d488e162
🐛 Fix swapped Move left and Move right shortcuts (#13068)
The `move-unit-left` shortcut was bound to the Right arrow and moved the
selection right, and `move-unit-right` the reverse, in both the workspace
and the path editor. Arrow keys behaved correctly, but the shortcuts panel
and the shortcut settings showed "Move left" next to the Right arrow, and
customizing "Move left" rebound the rightward move.

AI-assisted-by: claude-opus-5-5

Signed-off-by: Roshan Ramani <roshanramani.dev@gmail.com>
2026-11-17 09:45:02 +01:00
Pablo Alba
aa35bc78c9
🐛 Fix duplicate-file crash on nil keys in deleted component objects (#13045)
* 🐛 Fix duplicate-file crash on nil keys in deleted component objects

* ♻️ Withdraw the legacy-36 migration in favor of 0036

legacy-36 and 0036-clear-nil-keys-in-objects perform the same
cleanup of nil keys in `:objects`. Migrations only clean up
invalid data here, so they carry no order dependency between
them and running both is wasteful.

Remove the legacy-36 defmethod and its id from
`available-migrations`, the same way 0021-fix-shape-svg-attrs
was withdrawn in favor of 0035. Files that recorded legacy-36
re-migrate through 0036; their persisted `:migrations` lists
keep the old id, and the playwright fixtures recording it stay
untouched.

Add a migrate-file test proving a file that recorded legacy-36
is re-armed for 0036 and passes schema validation after.
Document the withdrawal pattern in the migration subtleties
memory.

AI-assisted-by: inkling-2-small-free

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-17 09:36:40 +01:00
Florian Schrödl
0ae02daae9
♻️ Extract and test how tokens are applied to shapes (#13034)
toggle-token and apply-token-from-input each worked out inside the
event which attributes a token goes on, and repeated most of it. The
decision now lives in pure helpers both events share: `toggle-action`
(remove the token or apply it), `apply-action` (on which attributes)
and `target-shapes` (a selected group stands for its children).
`action->event` turns the result into the event to run. Nothing
changes in behavior; apply-spacing-token-separated loses its `attr`
option, which every caller left empty.

Unit tests cover the decision, with the cases the store tests
lacked: a color token comes off the stroke too, a shape without the
token completes the others, and a spacing token on chosen attributes.
Store tests cover what had none: applying from the fill and stroke
rows, the opacity, position and size fields, the min width of a flex
child and the selection colors, choosing the token a shape already
has, and toggling a token on a group.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-17 09:31:47 +01:00
Andrey Antukh
114ce6ec03
✨ Make legacy-db file data read-only (#13102)
legacy-db is no longer a write backend: fdata/upsert! rejects it
(:legacy-backend-read-only), handle-persistence stops writing
file.data and file_change.data, and the update-params schema of
fdata no longer accepts it. Reading old payloads keeps working, and
the file-data-backend default stays db.

The configuration guide, the backend config (a startup warning reads
PENPOT_FILE_DATA_BACKEND=legacy-db as db instead of failing every
save) and the storage memories say so, with the upgrade note for
deployments that still set the variable.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: inkling-2-small-free
2026-11-17 09:24:23 +01:00
Elena Torró
1068421d0b
🐛 Keep the drop index when moving a child to the end of a flex layout (#13098)
Closes #12929

AI-assisted-by: claude-opus-5-5
2026-11-17 08:48:20 +01:00
Elena Torró
a050a21fae
🐛 Omit board fill on WASM export when Show in exports is off (#13073)
Send the flag to render-wasm and drop the root's fills while it renders.

Closes #12930

AI-assisted-by: claude-opus-5.5
2026-11-17 07:58:10 +01:00
penpot-renovate[bot]
23468f9e4f
⬆️ Update dhi.io/nginx image to ad079e1 (#13096)
- dhi.io/nginx 1.31.8-alpine3.24@ad079e1 (docker/images/Dockerfile.storybook)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 18:39:36 +01:00
Andrey Antukh
57b9b8463b
🐛 Keep app booting when SES bootstrap fails (#13085)
* 🐛 Keep app booting when SES bootstrap fails

Wrap repairIntrinsics in a guarded bootstrap so a SES Temporal check failure
no longer aborts bundle evaluation and blanks the whole app. When it fails,
plugins and MCP stay unavailable for that session, a telemetry event reports
the error, and trying to open a plugin shows an environment warning.

Add vitest coverage for both bootstrap paths and a CLJS regression test for
the runtime-unavailable toast.

Closes #12924

AI-assisted-by: hy4-flash-free

* 🐛 Fix CI format check and plugin test entry point

Wrap the long Prettier line in bootstrap.spec.ts so `pnpm run format:check`
passes again in the plugins module, and document that the duplicated SES
options are an intentional contract snapshot.

Switch the runtime-unavailable CLJS test to the public open-plugin! entry
point instead of the private load-plugin! function. The M2 test for
signal-runtime-ready was dropped: loading app.plugins in the Node test
runner needs packaging and DOM shim changes outside this fix.

Closes #12924

AI-assisted-by: hy4-flash-free
2026-11-16 17:52:55 +01:00
Andrey Antukh
d97972c81d
🐛 Build the SSO redirect target on the server (#12952)
* 🐛 Build the SSO redirect target on the server

The organization SSO flow took the destination URL from the client and
redirected to it after the identity provider round-trip, so any
authenticated profile could point that redirect at another domain: the
server never checked where the URL came from.

The RPC now takes the route token of the client (?screen=...&params)
and the server builds the destination out of it: the public URI plus
the query string, parsed and rendered back. The destination can no
longer leave the public URI, and the callback checks it again before
redirecting, which covers the state tokens minted before the change.

The state token also carries the profile that started the flow, so
only that session is granted the organization SSO entry: a state
prepared by someone else cannot mark the session of the one that
completes it. Whenever the RPC receives an organization id (it always
comes from the client), it also checks that the profile is a member of
the organization or its owner, instead of letting any profile start a
flow for an organization it does not belong to.

Tests cover the destination builder, a callback that ignores a
destination outside the public URI, a state that does not grant the SSO
entry to another session, and the organization access check.

AI-assisted-by: minimax-m3.1-flash

* 🐛 Check the SSO redirect target against the public URI

The callback compared the destination with a string prefix, so the two
bypasses the advisory documents (a userinfo prefix and a look-alike
host) were closed only by the trailing slash of the public URI. The
check now compares the parsed origin (scheme, host case-insensitive,
port with the default made explicit) and requires the path to be under
the base path of the public URI, so neither bypass passes.

The client sent `location.search`, the history token, whose leading `?`
is URL syntax and no part of a query string. It sends the query string
proper now (`screen=…&params`) through `rt/get-current-query-string`,
and `rt/token->query-string` is the single bridge between the two
shapes: every other consumer concatenates the token over the base path.
The server puts the separator back when it joins the destination, and
trims one that still arrives, so a browser holding the previous bundle
keeps its destination while the new one rolls out.

Building the destination can no longer break the flow: it catches its
own errors and lands on the public URI, which matters in the
deployments where the browser origin differs from the configured one.
A value that is not a route is ignored with a warning, decided by the
keys it parses into (a URL becomes a single key holding the whole URL),
and a route with no `screen`, like the root params of the application
(`template=…`), stays a usable destination.

The organization membership lookup is now shared by the check that
gates the SSO flow and the one that raises, so the two cannot drift
apart, and the docstring of the RPC lists `:no-organization-access`
next to `:no-team-access`.

`rt/nav-raw` returns its `:href`/`:uri` through `@event`, as its
sibling `rt/navigate` already did. Nothing in the app listens to
`::rt/nav-raw`, so the callers behave the same; the SSO test now reads
the destination from the event instead of intercepting the helper.

Tests cover the two bypasses and the base-path rule, the RPC with an
unusable destination, the value that is not a route, the request that
carries both ids, an organization that does not exist, the query string
the frontend sends, and the two router helpers.

AI-assisted-by: minimax-m3.1-flash

* ♻️ Deduplicate SSO destination URL building

The backend had an unreachable blank check and a one-line wrapper around
the URI helper in build-dest-url. On the client, the SSO retry and error
flows each assembled the destination from public-uri and the route params.

Flatten build-dest-url with a cond and call u/query-string->map directly.
Add rt/token->url as the single client helper for absolute route URLs, with
tests for both token shapes and for a blank token. Update the routing memory
with the new helper contract.

AI-assisted-by: hy4-flash-free
2026-11-16 17:34:49 +01:00
Florian Schrödl
699f79c693
♻️ Extract the token group's pill click and open sections (#12795)
Move what the token group works out into pure functions with tests:
whether a type's section is open, whether a text is being edited, and
what clicking a token pill does. Nothing changes in behavior.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-16 16:14:57 +01:00
Eva Marco
82421f9729
🐛 Show resolved value when opening an alias token edit form (#13038)
* 🐛 Show resolved color in alias token edit form swatch

The color token form opened every alias token with an empty swatch.
The resolved value only reached :color-result once the value field
was touched, so the swatch had nothing to show on open.

Keep the resolved color for the swatch when the field is untouched,
without showing the "resolved value" hint. Add an e2e test that loads
a fixture with a color token and an alias to it.

Closes #12672

AI-assisted-by: claude-opus-5-5

* 🐛 Show resolved value when opening an alias token edit form

The token edit form only showed "Resolved value" after typing in the
value field: the single-value inputs dropped the first resolution while
the field was untouched, and the typography/shadow reference field read
its first value from [:value :reference], which an alias token does not
have (its value is the reference string).

Show the hint on open when the resolved value differs from the written
one, so a plain `16` gets no `Resolved value: 16`. Start the reference
field stream with the alias string. Extend the e2e fixture with font
size and typography aliases and add a test for them.

Closes #12788

AI-assisted-by: claude-opus-5-5
2026-11-16 16:00:18 +01:00
Andrey Antukh
d2dfc09495
🐛 Carry the team id in every team scoped navigation (#12963)
* 🐛 Carry the team id in every team scoped navigation

The eleven `go-to-*` helpers of `app.main.data.common` resolved the
team with a plain `or` against `:current-team-id`, or read that key
alone, so an event emitted while no team was initialized built a
dashboard or workspace route without a `team-id`. `team-container*`
raises `:not-found :invalid-team-id` for such a route and
`exception-section*` does not report `:not-found`, so the user landed
on the not-found page and PostHog never saw it.

Add `resolve-team-id`, which falls back to the team of the route
already on screen and then to the profile `:default-team-id`. The
route goes first on purpose: an emitter that runs from a team screen
knows which team the user is looking at, and the personal team would
open their file under another team. The `:default` sentinel keeps its
meaning. With none of the four the helper still navigates with a nil
id, which is the documented last resort.

Harden the team-container test at the same time: it now renders the
same component with the store team the caller wants, captures markup
and errors per render, unmounts the root from a `finally`, and covers
the guard that compares the route id with the store team. React
reports a hook-order violation through `console.error` and recovers,
and only when a render adds hooks, so the harness captures that too
and the "id leaves" direction stays a guard test.

Closes #12962

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Measure the navigation tests by termination

The `emitted` helper of `common_test` waited for the first event, with a
budget of 50 macrotasks, so a helper that emitted anything before its
navigation could be disposed early, and a helper that stopped closing
its stream passed on the first event. Wait for termination instead:
`async/observe` over a subject, and make `go-to-dashboard-search`
complete by pushing the `::rt/navigated` it listens for, which is the
event it waits for in the app to focus the search box. Every helper of
the family is measured the same way now, `-search` included.

`nav-params` returns nil when no navigation matched instead of calling
`deref` on nil, as its docstring promised. Add the mirror test for the
`:default` sentinel of `go-to-dashboard-deleted`, the second copy of
the branch that `-recent` already had covered.

Correct two memories: the resolver decides for every team scoped
helper, not for every `go-to-*` and not alone, since two direct
`rt/nav` emitters are the residual; and `(rx/empty)` is for a stream
that must not emit, not for one that must stay open.

Relates to #12962

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 15:42:43 +01:00
Andrey Antukh
6e4e69c339
🐛 Read the media and thumbnails of a package one at a time (#13024)
The analysis of a package reads the metadata of every media and
thumbnail of its container to attribute the shared objects to each
file. It read them all at once, one promise per entry, and the
container of a big file holds thousands of them: every read opens its
own decompression stream, and Chromium fails the whole analysis under
that load while Firefox keeps up. The failure is masked by zip.js with
a generic "Cannot close a ERRORED writable stream".

The references now go through an observable, one entry at a time, and a
reference that cannot be read leaves its file without a size instead of
failing the package: a number that misses one of the objects of the
file is worse than no number.

Closes #13022

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 15:42:15 +01:00
Andrey Antukh
0fda857426
🐛 Invalidate all sessions on password recovery (#12967)
* 🐛 Invalidate all sessions on password recovery

recover-profile validated the recovery token and updated the
password, but never deleted the existing server-side sessions in
http_session_v2. A previously captured auth-token kept working
after the victim "recovered" the account (GHSA-3j2p-qr45-q365,
CWE-613).

The command now calls session/invalidate-all for the recovered
profile. The command already runs with ::db/transaction true, so
the delete joins the same transaction as the password UPDATE: both
commit or roll back together. There is no current session to
preserve — the RPC is unauthenticated — which is the same contract
the profile deletion path uses.

Tests in rpc_auth_test.clj pin the behavior: the victim's sessions
die, other profiles' sessions survive, and a failed recovery is a
no-op. Both invalidation tests failed before the fix.

AI-assisted-by: inkling-2-small-free

* 🐛 Pin recovery-transaction atomicity after review

Review of 92a65f544b found the tests pinned the end state but not
the rollback guarantee the fix relies on: the session DELETE joins
the password UPDATE in one transaction.

Add recover-profile-rollback-keeps-password-and-sessions: send-template
throws after the update, so the command must fail with password and
sessions both untouched. recover-profile-failure-keeps-sessions now
also asserts the password, matching its docstring. The invalidate-all
docstring mentions the recovery caller.

Addresses review findings M1, M2 and M3.

AI-assisted-by: inkling-2-small-free

* 🔥 Drop recovery test that never reached the fix

The failure test used a token with the :authentication issuer, so
tokens/verify rejected it before any write. It passed with or without
the invalidation fix and exercised no new code. The rollback test
already covers that a failed recovery leaves no partial effects.

AI-assisted-by: hy4-flash-free
2026-11-16 15:40:36 +01:00
penpot-renovate[bot]
99db795966
⬆️ update GitHub Actions (#13071)
- node 26.13.1 (.github/workflows/plugins-deploy-styles-doc.yml)
- node 26.13.1 (.github/workflows/plugins-deploy-api-doc.yml)
- pnpm/setup v3.2.0 (.github/workflows/plugins-deploy-styles-doc.yml)
- pnpm/setup v3.2.0 (.github/workflows/plugins-deploy-api-doc.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 15:38:28 +01:00
María Valderrama
c0f746ee34
🐛 Apply the invite policy to pending invitations (#13080)
* 🐛 Apply the invite policy to pending invitations

* 📎 Code review
2026-11-16 15:38:07 +01:00
Marina López
ba0eab57ab
🐛 Hide team plan from every member when owner has Enterprise (#13079) 2026-11-16 14:55:54 +01:00
Luke Lau
5d88cb3bb7
🐛 Don't copy margins past tile in draw_current_tile_into_backbuffer (#13066)
When dragging a shape each tile that needs repainted is repainted via
draw_current_tile_into_backbuffer in fast mode. So if you drag over a tile it
ends up dropping drop shadows etc.

IIUC, each tile is 512x512px but drawn on a 1024px surface with 256px margins
so that the background blur has something to read. In a regular render we
render the 1024px surface but only copy the center 512px of the tile into the
backbuffer. But in draw_current_tile_into_backbuffer we were copying the whole
1024px surface. This draws over the other tiles with a fast mode render, even
though those other tiles don't need repainted.

This fixes it by reusing draw_surface_src_rect_to_dst which clips the canvas
properly so we only draw the 512x512px of the tile.

AI-assisted-by: claude-fable-5-2
2026-11-16 14:47:27 +01:00
Luke Lau
a224608a0e
⚡ Memoize boards with grids in frame-grid* (#13075)
On a test file `mem_benchmark` with 30k boards on one page, around 11ms is
spent each time the view is panned/zoomed iterating over each board and
checking has-grid?, which after https://github.com/penpot/penpot/pull/12790 is
now slightly slower because it involves an array scan.

We don't need to compute the list of boards with grids on every step of the pan
and can memoize it, which saves 30k calls to has-grid? on each frame.

AI-assisted-by: claude-opus-5-5
2026-11-16 14:32:10 +01:00
Álvaro Tejero-Cantero
bd4af28d78
📚 Name a worker count when running the Playwright suite (#12799)
* 📚 Name a worker count in the frontend testing memory

The memory gave the command and left the count to the config, whose
default of one worker makes the whole suite slow, and CI's two per
shard reads as the number to copy.

Records the count to reach for and what it costs: about 8 GB of
browser memory at four workers, so a 64 GB host takes four beside a
watching devenv, and a 32 GB host takes two, or four on a slim devenv.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Gate the Playwright worker count on cores and memory

Four workers keep about nine hardware threads busy, measured as the
container's CPU seconds over wall time, and hold about 8 GB of browser
processes. States the cost per worker, so a reader on a smaller machine
does not oversubscribe it.

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 13:55:05 +01:00
Luke Lau
bbaebfb146
🐛 Send the team's custom fonts to the worker (#13044)
The worker draws local exports and looks custom fonts up in its own store
to find their files. Only the main thread fetches the team's fonts, so the
worker's store had none, and exports drew text set in a custom font with
the fallback font.

The main thread now sends the worker the team's fonts each time they
change, and the worker keeps them in its store.

AI-assisted-by: claude-opus-5-5
2026-11-16 13:53:43 +01:00
Luke Lau
1ccb9e9cb9
⚡ Move WASM component thumbnail rendering to the worker (#13046)
This PR moves component thumbnail rendering off the main thread
and onto the worker, which avoids the freezes.

The main thread will send the worker over a request containing the component's
shape data, and the worker will render it in batches. Once finished it will
both persist the thumbnails to the server if requested and send back the PNG
data to the main thread.

AI-assisted-by: claude-opus-5-5
2026-11-16 13:48:04 +01:00
Belén Albeza
b0dbbe8285
🐛 Fix blurry file thumbnails rendered with render-wasm (#13026)
The wasm path drew file thumbnails on a 252px wide canvas, wrapped the
PNG in an SVG and sent it to the rasterizer, which scaled it up to its
default 300px width. The result was blurry on the dashboard on high-DPI
screens and in the link preview card.

Render the wasm thumbnail at 600px wide, twice the old size, and upload
the canvas PNG as is, without the rasterizer round trip. The SVG render
path keeps its current size.

Closes #13025

AI-assisted-by: claude-opus-5-5
2026-11-16 13:32:38 +01:00
penpot-renovate[bot]
99497d3b7c
⬆️ Update dependency pnpm to v12.21.0 (#13063)
- pnpm 12.21.0 (docker/images/Dockerfile.media-processor)
- pnpm 12.21.0 (docker/images/Dockerfile.mcp)
- pnpm 12.21.0 (docker/images/Dockerfile.exporter)
- pnpm 12.21.0 (docker/devenv/Dockerfile)
- pnpm 12.21.0 (render-wasm/package.json)
- pnpm 12.21.0 (plugins/package.json)
- pnpm 12.21.0 (plugins/libs/plugins-styles/package.json)
- pnpm 12.21.0 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.21.0 (plugins/libs/plugin-types/package.json)
- pnpm 12.21.0 (plugins/apps/table-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.21.0 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/icons-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/example-styles/package.json)
- pnpm 12.21.0 (plugins/apps/e2e/package.json)
- pnpm 12.21.0 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.21.0 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.21.0 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.21.0 (package.json)
- pnpm 12.21.0 (media-processor/package.json)
- pnpm 12.21.0 (mcp/packages/server/package.json)
- pnpm 12.21.0 (mcp/packages/plugin/package.json)
- pnpm 12.21.0 (mcp/packages/common/package.json)
- pnpm 12.21.0 (mcp/packages/browser-extension/package.json)
- pnpm 12.21.0 (mcp/package.json)
- pnpm 12.21.0 (library/package.json)
- pnpm 12.21.0 (frontend/text-editor/package.json)
- pnpm 12.21.0 (frontend/packages/ui/package.json)
- pnpm 12.21.0 (frontend/packages/mousetrap/package.json)
- pnpm 12.21.0 (frontend/packages/draft-js/package.json)
- pnpm 12.21.0 (frontend/package.json)
- pnpm 12.21.0 (exporter/package.json)
- pnpm 12.21.0 (docs/package.json)
- pnpm 12.21.0 (common/package.json)
- pnpm 12.21.0 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 13:15:48 +01:00
penpot-renovate[bot]
4d91d8f098
⬆️ Update DHI base images (#13062)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@e3e8512 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@f90fd43 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@2168f2f (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@dee4bb1 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-16 13:15:17 +01:00
Belén Albeza
27784f70ac
🐛 Fix custom CJK fonts losing glyphs when converted to TTF (#13074)
FontForge opens a CID-keyed OpenType font as several sub-fonts, and
Generate() writes only the active one. Most CJK fonts use this layout,
so the TTF used by the render-wasm renderer kept only a few hundred
glyphs (the Latin sub-font) and lost its family name. Merge the
sub-fonts with CIDFlatten() before generating, both in the backend and
in the media-processor. The call is guarded because it fails on fonts
that are not CID-keyed.

Add a small synthetic CID-keyed font with two sub-fonts as a fixture
and check that the converted TTF keeps all its glyphs.

Closes #10976

AI-assisted-by: claude-opus-5-5
2026-11-16 13:14:40 +01:00
Andrey Antukh
909a29cf50 🔥 Remove unused file 2026-11-16 12:41:05 +01:00
Andrey Antukh
e95007a93b 🐛 Retry the publish of the websocket delivery assertions
The subscription to the bus reaches redis through an asynchronous agent
action, so a message published right after the connection opens can be
lost before redis knows the subscription. The delivery assertions of the
websocket tests published once and bet on winning that race, which fails
on a loaded CI runner.

`poll-published!` publishes, drains a window and publishes again only
when the window saw nothing, up to three attempts. The assertion still
proves that the client receives a message it published, and the negative
assertions keep the single drained window they need. A test pins the
retry so it does not go back to a single publish.

AI-assisted-by: deepseek-v4.1-flash
2026-11-16 11:59:00 +01:00
Andrey Antukh
239291539b 📚 Add the 2.21.0 changelog section
Add the 2.21.0 changelog section, generated from the milestone
issues and the advisory source of truth.

The 🔒 entry references GHSA-m53j-2766-6jqw with the fixing PR
#12189. The advisory now declares patched version 2.21.0 only: the
fix reached develop after 2.20.0-RC1, so the 🔒 section of
2.20.0, which claimed that release fixes the vulnerability, is
removed as well.

Entries cover the closed milestone issues after the dev-only and
exclusion filters; #9263 stays in 2.21.0 by maintainer decision and
is flagged in the anomaly report as milestone-mismatched with PR
#9958.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:58:05 +00:00
Andrey Antukh
18a827d65a Merge remote-tracking branch 'origin/staging' into develop 2026-11-16 11:38:09 +01:00
Andrey Antukh
65b882d044 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-16 11:37:52 +01:00
Andrey Antukh
b39276c148 📚 Update changelog for 2.19.2
Add the 2.19.2 section with the six bug fixes of the milestone, in
merge order, following the old 2.19.x layout: 🐛 Bugs fixed only,
no 💥 (no breaking changes, consistent with patch releases), no
🚀 (patches carry none), no 🔒 (no advisory declares
2.19.2). #12622 carries community attribution. The section stays
marked (Unreleased) until the release commit.

AI-assisted-by: inkling-2-small-free
2026-11-16 11:25:16 +01:00
Andrey Antukh
b8136da06a 🐛 Fix dev-only-bugs counting released same-minor versions as this milestone
`only_unreleased_envs` compared versions as major.minor, so for the
2.19.2 milestone the released 2.19.0 counted as "this milestone" and
bugs naming it as an affected version were listed as dev-only
candidates (e.g. #13019, "only seen on 2.19.0 and later").

Compare full semver instead, with a missing patch read as 0: only the
milestone version itself counts as an unreleased signal, and any other
version is a production hint that keeps the bug in the changelog, as
the documented contract already said.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:14:55 +00:00
Andrey Antukh
86eb0bf805 Merge remote-tracking branch 'origin/staging' into develop 2026-11-16 11:07:05 +01:00
Andrey Antukh
4a9f553c19 📚 Update the 2.20.0 changelog
Regenerate the 2.20.0 section for the new layout: `💥` and
`🔒` are generated from the breaking change notes and the
advisory (patched_versions includes 2.20.0); features and
enhancements move to their own `🎉` and `✨` sections
instead of one mixed section; `🚀` lists the two milestone
issues labelled `release highlight` (#11759 and #11761, no closing
PR in the milestone).

Add the issues closed since the section was written: #10436,
#12052, #12135, #12615, #12728, #12738, #12740, #12948 and #13028
as bugs, and #10465, #12265, #12796, #12902, #13029 and #13030 as
enhancements. Bugs that never reached production stay out per
`dev-only-bugs` (#12860, #12138) and manual review (#12901).

#12977 leaves the section: its fix (PR #12978, merged 2026-11-13
in develop) carries milestone 2.21.0, so the issue moves there and
will be picked up by the 2.21.0 changelog.

AI-assisted-by: inkling-2-small-free
2026-11-16 10:03:54 +00:00
Andrey Antukh
328e41e037 ♻️ Stop excluding release blocker from the changelog flow
The `release blocker` label marks work that blocks the release; it is
not a signal about whether users will notice a change. Drop it from
the changelog exclusions and keep only `no changelog`.

Closed bugs carrying `release blocker` are analyzed by `dev-only-bugs`
like any other bug: a bug found while testing an unreleased feature
never reached production, so the tool is the right filter for them.
Closed non-bug issues with the label are surfaced by a manual review
step in the flow: `gh.py issues --label "release blocker"` lists them
and the user decides between `no changelog` and keeping the entry.

Update the memories, the update-changelog skill and the tests to match.

AI-assisted-by: inkling-2-small-free
2026-11-16 09:17:27 +00:00
Andrey Antukh
7ef8c1cf36 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-16 10:05:00 +01:00
Andrey Antukh
635f7fc914
✨ Disable the admin-console by default in the devenv frontend (#13061)
* ✨ Disable the admin-console by default in the devenv frontend

The devenv does not run the admin-console service, but the frontend
showed its UI by default. The devenv containers now build the
frontend with PENPOT_DEVENV=true, which prepends disable-admin-console
ahead of the developer's config.js flags. Flags parse in order, so
enable-admin-console in config.js re-enables it; builds outside
devenv are unchanged.

Closes #13060

AI-assisted-by: inkling-2-small-free

* 📚 Require English in all code-facing text

Add a hard rule to AGENTS.md: everything in code must be in
English — identifiers, comments, docstrings, test descriptions,
log hints — with tests not excepted. The only allowed place for
another language is product copy rendered through the i18n
system. Also restated in mem:clojure/design-rules and mem:testing.

AI-assisted-by: inkling-2-small-free
2026-11-16 09:54:23 +01:00
Pablo Alba
77c87d16f1
🐛 Keep variant property order past eight properties (#13014) 2026-11-16 09:30:29 +01:00
Eva Marco
fa6af23d75
🐛 Disconnect a library that fails to load instead of breaking the file (#13048)
Opening a file connected to a broken shared library showed the
"Internal Error" page: every connected library was loaded in a single
stream, so one failing get-file broke the whole workspace.

Catch the error per library. When the backend fails to load it (a
server error, never a transport or transient error), a directly
connected library is disconnected and a toast tells the user; an
indirect one is only dropped from the state.

Closes #12429

AI-assisted-by: claude-opus-5-5
2026-11-16 09:02:31 +01:00
Filip Sajdak
557774e4f4 🐛 Call the team hook on every render of the team container (#12920)
team-container* called mf/deref only after returning early for a team id
that is not a uuid. When the same component rendered once without a team
id and then with one, the number of hooks changed between the two renders
and React threw "Rendered more hooks than during the previous render"
(error #310), which sent the user to the internal error page. The reverse
order throws "Rendered fewer hooks than expected".

Call the hook on every render and keep the uuid check in the condition
that builds the providers.

Two tests mount the component in jsdom and render it with a team id that
arrives and with one that leaves; both fail without this change.

Closes #12728

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-16 08:41:22 +01:00
Usama Sadiq
861bb5bb37 🐛 Fix database pool min-size falling back to max-size (#13052)
* 🐛 Fix database pool min-size falling back to max-size

Since 2.16 the pool minimum defaults to the maximum (60), so an idle
backend opens and holds 60 connections at startup. Restore the pre-2.16
default of 0 so the pool grows with load and idle connections expire.

Closes #12622

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

* 📚 Document database connection pool settings

Describe PENPOT_DATABASE_MIN_POOL_SIZE and PENPOT_DATABASE_MAX_POOL_SIZE,
their defaults, and sizing on a PostgreSQL shared with other apps.

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

---------

Signed-off-by: Usama Sadiq <usama7274@gmail.com>
2026-11-16 08:37:49 +01:00
Andrey Antukh
e23ca02aa2
✨ Reference the file data storage object with a foreign key (#12769)
* ✨ Reference the file data storage object with a foreign key

The file data pointer lived in `file_data.metadata.storage-ref-id`, a JSON
key. The storage GC parsed it to find live references, and nothing stopped it
from physically deleting an object a live row still pointed at: a failure in
that scan lost the blob.

Migration 0159 moves the pointer to a nullable `storage_object_id` column on
`file_data`, backfills it from the metadata key, removes the key from the
rows it backfills, indexes the column and rebuilds the foreign key to
`storage_object(id)` as `NO ACTION DEFERRABLE INITIALLY IMMEDIATE`.
`INITIALLY IMMEDIATE` is deliberate: the GC catches the violation inside the
chunk transaction, while a deferred check would fire at commit, outside it,
and the sweep would never move past the blocked chunk.

The column is the only source of truth. The `storage-ref-id` entry is gone
from the file metadata schema, and the single point that serializes the
metadata drops it, so no write can resurrect it. `upsert!` does not query
the row: the pointer of the row being replaced travels in its params, and
the object is touched before the row is written. Fragments and snapshots
pass nil because a modified pointer always gets a new id; the offload passes
its own row; `insert-file!` drops it, since an INSERT never replaces a
persisted object.

`update-file!` takes the file map read from the database plus the fields to
change and merges them, so the pointer comes from the read map and no caller
can lose it by building a partial map. A map without `:data` raises
`:missing-file-data`: it is not `assert`, which is compiled out in the
deployed binary. The params cannot override the pointer or retarget the
write with `:id`/`:file-id`.

`bfc/file->response` strips the pointer from every response (`get-file`,
`create-file`, `duplicate-file`, the view-only bundle, the team access
request, the comments read, the library summary, the snapshot read and the
debug download). A `file-data` object is authentication-only, so a leaked id
would let any logged-in profile fetch the blob from `/assets/by-id`.
`get-file-fragment` resolves through the file data backend, so a fragment on
cold storage no longer hands nil to `blob/decode`, and answers only the keys
of its result schema.

The storage GC reference scan, the restore union and `objects-gc` read the
column. The offload reads `SELECT fd.*`, so its rows go through a
`decode-row` that decodes jsonb metadata and drops a JSON null, which the row
schema rejected; a row without a readable object raises
`:missing-storage-object` with the file and row ids. A chunk blocked by the
foreign key (SQLState 23503) is a real anomaly, since the reference scan
should have frozen the object: it stays an error with its own hint and the
chunk is retried on the next run until the reference is fixed.

The release is a coordinated restart, not a rolling deploy, and not
rollback-safe for the rows it writes: the previous code writes and scans the
metadata pointer, so a key it writes misses the backfill and removing it
loses the reference. `operations/2.20.0-file-data-storage-object-id.md`
builds the column, the index and the foreign key online and runs the
backfill with every instance stopped, then marks the step as applied; a
fresh or small instance does the whole job in the migration. The runbook
keeps the validation scan out of the stopped window and adds a lock timeout
and retry to the online statements.

Tests cover the foreign key shape, the backfill and its corrupt and orphan
guards, the rebuild of a missing index and of a wrong, non-deferrable or
deferred constraint, the deletion refused by a live reference, the metadata
cleanup, the offload row normalizations, the touched objects of the unshare
and the restore, the response stripping on every call site and the chunk
blocked by the foreign key. The backend memories and the developer guide
document the pointer, the GC behavior and the two migration paths.

Closes #12768

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: inkling-2-small-free

* 🐛 Hold the file row lock across the migration write

`get-file` applies the pending migrations in memory and, when it finds
any, persists the result. That write read the row without a lock and
`update-file!` is an unconditional UPDATE, so a save that committed in
between was overwritten: the saved content, `revn` and `modified-at`
went back, the next save from that client failed with `:revn-conflict`
and the object the save left became an orphan the storage GC cannot
revisit once it is frozen.

The read now runs in a transaction of its own (`get-file` calls
`get-file*` inside `db/tx-run!` and accepts a cfg map only), so the
`file` row lock covers the re-check and the write. `get-file*` reads
the row once (`sql:get-file` plus `resolve-applied-migrations`) and
completes it with `attach-file-data`; only when a migration is pending
it reads the file again under `FOR UPDATE`, re-checks `need-migration?`
on that fresh map and persists it (`migrate-file`). The common case
stays at one row read and one migrations resolution, the migration case
keeps a single decoded copy in the heap, and a row that changed in
between is not overwritten. A migration found on the unlocked read is
applied in memory only, for the next read to persist.

`migrate-file` returns the reattached map when it persists, with the
`:storage-object-id` of the row the write left, and the realized map
when it does not; `:realize?` is what returns the read fully plain.
`fdata/upsert!` answers `{:written ... :storage-object-id ...}` instead
of a boolean, `update-file!` propagates it and the migrating read puts
it back in the map it returns, so a caller that persists again touches
the current object instead of the replaced one. `insert-file!` keeps
nil: an INSERT never replaces a persisted object.

The write also compares the stored pointer with the caller's value
(`IS NOT DISTINCT FROM`) and raises `:stale-file-data-pointer` when it
affects no row. Every writer holds the `file` row lock, so a pointer
that no longer matches is a protocol violation, not a race to retry.
The expected pointer is a separate argument because the storage branch
replaces `:storage-object-id` before the upsert, and the check lives in
`upsert-in-database`: the `legacy-db` branches do not use that SQL and
their count can legitimately be zero.

The save, snapshot and restore-backup reads take `FOR UPDATE` instead
of `FOR SHARE`: they inherit the lock, and the migration branch asks
for the exclusive one in the same transaction, so a shared lock that
upgrades deadlocks two concurrent readers. A data read rejects
`:lock-for-share?` for the same reason; only row-only reads accept it.
`sql:get-file-without-data` is a plain select now, not a CTE, which
makes locking legal without loading the data. Realization has a single
owner: `attach-file-data` only resolves and decodes, and `get-file*`
realizes once per path. The applied migrations no longer travel in the
`get-file-with-summary` response, cache hit and cache miss alike.

`embed-assets` reads each library without `:realize?`, so its
`:components` is an unloaded pointer map; `absorb-assets` walks it with
`vals`, `PointerMap.seq` calls `load!` and that raised with no loader
bound, so every export with `:merge-libraries` of a library with
components failed. The absorb runs under the library's own loader now.

The in-place binfile import (`import-file-and-overwrite*`,
`::overwrite`, `::bfc/file-id`) had no product caller and cannot carry
the previous `storage_object_id`, so overwriting a storage-backed row
would leave its object unreclaimed. `import-files!` rejects a cfg with
`::bfc/file-id` (`:not-supported`) and the helpers only that path used
go with it (`invalidate-thumbnails`, `*reference-file*`, the
`::overwrite` id branch of `index-object` and of `save-file!`, the
`on-conflict-do-nothing?` flag of the media insert).

Tests: a new `file-migration-read-test` pins the single read, the
locked re-read, the concurrent save interleaving, the double migration
and the read-only path; the upsert tests cover the return shape and the
stale pointer; the SQL recorder they share lives in
`backend-tests.helpers`, and the save, snapshot, restore and summary
sequences follow the new reads. The backend memories document the read
contract and the pointer.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Harden the get-file guard and skip its read-only transaction

The cfg guard of `get-file` was an `assert`, which `app.config` elides
when `:backend-asserts` is off, so a deployed binary accepted a bare
pool or connection while the docstring promised it raises. It throws
`IllegalArgumentException` now, the class the option guards of the same
namespace already use, and the existing test expects it instead of
`AssertionError`.

A read with `:read-only?` applies the pending migrations in memory and
writes nothing, so the transaction the migration write needs only costs
two round trips there. `get-file` runs it with `db/run!` instead: with
a cfg that carries a connection it still joins the caller's
transaction, and only the reads that would have opened their own skip
it. A lock option needs a read-write transaction from the caller,
because PostgreSQL rejects row locks in a read-only one; the docstring
states that.

Tests: the guard test expects the new class; a cfg marked
`::db/read-only` reads a pending migration without persisting it (the
config flag that fills that key is legacy, but the key is set by hand);
and a read-only read is pinned as not opening a transaction while the
same call without the option does. The backend memory documents the
split and the guard.

AI-assisted-by: deepseek-v4.1-flash
2026-11-14 19:27:18 +01:00
Belén Albeza
d9a9d0c2bc
📚 Update contributing guide with rules about auto-rejecting PRs (#13012) 2026-11-14 18:16:55 +01:00
Usama Sadiq
dc340f74af
🐛 Fix database pool min-size falling back to max-size (#13052)
* 🐛 Fix database pool min-size falling back to max-size

Since 2.16 the pool minimum defaults to the maximum (60), so an idle
backend opens and holds 60 connections at startup. Restore the pre-2.16
default of 0 so the pool grows with load and idle connections expire.

Closes #12622

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

* 📚 Document database connection pool settings

Describe PENPOT_DATABASE_MIN_POOL_SIZE and PENPOT_DATABASE_MAX_POOL_SIZE,
their defaults, and sizing on a PostgreSQL shared with other apps.

Signed-off-by: Usama Sadiq <usama7274@gmail.com>

---------

Signed-off-by: Usama Sadiq <usama7274@gmail.com>
2026-11-14 18:12:22 +01:00
penpot-renovate[bot]
a19fbbc826
⬆️ Update DHI base images (#13050)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@9cef390 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e9caa9 (docker/devenv/Dockerfile)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-14 18:10:55 +01:00
AJ Schmidt
3ba5a0cb15
📚 Fix typo in export presets docs
This PR fixes a typo:

`pressets` => `presets`

Signed-off-by: AJ Schmidt <ajschmidt8@users.noreply.github.com>
2026-11-14 18:10:24 +01:00
Andrey Antukh
9c5260314a
🐛 Revoke websocket subscriptions when access is revoked (#12189)
* 🐛 Revoke websocket subscriptions when access is revoked

GHSA-m53j-2766-6jqw. A websocket subscription was authorized once,
when it opened, and that decision was trusted for the rest of the
connection's life, so a member whose access was revoked kept receiving
live file content as long as the tab stayed open.

The revocation travels over the message bus instead of closing the
socket from the mutation: the connection registry is local to one
backend instance, while the RPC that revokes access runs on whichever
instance received the request. Every instance re-verifies the
subscriptions it owns with the same `has-read-permissions?` predicates
that `:subscribe-file` and `:subscribe-team` authorize with, so a
downgrade to viewer keeps its subscription and a non-member
organization owner keeps read-only access.

Two mechanisms cover each other:

- Access-changing mutations announce after the write. delete-team-
  member, leave-team, update-team-member-role, delete-file,
  delete-project, move-files, move-project and delete-team announce.
  delete-share-link is deliberately excluded: a share-link visitor
  never passes the subscribe permission check, so there is no
  subscription to revoke. Team-wide changes publish one event per
  team; the watcher resolves the members itself, and both publishers
  defer through `db/after-commit` so the watcher never decides from
  pre-commit state.
- Every relay re-checks access every 5 minutes by default
  (`subscription-revalidation-interval`), driven from the relay loop
  with `alts!` over the subscription channel and a one-shot tick
  channel, so a steady stream of edits cannot push the check away
  forever.

Supporting changes:

- The registry holds `:connections` and `:by-profile` in one atom,
  kept consistent by register/unregister in a single swap!; a profile
  lookup is one map access, and a profile whose last connection
  leaves is dropped from the index.
- `close-file-subscription` and `close-team-subscription` announce
  the :leave-* message, close the relay channel and purge the bus
  subscription, and are idempotent. Closing the channel ends the
  relay loop and its re-check task.
- The permission query the re-check uses filters soft-deleted files,
  projects and teams, matching its sibling queries.
- A transient database error fails open: it logs and keeps the
  subscription; the next tick retries.
- The revocation watcher's params schema declared the nitrate client
  as a bare keyword inside [:map ...], which malli resolved as a
  registry lookup and which took the whole system build down under
  enable-backend-asserts. It is now an explicit pair, optional
  because init-key yields nil with the admin-console flag off.

The tests reproduce the leak against the real message bus, including
the negative control for the interval re-check. The written records
live in mem:backend/websocket-subscriptions.

Closes #12155

AI-assisted-by: space-bunny-free
AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Mirror the subscribe gates in the revocation re-checks

Addresses the second review round on PR #12189 (pabloalba).

The re-checks only asked whether the profile could read. The membership
legs of sql:file-permissions filter soft-deleted rows since the F2 fix,
but the organization-owner fallback grants read-only access over HTTP
without looking at deleted_at, so a soft-deleted file or team kept an
expelled org owner subscribed indefinitely. The re-checks now gate on
existence as well: file-readable? and team-readable? require a live row
(db/get*, non-throwing, soft-deleted rows excluded) plus the read
permission, mirroring the :subscribe-file gates and extending them to
teams. Both relays and revalidate-profile-subscriptions use them; the
subscribe handlers keep their exact behavior.

Also from the same review:

- watcher-consumes-revocation-events-from-the-bus no longer races the
  command's after-commit announcement: it takes access away with a
  direct row delete and drives the watcher with a synthetic bus event,
  instead of deleting through the command and publishing a second
  notify-* that would mask a broken command announcement.
- moving-a-project-closes-source-team-subscriptions pins the
  move-project e2e: the file subscription drops, the source-team
  subscription survives (membership unchanged).
- soft-deleted-file/team-closes-org-owner-subscription cover the new
  gate with red-before/green-after tests.

mem:backend/websocket-subscriptions updated.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Refuse dead teams in resolve-team-id (PR #12189 review)

Second review round on PR #12189 (pabloalba).

The file re-check added in 87ca7aca53 stops at the file row, but the
organization-owner fallback grants read-only access to non-member org
owners via Nitrate over HTTP, which knows nothing about Penpot
soft-deletion. resolve-team-id resolved the team from unfiltered SQL,
so an expelled org owner kept the file subscription open for an
indefinitely soft-deleted file, project or team. Now both SQL lookups
filter deleted_at, and resolve-team-id checks the team row itself
(db/get*, which filters soft-deleted rows) for every caller — teams,
projects and files alike.

Also from the same review:

- file-readable? now uses the same sql:get-minimal-file as the
  subscribe gate (non-throwing via throw-if-not-exists false), and
  team-readable? projects id and deleted-at, so db/get* can filter.
- The test suite extracts the thrice-duplicated raw delete into
  delete-team-membership!; a sibling of the fail-open pin now redefs
  the existence probe to throw; memory and docstring aligned with the
  code.
- docs/technical-guide/developer/architecture/backend.md attributed
  the websocket to notifications.clj; it now names app.http.websocket
  as the owner and notifications.clj as the revocation publisher.
- mem:backend/auth-permissions-product-domains documents the fallback
  mechanism and the resolve-team-id contract.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Test resolve-team-id refusal and tidy review findings

Third pass over PR #12189 after review-code on 225eb98de2.

resolve-team-id now has a direct test: all three branches resolve
while rows are live, and file, project and team soft-deletes each
refuse their branch — the two new SQL predicates and the final team-row
check are all pinned by it.

From the same review:

- failing-existence-check-keeps-the-subscription and the sibling
  a-failing-check-keeps-the-subscription are deleted: they could not
  distinguish a live relay from a dead one. The fail-open policy
  remains documented in the docstring and the memory; the memory no
  longer claims the pins exist.
- team-readable? projects ::sql/columns like the rest of the backend;
  the fully-qualified keyword went away with the new alias.
- The dead when-not guard in get-organization-owner-viewer-team is
  removed (resolve-team-id guarantees the row live); the let binding
  in resolve-team-id no longer shadows the destructured team-id.
- docs/technical-guide names rpc/notifications.clj as the publisher
  and the tree lists the real top-level modules.

Closes #12155

AI-assisted-by: inkling-2-small-free

* 🐛 Make the revocation tests deterministic

Third review round on PR #12189 (pabloalba).

Both tests took access away through a command that announces, so the
live watcher could close the subscriptions before the test looked:

- revocation-closes-subscriptions-that-are-no-longer-authorized
  asserted an exact count of 2 closes after delete-team-member, and
  that count reads 0 whenever the watcher got there first. It now
  removes the member with delete-team-membership!, the raw row delete
  that publishes nothing, and asserts both subscriptions are still
  open before calling revalidate-profile-subscriptions.
- team-event-closes-every-member-subscription ran delete-team and then
  published a second notify-team-permissions-changed, so a missing
  announcement in delete-team still passed. It now soft-deletes the
  team row (sql:team-members joins team_profile_rel and profile only,
  so the watcher still resolves every member), asserts nothing closed,
  and drives one synthetic team event.

The command paths stay covered by the announce and end-to-end tests
for delete-team-member and delete-team. The file split the reviewer
suggested stays the optional follow-up, tracked outside this PR and
waiting for the merge.

Closes #12155

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 18:27:30 +01:00
Andrey Antukh
f3c1230b79
🐛 Allow OIDC registration with password login disabled (#12974)
* 🐛 Allow OIDC registration with password login disabled

The register-profile command rejected every registration without the
login-with-password flag, including SSO ones, while the OIDC callback
already allowed :oidc-registration as an alternative, so new OIDC
users were stuck at the register screen with a 400
registration-disabled error.

The gate now branches on the verified token backend: password tokens
keep the previous checks, SSO tokens only require :registration or
:oidc-registration through a predicate shared with the callback, so
the two gates cannot drift apart again. Flags and invitation checks
are split into single-responsibility helpers, and tokens without a
backend claim are rejected loudly by schema instead of defaulting
silently. Covered by regression tests for the SSO matrix, canceled
invitations, the missing-backend case and the no-password-fallback,
plus a configuration doc note.

Closes #12964

AI-assisted-by: Muse Spark 1.3 Contributor

* 🐛 Gate SSO registration by provider type, accept invitations

Follow up the review on #12974. The SSO registration gate now treats a
team invitation token as a substitute for :registration /
:oidc-registration, in both the OIDC callback and register-profile,
mirroring the password flow: the callback redirects to the register
screen when an invitation is present, and register-profile revalidates
the invitation row under a row lock before creating the profile when
no SSO flag is set, raising :invalid-token instead of creating an
account that cannot join a team.

The SSO branch is restricted to OIDC-typed backends
(oidc-provider-types); a prepared-register token carrying any other
backend is rejected with :invalid-token instead of silently taking
the SSO rules. The flags and invitation checks are shared by one
check-registration helper in register-profile; the prepare command
runs flags, then the email-mismatch restriction, then the
invitation row lookup, so the password login flag fails closed first.
The schema defs are private, the create-profile default uses
penpot-backend, the OIDC callback test decodes the redirect token to
assert the :prepared-register claims, and the docs and the auth
memory state the invitation path for SSO registration.

Regression tests cover invitation-only registration, canceled
invitations with and without SSO flags, unknown backends, and the
double-fault precedence in prepare-register.

AI-assisted-by: inkling-2-small-free

* 🐛 Require invitation email match for SSO registration

Close the hole left by treating any invitation token as a substitute
for the SSO registration flags. register-profile now requires the
invitation member-email to match the prepared-register email, raising
:email-does-not-match-invitation otherwise, mirroring the password
flow. The OIDC callback verifies the team-invitation JWT and the
email before redirecting to register instead of trusting raw
presence. Covered by the requested mismatch regression test plus
callback tests for mismatch and garbage tokens.

AI-assisted-by: Muse Spark 1.3 Contributor

* ♻️ Share registration gate between prepare and register-profile

After the email-match fix, check-registration runs flags, email
and invitation row in the same order as validate-register-attempt!.
The prepare command now calls the shared helper instead of inlining
the trio, and the stale docstring rationale is gone, so future
invitation rule changes land in one place.

AI-assisted-by: Muse Spark 1.3 Contributor
2026-11-13 17:33:42 +01:00
Marina López
bde53b4696
🐛 Hide team plan for standalone enterprise owners (#13047) 2026-11-13 14:16:30 +01:00
Eva Marco
b2c4b58b27
🐛 Keep the pages panel scroll when switching pages (#13004)
Switching pages swaps the workspace content for a loader until the new
page is ready, so the whole left sidebar mounts again. The pages list
started at the top and the selected page scrolled into view at the
bottom of the panel, even when it was already visible.

The sidebar scroll store now lives in workspace-inner*, which stays
mounted across page switches. The pages panel saves its scroll there and
restores it before the selected page scrolls into view. The layers,
assets and tokens panels keep their positions across page switches too.

Closes #13002

AI-assisted-by: claude-opus-5-5
2026-11-13 14:11:30 +01:00
Andrés Moya
0e4be21320
🐛 Ensure component name always matches root instance name (#12794)
* 🐛 Remove svg extension when creating a component from a svg file

* 🐛 Normalize root names to always match the component name

* ✨ Remove any extension ignoring case, not only .svg

* 🔧 Add some fixes and checks

* 🔧 Improve rename detection
2026-11-13 13:53:28 +01:00
Marina López
bb4556b705
🐛 Persist promotional banner collapse (#13040) 2026-11-13 13:07:05 +01:00
andrés gonzález
097ff41e03
✨ Send telemetry for WebGL renderer failures (#12987)
* ✨ Send telemetry for WebGL renderer failures

WebGL renderer failures left no trace in the audit log, so we could not
tell why people switch WebGL off.

- Send webgl-unavailable when the "WebGL is not available" modal shows,
  with the init error.
- Send webgl-context-lost, webgl-context-restored and webgl-reload-failed
  from the WASM renderer.
- Add the active renderer (wasm or svg) to handled-exception,
  unhandled-exception, exception-page and open-workspace-file.
- Add environment details to the WebGL events and to
  enable/disable-webgl-rendering: normalized GPU family and backend, max
  texture size, device memory, CPU cores and whether the app runs inside
  Electron. The full GPU renderer string is not sent, as it can help
  fingerprint a device.

Closes #12984

AI-assisted-by: claude-opus-5-5

* ⚡ Avoid throwaway WebGL contexts in renderer telemetry

WebGL events no longer create a context to read GPU details; the
settings, menu and unavailable modal create one at most once.

AI-assisted-by: claude-opus-5-5

* 🐛 Refresh remembered GPU details on each renderer init

A restored context can run on another GPU, so read it on every init
instead of only the first one.

AI-assisted-by: claude-opus-5-5

* ⚡ Never create a WebGL context for renderer telemetry

WebGL events send GPU details only when the renderer has already read
them on init, instead of probing a throwaway context.

AI-assisted-by: claude-opus-5-5

---------

Co-authored-by: Elena Torro <elenatorro@gmail.com>
2026-11-13 11:59:12 +01:00
Andrey Antukh
8715450f53 Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 09:49:27 +01:00
alonso.torres
976f016481 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-13 09:42:15 +01:00
Belén Albeza
a7b6ef0d41
🐛 Restore the WASM stack pointer after a failed call (#12978)
A failed render-wasm call traps without popping its shadow-stack frames.
Each failure leaked stack until it overwrote static data, and from then on
every call trapped as wasm-critical. h/call now saves the stack pointer
before the call and restores it on failure.

Closes #12977
Related to #12906

AI-assisted-by: claude-opus-5-5
2026-11-13 09:39:43 +01:00
Andrey Antukh
69c33c4877
♻️ Wait on signals instead of event-loop turns on frontend tests (#12986)
* ♻️ Wait on signals instead of event-loop turns

`wait-for` counted its budget in event-loop turns (50 x `setTimeout 0`,
about 54 ms idle), so a delivery that travelled through the RxJS async
scheduler could arrive after the wait gave up. It now accepts a
`:signal`: the predicate is re-checked on every emission and the wait
resolves on the event that changes the state. The ceiling (5000 ms)
only bounds failure when a signal is given, and stays a poll budget for
the waits that have nothing to listen to.

`wait-for-store` wraps the store's input stream, which notifies its
subscribers after applying the update. The five explicit `:max-ticks`
sites move to `:timeout-ms` in the same change, so the old option never
lives in a state where destructuring silently ignores it.

New `frontend-tests.helpers.async-test` pins the contract: a late
delivery resolves, a store signal resolves on the event without a poll,
an already-true predicate does not yield, a never-true predicate records
one failure within the ceiling, and a resolved wait stops listening.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Wait for the persistence requests on their own stream

The persistence tests waited for `(= N (count @requests))` by polling the
atom at every event-loop turn. The mocked transport now pushes each
request into a `requests$` subject, and those 59 waits pass it as
`:signal`, so they resolve on the request that makes the count true. The
two predicates that mix the request count with the store state merge
both signals.

The wait that flaked (`the other file's edit is sent next`, in the
rollback test) is one of these: it no longer depends on a turn budget,
and it cannot lose the delivery.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Wait for the store event in the store backed tests

153 waits whose predicate reads the store now go through
`wait-for-store`, so they resolve on the store event that changes the
state instead of polling at every event-loop turn. The count is 153, not
the 162 the plan estimated: nine of them read the store through a
helper (`fill-color`, `shape-name`, `copy-children`) rather than
`@store`. The five sites that carried an explicit ceiling drop it, since
with an event the ceiling only bounds failure.

Resolving on the event exposes the places where the poll was really a
pause for a side effect to advance:

- `tokens-source-test` emitted `unlink-file-from-library`, a watch that
  chains a second `rp/cmd!` after an async hop. The store signal fires
  before the chain runs (the event itself dissocs the library), so the
  mock scope closed in flight and the second call reached the real HTTP
  layer. The test now waits for that second call, through the mock.
- `persistence-rollback-test` drives the persistence pipeline step by
  step; two of its waits are the pause before the next emit, not an
  effect, so they keep the poll.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Document the wait contract and what still polls

`mem:frontend/testing` described `wait-for` as an immediate check plus a
bounded poll. It now waits on a signal, and the ceiling only bounds
failure; `wait-for-store` and the persistence `requests$` subject are
how a wait resolves on the event that changes the state. The waits with
nothing to listen to keep the poll, and the memory names them.

The two traps that cost a red suite are recorded too: a wait that means
"the side effect advanced" has no event and keeps the poll, and a
store-state wait can close the mock scope before the flow it observes
ends, so the chain's last RPC call has to be awaited through the mock.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Give the wait wrapper a single signal

`wait-for-store` built its options with `assoc`, so it replaced the
`:signal` a caller passed. The two waits whose predicate reads both the
store and the request counter relied on a merged signal that was being
discarded; they now pass it to `wait-for` themselves, and the wrapper
takes no options. That merge was the safety net for a request counted
after the store event, and it was inert.

The helper also reports why a wait failed (ceiling, ended signal, or an
error with its cause) instead of calling all three a timeout, counts
the predicate evaluations in the late-delivery case to prove the signal
path does not poll, reads the clock through `app.common.time` in the
helper and the test, and the test's turn probe cancels its timer. The
copyright header of `helpers/async.cljs` stays as the previous commit
left it, matching the majority spelling.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Give the cycle counter a signal too

Two waits in the retry tests read the store and the `cycles` atom, which
the `slow-cycle-timer` double fills inside the persistence watch. They
passed only the store signal, so a counter change could not wake them:
the same latent race the previous commit removed from the request count.
The double now also emits each new cycle into a `cycles$` subject, and
both waits pass the merged signal to `wait-for`.

The public `wait-for` docstring names the three failure reasons, and the
message for a failed signal bounds the cause text: the exception message
when there is one, otherwise the printed value inside a try, cut at 500
characters.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Stop the error handlers flooding the test log

Four tests in `main-errors-test` run a real error handler end to end.
Those handlers print the cause with `ex/print-throwable` before
notifying, which is what a developer wants in the app; in a test it only
writes console noise. One of them replays 10,000 offline errors, so a run
carried 10,006 dumps: 190,010 of its 192,682 lines, 13.8 MB, well past
the ~4 MB where GitHub truncates a step log and takes the summary and any
failing assertion with it.

The tests stub `ex/print-throwable` with a no-op for the duration of
their mock scope. The handlers still run, the assertions do not change,
and the 10,000-error loop still proves the governor emits once. A full
run now logs 2,583 lines and 384 KB. The single block that remains comes
from the re-entrancy guard, which prints its own notice on purpose.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Silence the deliberate WASM panic in its test

`get-svg-markup` logs a WASM panic with `console.error` before returning
empty, and `get-svg-markup-swallows-wasm-errors` sends two panics through
it on purpose. That was two 35-line dumps in which the exception printed
its own `data`, `arr` and protocol masks, and it reads like a failure that
the suite is hiding.

The test replaces `(.-error js/console)` for its body and restores it in a
`finally`. The assertions and the helper stay as they are: the same
panic answered with a different stack would still show up, because only
this test silences the console.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Use real timer for wait-poll ceiling

wait-poll read the app clock (ct/now), so tests freezing that clock
never reached the deadline and hung instead of failing. Track expiry
with a real setTimeout flag, pass timeout-ms through wait-signal and
wait-for, and cover the frozen-clock case with a regression test.

AI-assisted-by: muse-spark-1.3-contributor
2026-11-13 09:18:43 +01:00
Andrey Antukh
679e70e48c 🐛 Keep file_change kind nullable so rollback keeps writing
Migration 0158 created file_change with kind NOT NULL, which blocks
any emergency rollback to the previous version: the old code inserts
without kind and every save would fail. kind is still a reserve
(reads use label IS NOT NULL), so it stays nullable until the
rollback window closes; the NOT NULL comes in a later migration.
The runbook keeps parity, the schema and migration tests cover the
nullable column and the rollback insert, and the storage memory
records the decision.
AI-assisted-by: muse-spark-1.3-contributor
2026-11-13 08:05:15 +00:00
Alonso Torres
3e48ab96e4
🐛 Limit token propagation to the new instance on swap and instance (#13021)
* 🐛 Limit token propagation to the new instance on swap and instantiate

Switching a variant and adding a component instance re-applied every
active token to every shape of every page. In files with thousands of
token-styled shapes this took over 10 seconds, while only the new
instance needed the file's token values.

propagate-workspace-tokens takes an optional scope (page id and root
ids) that limits the propagation to those shapes and their descendants.
component-swap and instantiate-component pass the new instance's root.
Calls without a scope still cover the whole file.

Closes #13019

AI-assisted-by: claude-opus-5-5

* ✨ Add token tests for instantiate, swap and multi-page propagation

Cover instantiate-component and component-swap with components whose
stored values differ from the resolved tokens, checking that the new
instance gets the file's token values and other shapes are untouched.
Add a second page to the propagation tests to check that unscoped
propagation reaches other pages and scoped propagation does not.

AI-assisted-by: claude-opus-5-5
2026-11-13 09:02:44 +01:00
Andrey Antukh
335c6782a0 Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 08:33:31 +01:00
Andrey Antukh
7dcc0fde7a Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-13 08:33:01 +01:00
Andrey Antukh
a39dee6d79 Merge remote-tracking branch 'origin/main' into staging-2.19.x 2026-11-13 08:32:45 +01:00
Roshan Ramani
7516f1d163 📚 Fix the telemetry flag name in the configuration guide (#13032)
The guide told self-hosters to add enable-telemetries to PENPOT_FLAGS.
flags/parse turns that into :telemetries, which nothing reads; the
telemetry task checks :telemetry, so the documented flag never enables it.

AI-assisted-by: claude-opus-5-5
2026-11-13 08:32:29 +01:00
Roshan Ramani
e202612469
📚 Fix the telemetry flag name in the configuration guide (#13032)
The guide told self-hosters to add enable-telemetries to PENPOT_FLAGS.
flags/parse turns that into :telemetries, which nothing reads; the
telemetry task checks :telemetry, so the documented flag never enables it.

AI-assisted-by: claude-opus-5-5
2026-11-13 08:31:37 +01:00
penpot-renovate[bot]
c19f7a6e5d
⬆️ Update dhi.io/node image to v24.22.3 (#13031)
- dhi.io/node 24.22.3@ (docker/devenv/Dockerfile)
- dhi.io/node 24.22.3@ (.nvmrc)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.3-debian13@84adaf3 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.3-debian13-dev@aee95f1 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-13 08:30:10 +01:00
Andrey Antukh
5c45c8ddaf Merge remote-tracking branch 'origin/staging' into develop 2026-11-13 08:18:50 +01:00
Andrey Antukh
ad710ed143 📚 Document the squash-merge default and the no-squash opt-out
Squash merge appeared twice in the guide, as a review-process bullet and
in branch naming, with no reason and no way out. Add a Merging
subsection: one PR lands as one atomic commit, so the history stays
readable and git bisect never stops on an intermediate commit that fails
the tests or the linter. A contributor who keeps every commit
self-contained may ask for a regular merge in the first line of the PR
description.

Record the same contract in mem:workflow/creating-prs (merge strategy,
and the Merge line above the AI Note line) and in
mem:workflow/creating-commits (what a self-contained commit means when a
regular merge is requested).

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 07:10:24 +00:00
Andrey Antukh
9bac1d290b 📚 Clarify when to run the file_change partition procedure
The procedure straddles the upgrade to 2.20.0 and the document said
nothing about the order. The preparation (phases 0 and 1) goes before
the version update, the bulk and the delta passes after it, and only
the cutover needs a scheduled maintenance window. The subtitle claimed
the whole thing was online, which is not true.

The copy marks no longer travel as a session GUC with a date typed by
hand. The delta mark is seeded at the bulk with `now()`, and a CHECK
keeps it from ending up empty; the instant of the cutover is recorded
in the same table after the commit, so the checks of phase 5 read it
instead of comparing against a date the operator retypes in every
session.

The procedure was run end to end on a scratch database, with the 23
SQL blocks extracted from the document and the application writes
simulated between the phases.

AI-assisted-by: deepseek-v4.1-flash
2026-11-13 06:58:46 +00:00
deni-ing
d774e197d0
🐛 Make GitHub logo follow the theme on the login page (#12943)
The GitHub icon had a hardcoded white fill, so it was almost
invisible on the light theme. Use currentColor so it follows
the button text color.

Fixes #12938

Signed-off-by: deni-ing <242363192+deni-ing@users.noreply.github.com>
2026-11-12 18:26:57 +01:00
Luis de Dios
24a5fd2043
♻️ Refactor the share link dialog to the design system (#12965)
* ♻️ Split the share link dialog into smaller components

Move the dialog state into a content component and extract the
permissions section, the page selection, the select rows, the
collapsible panel and the destroy confirmation.

The confirmation now keeps the link to destroy instead of matching it by
flags, so deleting a link no longer depends on the options snapshot.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ♻️ Use the design system controls in the share link dialog

Replace the deprecated select, the raw text input and the hand-written
page checkboxes with the design system select, input field and checkbox
components, and drop the deprecated icons and the unused uuid parsing.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ✨ Add dialog semantics to the share link dialog

Give the dialog and the destroy confirmation proper dialog and
alertdialog roles and labels, cancel the confirmation with Escape, and
connect the disclosure button with its panel through aria-controls.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* ✨ Toggle the share dialog from the viewer header

Let the share button close the dialog when it is already open, report
the dialog state with aria-expanded and aria-pressed, and mark the
button with data-allow-click-modal so the closing click does not count
as an outside click.

Drop the click-outside guard on close; it is not needed once the button
can reopen the dialog.

AI-assisted-by: deepseek-v4.1-flash
(cherry picked from commit 3ce7f9d09765e4f6afc033db544aabca379c0ca4)

* 🐛 Move focus into the share dialog and restore it on close

Move focus into the dialog when it opens and into the Cancel button when
the delete confirmation opens, so Escape reaches the confirmation handler
instead of closing the whole dialog.

Restore focus to Destroy link when the confirmation closes and to the
Share button when the dialog closes.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Apply review follow-ups to the share link dialog

Use aria-haspopup instead of aria-pressed on the Share toggle, drop the
raw z-index, and express the small spacing values with px2rem. Restore
the collapsible pointer cursor and merge its two classes.

Memoize the permission options, expose aria-controls only while the
panel is open, and label the read-only URL field.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 17:53:39 +01:00
Danny Shirely
192e76781a
✨ Add export previews to Design and Inspect (#12059)
* ✨ Add export previews to Design and Inspect

Preview a selected shape's export presets in Design and Inspect, with
output dimensions and a shared persisted expansion preference.

Bound native previews to 240px and debounce subtree updates while keeping
the current image until its replacement is ready. Release replaced URLs
and cancel pending rendering when the preview closes.

Keep this change focused on previews and track native export fill
handling separately.

Closes #7561

AI-assisted-by: gpt-6
Signed-off-by: Danial Shirali <danialshirali16@gmail.com>

* 🌐 Add Spanish translations for the export preview

The export preview added four new keys only to en.po, so Spanish users
saw them in English.

AI-assisted-by: claude-opus-5-5

* 🌐 Improve the Spanish export preview labels

AI-assisted-by: claude-opus-5-5

---------

Signed-off-by: Danial Shirali <danialshirali16@gmail.com>
Co-authored-by: Eva Marco <evamarcod@gmail.com>
2026-11-12 17:23:46 +01:00
Andrey Antukh
ec1e2c9e1a Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-12 15:36:44 +01:00
Andrey Antukh
92c478b0e2 Merge remote-tracking branch 'origin/merge-staging' into staging 2026-11-12 15:32:33 +01:00
Andrey Antukh
dc38eaa843
🐛 Fix font upload crash and preserve font metadata (#11951)
* 🐛 Fix font upload crash and preserve variant metadata

opentype.js 2.0.0 returns no fontSubfamily name for fonts whose
Macintosh name table only holds custom records. That made
parse-font-weight/parse-font-style call re-seq on nil and crash the
upload.

A nil or blank variant is now treated as no match in
app.common.media, defaulting to weight 400 and style normal. The
variant parsing also accepts oblique as italic, and the new
parse-font-family helper derives the family from a filename by
stripping the known weight/style tokens, separating a style token
glued to a weight token ("Roboto-BoldItalic" -> "Roboto").
first-nonblank-string returns the first nonblank candidate, so a
blank preferredFamily/preferredSubfamily no longer shadows a valid
fontFamily/fontSubfamily.

app.main.data.fonts gains resolve-font-metadata and
prepare-font-entry. opentype.js values win per field; any missing
value falls back to the filename-derived family, weight and style.
The parsed and woff2 branches share the same helper, so metadata is
preserved instead of collapsing every face to weight 400, and
fonts with missing vertical-metric tables upload with no height
warning instead of failing.

process-upload reports per-file failures (unknown signature,
unparseable file, preparation error) with a single bad-font
notification instead of silently dropping them, and never errors
for per-file failures.

The upload now sends :variant-name to create-font-variant, which
accepts it as an optional param (ships in 2.20), so the foundry
name survives the upload.

Closes #10436

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: muse-spark-1.3-contributor
Signed-off-by: CHOI JUNSOO <junsoo1172@gmail.com>

* 🐛 Parse glued font style tokens in the font parsers

EvaMarco's review on #11951: parse-font-family and
parse-font-weight handled glued tokens like "BoldItalic", but
parse-font-style did not. For names such as
"opensans-bolditalic-webfont.woff2" the three parsers disagreed:
the italic face resolved to style normal and collided with the
bold-only file on the (family, weight, style) join key. One
binary silently overwrote the other during upload, losing one
font face without any warning.

A new private normalize-font-name helper is applied by all three
parsers. It splits a style token glued to a weight token, now
with a lookahead boundary so "LightItalic_v2" splits too, and
strips a trailing technical suffix (-VF or _v<number>).
parse-font-family, parse-font-weight and parse-font-style now
interpret the name consistently.

Tests cover the three cases from the review, the bold-only
control case, and an end-to-end process-upload check that a bold
and a bold-italic WOFF2 survive as two variants.

Relates to #10436 and PR #11951.

AI-assisted-by: inkling-2-small-free

* Remove duplicate frontend test registration

---------

Signed-off-by: CHOI JUNSOO <junsoo1172@gmail.com>
2026-11-12 15:29:30 +01:00
Marina López
5803f2594c
🐛 Show the organization in the new team form (#13023) 2026-11-12 15:11:43 +01:00
Marina López
029283b1f9
🐛 Hide standalone Enterprise plans for team owners (#13017) 2026-11-12 14:54:56 +01:00
Luis de Dios
185fef889b
🐛 Fix collapsing token set groups after reordering (#12671)
* 🐛 Fix collapsing token set groups after reordering

The handler captured the first render path, and the tree is keyed by
index, so after a reorder it toggled the path of another group.

Closes #12625

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Refresh double click handler of token set groups

Add the start edition callback to the hook deps so the handler does
not keep the initial closure.

AI-assisted-by: deepseek-v4.1-flash

* ✨ Add regression test for folding reordered token set groups

Reorder a group in the sets tree, fold it, and assert that only that
group collapses. The tree is keyed by index, so the test also proves
the collapse handler does not keep a stale path after a reorder.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Refresh double click handler of token set nodes

Add the start edition callback to the hook deps so the handler does not
keep the initial closure.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Key token set tree nodes by identity instead of index

React reuses a node by its key. Keyed by the tree index, a reordered
group moved the instance to another node and its handlers kept the
previous paths. Groups are now keyed by path and sets by id, so each
node keeps its own handlers.

The set being created has no persisted id yet, so the tree walk
generates a new one on every render. Keep one uuid for its whole
creation instead.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 13:49:43 +01:00
Miguel de Benito Delgado
ef725c0cc2
🔧 Enable checking commit messages from files (#12951)
Adds a new flag --message-file to check-commit

AI-assisted-by: deepseek-flash
2026-11-12 13:28:35 +01:00
Eva Marco
d829e47a04
🔧 Run the Playwright tests with a dark color scheme (#12989)
* 🔧 Run the Playwright tests with a dark color scheme

Since #12480 a profile without a theme follows the system one, and
headless Chrome reports a light scheme, so the e2e tests ran with the
light UI. Set colorScheme to dark in the shared Playwright options so
the tests use the default dark UI again. Tests that need another scheme
still set it with page.emulateMedia.

AI-assisted-by: claude-opus-5-5

* 🔧 Drop the dark color scheme overrides now set for every test

The shared Playwright options now set the dark color scheme, so the ds
project override and the emulateMedia call in the selection size badge
test do nothing.

AI-assisted-by: claude-opus-5-5
2026-11-12 13:27:52 +01:00
penpot-renovate[bot]
88e1b29e5c
⬆️ Update DHI base images (#13008)
- dhi.io/debian-base trixie-debian13@c12f605 (docker/imagemagick/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@a5fc6ab (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@fb9c8a5 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@8a0d16b (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-12 13:10:05 +01:00
andrés gonzález
9d42ed29ce
📚 Document how to order variant properties and values (#12996)
Users ask how to choose the order of values in the variant property
dropdown. The values follow the order of the variants in the Layers
panel, and this was not documented. Also document that properties can
be reordered by dragging them in the Design tab.

AI-assisted-by: claude-opus-5-5
2026-11-12 13:04:13 +01:00
JiMyung Lee
61154e6bfe
✨ Keep the swap component search when changing library (#12873)
Changing the library in the Swap component menu cleared the search
field, so checking several connected libraries for the same component
meant typing the name again for each one. Keep the term and reset only
the library id and the group path, so the new library is shown already
filtered.

Closes #12335

AI-assisted-by: claude-opus-5-5

Signed-off-by: JiMyung Lee <lee.ji.myung@gmail.com>
2026-11-12 12:45:36 +01:00
Marina López
b5eab6a949
💄 Update promotional banner (#12944) 2026-11-12 12:35:13 +01:00
Belén Albeza
86a8ac34cd
🎉 Add basic bullet lists to the v3 text editor (#12720)
* 🎉 Add UI for toggling bullet lists (under config flag)

* 🎉 Store list-style-type as a paragraph attribute

The bullet list toggle now saves list-style-type ("none" or "disc") on
the selected paragraphs and reads it back. A paragraph without the attr
counts as "none", so a bullet paragraph next to an older one shows as
mixed.

WASM keeps the value on each paragraph: the paragraph bytes grow from
12 to 16, Enter keeps it on the new paragraph, and the v3 editor reports
it in the current styles, so the sidebar shows it while editing too.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Draw bullets for list paragraphs

A paragraph with list-style-type "disc" now gets a solid round bullet,
sized as Chrome draws a CSS disc marker (measured with Source Sans Pro):
a radius of 0.16 font sizes, centered 0.32 font sizes above the first
baseline, with a gap of 0.33 font sizes before the text. The bullet
hangs outside the text box, before its start edge (after it for RTL),
like CSS list-style-position: outside, so the text layout, caret and
selection stay as they were.

The bullet takes the paint of its line's first style run, so fills,
strokes, shadows and masks apply to it as they do to the glyphs. An
empty item, which has no run, borrows the previous item's paint. The
cached paint, the text extrect and text-to-path share one walker over
the paragraphs, so they always agree: the extrect grows to hold the
bullets, like shadows and blur, and flattening a text keeps them.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Show the list style in Inspect

Text paragraphs with list-style-type "disc" now show a "List Style" row
in the Inspect Styles and Computed views, after Text Transform. Its copy
action gives "list-style-type: disc;". Text without a list shows no row.
The list item check lives in common as list-paragraph?, so every place
that needs it shares one rule.

Refs #12630

AI-assisted-by: claude-opus-5-5

* ✨ Generate list HTML and CSS for bullet paragraphs

In the Inspect code, neighbouring paragraphs with list-style-type
"disc" become one <ul> with an <li> each, keeping the paragraph's class
and dir. Other paragraphs stay <p>. Only code mode groups them; text
rendering still uses <p>.

The CSS sets list-style-type on the items and resets the list's margin
and padding, so the browser hangs the bullet before the text as the
canvas does. Paragraph rules set font-size: 0, so each item also gets a
::marker rule with its first span's font size and text color. A
::marker can't paint a gradient, so a gradient fill gives its first
stop.

Refs #12630

AI-assisted-by: claude-opus-5-5
2026-11-12 11:50:05 +01:00
Belén Albeza
00e77a0f53
🐛 Fix slow variant switch with the WASM renderer (#12985)
Changing a component inside a big main component made the WASM renderer
draw that component's thumbnail at full size on the main thread, before
the viewport redraw. On the Fauna file the 2140x1642 thumbnail took about
1.2 s to render and PNG-encode, so the switch showed up late.

- Fit the thumbnail's longest side between 200 and 512 px. Small
  components still scale up; big ones now scale down.
- Draw the thumbnail only on a frame with no viewport render pending

Closes #12968

AI-assisted-by: claude-opus-5-5
2026-11-12 11:49:01 +01:00
Andrés Moya
2c86db80bd 🔧 Unify migration tests 2026-11-12 11:41:27 +01:00
andrés gonzález
1eccb3b4bb
✨ Track library token usage in analytics events (#12958)
Add a from-library property to the apply-tokens event, true when the
file takes its tokens from a linked library.

Emit a link-file-to-library-tokens event with tokens-count when the
linked library has tokens, mirroring link-file-to-library-variants.
Both events share one get-library-usage call.

Closes #12915

AI-assisted-by: claude-opus-5-5
2026-11-12 11:41:11 +01:00
Miguel de Benito Delgado
756541939a
🔧 Add renderer benchmark runner (#12756)
Add a CLI under frontend/performance/benches/render_wasm that builds the
renderer, runs seeded scenes (rects, plus, paths, masks) in headless
Chromium and saves per-case timings.

- build, run: prepare or reuse builds; measure cases with warmups,
  repetitions, deadlines
- ab: build and compare Cargo feature configurations
- compare, merge: median changes with bootstrap CIs between saved runs,
  guarded by scene, build and settings checks
- purge: delete managed builds, runs or comparisons
- GPU runs through `run-devenv --gpu`

Lint, format and pnpm scripts now cover performance/ and its tests.

AI-assisted-by: gpt-6.1-sol, deepseek-flash, muse-spark, glm-5.3, gpt6-astra, claude-5.5-sonnet, claude-haiku-5.5
2026-11-12 11:02:34 +01:00
Andrey Antukh
8b9558b380
♻️ Reduce and speed up the frontend e2e token suite (#13001)
* ✨ Add unit tests for the token name schema

Cover make-token-name-schema, the schema that rejects a token name whose
path clashes with an existing token or group. The lower-level
token-name-path-exists? already had a test; the schema that composes it
did not.

The cases pin an exact duplicate, a token inside a token, a token name
that collides with a group, an existing child path, valid siblings, the
fail-closed nil tree, invalid syntax and the length limit. This is the
unit-test base for reducing the 15 group-name-clash e2e cases to two.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Remove duplicate token tests and fix the spread step

The three top-level tests duplicated the ones inside the creation
describe; only the indentation differed. Keep the originals.

The shadow token test labelled "Valid spread" filled the blur field, so
the token under test ended with blur 0 and the spread never changed.
Fill spread instead and expect the referenced token to resolve with
Blur: 4.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Reduce the token group-name-clash e2e matrix to two tests

The fifteen cases repeated the same setup and only changed the token
type and value. The rule is now covered by the token name schema unit
test, so keep two representatives: thirteen simple types in one test
with a test.step each, and the two composite forms (Typography and
Shadow) in a second test.

The helper no longer re-runs the file setup, so each test loads the
page once: 15 page loads become 2 and the block drops from 433 s to
about 21 s.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Cover the numeric token creation types in one test

The nine creation tests repeated the same validation ladder and only
changed the token label and placeholder. Run the ladder once in the
border radius step, keep the reference token creation, and create the
other eight types with a test.step each.

The new helpers split the flow: open the form, assert the ladder, save
a token. The e2e drops from nine page loads to one.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Reduce the remapping e2e spec to three tests

Keep one remapping flow per token shape: box shadow with alias
references, border radius with a value change, and a group rename with
undo. The two cancel paths lose their own page loads and become steps
inside the shadow test: close keeps the original name, and "don't
remap" renames the token and breaks the reference.

The deleted typography and shadow applied-shape cases exercise the same
data path, which the remapping unit tests already cover.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Shrink the tooltip and numeric-revert e2e tests

The tooltip test asserted five message shapes over eleven layer
selections. Keep the three that have no unit test: a deleted token, a
reference outside the active sets, and a token in an inactive set. Each
is one step with a pill label and its tooltip text.

Three of its five messages were already covered by the token pill unit
test, including both resolved-value cases.

Keep one numeric-revert test and drop the fixed sleeps: await the value
with toHaveValue instead. Division by zero and the min/max clamp are
unit-tested in the numeric input tests.

AI-assisted-by: deepseek-v4.1-flash
2026-11-12 10:58:52 +01:00
Andrés Moya
a35dd49b6a Merge branch 'staging-2.19.x' into staging 2026-11-12 10:36:04 +01:00
makesomethingshit
dfc7919f7b
🐛 Preserve HSL values while editing lightness (#12629)
* 🐛 Preserve HSL values while editing lightness

Keep HSL editing components through black, white, and grayscale.
Use the same draft for numeric inputs, sliders, and gradient previews.
Strip the draft before emitting color data and reset it on other edits.

Closes #12615

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
AI-assisted-by: GPT-6

* 🐛 Keep HSL drafts consistent across gradient and ramp edits

Match RGB components to HEX so gradient echoes preserve HSL drafts.
Discard stale HSL values when editing Ramp hue, saturation or value,
while preserving them for opacity-only changes.
Cover grayscale gradient echoes and saturation restoration.

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
AI-assisted-by: GPT-6

---------

Signed-off-by: makesomethingshit <junsoo1172@gmail.com>
2026-11-12 10:11:56 +01:00
Andrey Antukh
64a51e86fd
🐛 Fix files blocked by dangling child references in migrations (#12936) (#12979)
The 0033-normalize-main-instance-names migration walks the shape tree
following the :shapes vectors. When a parent listed a child id missing
from the container :objects map (a dangling reference), the traversal
stored the nil child into :objects, creating a nil key that made
check-file-data reject the whole file: get-file answered 500 "invalid
file data" and the file could not be opened or migrated further.

Fix it in two parts:

- 0033 returns the container untouched when a listed child does not
  exist, so the traversal never stores a nil shape.
- The append-only migration 0035-fix-dangling-children prunes from
  every :shapes vector, in pages and components, the ids absent from
  the owning container :objects map, logging each fix at trace level.
  It mirrors 0032-fix-children-not-in-parent, which repairs the
  opposite inconsistency.

Both parts are needed: 0033 runs before the repair migration, so
without the guard the same file fails again in the same migrate call.

Add regression tests for the traversal guard and for the repair
migration, and record in mem:clojure/logging the convention that
:hints start lowercase.

Closes #12936

AI-assisted-by: qwen3.8-flash
2026-11-12 09:41:49 +01:00
Marina López
df0092fc60
🐛 Enforce sso before creating organization teams (#12980) 2026-11-12 08:35:32 +01:00
Andrey Antukh
75b55b567f 📚 Document test database and valkey env discipline
Agents must respect the PENPOT_TEST_* variables already set in the
shell and not override them with values copied from CI logs (the CI
endpoints only exist inside the CI docker network) unless the user
asks otherwise, such as a temporary database.

The shared test database may carry migrations applied by newer
versions; when that blocks a test run, dropping and recreating only
the database from PENPOT_TEST_DATABASE_URI is explicitly permitted.

AI-assisted-by: inkling-2-small-free
2026-11-12 06:27:44 +00:00
Andrey Antukh
12d73d0f63 🐛 Fix flaky assertion in climit waiting-count test
The test expected exactly one publication of the :rpc-climit-waiting
gauge inside the with-redefs window. The republisher daemon thread,
started at component init with no delay and rescheduling every 15 s,
publishes the same counter and can land inside that window, so the
recorded sequence could be [0 0] and the assertion failed with
actual [0 +0] (Backend Tests job in #12979).

Assert on the last published value instead of the whole sequence,
the contract waiting-last documents and the sibling tests use.

AI-assisted-by: inkling-2-small-free
2026-11-12 06:27:40 +00:00
Andrey Antukh
e57d358c53 📎 Update .gitignore 2026-11-12 07:09:42 +01:00
Pablo Alba
34f0d12b99
🐛 Ignore float noise when syncing moved library components (#12976) 2026-11-11 17:09:33 +01:00
penpot-renovate[bot]
b438676d6c
⬆️ Update dhi.io/node image to f6ba5e2 (#12975)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@f6ba5e2 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-11 16:55:19 +01:00
andrés gonzález
0d05cfaf5e
✨ Add the file to plugin and MCP audit events (#12972)
* ✨ Add the file and MCP session to plugin audit events

Events sent by plugins and the MCP plugin only carried their origin, so
the multi-file and dashboard MCP workflows could not be measured, nor
duplicated events from several open tabs spotted.

The context of se/event and se/add-event now carries the file-id of the
file the action ran in (absent on the dashboard) and, for the MCP, the
mcp-session-id of the tab connection that ran it.

Closes #12902

AI-assisted-by: claude-opus-5-5

* ♻️ Drop the MCP session id from plugin audit events

Every audit event already carries the session-id of the tab in its
context, and the MCP session id is only a hash of it, so it adds
nothing to group events by tab.

AI-assisted-by: claude-opus-5-5

* 🐛 Use a reified event in the system events test

ptk/data-event does not implement IWithMeta, so add-event failed on it.
Plugin actions emit reified events, which carry metadata.

AI-assisted-by: claude-opus-5-5
2026-11-11 16:18:15 +01:00
andrés gonzález
ca616a39ed
🐛 Send the plugin origin with penpotMgmt dashboard actions (#12971)
Dashboard actions made by a plugin or the MCP through penpotMgmt reached
the backend audit log without event origin, so they looked like actions
made by hand in the dashboard. The backend reads the origin from the
x-event-origin header, which rp/cmd! only sends when the params carry
::ev/origin in their meta.

Tag the params of every penpotMgmt mutation with the same meta that
se/add-event sets (mcp or plugin). The rename, share and team update
events now keep that meta on the RPC params, so setters send it too, and
set-file-variants-shared takes the caller origin instead of a fixed
dashboard one.

Closes #12901

AI-assisted-by: claude-opus-5-5
2026-11-11 16:02:29 +01:00
Pablo Alba
bb672a3723 🐛 Block grid track duplication and paste in component copies 2026-11-11 15:32:10 +01:00
Pablo Alba
3eebe92606 🐛 Keep the swap slot when duplicating a swapped nested copy 2026-11-11 15:32:10 +01:00
Gabriel Ribeiro
7e9a3c3c28 📚 Fix typos in technical and user guides
Fix five misspelled words in the docs: "notifiy" and "occured" in the
integration guide, "enviroment" in the developer guide index, "succesful"
in the authentication subsystem page and "achive" in the layers page.

AI-assisted-by: claude-opus-5-5
Signed-off-by: Gabriel Ribeiro <267068126+gabrielribeiro3105@users.noreply.github.com>
2026-11-11 15:24:13 +01:00
andrés gonzález
a301c2b43f
✨ Keep board names visibility after reloading the workspace (#12872)
Hiding board names from the View menu reset on every reload, while
rulers and comments kept their state. Add the flag to the layout flags
persisted in local storage so it behaves the same way.

Closes #12871

AI-assisted-by: claude-opus-5-5
2026-11-11 14:52:35 +01:00
andrés gonzález
54b1eb5d5f
✨ Add shape count to the apply-tokens audit event (#12970)
The apply-tokens event fires once per apply call, so a plugin or MCP
script that loops over shapes sends one event per shape while the UI
sends one per selection. Add a shape-count prop with the number of
shapes the token was applied to, so analytics can compare origins in
the same unit.

Closes #12900

AI-assisted-by: claude-opus-5-5
2026-11-11 14:43:11 +01:00
Alonso Torres
180b1ef7cb
🐛 Fix default theme for penpot plugins (#12982) 2026-11-11 14:16:20 +01:00
Andrey Antukh
bcac14a4a8
🐛 Skip circular library links on binfile import (#12062)
* 🐛 Skip circular library links on binfile import

File import recreated circular library references that the
link-file-to-library command already rejects. The v3 importer
now skips manifest relations that would close a cycle, or link
a file to itself, and logs each skipped link. Closes #12052.

AI-assisted-by: muse-spark-1.3-contributor-free

🐛 Address review findings on circular import guard

F1 extracts the cycle check into bfc/library-link-status and reuses
it in link-file-to-library and v3 import, keeping UI errors intact.

F3 orders get-files-rels and manifest relations so the first edge
wins deterministically, and documents it.

F2 adds a three-node transitive cycle test; F4 asserts file_library_sync
has no orphan rows for skipped links and matches rels otherwise.

AI-assisted-by: muse-spark-1.3-contributor

🐛 Pass team-id in binfile v3 circular library tests

The four circular library relation tests (import-skips-circular,
import-skips-transitive-cycle, import-skips-self-referencing,
import-keeps-acyclic) called v3/import-files! without ::bfc/team-id.
resolve-and-link-libraries asserts (uuid? team-id), so with
enable-backend-asserts the tests errored instead of running.

Add (assoc ::bfc/team-id (:default-team-id profile)) to each test,
mirroring the production RPC import path and the earlier fix in
a36193e54b for the other binfile v3 import tests.

Verified: 1338 tests, 6910 assertions, 0 failures.

AI-assisted-by: mistral-large-4

* 🐛 Guard the v1 import and restore the RPC check order

The version 1 importer recreated the circular library references that
the version 3 importer already skips, and a user reaches it through
`import-binfile` and the import job. It now asks
`bfc/library-link-status` before inserting each relation, so a self
link or an edge that would close a cycle is skipped and logged, and it
processes the relations in sorted order like v3 does. Closes #12052.

The version 2 importer stays out on purpose: only the developer REPL
reaches it, and a note in the code records the gap.

`link-file-to-library` keeps the shared helper but restores the order
of the base commit: the self link guard, then the edition permission
and team ownership checks, and only then the cycle check. A caller
without permission gets `not-found` again instead of the circular
error, and a rejected request no longer runs the recursive library
query first.

Tests: a v1 import that recreates a cycle, the self link error of the
command, and the permission check running before the cycle check.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Tolerate repeated file entries in the v1 binfile import

The v1 exporter unioned the requested files with their libraries
without deduplicating, so a package exported from a file inside a
library cycle (see #12052) lists that file twice in the metadata. The
v1 import created one file per entry, so the second insert hit the
primary key of `file` and aborted the whole import with "file already
exists".

The importer now reads the pair of objects of every entry to keep the
stream in sync but imports the first occurrence only, warns about the
repeats, and answers with one id per imported file.

The v1 export keeps its behavior: it has no caller in the product, and
only the tests and the developer REPL use it.

Tests: a package with a repeated entry, built with the v1 export from
a cyclic library pair, imports the file once and leaves no duplicated
rows.

AI-assisted-by: deepseek-v4.1-flash
2026-11-11 13:02:57 +01:00
andrés gonzález
0efa550a99
✨ Improve the changelog flow: dev-only bugs and feature sections (#12981)
* ✨ Keep bugs that never reached production out of the changelog

The team rarely adds `no changelog` to bugs found while building a
feature of the same release, so they end up in the changelog and make
Penpot look less stable than it is.

Add `changelog.py dev-only-bugs`. Bugs whose parent is a Feature or
Enhancement of the same milestone are sure: `--apply` labels them
`no changelog` on GitHub. Bugs under an epic of the milestone, with the
`regression` label, or whose version field names only hourly, develop,
staging, PRE or this milestone, are candidates for a human to confirm;
the anomaly report lists them as warnings. `gh.py issue` now returns
body and parent. The flow memory documents it as step 3.

Closes #12747

AI-assisted-by: claude-opus-5-5

* ✨ Split features and enhancements in the changelog flow

Features were listed together with enhancements under "✨ New
features & Enhancements", so they did not stand out. Map the Feature
issue type to its own "🎉 Features" section, keep Enhancement under
"✨ Enhancements", rename the bug section to "🐛 Bugs" and
set the order: boom, rocket, lock, tada, sparkles, bug.

The new layout applies from 2.20.0: 2.19.0 and 2.19.1 are already out
with the old one, so 2.19.x patches keep it.

Closes #12741

AI-assisted-by: claude-opus-5-5
2026-11-11 13:00:06 +01:00
Juanfran
88339e41a5 🐛 Build plugin scripts before Angular copies assets (#12815)
Generate plugin.js before Angular builds each of the eight Angular plugins.
The previous order wrote the script to src/assets after Angular had copied
assets to dist, so clean deploys published manifests without their code.

Verified clean builds include all declared manifest assets for all eight
plugins. Plugin lint, format checks and all 101 tests pass.

AI-assisted-by: gpt-6.1-sol
2026-11-11 12:51:24 +01:00
Alonso Torres
1599e096da
🐛 Fix MCP session discovery for plugins connected from the dashboard (#12865)
* 🐛 Fix MCP session discovery for plugins connected from the dashboard

With Redis-backed routing, execute_code without a sessionId timed out
with SessionDiscoveryError when the plugin ran on the dashboard. The
discovery reply schema required a string fileId and fileName, so replies
listing a session with no open file were dropped, and discovery waited
for its timeout.

Share one session schema between the plugin handshake and discovery
replies. It accepts a null file and keeps workspaceState. An unreadable
reply now fails discovery at once with an error that names the problem
and says a retry will not help. Timeout and Redis failures state their
reason.

Closes #12860

* 🐛 Stop blaming server load when MCP discovery fails on shutdown

Closing the dispatcher rejected pending discoveries with the default advice,
which blamed temporary server load. Shutdown now gives its own advice.

The dispatcher tests now cover each discovery failure path: an unreadable
reply alongside a valid one rejects with InvalidDiscoveryResponseError and
does not dispatch, a late unreadable reply leaves a settled discovery
unchanged, and timeout, Redis failure and shutdown each state their reason.
2026-11-11 12:10:34 +01:00
Alonso Torres
adda9ad908
✨ Restrict global plugins to MCP and localhost (#12949)
Plugins with global scope or a management permission (allow:global,
manage:projects, manage:teams, manage:delete) keep running outside a file
and can drive the management API. That feature is still in progress, so
only the bundled MCP plugin and plugins served from localhost, 127.0.0.1
or [::1] get it. The host check ignores letter case and a trailing dot.

The check runs at three points:

- parse-manifest throws global-not-allowed, so the plugin manager and the
  dashboard install link refuse remote global plugins with a specific
  message. Opening an installed plugin fetches its manifest again, so
  plugins stored before this change are refused too.
- start-plugin! and load-plugin! refuse to run such a plugin and warn the
  user; this also covers bundled plugins without a URL.
- has-permission? withholds the global permissions from stored remote
  plugins, so the management API rejects them.

AI-assisted-by: claude-opus-5-5
2026-11-11 10:28:11 +01:00
andrés gonzález
37f4556b0b
📚 Document MCP sign-in with OAuth (#12950)
Explain how MCP clients with OAuth support connect to the remote MCP
server with a URL that has no key: the user signs in to Penpot, allows
access, and the client receives the MCP key. Keep the key-in-URL path
for clients and instances without sign-in.

Describe the approval page, what allowing access changes, and how
regenerating or expired keys affect signed-in clients. Document
PENPOT_MCP_AUTH_URI for self-hosted instances.

Relates to #12034

AI-assisted-by: claude-opus-5-5
2026-11-11 09:54:27 +01:00
Alonso Torres
f89879cca1
🐛 Fix plugin component lookups for missing and deleted components (#12755)
* 🐛 Fix plugin component lookups for missing and deleted components

Shape.component() threw "Assert failed: (uuid? id)" on a copy whose
library is not available, because the head lookup returns a pair with a
nil component and the proxy was built anyway. It now returns null.

Shape.componentRoot() built its proxy with the component's file and main
page, so a copy on another page or file than its main got a shape that
pointed at nothing, and a copy of a missing library threw. It now
resolves the root on the shape's own file and page, like componentHead().

LibraryComponent.mainInstance() returned a proxy for the root kept in a
deleted component's objects, which is on no page, so every field read
empty and parent threw. It now returns null for a deleted component.

* 📚 Fix the variant example in the MCP instructions

The createVariantContainer example called penpot.currentPage.findAllShapes
and Shape.isMainComponent, which do not exist, so running it as written
failed. Use penpotUtils.findShapes with isComponentMainInstance instead.
2026-11-11 09:39:04 +01:00
Pablo Alba
c466c289de 🐛 Fix integrity error copying variants across files 2026-11-11 09:01:09 +01:00
Andrey Antukh
34c702bc80
✨ Add table partitioning for file-change (#12417)
* ✨ Partition file_change by file_id

Partition `file_change` by `HASH (file_id)` in 16 partitions, with the
same bounds as `file_data`, so a join between them is resolved partition
by partition. The primary key becomes `(file_id, id)`, which a
partitioned table needs, and every access filters by `file_id`.

The new table carries a `kind` column (1 snapshot, 2 revision) as a
reserve: the queries still tell them apart by the label, and `kind`
takes over when they are rewritten. The payload of a snapshot stays in
`file_data`; the `data` column is kept for reading old `legacy-db`
payloads.

The four snapshot commands require `file-id`, because it prunes the
partition. The objects GC locks and deletes by the pair, in batches. The
old table is kept as `file_change_legacy` as the way back.

The automatic migration refuses to run above 10 GiB or 2M rows and
points to the runbook; big instances partition online with that runbook,
which copies by id, follows the `updated_at` watermark in delta passes
and checks that no row of the source is missing from the target.

Closes #12416

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Address the review of the file_change partition

The production runbook promised a rollback it did not have: the section
was a sketch, and its statements could write to the table about to be
dropped. The rollback is removed, and so is every comment that described
it. `file_change_legacy` stays as a frozen copy of the previous layout.

The cutover check now aborts the transaction when a row of the old table
is missing in the new one, instead of leaving the decision to the
operator.

The five secondary indexes of the new table are built after the last
delta pass: the bulk and the delta passes only need the primary key, so
the batches no longer maintain them.

The copy of 0158 is now replayed in a test against the pre-0158 shape
with a labelled and an unlabelled row, plus an empty instance that must
leave no legacy table behind. The updated_at assertion of the snapshot
mutations is bounded by the test clock, and the repair revision stops
writing the `features` of the file, like the normal save.

AI-assisted-by: deepseek-v4.1-flash

* 📎 Fix linter issues

* 🐛 Fix stale helper call in rpc_file_snapshot_test

The suite does not load: `th/run-pending-tasks!` does not exist in
backend-tests.helpers. The pending jobs runner is `run-pending-jobs`
(no arguments), the name every other call site of the suite uses.

AI-assisted-by: inkling-2-small-free

* 📚 Align the partition change with the 2.20.0 milestone

2.19.0 is already shipped; this work targets 2.20.0. The operations
file of the partition procedure carries the 2.20.0 prefix, the
references that point at it (the migration, the runbook table and the
memory) follow, and the RPC doc/changes tags of the snapshot commands
record 2.20 as the release that makes `file-id` required.

AI-assisted-by: inkling-2-small-free

* 🐛 Fix review findings on the partition rebase

The 0157 legacy test was missed by the rebase and inserted file_change
rows without the new mandatory `kind` column, so the backend suite fails
against the migrated schema. The inserts now carry `:kind`
(snapshot-kind for the labelled rows, revision-kind for the rest).

The post-cutover verification block of the runbook terminates the
first query with a semicolon so the block pastes into psql without a
syntax error.

The guard message of the migration names the step
(0158-partition-file-change) instead of "0158", ambiguous since two
migrations share that number.

Closes #12416

AI-assisted-by: inkling-2-small-free

* 🐛 Fix the stale label in the snapshot restore test

The rename block of snapshots-crud runs before the restore block,
so the restored version is already named "label1-updated" when the
restore publishes its message. The command names the version by the
label the row has at that moment, not by the one it had when it was
created, so the test was asserting a stale value.

Assert the current label, and say why in the comment.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Split the version event tests into one test per command

The workspace version events and the plugin proxy send the file-id
of the open file on every snapshot command, because the command
needs it to prune the partition of file_change. Both tests checked
that with a doseq and t/testing blocks, so a failure named the whole
group and not the command that broke.

Give each command its own test. The workspace ones share a helper
that runs the event and returns the command and params it sends; the
plugin ones keep their own mock of rp/cmd!.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 23:27:17 +01:00
Filip Sajdak
58b349fb77 🔧 Keep the CI expect timeout in the default and ds e2e projects (#12922)
The Playwright config gives the retrying expects 40 seconds on CI and 5
seconds elsewhere, but the default and ds projects set their own expect
block for the screenshot tolerance, and a project block replaces the
top-level one instead of merging with it. On CI those projects have run
with Playwright's 5 second default.

That is too short for the first dashboard load on a busy runner, where the
dashboard module is fetched lazily: specs that wait for the dashboard
heading time out and pass on the retry (flaky), or fail after the
retries. In the shard of one recent run that was 7 flaky and 4 failed
specs out of 112, and develop itself failed the same families of specs on
several runs the same day.

Name the timeout once and repeat it in both projects. A probe config that
imports this one and polls a condition that never holds reports 40000 ms
for both projects with this change, and 5000 ms without it. The
render-wasm project already repeats its own value.

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 23:00:14 +01:00
Andrey Antukh
671ff4a05f Merge remote-tracking branch 'origin/staging' into develop 2026-11-10 20:06:46 +01:00
Andrey Antukh
8172059b2e Merge branch 'staging-2.19.x' into staging 2026-11-10 19:53:23 +01:00
Andrey Antukh
5e5722dfe7 📚 Update changelog 2026-11-10 19:24:39 +01:00
Andrey Antukh
a58ace4df4 Merge branch 'staging-2.19.x' 2026-11-10 19:18:41 +01:00
Andrey Antukh
d5c4bfb234 ✨ Add security fixes section to the changelog flow
The 🔒 section is generated from advisories instead of being written
by hand: an advisory becomes an entry when its patched_versions include
the release version, so unpublished (triage/draft) advisories are valid.

- gh.py advisories --patched-version filters on the exact token of
  vulnerabilities[*].patched_versions; the list output keeps the summary
  schema.
- changelog.py security renders the entries sorted by GHSA id, links the
  merged milestone PRs labelled "pr pt" that reference the GHSA id and
  warns when a reference has no matching patched version. The subsection
  follows 🚀 (or 💥, or the version header) and is removed
  when the version has no advisories.
- Tests cover the filter, the entry rendering, PR discovery and the
  subsection placement.
- Memories and the update-changelog skill document the contract.
- CHANGES.md: add 2.19.1 with its security fixes and the remaining bug
  fix.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 18:18:00 +00:00
Andrey Antukh
e4fcc663c0 Merge remote-tracking branch 'origin/staging-2.19.x' into staging 2026-11-10 18:09:07 +01:00
Andrey Antukh
88b09efd87 📚 Note breaking changes must not be hard-wrapped
The changelog generator copies the note lines verbatim, so a note
hard-wrapped at 80 columns ends up wrapped in CHANGES.md. Record
that the breaking-change note must keep one line per paragraph or
bullet, in both the issue and changelog workflow memories.

AI-assisted-by: inkling-2-small-free
2026-11-10 16:58:21 +00:00
Yamila Moreno
a3f50887b3 📚 Improve 2.19 documentation regarding admin-console
Signed-off-by: Yamila Moreno <yamila.moreno@kaleidos.net>
2026-11-10 16:13:06 +01:00
Andrey Antukh
1db1487c7f
🐛 Require thread access when marking threads as read (#12894)
* 🐛 Require thread access when marking threads as read

mark-all-threads-as-read took any list of thread ids and wrote the read
status of the caller on each of them, without checking that the caller
can access the file that owns the thread. It was the only comment
mutation that did not check permissions, and it took the ids as an
unbounded array, so a single request could keep a database connection
and a transaction open for a long time.

The rpc now resolves the threads in one query and checks the comment
permissions of every distinct file, answering not-found when a thread
does not exist or is not accessible. Both cases answer the same error
on purpose, so the response cannot be used to tell existing threads
apart.

The status rows are now written with a single insert, keeping the
modified-at date one second in the future that makes the thread count
as read. Repeated ids are deduped, the command returns nil again like
the sibling commands with no result, and the schema limits the number
of threads per call to 500 while the dashboard sends the ids in batches
of that size.

Tests cover access rejection, unknown threads, mixed batches that write
nothing, repeated ids, empty lists, marking twice, and the frontend
batch split.

Relates to GHSA-j4vc-cmf6-jhr7

AI-assisted-by: minimax-m3.1-flash

* 🐛 Refresh the unread list when a batch fails

The dashboard sends the thread ids in batches of 500 and each batch
commits on the server. A failure in a later batch left the unread list
stale: the re-read only ran when every batch succeeded, so the threads
already marked as read stayed visible as unread.

The re-read now runs in both paths. On failure the event is emitted
before the error is propagated, so the error toast still shows and the
success toast stays limited to a full success.

The backend also answers an empty list without opening a transaction,
since there is nothing to check and nothing to write.

Relates to GHSA-j4vc-cmf6-jhr7

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:54:11 +01:00
Andrey Antukh
8510daef52
🐛 Restrict the team invitation token and list to admins (#12892)
* 🐛 Restrict the team invitation token and list to admins

The token of a team invitation is a bearer credential: whoever holds it
accepts the invitation with the role of the invitation row. The RPC that
mints it only required edition permissions, while the rest of the
invitation commands require admin, so a team editor could get the token of
any pending invitation, including the ones that grant the owner role.

The token now requires admin, and the token of an owner invitation is
restricted to the team owner: an admin cannot create nor promote an owner
invitation, so it cannot keep its token either. It reuses the error code
already used for that rule.

Minting the token now leaves an audit event named as an action, carrying
the granted role and the id of the invitation, so the extraction does not
go unnoticed.

The list of pending invitations is admin only as well: it carries the
email of the invited person and the role offered, and the dashboard was
painting those emails to any member. The frontend catches the permission
error and shows the permission notice instead, and the copy action is not
offered for an owner invitation, which the backend rejects for an admin.

The five invitation commands now answer with the same permission error,
so they speak one language.

Relates to GHSA-4mmc-j8rh-74pf

AI-assisted-by: minimax-m3.1-flash

* 🐛 Clear the cached invitations and gate the list on team admin

The pending invitations are a team admin only list. The dashboard still
offered the entry to every member and keyed the section on the invite
policy of the organization, so a team admin blocked by that policy saw
the no-permission notice although the backend lets them list.

- `fetch-invitations` answers a denied refetch with an
  `invitations-fetched` event carrying an empty list, so a demoted admin
  does not keep the cached emails and the header no longer paints the
  invite button from a stale list.
- The header tabs and the team options dropdown hide the invitations
  entry without `:is-admin`, which already includes the team owner.
- `invitations-section-content` keys the section on `:is-admin`, and the
  invite policy only gates the invite actions: resending keeps the
  `can-invite?` check, deleting and row selection only need team admin.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:53:43 +01:00
Andrey Antukh
0420db1ca9
🐛 Apply the share-link page scope to the comment queries and writes (#12888)
* 🐛 Apply the share-link page scope to the comment queries and writes

A share link can be restricted to a subset of the pages of a file, and
the comment queries enforced that scope. But the RPC that resolves the
profiles of the commenters authorized the request with
check-comment-permissions! and dropped the returned perms, so its query
filtered only by file id. The holder of a page scoped link got the id,
email, fullname and photo of every profile that commented on any page
of the file, including the pages the owner left out of the link.

The query now takes the pages of the share link and only selects the
authors of the threads on them. The profile of the caller is still
added, so the viewer keeps its own avatar.

The scope was missing as well on the comment writes that take a page id
or a thread id: create-comment-thread accepted a page outside the link,
create-comment accepted a thread of an unshared page, and
update-comment-thread, update-comment-thread-position and
update-comment-thread-frame accepted a thread of an unshared page. They
now raise not-found, the same way the comment queries already did.
Replying to a thread of an unshared page also sent the emails to the
thread participants.

The page scope rule now lives in one place: share-link-pages returns
the pages granted by a share link and check-page-scope raises not-found
when a page is outside of it, so get-comment-threads reuses the helper
instead of checking the pages inline.

The docstring of get-profiles-for-file-comments said that it returns
the participants of all the comment threads of the file, which is not
true anymore for a share link. The docs are public, so the text and
the changes list are updated.

The tests of the share link page scope are hardened: the setup asserts
that the comment threads are created, the membership case has a second
commenter so it can no longer pass for the wrong reason, and the link
without pages, the allowed paths of the thread updates and the
who-comment=team denial are covered.

Relates to GHSA-89cq-3wv7-987r

AI-assisted-by: minimax-m3.1-flash

* 🐛 Enforce the share-link page scope on the remaining comment writes

Check the page scope of the share link before mutating in the four
comment writes that still skipped it: update-comment-thread-status,
update-comment, delete-comment-thread and delete-comment. The check
runs before the ownership checks, so an out-of-scope write fails with
not-found and the response does not reveal that the comment or thread
exists. Load the file in create-comment only after the permissions and
page scope checks, matching create-comment-thread, so a rejected reply
does not load the file data.

Cover the denial and the in-scope paths of the four commands, and the
fail-closed case of a link without pages on the comment reads. Move
the outsider thread creation after the profiles assertion so the
caller-union assertion fails when the union is removed. Document the
page scope invariant in the backend permissions memory.

AI-assisted-by: deepseek-v4.1-flash
2026-11-10 15:53:07 +01:00
Andrey Antukh
4874da450f
🐛 Escape SVG values and sanitize every SVG upload (#12887)
* 🐛 Escape SVG values and sanitize every SVG upload

The SVG sanitizer removed dangerous elements and attributes by name and
then wrote the tree back with clojure.xml/emit, which escapes nothing.
A payload hidden in an attribute value or in a text node became live
markup again: an id of "x' onload='alert(1)'" was stored as a real
onload handler, and an entity encoded script tag was stored as a real
script element.

Write the filtered tree with a serializer that escapes text and
attribute values, and route every path that stores an uploaded SVG
through it. The thumbnail writers (create-file-object-thumbnail and
create-file-thumbnail) and the organization logo stored the upload
untouched, so they could plant live markup in a public bucket.

The filter only knows about SVG elements, so it also accepted
documents that are not SVG at all: a page with an iframe or a meta
refresh was stored as an image. The sanitizer now rejects any document
whose root is not an svg element in the SVG namespace, and it compares
the scheme of an url the way the browser does, so a scheme split with
a tab or a line break no longer slips through the javascript: filter.
The attributes are written sorted by name, because the parser hands
them over in the reverse order of the document and the output changed
on every sanitization.

The profile picture of an identity provider is not stored either when
it is an SVG: the avatar is always rendered as an image, so an SVG
would only add the risk of keeping live markup in the storage. The
dimensions of an SVG are now read from its local name, so an SVG with
a namespace prefix can be uploaded like any other.

The logo had no size limit and the sanitizer reads the whole file, so
it now honors the media max file size.

The output shape changes with it: no line breaks, double quotes, and
the single quote of an attribute value escaped as an entity. The hash
of every stored SVG object changes with it.

The media tests use cuerdas.core instead of clojure.string, as the rest
of the codebase does.

Relates to GHSA-qwhj-rchj-58mg

AI-assisted-by: minimax-m3.1-flash

* 🐛 Drop non-SVG namespace subtrees in the SVG sanitizer

The sanitizer only checked the namespace of the root, so a child could
rebind xmlns to XHTML and smuggle live HTML (iframe/srcdoc/meta) past the
blocklist. The serializer escapes attribute values, but the browser decodes
them again, so srcdoc stays live.

Track the namespace bindings through the recursion and drop any element
that is not in the SVG namespace, with its subtree. The root is validated
up front with the same model, so a document that is not an svg document is
rejected with an honest hint instead of a misleading one.

The default namespace is stored under a namespaced key, because a prefix
literally named `default` collided with the internal key and re-opened the
same hole through a prefixed binding. The attribute filter now keeps
`xmlns`/`xmlns:*` declarations, which are not event handlers, so a prefix
named like one does not end up undeclared in the output.

The blocklist also drops HTML-only elements written in the svg namespace
(iframe, meta, embed, link, base).

Add regression tests for the child rebinding, for the prefixed variants,
for the `default` prefix, for the root local name, for the
create-file-media-object-from-url path (mocked download), and for the
HTML-only tags. Assert double-quoted attributes in the clean content test.

Relates to GHSA-qwhj-rchj-58mg

AI-assisted-by: longcat-2.5-free
2026-11-10 15:52:28 +01:00
Andrey Antukh
c39e3e8301
✨ Strictly validate uploads against file bytes on disk (#12801)
* ✨ Strictly validate uploads against file bytes on disk

Sniff the real content type of every upload with simplemagic magic
bytes instead of trusting the client-supplied mtype header. Declared
types outside the allowlist keep the not-allowed code; allowed
declarations whose bytes differ are rejected as mismatch. Covers SVG
by probing its root tag over leading bytes (magic never matches XML
text), TTF/OTF by magic name and WOFF/WOFF2 by four-cc.

Read the real file size from disk instead of trusting the declared
:size. Files over the limit are rejected even when the declared size
lies downwards, and small files pass even when it lies upwards.

Run the path check first everywhere so forged server paths keep the
params-validation code. Recheck from-url downloads after the write,
with a declared-only fast fail at header time. Thread the assembled
path through font validation.

Refresh the affected tests around real and sparse fixture files,
including a PNG-as-font chunked-upload rejection case.

Closes #12796

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Accept BOM-prefixed SVG uploads and cover WOFF/WOFF2/OTF/WEBP

Address review comments on #12801:

- SVG files saved with a UTF-8 BOM were rejected as
  :media-type-mismatch: read-head kept U+FEFF and the <svg probe
  never ran. A leading BOM is now stripped before the probe, and a
  generic XML sniff (application/xml, text/xml) is treated as
  "no magic evidence" for declared image/svg+xml, so the root-tag
  probe decides whether the file opens with an <svg root.
- sniff-match?/svg-file? docstrings claimed the magic database never
  matches SVG; simplemagic 1.17 does match an SVG with an XML prolog
  as image/svg+xml. Reworded to match reality.
- Added matching-pass tests for WOFF, WOFF2, OTF and WEBP (they hit
  the read-four-cc, magic-name and direct magic branches) and three
  BOM SVG regression cases.

AI-assisted-by: inkling-2-small-free
2026-11-10 15:51:48 +01:00
Elena Torró
7f0b62b994
🐛 Keep drop shadows out from under translucent shapes (#12904)
* 🐛 Keep drop shadows out from under translucent shapes at every zoom

Cut the shape out of its drop shadow, as CSS box-shadow does, and stop
large shadows from being cut off at high zoom.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep drop shadows out from under translucent shapes in SVG and PDF

Match the canvas: clip the shape out of its drop shadow on export.

AI-assisted-by: claude-opus-5-5
2026-11-10 15:05:08 +01:00
Juanfran
e1f924cf00 🐛 Send organization invitations to unregistered emails (#12937)
send-template drops nil values before enqueuing the mail job, but the
invite-to-organization schema required the :user-name key. Inviting an
email with no Penpot account left :user-name nil, the key was dropped
and the job failed its schema check, so the invitation email was never
sent. :user-name is now optional in invite-to-organization and in
renewal-notice, which has the same shape.

New test renders both templates through the queue with a nil
:user-name.

AI-assisted-by: claude-opus-5-5
2026-11-10 14:12:27 +01:00
penpot-renovate[bot]
c9d3cbb383
⬆️ update GitHub Actions (#12898)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker-devenv.yml)
- docker/login-action v4.8.0 (.github/workflows/build-docker-devenv.yml)
- zizmor 1.32.0 (.github/workflows/lint-workflows.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-10 13:48:10 +01:00
Miguel de Benito Delgado
cbd4c9634c
🔧 Use correct :cache-root in shadow-cljs.edn (#12883) 2026-11-10 13:47:50 +01:00
Marina López
4e8f05171e
🐛 Forward attribute deletion events to admin console (#12932) 2026-11-10 13:47:13 +01:00
Alonso Torres
79d4936cae
⬆️ Release plugins and mcp (#12934)
* ⬆️ Updated plugins release to 1.6.0

* ⬆️ Updated mcp release to 2.19.1
2026-11-10 13:44:51 +01:00
Juanfran
ae24118caf
🐛 Send organization invitations to unregistered emails (#12937)
send-template drops nil values before enqueuing the mail job, but the
invite-to-organization schema required the :user-name key. Inviting an
email with no Penpot account left :user-name nil, the key was dropped
and the job failed its schema check, so the invitation email was never
sent. :user-name is now optional in invite-to-organization and in
renewal-notice, which has the same shape.

New test renders both templates through the queue with a nil
:user-name.

AI-assisted-by: claude-opus-5-5
2026-11-10 13:44:26 +01:00
Andrés Moya
10b08d5224 🐛 Improve sync of layout item attributes in components (#12851)
* 🐛 Preserve layout item touched groups when switching variant

* 🔧 Add systematic tests to check layout items component sync

* 🔧 Add small updates after comments

* 🔧 Add tests for partial overrides
2026-11-10 13:41:54 +01:00
penpot-renovate[bot]
2e6c799bba
⬆️ Update dhi.io/node image to f943257 (#12931)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@f943257 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-10 11:45:57 +01:00
Filip Sajdak
ee3e2a0e61
🔧 Keep the CI expect timeout in the default and ds e2e projects (#12922)
The Playwright config gives the retrying expects 40 seconds on CI and 5
seconds elsewhere, but the default and ds projects set their own expect
block for the screenshot tolerance, and a project block replaces the
top-level one instead of merging with it. On CI those projects have run
with Playwright's 5 second default.

That is too short for the first dashboard load on a busy runner, where the
dashboard module is fetched lazily: specs that wait for the dashboard
heading time out and pass on the retry (flaky), or fail after the
retries. In the shard of one recent run that was 7 flaky and 4 failed
specs out of 112, and develop itself failed the same families of specs on
several runs the same day.

Name the timeout once and repeat it in both projects. A probe config that
imports this one and polls a condition that never holds reports 40000 ms
for both projects with this change, and 5000 ms without it. The
render-wasm project already repeats its own value.

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 11:33:42 +01:00
Filip Sajdak
90d98b1989
🐛 Call the team hook on every render of the team container (#12920)
team-container* called mf/deref only after returning early for a team id
that is not a uuid. When the same component rendered once without a team
id and then with one, the number of hooks changed between the two renders
and React threw "Rendered more hooks than during the previous render"
(error #310), which sent the user to the internal error page. The reverse
order throws "Rendered fewer hooks than expected".

Call the hook on every render and keep the uuid check in the condition
that builds the providers.

Two tests mount the component in jsdom and render it with a team id that
arrives and with one that leaves; both fail without this change.

Closes #12728

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-11-10 11:32:58 +01:00
Alonso Torres
4fb0ea9eee
🐛 Give unique values to same-name components combined as variants (#12881)
When components whose name matches the variant container are moved into
it, removing the container name leaves an empty value. Combining several
components named the same produced "Value 1" for the first variant and
empty values for the rest, so variants shared the same combination.

Assign the first unused "Value N" to the last property when all computed
values are blank, tracking values given within the same batch.

Combine moves the remaining mains into the container one at a time, in
the combine order, so the generated values follow that order instead of
the layer order.

Closes #12878
2026-11-10 09:16:08 +01:00
Eva Marco
72575e5c05
🐛 Make Add to an organization a keyboard button (#12722)
"Add to an organization" in the team settings opens a modal but was an
<a> without href: the keyboard could not reach it. It is now a
<button type="button"> that keeps its text-link look (accent color,
medium weight, underline on hover, the same box), so Tab reaches it and
Enter or Space opens the modal.

New visual test "Shows the Add to an organization link", with fixtures
for a team in no organization and an organization the user belongs to:
0 px of difference at rest and on hover, and 0 px on the whole settings
page.

AI-assisted-by: claude-opus-5-5
2026-11-10 08:33:04 +01:00
Luke Lau
cd1ece4beb
⚡ Keep the translation flag on undo and redo of a move (#12870)
* ⚡ Keep the translation flag on undo and redo of a move

Previously undos and redos always forced a component sync + thumbnail
rerender for every component touched even if the underlying commits were just
translations. This propagates the translation flag to undos and redos and
propagates it to commit-changes. An entry merged from several undos/redos is
a translation only if all of its undos/redos are translations.

This reduces a UI freeze when undoing a move of a board with 453 components
from 2.42s -> 0.22s.

AI-assisted-by: claude-sonnet-5-5

* :translation? -> :translation

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 20:47:42 +01:00
penpot-renovate[bot]
bdf16bcfda
⬆️ Update DHI base images (#12897)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@5c9d8c2 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@94571f3 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@e8db455 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@7f8db65 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@ee39b15 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-09 17:39:54 +01:00
Luke Lau
6846afd08c
⚡ Send objects-map pages to the worker without a second encode (#12811)
* ⚡ Send objects-map pages to the worker without a second encode

When switching between pages on a large file e.g Material Design Icons, there's
some UI freeze from :index/initialize encoding object map shapes to send to the
worker. An objects map already stores the encoded JSON for each shape, and
`encode-str` was wrapping those strings in a second layer of JSON on the main
thread.

We can just transmit the already encoded JSON in a js/Map alongisde the
payload, similar to :blob, so structured clone just copies the strings verbaitm
without re-encoding. To do this this PR adds two functions to ObjectsMap that
converts to and from a js/Map.

The "Main components" page of the Material Design Icons has ~30k objects, so
this reduces the freeze when switching pages from 306ms -> 189ms (1.63x) on a
release build.

AI-assisted-by: claude-sonnet-5-5

* Split property tests out into new deftest

---------

Signed-off-by: Luke Lau <luke_lau@icloud.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:35:20 +01:00
Luke Lau
55360c52f5
✨ Cache the transit reader when decoding strings (#12704)
* ✨ Cache the transit reader when decoding strings

On the cljs side, we create a new transit reader on every call to decode-str.
This turns out to have a fair bit overhead in both transit-cljs and transit-js,
since it has to merge the handlers, call clj->js, allocate a new ReadCache,
etc.

By reusing it the measured decode time goes from ~58us -> ~41us, so about 1.4x
faster. This particularly helps for some operations like whenever a component
is edited and every object needs to be decoded, see
https://github.com/penpot/penpot/issues/9344, although they could probably be
made more efficient in other ways.

The reader usually clears its read cache after a successful read, but whenever
it throws it hangs around. So in this case we need to reset the cache
ourselves, otherwise the next read will be from a clobbered cache.

The tests were written by Claude Sonnet 5.5, and should cover the clobbering
behaviour above.

Signed-off-by: Luke Lau <luke@igalia.com>

* Keep one transit reader per type and rebuild it on add-handlers!

---------

Signed-off-by: Luke Lau <luke@igalia.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:34:24 +01:00
Luke Lau
3de7e62aa9
⚡ Don't send file data contents back from duplicate-file (#12911)
The client calls the duplicate-file command in two places, the dashboard and in
the plugin API. In both places the :data is dropped via `(dissoc file :data)`,
so the client never uses that field.

However the file data takes a while to send over the wire and decode. When
duplicating penpot-app there's a roughly 1s freeze whenever the response is
returned from the server with the file data. We can avoid it by dropping the
data from the response.

I also did a quick check and neither the MCP server nor any public client of
the API on GitHub seem to use the :data field.

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:23:39 +01:00
Belén Albeza
fb7468c3f5
🔧 Make all CI jobs use Rust 1.99 (#12903)
* 🔧 Bump Rust version of the devenv image to 1.99

* 🔧 Run the pinned Rust setup from the lint and format scripts

./build and ./test switch to the Rust version in render-wasm/_build_env
through setup, but ./lint did not, and formatting called cargo fmt
directly. In CI both ran on the devenv image's default toolchain.

Make ./lint run setup and add a ./fmt script that does the same, so all
render-wasm entry points use the pinned version. CI and scripts/ci now
call ./fmt instead of cargo fmt.

AI-assisted-by: claude-opus-5-5
2026-11-09 16:23:47 +01:00
Miguel de Benito Delgado
01191051c7
🐛 Replace stale atlas pixels when zooming (#12884)
Copy finished tiles with BlendMode::Src so transparent pixels replace what the atlas slot held before
2026-11-09 14:28:13 +01:00
Miguel de Benito Delgado
33602b9114
🔧 Allow skipping renderer build setup via env. var (#12882)
Set PENPOT_WASM_PREPARED=1 to speed up repeated builds
2026-11-09 12:53:18 +01:00
Andrey Antukh
4b5dd51b39 📚 Add 2.20.0 changelog section
Populate the 2.20.0 release notes from the milestone: 19 bugs fixed
and 31 new features & enhancements, classified by issue type with the
community contribution attributions and the merged PR references. No
💥 or 🚀 sections: no issue carries the breaking change or
release highlight labels. No GHSA entries: no published advisory lists
2.20.0 as patched version.

AI-assisted-by: inkling-2-small-free
2026-11-09 11:19:50 +00:00
Luke Lau
943e63a99e
⚡ Group color updates in selections (#12875) 2026-11-09 12:08:36 +01:00
Andrey Antukh
4c31e90665 Merge remote-tracking branch 'origin/develop' into staging 2026-11-09 11:15:38 +01:00
Hosted Weblate
9034e707bf
🌐 Merge branch 'origin/develop' into Weblate. 2026-11-09 11:09:57 +01:00
Alexis Morin
968edc6a55
🌐 Add translations for: French (Canada)
Currently translated at 96.7% (2285 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-09 11:07:01 +01:00
AntonPalmqvist
a2dfcb8669
🌐 Add translations for: Swedish
Currently translated at 95.9% (2266 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/sv/
2026-11-09 11:06:59 +01:00
Andrey Antukh
f66e5afa70 📚 Correct link-issue closing behavior in workflow memories
The memories claimed link-issue does not close an issue
retroactively when its PR is already merged. Adding 'Closes #12896'
to merged PR #12067 and running link-issue closed the issue
immediately instead (observed 2026-11-09). Update the
mem:workflow/creating-issues, mem:workflow/creating-prs and mem:scripts/gh
entries to describe the observed behavior.

AI-assisted-by: inkling-2-small-free
2026-11-09 09:56:57 +00:00
Andrey Antukh
212037f368 📚 Add upload path hardening entry to 2.19.0 changelog
Add the missing 2.19.0 entry for #12896: upload endpoints accepted
non-multipart upload paths and validation error bodies arrived
truncated. Fixed by #12067. The issue did not exist when the
changelog section was last generated.

AI-assisted-by: inkling-2-small-free
2026-11-09 09:56:27 +00:00
Andrey Antukh
1df2941576 Merge remote-tracking branch 'origin/staging' into develop 2026-11-09 10:42:28 +01:00
Andrey Antukh
b766e88ed8 Merge remote-tracking branch 'origin/main-2.18.x' into staging 2026-11-09 10:31:58 +01:00
Andrey Antukh
f90ef83da0 📚 Update changelog 2026-11-09 10:28:54 +01:00
Andrey Antukh
341e3dfa65 Merge remote-tracking branch 'origin/staging' into staging 2026-11-09 10:17:47 +01:00
Andrey Antukh
e86131cf16 Merge branch 'main-2.18.x' into staging 2026-11-09 10:17:33 +01:00
Andrey Antukh
0ea6d166f0
✨ Add a session origin model to auth events (#12856)
* 🐛 Return the full profile from the LDAP invitation branch

The invitation branch responded with only {:invitation-token token},
so the session start received a map without id or auth-backend. With
the new auth-origin vocabulary that map fails validation and the login
breaks in development and tests; in production the event would be
emitted without a channel.

Return the profile plus the token, like the regular branch, and cover
both branches with RPC-level tests.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Mint an auth token in the OIDC callback with the invitation inside

The callback used the invitation JWE as the session token, so the
frontend landed directly on token verification and never ran the
regular session start: no signin event, no teams, no host hook.

Always mint an :auth token and carry the invitation in its claims.
Token verification returns the claims verbatim, so login-from-token
attaches the invitation to the profile and logged-in does the rest,
the same pattern the verify-email token already uses.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* ✨ Keep the session origin in the telemetry shadow rows

The anonymisation filter drops every string unless it is on a short
allowlist. Add source and activation next to lang and auth-backend so
the shadow copy keeps them; the email and the full name are still
dropped and the email domain is still derived from the email.

The filter keeps its two bindings: the transducer discards everything
that is not a UUID, a boolean or a number, so the allowlist is applied
on the filtered map, never on the raw props.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* ✨ Describe the signup outcome with activation, not with action

The three success branches of register-profile labelled a new account
as "login", "email-verification" or "accept-invitation", so the event
contradicted its own name and the value only lived in context, where
the analytics consumers never read it.

Add activation to the event props and drop action from those branches:
the name already says it is a signup, and a constant action would only
add noise. The retry branches keep their action in context because
there it distinguishes four different outcomes.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* ✨ Describe the session origin in the signin event

The event carried the profile but never said why the session started:
the only field that tried was frontend metadata that never leaves the
browser, and several flows filled it with the wrong value.

Add a small closed vocabulary (source, auth-backend, activation,
from-invitation) and compose it from the profile when signin is
emitted. The assertion fails in development and tests and is elided in
the release build, so a bad value cannot break a real login.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* ✨ Emit the session origin from every login entry point

The four emitters stamped a free-form string as metadata, so the
session event could not rely on it: two flows that were signups wrote
the same value as a login.

Route them through origin-meta: password and LDAP logins declare
source login, register declares source register with activation
immediate, and login-from-token derives both from the token issuer:
:auth for a login and :verify-email for a signup that needed the
email confirmation.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Send the invitation signups through the regular session start

Registering from a team invitation link navigated by hand to token
verification, so the session start never ran: no signin event, no
teams, no host hook. The same navigation was copied in four places.

Teach login-from-register to carry an optional invitation token,
extract the profile decoration into with-register-origin, and call it
from the four sites instead of the hand-written navigation.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Resolve the invitation without waiting for the teams

Registering from an invitation link now goes through the session
start, and that start only navigates after the teams request
completes. fetch-teams has no error handling, so a network failure
there would leave the invitation unaccepted and the user on the signup
screen with a session already minted.

Move the invitation navigation out of the teams path: it emits as soon
as the profile carries a token, and the teams path emits nothing for
an invited profile so it cannot move the user away from the token
verification screen.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* 📚 Document the session origin vocabulary and the invitation flow

Add a focused memory with the four fields, where each value comes from,
the table of flows, the LDAP exception and the invitation path through
logged-in, and link it from the frontend routing.

Update the audit-log memory: source and activation are now on the
telemetry allowlist, activation lives in props and action only exists
on the retry rows.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash

* 📎 Cover the invitation session start and fix the auth memories

The invitation flows and the token decoration in login-from-token were
only covered by helper tests, so removing the guard that keeps the
teams path from navigating, or the token attach, broke the behaviour
silently.

Run the events through ptk/watch with a teams-fetched event as input:
an invited profile navigates to the token verification screen and
never to the dashboard, and the signin keeps from-invitation. Add the
login-from-token cases with and without the token, and fix the memory
statements about action, the flow table and from-invitation.

Closes #12849

AI-assisted-by: deepseek-v4.1-flash
2026-11-09 10:11:54 +01:00
Andrey Antukh
9b1e18a478 🐛 Validate binfile library relations against the package (#12862)
Ignore imported library relations that point outside the
package or self-link a file on the v3 import path. Delete
leftover cross-team and self relations from file_library_rel
and their matching file_library_sync rows with migration
0158, sync rows first.

Move the get-files-rels SQL to a var, without behaviour change.

Cover the write-side invariants with state-based tests:
tampered manifests, package containment, self links and the
migration cleanup.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: muse-spark-1.3-contributor
(cherry picked from commit 0b8a428d6b28c35601793f8d05570dae614a2fd5)
2026-11-09 08:46:19 +00:00
Alexis Morin
5b9bf3cbba
🌐 Add translations for: French (Canada)
Currently translated at 96.5% (2279 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-08 10:51:26 +01:00
Danial Shirali
44a0cc7147
🌐 Add translations for: Persian
Currently translated at 94.5% (2233 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fa/
2026-11-08 10:51:22 +01:00
penpot-renovate[bot]
904cdd9716
⬆️ Update DHI base images (#12814)
- dhi.io/debian-base trixie-debian13-dev@61025ce (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@61025ce (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@61025ce (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@61025ce (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-alpine3.24@34c5f1f (docker/images/Dockerfile.storybook)
- dhi.io/nginx 1.31.8-debian13-dev@f88f28c (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@24214dd (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@24214dd (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@24214dd (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-07 22:53:06 +01:00
penpot-renovate[bot]
75b3da5290
⬆️ Update dependency pnpm to v12.18.0 (#12876)
- pnpm 12.18.0 (docker/images/Dockerfile.media-processor)
- pnpm 12.18.0 (docker/images/Dockerfile.mcp)
- pnpm 12.18.0 (docker/images/Dockerfile.exporter)
- pnpm 12.18.0 (docker/devenv/Dockerfile)
- pnpm 12.18.0 (render-wasm/package.json)
- pnpm 12.18.0 (plugins/package.json)
- pnpm 12.18.0 (plugins/libs/plugins-styles/package.json)
- pnpm 12.18.0 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.18.0 (plugins/libs/plugin-types/package.json)
- pnpm 12.18.0 (plugins/apps/table-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.18.0 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/icons-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/example-styles/package.json)
- pnpm 12.18.0 (plugins/apps/e2e/package.json)
- pnpm 12.18.0 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.18.0 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.18.0 (package.json)
- pnpm 12.18.0 (media-processor/package.json)
- pnpm 12.18.0 (mcp/packages/server/package.json)
- pnpm 12.18.0 (mcp/packages/plugin/package.json)
- pnpm 12.18.0 (mcp/packages/common/package.json)
- pnpm 12.18.0 (mcp/packages/browser-extension/package.json)
- pnpm 12.18.0 (mcp/package.json)
- pnpm 12.18.0 (library/package.json)
- pnpm 12.18.0 (frontend/text-editor/package.json)
- pnpm 12.18.0 (frontend/packages/ui/package.json)
- pnpm 12.18.0 (frontend/packages/mousetrap/package.json)
- pnpm 12.18.0 (frontend/packages/draft-js/package.json)
- pnpm 12.18.0 (frontend/package.json)
- pnpm 12.18.0 (exporter/package.json)
- pnpm 12.18.0 (docs/package.json)
- pnpm 12.18.0 (common/package.json)
- pnpm 12.18.0 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-07 22:52:43 +01:00
Alexis Morin
dac470cc94
🌐 Add translations for: French (Canada)
Currently translated at 94.4% (2231 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-07 08:51:35 +01:00
Stephan Paternotte
da8cdb734b
🌐 Add translations for: Dutch
Currently translated at 100.0% (2361 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/nl/
2026-11-07 08:51:32 +01:00
Elena Torró
7ecf48c52c
⚡ Speed up uploading page shapes to wasm (#12854)
* ⚡ Compute text fonts once per shape when uploading to wasm

* ⚡ Write the shapes batch in place and coerce fills once

* 🐛 Free the upload buffer when a batch record write fails
2026-11-06 15:45:34 +01:00
Belén Albeza
988cde3c9d
✨ Make WebGL the default renderer for new accounts (#12880)
New profiles now store renderer `wasm` in their props at creation, so
every signup path (email, OIDC, LDAP, Nitrate, demo, CLI) starts on
WebGL. A renderer passed by the caller still wins.

Existing profiles without a stored renderer keep the frontend fallback
to SVG, so no backfill or version check is needed to leave them as they
are.

The `register-profile` audit event gains `renderer-default` when the
profile is created in that call, so signups before and after this
change can be told apart.

Closes #11855

AI-assisted-by: claude-opus-5-5
2026-11-06 14:44:24 +01:00
Álvaro Tejero-Cantero
87fc15b5aa
🐛 Open the first page in view mode when the URL has no page id (#12687)
A view URL without a page-id crashed builds with assertions enabled:
the viewer passes a nil page-id to `initialize`, whose schema accepted
a missing key but not a nil value. Release builds skip that check and
draw the first page, but every viewer event reads the page from the
route, so zoom to fit and fill, the next-board arrow, Navigate To
interactions, and comment pins did not work.

`initialize` now accepts a nil page-id, `fetch-bundle` makes it
optional, and `bundle-fetched` replaces a route without page-id by one
naming the file's first page. The viewer then initializes again on the
completed route, as it does when the user changes page. A file with no
pages keeps its route.

AI-assisted-by: claude-opus-5-5

Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
2026-11-06 13:05:39 +01:00
Andrey Antukh
a77cb2608e
🐛 Normalize stored SVG attribute keys to kebab-case (#12006)
* 🐛 Normalize stored SVG attribute keys to kebab-case

The v3 reader already emits kebab-case keys, but birth stored the
camelCase React prop names, so import and export never converged:
fill rules painted wrongly and filter defs lost their blur.

Birth now normalizes every stored attribute key to kebab-case
through the `svg-props` whitelist (`attrs->kebab-props`), applied on
all element paths of the svg importer, on the file builder and on
defs. svg-raw `:content` is kebab-ized in spelling only and keeps
unknown keys, because the sidebar reads them. Non-spec attrs Penpot
must keep live in `penpot-extra-attrs`, seeded with `:stroke-style`.
Render boundaries translate back to camelCase for React and for the
wasm props, and the shape schema declares the closed kebab contract
derived from that same whitelist, so birth and validation cannot
drift.

Migration `0035-fix-shape-svg-attrs-and-defs` converts old files
once with the birth transforms, and carries back the svg-viewbox
rect normalization the withdrawn `0021` had. That old id leaves
`available-migrations`, so files recording it migrate again through
this one, and the binfile cleaner drops its own svg repair.

Review fixes included here: `setup-opacity` and the blend mode read
the stored keys, gradient stops accept `:style` as a CSS string or a
map, `inherit-attributes` reads `:style` so a child's inline style
beats the inherited group attr per CSS, and the svg-attrs menu shows
spec spelling (`viewBox`, `class`) through `stored-attr-display-name`.

Tests cover the migration, the closed svg-attr schemas and the
import/export round trip; the svg-content schema test pins its
sample seed, which dealt no svg-raw shape on unlucky runs.

Closes #11947

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Keep legacy svg node tags and transforms valid in migration 0035

The closed svg node schemas require keyword tags and map attrs, but
stored files hold def and svg-raw content tags as strings, and some
node `:attrs` as nil. Migration 0035 left them as they were, so the
file data check that runs after migrating rejected those files and
they could not be opened. `normalize-node-tree` now turns string tags
into keywords and nil attrs into an empty map, at birth and in 0035.

With keyword tags, the svg defs renderer applies the shape transform
to user-space defs. Legacy `:svg-transform` values are plain maps,
which multiply into NaN matrices. `gmt/coerce` turns them into
matrices, in 0035 and in the binfile cleaner, because every v3 import
decodes the transform as a plain map again. Legacy masks and gradients
now render as designed in the svg renderer.

The binfile test helper calls `export-files!` with the current config
keys.

AI-assisted-by: claude-opus-5-5

---------
2026-11-06 12:52:53 +01:00
Andrés Moya
4f02995a3c
🐛 Re-enable adding layout to a variant container (#12879) 2026-11-06 12:11:48 +01:00
Elena Torró
93ed713a29
🐛 Free the wasm renderer state on clean up (#12877)
Each page switch leaked the six wasm states (~200 MB on a large page).
Null state getters now panic instead of reading address 0.
2026-11-06 11:43:50 +01:00
María Valderrama
2df4d10bd5
✨ Enforce attribute-based organization permissions (#12868)
* ✨ Enforce attribute-based organization permissions

* 📎 Code review
2026-11-06 11:40:50 +01:00
Elena Torró
7fa225115e
⚡ Reuse uuid instances when decoding transit on the frontend (#12792) 2026-11-06 11:33:37 +01:00
Andrey Antukh
6a1b58c310 Merge remote-tracking branch 'origin/staging' into develop 2026-11-06 10:16:02 +01:00
Andrés Moya
018b857de1
🐛 Improve sync of layout item attributes in components (#12851)
* 🐛 Preserve layout item touched groups when switching variant

* 🔧 Add systematic tests to check layout items component sync

* 🔧 Add small updates after comments

* 🔧 Add tests for partial overrides
2026-11-06 09:51:12 +01:00
Andrey Antukh
0333410482 Merge remote-tracking branch 'origin/staging' 2026-11-06 09:50:49 +01:00
Andrey Antukh
22864cbfd7 📚 Update changelog 2026-11-06 09:50:15 +01:00
Andrey Antukh
55fb8c46fc 📎 Updat elibrary and mcp package version 2026-11-06 08:49:30 +01:00
Andrey Antukh
0b8a428d6b
🐛 Validate binfile library relations against the package (#12862)
Ignore imported library relations that point outside the
package or self-link a file on the v3 import path. Delete
leftover cross-team and self relations from file_library_rel
and their matching file_library_sync rows with migration
0158, sync rows first.

Move the get-files-rels SQL to a var, without behaviour change.

Cover the write-side invariants with state-based tests:
tampered manifests, package containment, self links and the
migration cleanup.

AI-assisted-by: deepseek-v4.1-flash
AI-assisted-by: muse-spark-1.3-contributor
2026-11-06 08:37:02 +01:00
alonso.torres
1a80e98e62 🐛 Keep Inspect from crashing on a shape whose parent is missing
In a damaged file a shape may point to a parent that does not exist.
The Inspect tab computes the position relative to the parent, so
inspecting such a shape replaced the workspace with the internal error
page. Inspect leaves the position out and logs a warning instead.
2026-11-05 23:59:32 +01:00
alonso.torres
eadd1cce70 🐛 Keep a copy linked when the repair finds its component's main
A component whose :main-instance-id points to a missing shape fails
validation on its copies with :ref-shape-not-found. A repair scoped to
the copy's page never sees the main's own error, so it detached the
copy, even when the main was still on the component's page.

The repair of :ref-shape-not-found now looks for the component's main
on its page first and points :main-instance-id back to it, which keeps
the copy linked. A copy is still detached when no main is found.
2026-11-05 23:59:32 +01:00
alonso.torres
7eacbec245 🐛 Keep comments, focus, thumbnails and text in step with a rollback
A rollback reverts the local data with a rebase, which is not a commit,
so the parts of the workspace that follow commits missed it:

- Comment pins moved in the reverted action stayed where they were, on
  screen and on the server, and pins on a board the rollback removed
  kept pointing to it. They go back to the reverted position, and pins
  on a removed board move to the page root.
- Focus mode stayed on with no shape left when the rollback removed the
  focused ones, and the next shape drawn turned it on again. It ends
  when its last shape goes, and the selection drops removed shapes.
- Component thumbnails kept the rejected look. The thumbnails of the
  reverted mains are rendered and uploaded again, and an upload whose
  shapes changed during the render is skipped.
- Under WASM, an open text editor reset to the reverted content but the
  canvas kept the old layout. It is laid out again.
- A library sync whose save failed showed no update notice again once
  the release undid it. The notice is checked again after that
  rollback.
2026-11-05 23:59:32 +01:00
alonso.torres
88416d2542 ✨ Label the copy kept before a repair in the History panel
Before it repairs a damaged file, the backend keeps a copy of it as a
system snapshot, and the repair notice points to the version history to
find it. The History panel listed that copy among the autosaves, so it
could not be told apart.

The History panel lists the copies kept before a repair as their own
entry per day, "Before automatic repair, <date>", above that day's
autosaves, and the preview of such a copy uses the same label.
2026-11-05 23:59:32 +01:00
alonso.torres
61bf66342b ✨ Tell repairs, damaged data and other files apart in save notices
The notice of a rejected save always said the same thing. It did not
name the file when the user had moved to another one, gave no hint when
the page held data that was already invalid, and replaced the notice of
a backend repair still on screen, since the store holds one toast.

The backend flags a rejection whose first commit touches pages or
components that failed validation before it (:preexisting-damage). The
rejection notice names a file other than the open one, says when the
data was damaged before, and folds into a repair notice still on
screen; a later repair notice folds a rejection notice in the same way.
The notice says that the latest changes were undone, no longer that the
rest of the work is saved.
2026-11-05 23:59:32 +01:00
alonso.torres
27c1c38360 🐛 Apply the changes stored before a save before acknowledging it
The answer to a save lists the changes other sessions stored before it,
such as the backend's repair of a damaged file. When the answer came
before the message with such a change, the saved commit was
acknowledged first, and the late change was then applied on top of the
local edits instead of under them, out of the server's order.

A saved commit of the open file now stays pending until this tab has
received every change the answer lists from other sessions, so a late
change is applied under the local edits. If one does not arrive within
10 seconds, the file reloads.
2026-11-05 23:59:32 +01:00
alonso.torres
302f83f072 🐛 Drop the unsaved edits made on a version a peer restored
When a peer restores a version, the file reloads to it. Edits made on
the replaced version stayed queued: their save failed for good and
locked the workspace, and a rollback or a rebase replayed them on the
restored data.

Loading another version forgets the queued, buffered and saved commits
made on the old one. A save of such a commit, in flight or answered, is
dropped instead of failing or being rolled back, and one save carries
the edits of one version only.
2026-11-05 23:59:32 +01:00
alonso.torres
1d862c0dd5 🐛 Refuse every edit while saves fail
While saves fail the workspace turns read-only, but only some tools
checked it. Edits from the asset, color and token panels, and a drag or
a drawing in progress when the lock came, still made commits that
queued behind the failed save.

commit-changes refuses commits while the save-failure lock holds,
except those that are not user edits: the undo of rolled-back commits,
text position data and component touches. Undo and redo do nothing
while locked. When the lock comes, moves, resizes, rotations, drawings
and the drags of flex, guide and grid handles in progress are cancelled
and commit nothing, and the design panel shows the Inspect view.
2026-11-05 23:59:32 +01:00
alonso.torres
e10ceefb5d 🐛 Keep a file reload from doubling the workspace streams
Reloading the open file (a peer's restore, the fallback of a rollback)
runs initialize-workspace again without leaving the workspace. The
streams the first run started kept running next to the new ones, so a
commit added one undo entry per reload and each notification was
handled twice. The old subscription also sent its unsubscribe after the
new subscribe, which ended the new one on the server.

A new initialize-workspace, text reflow, shape layout or notification
subscription stops the one before it, and a subscription replaced by
one to the same file sends no unsubscribe.
2026-11-05 23:59:32 +01:00
alonso.torres
cc4788aabf 🐛 Keep reporting rollbacks the MCP agent has not heard about
The MCP plugin sends the rollbacks it saw with its next task response
and then forgets them. When that response came after the server had
given up on the task, the server dropped it and the agent never heard
about the rollback. A reload of the tab or of the plugin lost them too.

The server now sends the rollbacks of such a late response back to the
plugin, which reports them again with its next response. The plugin
keeps the rollbacks not reported yet in its local storage, one entry
per file, so a reload does not lose them. Both MCP plugin manifests ask
for the allow:localstorage permission for that.
2026-11-05 23:59:32 +01:00
alonso.torres
a6255880da 🐛 Let plugins write in Inspect mode and stop them as viewers
Plugin writes failed whenever the workspace read-only flag was set. The
Inspect tab sets it too, although it only hides the editing tools, so an
MCP agent could not edit while the user was inspecting. A user who
became a viewer had the flag cleared, so plugin writes did not throw and
silently did nothing.

Writes now fail on what makes them unsafe, not on the flag: failing
saves, a user who cannot edit, a version preview or a lost render
context. The error says which one, and the MCP instructions tell the
agent which of them pass by waiting.
2026-11-05 23:59:32 +01:00
alonso.torres
7538d2c187 ✨ Tell plugins and the MCP agent when changes are rolled back
The backend rejects a save after the call that made the changes has
returned, and the workspace then undoes them. A plugin, or the MCP
agent, went on working with shapes that were no longer there and never
learned why.

The rollback event now names the files that lost commits and the pages
and page shapes their changes touched, and plugins get it as the new
`contentrollback` event (`ContentRollback` type) through a subscription
to the store stream, so no rollback is dropped by debouncing.

The MCP plugin keeps the rollbacks it is told about and sends them with
its next task response, failed or not. The server collects them per tool
call and puts a warning first in that tool's response, naming what was
affected; the instructions tell the agent to read the pages again and
redo what is missing.
2026-11-05 23:59:32 +01:00
alonso.torres
9368c83dca ✨ Make plugin writes fail while the workspace is read-only
A read-only workspace (saves failing, a user who cannot edit, a version
preview) only hid the editing tools: plugins, the MCP among them, still
changed shapes, and those edits were never saved. Every plugin write now
throws a read-only error, whatever the throwValidationErrors flag, the
same way a write to a removed shape does.

`check-permission` with a write permission runs the new `check-writable`,
which covers most writers. Writers without a permission check (most
context methods such as create*, group, align or flatten, the z-order
methods, applyTypography on a text range, removing a ruler guide) call it
themselves, and shape setters get it through `check-shape-writable`.
Reads, comments and undo blocks keep working, so a block opened before a
lock still closes.

The MCP instructions tell the agent what the read-only error means.
2026-11-05 23:59:32 +01:00
alonso.torres
c8ef46a0a4 💄 Put the team organization update handler on its own line
`ptk/UpdateEvent` of `handle-change-team-organization` sat on the same
line as the end of its watch body.
2026-11-05 23:59:32 +01:00
alonso.torres
8ef3ba628d 🐛 Keep the workspace open when closing the v2 text editor
Closing the v2 text editor with Escape could crash the workspace. The
text options panel renders once more with the editor it saw last, after
closing has disposed it, and reading the style of a disposed editor
throws.

The panel now takes the values from the shape when the editor is gone.
2026-11-05 23:59:32 +01:00
alonso.torres
6426b17c9c 🐛 Fix small save bugs found by the rollback test plan
- Ending a drag inside the frame the shapes already belong to sent an
  empty reparent save and bumped the revision. The move commits only
  when some shape changes parent.
- When a user restored a version, other sessions reloaded the file with
  no notice. They now see a dialog naming who restored which version
  (by its name, else its date), saying that later changes are no longer
  in the file and, when they had changes not saved yet, that those were
  lost.
- Exports drawn in the browser (PDF, SVG, and PNG when the WASM renderer
  draws it) and duplicating a file from the dashboard started at once,
  so an export could show changes a pending save then rejected, and a
  copy could miss queued edits. Both wait for the file's pending saves.
  A client export warns that it includes unsaved changes; a copy warns
  that it misses them.
2026-11-05 23:59:32 +01:00
alonso.torres
94c05589cf 🐛 Keep the UI in step with a rollback
A rollback can remove or change shapes the UI still refers to.

- The selection drops shapes that are gone and the color picker that
  edited them closes.
- The undo history is cut at the reverted action; it stays when no
  entry holds a reverted change. Undo and redo commits carry real undo
  changes and the history index they moved from, so a reverted Ctrl+Z
  puts the change and the index back.
- An open path edition whose shape the rollback changed closes without
  saving the editor's stale copy. The v2 text editor loads the reverted
  text again; the WASM editor already showed it.
- The Alt measurements look the hovered shape up in the current data.
- Plugin setters, resize() and rotate() on a removed shape throw
  "Shape not found", whatever the plugin's throwValidationErrors flag.
2026-11-05 23:59:32 +01:00
alonso.torres
cb26e8a39b ✨ Keep autosave and the workspace alive when saves fail
- A save that failed on a short platform failure (a lost or refused
  database connection, lack of resources, a server shutting down, a
  serialization failure or deadlock, a socket or HTTP timeout, a host
  that did not resolve, an interruption) answered 500, which stops
  autosave. It now answers 503, which the frontend retries. Other I/O
  and database errors keep answering 500, timeouts 504.
- A save answered not-found no longer replaces the workspace with the
  access page. It is a terminal save error: the workspace stays open
  with the failed save notice, report and reload link. When edit rights
  are lost, the notice says how many changes could not be saved.
- A library sync marked the library synced three seconds later,
  whatever happened to its save. A rejected sync was undone but stayed
  marked: copies stayed out of date and the "updates available" notice
  never came back. The mark now waits for the save, a rollback of the
  sync brings the notice back, and a failed mark (e.g. the user just
  became a viewer) is logged instead of leaving the workspace.
- While saves fail the workspace is read-only. The view-mode banner
  says editing is disabled because changes can't be saved, with the
  Inspect banner's size and bold lead and no Done button. Users can
  still switch pages, pan, zoom and inspect. The lock lifts when a save
  lands and brings back the mode the user had or picked meanwhile; the
  WebGL context lock keeps the workspace read-only if it lasts longer.
  The lock lives in app.main.data.persistence.lock.
- A viewer is never locked. A role change that leaves the user a viewer
  keeps the edits that lost permission, queued and on screen, and shows
  the count notice again. The role change that brings edit rights back
  releases them and rolls them back on screen, as a rejected save would.
  Any other terminal failure is released by the next role change or file
  opened.

The debug faults `io` (socket timeout, 503) and `eof` (500) cover both
sides of the I/O rule.
2026-11-05 23:59:32 +01:00
alonso.torres
f4b20ed08f 🐛 Revert whole user actions in a rollback
A rollback reverts whole undo groups, across files and saves, so that
no user action is left half done. Some actions still split:

- An action that commits several times inside one undo transaction
  (e.g. applying a token to several shapes) gave each commit its own
  group, so a rejection could revert part of it: shapes ended up linked
  to a token whose value was never applied. An undo transaction now
  opens with a commit group that every commit made inside it takes.
- The commit that undoes saved commits of a rejected action joined an
  open transaction's group, so a later rejection there reverted the
  undo too. It gets a group of its own.
- Emptying the open transaction during a rollback dropped its group, so
  its later commits were no longer one unit. It keeps its group.
- Waiting for a save ended as soon as the queue emptied, while later
  commits could still wait in the buffer. Waiters now also wait for an
  empty buffer.

A test covers a token action of three commits in one save whose third
commit is rejected.
2026-11-05 23:59:32 +01:00
alonso.torres
989f68abfd ♻️ Share the change helpers of the update-file tests
The update-file tests each built the same rect change and the same
change that fails while applied. The rect change moves to the test
helpers as th/add-rect-change and the failing one gets a named helper.
The test of a database failure during the retry no longer counts calls
to process-changes: it fails every attempt that leaves the failing
commit out.
2026-11-05 23:59:32 +01:00
alonso.torres
2f3b928a60 ⚡ Check the schema of only the pages a save touches
update-file checked the schema of the whole file on every save, which
loads and decodes every page even when the changes touch one. On files
with several pages that was most of the time of a save.

validate-file-schema-affected! checks the file data with the pages index
reduced to the pages the changes modify; the data outside pages is still
checked whole, and a change type it does not know makes it check the
whole file. update-file uses it for the strict and the soft schema check.

A save of 10 commits on a file of 10 pages of 2k shapes goes from ~1.1 s
to ~240 ms; a single page file costs the same.
2026-11-05 23:59:32 +01:00
alonso.torres
4f42ab1d2d ✨ Keep the valid commits of a save that fails validation
A save whose changes failed validation was rejected as a whole: autosave
stopped, the edits made after it piled up unsent, and a reload lost
them all, the valid ones included.

The frontend now sends each commit of a save as its own group with its
commit id (:changes-with-metadata). When applying and validating all of
them at once fails, the backend searches by bisection for the first
commit that fails, keeps the commits before it and answers with
:rejected (that commit and every later one). Only the kept changes are
stored, snapshotted and broadcast, so other users never see the rejected
ones. Flat :changes and groups without ids stay all-or-nothing; database
and network errors still fail the whole request. A rejection without an
error type is reported as an internal error.

The frontend rolls the file back to what the server holds: it undoes
every pending commit of that file, applies again the remote changes that
arrived meanwhile, and redoes the kept commits. Undoing only the
rejected commits would write stale values over a peer's later change to
the same attribute. The rejected and later commits leave the queue, the
buffer and the undo history, and a warning that hides after ten seconds
tells the user. The pure bookkeeping lives in
app.main.data.persistence.commits.

The pending notice of a commit the rollback dropped no longer leaves the
save status at pending, which made "Pin this version" hang and fail.

Debug tools drive these cases by hand: debug.commitInvalidChange() makes a
save fail validation, holdSaves() delays the answers, failNextSave() and
failNextRollback() fail them on purpose (backend flag debug-save-faults),
and persistenceState() shows the queue.
2026-11-05 23:59:32 +01:00
alonso.torres
ef4d66c3aa 🐛 Apply remote changes under buffered local edits too
A remote commit undoes the pending local commits, applies the remote
changes and replays the local ones, so the local file ends like the
server will: remote first, local last. Only queued commits took part;
the ones still waiting in the persistence buffer (up to ~3s of edits)
stayed under the remote change. When both touched the same attribute,
the remote value won locally but lost on the server.

The buffer now lives in the store ([:persistence :buffer]) instead of
an rx buffer, so the rebase can reach it. A flush merges it per file
into queue entries that keep the list of commits they carry, and the
leave-page guard reads buffer and queue from the store. A buffer left
by a previous session is queued when persistence starts.
2026-11-05 23:59:32 +01:00
Andrey Antukh
45721bd836 🌐 Rehash and validate translations 2026-11-05 23:21:26 +01:00
Andrey Antukh
7d596f8197
🌐 Add translations for: French (Canada)
Currently translated at 92.5% (2185 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-05 22:20:21 +00:00
Andrey Antukh
fb49a992b0
🌐 Add translations for: Hindi
Currently translated at 73.3% (1731 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/hi/
2026-11-05 22:20:19 +00:00
Andrey Antukh
ebedd604e8
🌐 Add translations for: Swedish
Currently translated at 94.7% (2238 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/sv/
2026-11-05 22:20:18 +00:00
Andrey Antukh
c5a29c69e8
🌐 Add translations for: Serbian
Currently translated at 48.0% (1135 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/sr/
2026-11-05 22:20:16 +00:00
Andrey Antukh
5fcc1e375e
🌐 Add translations for: Yoruba
Currently translated at 41.1% (972 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/yo/
2026-11-05 22:20:14 +00:00
Andrey Antukh
dda0445314
🌐 Add translations for: Malay
Currently translated at 22.4% (530 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ms/
2026-11-05 22:20:12 +00:00
Andrey Antukh
3576248fb0
🌐 Add translations for: Hausa
Currently translated at 43.7% (1032 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ha/
2026-11-05 22:20:11 +00:00
Andrey Antukh
79b1e6e8ab
🌐 Add translations for: Dutch
Currently translated at 94.7% (2238 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/nl/
2026-11-05 22:20:09 +00:00
Andrey Antukh
21172c4f0e
🌐 Add translations for: Latvian
Currently translated at 67.8% (1603 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/lv/
2026-11-05 22:20:07 +00:00
Andrey Antukh
fa7834c508
🌐 Add translations for: Korean
Currently translated at 73.3% (1731 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ko/
2026-11-05 22:20:05 +00:00
Andrey Antukh
cd2c4332f3
🌐 Add translations for: Ukrainian (ukr_UA)
Currently translated at 75.3% (1779 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ukr_UA/
2026-11-05 22:20:04 +00:00
Andrey Antukh
0d80598ada
🌐 Add translations for: Croatian
Currently translated at 57.0% (1348 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/hr/
2026-11-05 22:20:02 +00:00
Andrey Antukh
24a191edd7
🌐 Add translations for: Portuguese (Portugal)
Currently translated at 56.2% (1327 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/pt_PT/
2026-11-05 22:20:00 +00:00
Andrey Antukh
f5fce6c5c5
🌐 Add translations for: Czech
Currently translated at 56.8% (1342 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/cs/
2026-11-05 22:19:59 +00:00
Andrey Antukh
0b72372e5d
🌐 Add translations for: Basque
Currently translated at 41.6% (984 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/eu/
2026-11-05 22:19:57 +00:00
Andrey Antukh
855352743a
🌐 Add translations for: Japanese
Currently translated at 94.6% (2235 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ja/
2026-11-05 22:19:56 +00:00
Andrey Antukh
50d58566cb
🌐 Add translations for: Polish
Currently translated at 40.9% (967 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/pl/
2026-11-05 22:19:54 +00:00
Andrey Antukh
8651e58aed
🌐 Add translations for: Italian
Currently translated at 79.2% (1870 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/it/
2026-11-05 22:19:52 +00:00
Andrey Antukh
50c716a0ae
🌐 Add translations for: Persian
Currently translated at 93.3% (2205 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fa/
2026-11-05 22:19:50 +00:00
Andrey Antukh
5fa7edb8e3
🌐 Add translations for: Chinese (Traditional Han script)
Currently translated at 92.9% (2194 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/zh_Hant/
2026-11-05 22:19:48 +00:00
Andrey Antukh
fae5036252
🌐 Add translations for: Hebrew
Currently translated at 80.0% (1889 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/he/
2026-11-05 22:19:47 +00:00
Andrey Antukh
cd95407035
🌐 Add translations for: Indonesian
Currently translated at 61.0% (1441 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/id/
2026-11-05 22:19:45 +00:00
Andrey Antukh
82be9680b2
🌐 Add translations for: Arabic
Currently translated at 40.7% (961 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ar/
2026-11-05 22:19:44 +00:00
Andrey Antukh
b27df30e36
🌐 Add translations for: Romanian
Currently translated at 70.5% (1666 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ro/
2026-11-05 22:19:42 +00:00
Andrey Antukh
bca9dfe906
🌐 Add translations for: German
Currently translated at 79.6% (1880 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/de/
2026-11-05 22:19:40 +00:00
Andrey Antukh
7351bac796
🌐 Add translations for: Portuguese (Brazil)
Currently translated at 50.3% (1188 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/pt_BR/
2026-11-05 22:19:39 +00:00
Andrey Antukh
3df670f291
🌐 Add translations for: Chinese (Simplified Han script)
Currently translated at 85.9% (2030 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/zh_Hans/
2026-11-05 22:19:36 +00:00
Andrey Antukh
8292320e30
🌐 Add translations for: Turkish
Currently translated at 94.7% (2238 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/tr/
2026-11-05 22:19:35 +00:00
Andrey Antukh
5c6f3ddb8b
🌐 Add translations for: Russian
Currently translated at 95.4% (2254 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ru/
2026-11-05 22:19:33 +00:00
Andrey Antukh
b58fa9b1bd
🌐 Add translations for: French
Currently translated at 94.7% (2238 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-11-05 22:19:31 +00:00
Andrey Antukh
0032642399
🌐 Add translations for: Spanish
Currently translated at 99.6% (2352 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/es/
2026-11-05 22:19:29 +00:00
Andrey Antukh
2f19f68296
🌐 Add translations for: Catalan
Currently translated at 93.2% (2201 of 2361 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/ca/
2026-11-05 22:19:27 +00:00
Hosted Weblate
8a59ffd886
🌐 Update translation files
Updated by "Cleanup translation files" hook in Weblate.

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/
2026-11-05 22:45:28 +01:00
Hosted Weblate
0c3a5de957
🌐 Merge branch 'origin/develop' into Weblate. 2026-11-05 22:45:23 +01:00
Andrey Antukh
97fe51e08b Merge remote-tracking branch 'origin/staging' into develop 2026-11-05 22:43:31 +01:00
andrés gonzález
84e0b33b74
📚 Explain what to do after losing access to an account (#12866)
Add a section to Your account for cloud users who can no longer sign
in, for example after their Google account was suspended. It points them
to support@penpot.app, lists what to send so support can confirm the
account is theirs, and suggests exporting important files.

AI-assisted-by: claude-opus-5-5
2026-11-05 22:30:20 +01:00
Elena Torró
afc40a3309
🐛 Refresh wasm heap views after allocations and wasm calls (#12833)
Growing the heap detaches typed views taken before an alloc or a call.
Take them after, so reads and writes never hit a detached buffer.
2026-11-05 18:09:59 +01:00
Elena Torró
a32f5a9811
⚡ Index fonts by family when fixing deleted fonts on page load (#12853) 2026-11-05 17:51:42 +01:00
Andrey Antukh
9234e829d9
✨ Render email templates inside the mail job (#12448)
* ✨ Render email templates inside the mail job

The web process rendered each email to full subject and HTML body
before submitting it, so every queued job carried a huge params
payload. The queue now stores only the template id plus the raw
template data, and the new :send-templated-mail job renders and
sends the email inside the worker.

The old :sendmail job stays in place to drain jobs submitted in the
previous format; the quota notice and the REPL test mail keep using
it. All templated callers (auth, profile, feedback, invitations,
comments, Nitrate) submit recipes through eml/send-template.

Closes #12445

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Unify templated mail on template id strings

The ::eml/template value was a namespaced keyword whose name had to
match the registry key, with one live mismatch (user-feedback vs
feedback). Callers now pass the registry id string itself, so keys
stay ::eml/-namespaced and values stay plain data, and the single
templates set rejects unsendable mail both at submit and in the
worker.

Closes #12445

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Encode templated mail data with the template schema

The job params column is plain JSON, so the URIs of an organization
(:logo, :avatar-bg-url) were stored as record maps and the template
check rejected them in the worker. Invites for organization teams,
organization invites and renewal notices failed without a retry.

Each registry entry now carries the JSON encoder and decoder of its
template schema: send-template encodes the data before submit and the
worker decodes it before rendering. The job schema keeps only the mail
envelope, as the template schemas own the typed data.

Closes #12445

AI-assisted-by: claude-opus-5-5

---------
2026-11-05 17:46:48 +01:00
Elena Torró
02223a67fa
🐛 Fix unmoved layout child stretched during wasm gap drag (#12831)
Follow-up to 419513011c: only pure moves pass down to children in wasm.
Remove unused set-wasm-rotation-modifiers.
2026-11-05 17:33:07 +01:00
Eva Marco
eb6e976c01
♻️ Move the dashboard header styles to the DS (#12676)
* ♻️ Move the dashboard header styles to the DS

The page header of the dashboard and settings pages took its styles from
common/refactor/common-dashboard.scss. Move the rules still in use to
dashboard/common/header.scss, written with DS tokens:

- The page titles become heading* (title-large) with a class instead of
  a bare h1; the team section links get a class, and the active class
  moves from the li to the link.
- Drop the rules that matched nothing: .element-name, .context-menu,
  .icon and .btn-secondary inside the header, and the global .btn-primary
  and .btn-secondary (only the fonts page uses them; it keeps the old
  import until its open change lands).
- The sidebar and subscription styles imported the file without using
  it; drop the import.

The header looks the same (0 px in the new visual test). The title line
height goes from 1.15 to 1.4, which does not move the text.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the dashboard libraries and search styles to the DS

Replace the deprecated spacing, size and border variables with DS
tokens and logical properties, and drop the refactor/ import.

- The search placeholder icon gets a class for its color instead of a
  .icon svg rule; its xl size already gives the 32px box.
- Delete the selectors no markup uses: .dashboard-projects in both
  files and .dashboard-shared in search.

No visual change (0 px in the new libraries and search visual test).

AI-assisted-by: claude-opus-5-5

* 🔥 Move the fonts page header to the DS and delete common-dashboard

The fonts page was the last user of common/refactor/common-dashboard.scss:
its header now uses the shared DS header styles and a DS heading, like
the other dashboard pages. Nothing else imports common-dashboard, so it
is deleted.

No visual change.

AI-assisted-by: claude-opus-5-5

* 📎 Remove a double blank line in the dashboard sidebar styles

Prettier rejects the two blank lines left after the imports, which made
the frontend lint fail in CI.

AI-assisted-by: claude-opus-5-5

* 📎 Remove a blank line between the subscription style imports

Stylelint (at-rule-empty-line-before) rejects the blank line left between
the @use rules after the common-dashboard import was dropped.

AI-assisted-by: claude-opus-5-5
2026-11-05 17:01:04 +01:00
Pablo Alba
f4a0d42363
🐛 Keep multi-select variant switches refreshing every instance (#12857)
* 🐛 Keep multi-select variant switches refreshing every instance

Multi variants-switch ended each swap with select-shape, which replaced
the selection, interrupted layout/WASM work, and navigated once per copy
so only one instance looked updated under flex. Skip per-swap select and
layout in the batch, re-read live geometry, coalesce layout ids via a
companion DataEvent, then restore selection once. The same path covers
Assets component-multi-swap. Frontend tests lock select-shape count,
layout coalesce, and mixed no-op selections.

Closes #12829

AI-assisted-by: Composer
2026-11-05 16:52:52 +01:00
Eva Marco
0f7e3fe92e
♻️ Use DS icon buttons in the path editing bar (#12682)
* ♻️ Use DS icon buttons in the path editing bar

The path editing bar now uses DS icon buttons (ghost, tooltip below) and
DS icons, like the top toolbar. The toggles (draw/move mode, handler
types, snap) use aria-pressed, and the handler type trigger uses the DS
flyout indicator, with no tooltip because its menu opens on hover.

This also shows the handler and snap icons again: since #12659 their
SVGs draw with currentColor, and the legacy %button-icon set color to
transparent, so they were invisible.

Add a visual test of the bar, its hover and the handler menu.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the path editing bar styles to the DS

Drop the deprecated styles import, resolve --panel-border-color to
--color-background-quaternary and use logical properties, except top and
height of the bar, which its transition names.

Delete the selectors the DS buttons left unused: .handler-trigger and
.flyout-indicator (the DS icon button draws its own indicator).

No visual change.

AI-assisted-by: claude-opus-5-5

* 📎 Add the blank line stylelint expects in the path bar styles

Stylelint (scss/double-slash-comment-empty-line-before) rejects the
comment right after a declaration, which made the frontend lint fail in
CI.

AI-assisted-by: claude-opus-5-5
2026-11-05 16:28:56 +01:00
Belén Albeza
fa1b3d4708
⚡ Keep only the shape data the worker reads (#12840)
* ⚡ Keep only the shape data the worker reads

The worker kept a full copy of the current page for its selection and
snap indexes. It now keeps only the attributes the indexes and the
change replay read, and re-indexes only the shapes that changed.

On a 52k-shape page the worker heap goes from 698 MB to 286 MB.

Closes #12740

AI-assisted-by: claude-opus-5-5

* ♻️ Share the worker re-trim with its replay test

The worker and its replay test now share `retrim-objects`.

AI-assisted-by: claude-opus-5-5
2026-11-05 16:25:29 +01:00
Alexis Morin
d685163260
🌐 Add translations for: French (Canada)
Currently translated at 97.6% (2222 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-05 14:51:23 +00:00
Nimesh
9d3d6da0d0
♻️ Migrate inspect components to modern syntax (#12435)
Migrate the remaining legacy components in the inspect module to the
modern rumext syntax: typography-block, text-block, svg-attr, svg-block,
selection-rect, selection-feedback, layer-item and left-sidebar.

Rename the ?-suffixed props of layer-item (component-child? to
is-component-child, hide-toggle? to hide-toggle) and drop the unused
sortable? and filtered? props passed from left-sidebar.

Relates to #9260

AI-assisted-by: claude-opus-5-5

Signed-off-by: Nimesh Kulkarni <167794819+GitNimay@users.noreply.github.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-05 15:47:59 +01:00
Marina López
eef6868e9f
✨ Forward attribute update events (#12861) 2026-11-05 15:12:47 +01:00
Elena Torró
d82b5a4b1e
🐛 Fix pasted text invisible in new text box (#12859) 2026-11-05 15:00:46 +01:00
Belén Albeza
eae2c5bc95
🐛 Fix flaky plugin creation parent test (#12858)
The svg-with-images case asserted that the stubbed rp/cmd! saw only the
URL upload RPC. clear-thumbnail sends :delete-file-object-thumbnails after
a 200 ms timer, so on slow CI runners that call was recorded too and the
assertion failed.

Count only the :create-file-media-object-from-url calls and expect one,
so unrelated RPCs no longer break the test.

AI-assisted-by: claude-opus-5-5
2026-11-05 13:26:19 +01:00
Marina López
e42bae34cf
✨ Preserve subscription audit properties (#12850) 2026-11-05 13:21:23 +01:00
Belén Albeza
c6a9e62a4a
🐛 Fix typography in Settings (#12841) 2026-11-05 11:39:43 +01:00
Eva Marco
9349c3b778
♻️ Make the workspace logo and project name real links (#12699)
* ♻️ Give the workspace logo a real link and the DS logo asset

The Penpot logo in the workspace header was an <a> without href, so the
keyboard could not reach it and it could not open in a new tab. It now
links to the dashboard Recent page; a plain click still runs the in-app
navigation (prevent default), and Ctrl/Cmd-click opens the link in a
new tab.

The logo uses the DS raw-svg* asset instead of the legacy icon (same
drawing), with its own 32px class. The legacy svg kept its 500px height
attribute, so it overflowed the header by 222px above and below. This
was the last legacy icon in left_header.cljs.

New visual test "Shows the Penpot logo link": 0 px of difference at rest
and on hover.

AI-assisted-by: claude-opus-5-5

* ♻️ Make the workspace project name a real link

The project name in the workspace header was a div with on-click that
opened the project files in a new tab, so the keyboard could not reach
it. It is now a link to the same page with target "_blank": the browser
opens the new tab, and Tab, Enter and "Open link" work as for any link.

The left header visual test gives 0 px of difference.

AI-assisted-by: claude-opus-5-5
2026-11-05 11:32:08 +01:00
Eva Marco
1d8f2fc717
♻️ Give the viewer logo a real link and the DS logo asset (#12702)
The Penpot logo in the viewer header was an <a> without href. For team
members it now links to the dashboard Recent page of the file's team: a
plain click still runs the in-app navigation (prevent default), and
Ctrl/Cmd-click opens the link in a new tab. Without team permission it
stays as before: no href and no click.

The logo uses the DS raw-svg* asset instead of the legacy icon (same
drawing, same 28px class). This was the last legacy icon in
viewer/header.cljs.

The "Viewer header" visual test gives 0 px of difference.

AI-assisted-by: claude-opus-5-5
2026-11-05 11:31:50 +01:00
Eva Marco
f827b7b4a6
🐛 Make the dashboard template cards keyboard buttons (#12710)
A template card was an <a> without href inside a focusable div: Tab
reached the div, and neither Enter nor Space imported the template, so
the cards could not be used from the keyboard. Importing is an action,
so the card is now a <button> with the same look; it takes the tab
index the div had, and the manual Enter handler goes away.

Keyboard focus looks like hover (accent border and "Add to your
project").

The templates visual test gives 0 px of difference at rest, on hover and
for the scroll and collapse buttons; a new shot covers the focus.

AI-assisted-by: claude-opus-5-5
2026-11-05 11:31:33 +01:00
Eva Marco
686b47742d
🐛 Make the profile page modal links keyboard buttons (#12712)
"Change email" and "Want to remove your account?" were <a> without href
that open modals: the keyboard could not reach them. They are now
<button type="button"> that keep their text-link look (accent color,
underline on hover), so Tab reaches them and Enter or Space opens the
modal.

The profile visual test gives 0 px of difference, now also with a shot
of the hover on "Want to remove your account?".

AI-assisted-by: claude-opus-5-5
2026-11-05 11:31:15 +01:00
Eva Marco
8ab9624f0b
🐛 Make Download report.txt a keyboard button (#12723)
On the internal error page and the feedback page, "Download report.txt"
was an <a> without href, so Tab skipped it. It is an action, so it becomes
a <button type="button"> with the same look, underline on hover included.
The type keeps it from sending the feedback form.

A new visual test reaches both pages through a mocked server error and
checks 0 px of difference and the download with Enter.

AI-assisted-by: claude-opus-5-5
2026-11-05 11:30:39 +01:00
Eva Marco
657ce8b054
🐛 Make the login dialog section links keyboard buttons (#12724)
"Forgot password?", "Create an account" and "Login here." switch the
section of the login dialog (viewer and MCP authorize) and of the dialog
on the not found page. They were <a> without href, so Tab skipped them.
They become <button type="button"> with the same look; a class replaces
the bare a selector in static.scss.

A new visual test opens both dialogs, uses the keyboard to switch
sections and checks 0 px of difference at rest and on hover.

AI-assisted-by: claude-opus-5-5
2026-11-05 11:30:23 +01:00
Andrey Antukh
9d898b8b31
🔧 Normalize the tenant to lowercase (#12843)
An operator writing `PENPOT_TENANT=ACME` and one writing `acme` name the
same tenant: the value is compared as an exact string in the keys and
rows it lands in, and one uppercase letter would otherwise split one
tenant into two.

The value is normalized when the configuration is read, so nothing
downstream has to care about case, and the validated form accepts only
lowercase labels from then on.

AI-assisted-by: deepseek-v4.1-flash
2026-11-05 08:45:01 +01:00
Alonso Torres
eb9594a614
🐛 Fix ChatGPT tool discovery with the submit_feedback tool (#12845)
* 🐛 Fix ChatGPT tool discovery with the submit_feedback tool

ChatGPT compiles the regex patterns advertised in tool input schemas
with Python, which rejects Unicode property escapes such as \P{Cc}. The
submit_feedback subject pattern used one, so ChatGPT dropped the whole
tool list ("action discovery failed") on deployments that register the
tool.

The subject pattern now spells out the Cc range as a character class,
which accepts the same input. A test asserts that no advertised pattern
uses \p{...} or \P{...}.

AI-assisted-by: claude-opus-5-5

* 🐛 Fix MCP plugin test stub for the version check

plugin.ts checks version compatibility with extractMinorVersion, but the
tests loaded it with a require stub that returned the same dummy object
for every import, so the check threw once the UI initialized.

The plugin.ts tests now share a stub that supplies the real
VersionCompatibility, ErrorUtils and TaskHandler modules, stubs only the
code execution handler and rejects unexpected imports. TaskHandler is
transpiled for the sandbox because Node's type stripping cannot load its
parameter properties.

AI-assisted-by: claude-opus-5-5

* 🐛 Fix Management tests for backend-confirmed setters

Management setters save to the backend first and update the object only
when the save succeeds, so seven live tests failed:

- setter tests now wait for the new value before reading it
- createProject checks fileCount by listing the project again, as
  fileCount is the count from when the project was listed
- project moveTo no longer calls remove on the target team, which teams
  do not have; the live CI driver deletes temporary teams
- renaming the open file waits for its restore, which made the next
  navigation test fail

The README notes on setters, fileCount and team cleanup match this.

AI-assisted-by: claude-opus-5-5
2026-11-04 21:47:55 +01:00
Pablo Alba
7666d7d500 🐛 Fix Creating or combining a variant set renames the component first 2026-11-04 21:44:44 +01:00
Pablo Alba
eeda9c1d18 🐛 Fix renaming a variant set saves half-finished steps 2026-11-04 21:44:44 +01:00
Pablo Alba
d47f366c70 🐛 Fix Creating or combining a variant set saves half-finished steps 2026-11-04 21:44:44 +01:00
Francis Santiago
c7fee6a9e2 ⬆️ Update DHI base image digests
Refresh the pinned DHI digests used by Penpot's container builds.

Stale Debian bases conflict with current APT package versions.

AI-assisted-by: gpt-6
Signed-off-by: Francis Santiago <francis.santiago@kaleidos.net>
2026-11-04 17:51:15 +01:00
makesomethingshit
d42217a32b
🐛 Create Plugin API shapes at the page root (#12499)
* 🐛 Create Plugin API shapes at the page root

Create component instances and SVG shapes at the page root so a board at
the page origin cannot capture them. Continue using the existing forced
frame option for component instances and an explicit parent frame for
SVG imports. Interactive placement retains its current behavior.

Cover component instances, SVG imports and SVG imports with images on
ordinary, flex and grid origin boards, including undo, redo, position
changes and explicit reparenting.

Closes #12086

AI-assisted-by: deepseek-v4.1-flash

Signed-off-by: choi junsoo <junsoo1172@gmail.com>

* 🐛 Preserve interactive creation placement

Cover component and SVG placement inside and outside an origin board.
Exercise the sidebar move lifecycle and verify parent preservation.
Correct the Plugin API changelog category.

AI-assisted-by: gpt-6
Signed-off-by: choi junsoo <junsoo1172@gmail.com>

* 🔧 Cover page-root creation in the Plugin API test suite

Check that LibraryComponent.instance(), createShapeFromSvg() and
createShapeFromSvgWithImages() create shapes at the page root when a
board covers the page origin. Each test removes the shape it creates,
since it lives outside the scratch board.

AI-assisted-by: claude-opus-5-5

---------

Signed-off-by: choi junsoo <junsoo1172@gmail.com>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
2026-11-04 17:40:14 +01:00
Pablo Alba
69fa682217 🐛 Keep variant sets intact under group, mask, board, and layout 2026-11-04 16:45:23 +01:00
Pablo Alba
b2c9ca1d23 🐛 Fix mask shortcut works on variant main 2026-11-04 16:45:23 +01:00
Elena Torró
4be8df5877
⚡ Load the rasterizer iframe only when the SVG renderer needs it (#12835)
Create it once features say SVG, never in wasm mode. Also skip SVG frame
thumbnails in wasm mode.
2026-11-04 15:50:13 +01:00
Eva Marco
2f060e9baf
✨ Add skills for the DS migration epic (#12739)
* ✨ Add skills for the DS migration epic

Two skills that drive the DS migration (#12375) one area at a time, so
anyone working with an agent follows the same flow.

refactor-scss moves one .scss off frontend/resources/styles/common/
refactor/ to the DS tokens, mixins and px2rem as a pure refactor, with
a self-review for the issues reviewers flag and a zero-tolerance visual
check.

migrate-ds-area takes one area of a group issue (buttons and icons,
selects, ...): claim it in the group's PRs table, add a visual test,
change the controls, run refactor-scss, compare before and after with
the bundled diff scripts, then open the draft PR and post the QA
comment. It also lists devenv traps seen while migrating the first
areas.

AI-assisted-by: claude-opus-5-5

* ✨ Add the claim rules to the migrate-ds-area skill

The group issue body is saved whole, so two people claiming at the same
time can lose a row. The skill now rereads the body after saving and
puts the row back if it is gone. A claim with no PR for 7 days is free
once its owner has been asked in a comment and did not answer.

AI-assisted-by: claude-opus-5-5
2026-11-04 15:29:07 +01:00
Alexis Morin
48cb2a83b4
🌐 Add translations for: French (Canada)
Currently translated at 96.9% (2205 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-04 14:51:22 +01:00
María Valderrama
f532b8c12e
✨ Add get-profiles-by-ids management method (#12832) 2026-11-04 14:01:27 +01:00
Juanfran
757ec974df
🐛 Build plugin scripts before Angular copies assets (#12815)
Generate plugin.js before Angular builds each of the eight Angular plugins.
The previous order wrote the script to src/assets after Angular had copied
assets to dist, so clean deploys published manifests without their code.

Verified clean builds include all declared manifest assets for all eight
plugins. Plugin lint, format checks and all 101 tests pass.

AI-assisted-by: gpt-6.1-sol
2026-11-04 13:03:10 +01:00
Elena Torró
8d7e275b6b
🐛 Count id and type in the wasm shape proxy (#12793)
The proxy counted one entry fewer than it yields, so it was never equal to a
plain map with the same entries. Count both fields and test it.
2026-11-04 13:01:05 +01:00
Luke Lau
2573908315
⚡ Drop unused used-names sets from component duplication (#12809)
unames or update-unames! are no longer used after a1a3d09998, and each call to
cfh/get-used-names is a scan across all objects
2026-11-04 13:00:24 +01:00
Francis Santiago
21d2dd207b
Merge pull request #12830 from penpot/fc-debug-staging-apt-deps
⬆️ Update DHI base image digests
2026-11-04 12:57:46 +01:00
Francis Santiago
030509c6fd ⬆️ Update DHI base image digests
Refresh the pinned DHI digests used by Penpot's container builds.

Stale Debian bases conflict with current APT package versions.

AI-assisted-by: gpt-6
Signed-off-by: Francis Santiago <francis.santiago@kaleidos.net>
2026-11-04 12:46:37 +01:00
Andrey Antukh
35baa06606
✨ Migrate legacy file payloads to file_data and delete orphans (#12785)
* ✨ Migrate legacy file payloads to file_data and delete orphans

Copy file.data and labelled file_change.data to file_data on boot
(type main/snapshot, backend db, deleted_at preserved), then delete
file_change rows with data but no label, which no read path reaches.
Large instances pre-migrate with batched statements and fake this
step. Closes #12784.

AI-assisted-by: muse-spark-1.3-contributor

* 📚 Add 2.19.0 operations runbook for legacy file_data migration

Manual pre-migration procedure for big instances, same statements as
migration 0157 in batched transactions, following the convention of
the file-change partition runbook. Closes #12784.

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Address review feedback on the legacy file data migration

Fix the issues raised in the review of the migration that moves the
legacy file payloads to file_data.

The migration deletes the unlabelled file_change rows that carry data,
but restore-files still cleared their date, and the retention test
asserted they came back. A row without a date never reaches objects-gc,
which is the leak fixed in #12419. Restore now unmarks only snapshots,
the rows that carry a label, and the retention test checks that the
legacy payload keeps its date instead of coming back.

Cover the conflict path that had no test: a pre-existing snapshot that
collides with a labelled change row, and a second run that must leave
the legacy columns null and the orphan row absent.

The runbook analyses file_data after the inserts, so the planner does
not wait for autovacuum.

Drop app.srepl.procs.fdata-storage: the copy it performed is what the
migration replaces and nothing requires it.

Closes #12784

AI-assisted-by: deepseek-v4.1-flash
2026-11-04 12:25:01 +01:00
Andrés Moya
177c3c6551
🔧 Add one more test case (#12827) 2026-11-04 12:05:38 +01:00
Florian Schrödl
4b2061f399
✨ Add unit tests for the token forms and token tree (#12781)
* ✨ Add unit tests for the token form and combobox helpers

Several helpers of the token forms and the token value combobox were
already pure but had no tests. Cover the reference under the cursor,
the order of the token tree, moving through the combobox options, the
shadow and typography form values and their validation, which value a
form tab validates, and that a token import needs a json file.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

* ✨ Test the shadow and typography form rules

Add tests for the shadow and typography form schemas: a negative blur
is rejected, a negative spread and a blank blur or spread are accepted,
every shadow needs its offsets and color, a line height needs a font
size, and a token can't reference itself. A typography token submitted
without any field is rejected.

They replace the tests of the validator wrappers, the combobox focus
order and the import file check.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>

---------

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-04 12:04:52 +01:00
andrés gonzález
4ee2a812be
🔥 Remove Beta badge from MCP server in Integrations (#12813)
The Integrations settings page labelled the MCP server as Beta, while
the MCP docs give no maturity status. Drop the badge, its style and the
unused translation key so the UI no longer presents it as beta.

Closes #12800

AI-assisted-by: claude-opus-5-5
2026-11-04 11:30:53 +01:00
Luke Lau
9b672785af
🐛 Fix gap handle drag leaving a stuck preview in wasm (#12782)
AI-assisted-by: claude-sonnet-5-5
2026-11-04 11:29:39 +01:00
Florian Schrödl
e6153b61a5
♻️ Extract the token pill state and tooltip from its component (#12691)
The token pill worked out inside the component whether its token is
applied to the selection, whether it can apply to it at all and why
the token is invalid, so no test could reach that logic. Move it into
`pill-state`, and split the tooltip into `tooltip-info`, which decides
what it says, and `render-tooltip`, which renders it. Both are pure
and get unit tests for the applied, partial, disabled and error states
and each tooltip case.

The two chained `on-click` handlers become one. Nothing else changes
in behavior.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-04 10:56:50 +01:00
Luke Lau
a089f7a5a1
🐛 Fix library telemetry counters (#12787)
At some point the code around this changed, I believe this used to reduce over
file ids. However today it reduces over entire library maps, so it passes a map
instead of an id to `dm/get-in files`. This then hashes the library which
forces it to decode all of its shapes, and then it fails to lookup the data. So
the component/colors/typography counts were always returning 0.

Using reduce-kv fixes this, and we can just get the library data directly with
`(:data library)`.

On the backstage file in penpot-files this brings the time to the telemetry
snapshot down from 3.8s to 1.5s

AI-assisted-by: claude-5.5-sonnet

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-04 10:44:30 +01:00
andrés gonzález
59b2d33824
📚 Link the libraries video directly from the contributing guide (#12810)
The "watch a video explanation" link pointed to the how-to-use section
of penpot.app/libraries-templates, which is broken and will change.
Point it to the YouTube video instead.

AI-assisted-by: claude-opus-5-5
2026-11-04 09:53:16 +01:00
David Barragán Merino
0e9a9a8950
💥 Enable admin console by default (#12475)
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-03 23:34:16 +01:00
Andrey Antukh
68c85c3c69 ✨ Add changelog flow memory and breaking-change generator
Add a `boom` subcommand that generates the `💥` section of CHANGES.md
from the closed, non-excluded milestone issues labelled `breaking change`,
using the last `## Breaking change note` comment of each issue. The whole
comment is copied (headings demoted to `#####` outside code fences), and
each issue becomes a `#### <title>` block with the issue and merged
closing PR references on their own line. `--write` replaces only that
subsection and is idempotent.

Add a `comments` subcommand to gh.py (REST, paginated) so the generator can
read issue comments, and report anomaly type G (`breaking change` issue
missing from `💥`), mirroring the `release highlight` check.

Document the flow in `mem:workflow/updating-changelog` (also readable by
release managers, like `mem:audit-log`), turn the `update-changelog` skill
into a router to it, and update the references in `critical-info`,
`creating-issues` and the scripts memories.

AI-assisted-by: deepseek-v4.1-flash
2026-11-03 22:31:27 +00:00
Andrey Antukh
d0a4ffc2c0 📚 Make the no-wrap rule cover comments and review replies
The writing principles of the PR memory carried the no-wrap rule inside
the PR description section, so it read as description-only and comments
kept getting hard-wrapped. Say that it applies to PR and issue comments
and to review replies too.

AI-assisted-by: deepseek-v4.1-flash
2026-11-03 20:28:25 +00:00
Andrey Antukh
4173cc34ab Merge remote-tracking branch 'origin/staging' into develop 2026-11-03 20:21:46 +01:00
Andrey Antukh
d5e7fe59c9 Merge remote-tracking branch 'origin/main-2.18.x' into staging 2026-11-03 20:17:01 +01:00
Andrey Antukh
59c61dab59 📎 Update changelog 2026-11-03 18:58:56 +01:00
Elena Torró
51cbae6ea5
⚡ Improve memory consumption on shape decoding (#12790)
* ⚡ Avoid decoding the opened page twice for its frame index

The frame index was built over a with-meta copy, so pages got decoded again.
Read it from the page objects themselves and fix the with-meta identity check.

* ⚡ Intern keywords when decoding transit on the frontend

Each shape is decoded on its own, so every shape had its own key keywords.
Cache them in the read handler so all decoded data shares one instance.

* ⚡ Use an array map as the wasm shape proxy delegate

A hash map with ~35 keys costs ~3 KB of nodes per shape on large pages.
An array map holds the same entries in one array, at a small lookup cost.
2026-11-03 17:41:36 +01:00
Nimesh
88cd0933bd
🐛 Raise and make configurable the MCP request body size limit (#12450)
The MCP endpoint parsed request bodies with express.json() and its
default 100kb limit, so large tool calls such as execute_code failed
with HTTP 413 (PayloadTooLargeError). Read the limit from the new
PENPOT_MCP_MAX_BODY_SIZE env var, defaulting to 10mb, and document it.

Closes #12080

Signed-off-by: Nimesh Kulkarni <167794819+GitNimay@users.noreply.github.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-03 16:21:28 +01:00
GennoBou
029d9c0af7
🐛 Fix Japanese autodetection fallback on "ja" locale (#12283)
Most browsers return "ja" rather than "ja-JP" in navigator.language.
Because parse-locale expands "ja-JP" to ["ja_jp" "ja"] but keeps "ja"
as ["ja"], using "ja_jp" causes autodetection to fall back to English.

Normalize the Japanese locale identifier to "ja" and map jpn_JP.po to
translation.ja.js, matching other single-region locales like Ukrainian.

AI-assisted-by: gemini-3.8-flash
2026-11-03 16:11:22 +01:00
Robert
2d649bd53c
🐛 Ignore patch versions in the MCP version compatibility check (#12009)
The plugin compared the full major.minor.patch of the running Penpot
version with the MCP package version. The MCP package version is only
set per minor release (2.15.0, 2.16.0, 2.17.0), so every Penpot patch
release (for example 2.16.2 or 2.17.2) showed the "Version mismatch
detected" warning although nothing changed for the MCP.

Compare only major.minor, move the check into a small module with
unit tests, and update the README versioning note to match.

AI-assisted-by: claude-opus-5-5

Signed-off-by: robertkodra <36516516+robertkodra@users.noreply.github.com>
2026-11-03 16:08:54 +01:00
Álvaro Tejero-Cantero
b19ad65581
🔧 Store devenv workspace clones in the XDG data directory (#11986)
manage.sh has kept the ws1+ workspace clones in
~/.penpot/penpot_workspaces since parallel instances landed (#9906).
That directory sits outside the XDG base directory layout. The default
is now $XDG_DATA_HOME/penpot-devenv/workspaces, with ~/.local/share
standing in when the variable is unset or empty. It is the data
directory and not the cache, because a clone can hold commits that
exist nowhere else.

A directory that already exists at the former default keeps being
used, so existing clones are neither orphaned nor seeded again. An
exported PENPOT_WORKSPACES_DIR still overrides both.

AI-assisted-by: claude-opus-5-5

Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
2026-11-03 16:06:44 +01:00
Florian Schrödl
d1c8b0f959
🐛 Report tokens named below another token with tokenscript (#12529)
A token cannot contain tokens: next to a token `a`, the tokens `a.b`
and `a.b.c` are now invalid with tokenscript, whatever the order of
their sets. As with StyleDictionary, their pills and the token form
show "Group name of a.b conflicts with a token of the same name in
another active set." and `{a.b}` is a missing reference.

The interpreter reports them as :processor/token-name-collision, which
the adapter maps to :error.token/name-collision. frontend/deps.edn
updates penpot/tokenscript-interpreter to v0.2.0, which adds the check.
Since v0.1.0 its errors carry codes and data only, which the adapter
already relies on.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-11-03 16:06:16 +01:00
penpot-renovate[bot]
2bf9f4c78b
⬆️ Update DHI base images (#12767)
- dhi.io/debian-base trixie-debian13-dev@af61a32 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@af61a32 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@834d20c (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@af61a32 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@af61a32 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-debian13-dev@f69a0ea (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@df819cf (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13-dev@df819cf (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@df819cf (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-03 16:03:28 +01:00
penpot-renovate[bot]
b2be633b91
⬆️ Update pnpm to v12.16.1 (#12778)
- pnpm 12.16.1 (mcp/packages/browser-extension/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-03 16:02:56 +01:00
Andrey Antukh
be37341f36 Merge remote-tracking branch 'origin/staging' into develop 2026-11-03 15:07:18 +01:00
Andrey Antukh
610813281b Merge remote-tracking branch 'origin/main-2.18.x' into staging 2026-11-03 15:06:32 +01:00
Andrey Antukh
41654aa1b7
✨ Restrict management RPC methods to their declared shared keys (#12736)
* ✨ Restrict management RPC methods to their shared keys

Management methods now declare the shared-key ids allowed to call them
with ::rpc/shared-keys. A wrapper in wrap-management checks the key id
from the request before opening a transaction, so a valid key outside
the declared set answers 403 shared-key-not-allowed. Methods without
the metadata keep accepting any configured key.

Scope upload-tempfile to exporter and media-processor, file analysis to
nexus, durable jobs to exporter, and the Nitrate methods to
admin-console. The test helper injects a declared key for the allowed
path, and a coverage test fails when a management method is missing or
declares the wrong set.

Closes #12730

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Harden shared-key authorization scope and tests

Rename the per-method metadata to ::rpc/allowed-shared-keys so it reads
as the allowed key ids, not the secrets. Fix the management test helper
to resolve the key from the supplied request and to keep an explicit nil,
which prevents a rejection test from silently using an allowed key.

Strengthen coverage: assert every management namespace is scanned, check
all four job methods reject a key they do not declare, verify a rejected
complete-job leaves the row and its events untouched, and exercise both
keys allowed on upload-tempfile. Log denied keys with the method name and
key id, without the secret, and update the audit-log memory for
push-audit-events.

Closes #12730

AI-assisted-by: deepseek-v4.1-flash
2026-11-03 13:49:36 +01:00
Alonso Torres
410bde8103
🐛 Move displaced path content back onto its box (#12754)
Some files have component copies whose path drawing (:content) sits away
from its box (:selrect and :points). The renderer draws the content, while
the sidebar, layouts and syncs use the box, so these icons render outside
their boards and are clipped or drawn over other shapes. Moving the shape
keeps the gap, and sync never repairs it because both groups are touched.

Add the migration 0034-fix-path-content-offset. For every path in pages
and in component objects, deleted components included, it derives the box
from the content as update-geometry does and, when the sizes agree but the
position does not, moves the content onto the stored box. The box, the
transform and the touched groups stay as they are; paths whose drawing has
another size are left untouched.

Closes #12752
2026-11-03 12:12:08 +01:00
Andrey Antukh
afe6c1b99c
🐛 Confine upload paths to temp dirs (#12772)
* 🐛 Confine upload paths to temp dirs

Reject upload maps whose path leaves the server temp dirs.
Transit bodies can forge a Path such as /etc/passwd and the
handlers open it directly. Centralize the realpath prefix
check in media validation and apply it to all nine upload
fields. Legit multipart files under /tmp/undertow/ or
/tmp/penpot/ still pass.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Rename upload validators to check-*

Drop the validate prefix and the bang from the media
validation helpers for a consistent check naming. Move
the temp dir confinement into validation and cover all
nine upload fields. Add unit tests for each helper.

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Fix svg upload tests to use penpot tmp dir

SVG upload tests wrote tempfiles to the system tmp dir,
outside the allowed upload prefixes, so check-upload-path
rejected them. Point write-temp-svg at /tmp/penpot like
the other upload tests.

AI-assisted-by: muse-spark-1.3-contributor
2026-11-03 12:07:52 +01:00
Juan de la Cruz
4ad8d1d36a
📚 Update outdated screenshots and UI labels in the docs (#12753)
* 📚 Update interface, account and plugin screenshots in the docs

Replace screenshots that showed an older UI: the dashboard sidebar
without Sources and Pinned projects, the account sidebar with Access
tokens, and the workspace without the Tokens tab or the grouped
toolbar.

Covers the interface overview (annotated workspace and dashboard),
the account pages, the plugin manager entry points and the access
token dialogs, which now live under Integrations.

Signed-off-by: elhombretecla <delacruzgarciajuan@gmail.com>

* 📚 Update design and design system screenshots in the user guide

Replace screenshots and videos that showed an older UI: the 1.x custom
fonts pages and context menu, the toolbar with separate ellipse and
pencil tools, the sidebar without the Tokens tab, the old Libraries
button, the color picker without the tokens switch and the token menus
with a shorter list of types.

Covers workspace basics, layers, text and fonts, color, export,
assets, libraries, components and design tokens.

Signed-off-by: elhombretecla <delacruzgarciajuan@gmail.com>

* 📚 Match docs text to the current UI labels

Access tokens now live under Your account > Integrations, and the
buttons read "Create new access token" and "Create access token".
The workspace main menu lists Plugins and Help & info. The libraries
panel opens from the "Add library" or "Manage library" button. The
dashboard sidebar names its sections Libraries and Fonts.

Signed-off-by: elhombretecla <delacruzgarciajuan@gmail.com>

---------

Signed-off-by: elhombretecla <delacruzgarciajuan@gmail.com>
2026-11-03 10:59:09 +01:00
Alonso Torres
43f3bda8b5
✨ Add mcp browser extension (#12627)
* 🐛 Fail MCP tasks at once when their plugin connection closes

A response only counts when it comes from the connection the task went
to, so a task sent over a connection that then closes can never finish.
It used to wait for its full timeout. The plugin bridge now fails such
tasks as soon as their connection closes.

Connections also log their close code and reason, to tell a page
closing its socket from a browser or network failure.

* ✨ Announce the MCP session to a browser extension

Let a browser extension learn which tab holds an MCP session, so it can
keep the browser from discarding that tab. While the MCP plugin runs,
the page listens for the extension's hello message, and says it is
ready once it listens, so that a content script that said hello too
early says it again. After a hello, the page posts its session ID and
connection status, and posts them again on every status change. When
the plugin stops, it reports the session as disconnected.

The plugin also reports "disconnected" while it reconnects on its own,
for instance after a page resume, so the page announces "connecting"
then and "disconnected" only once the user has stopped the connection.
Otherwise the extension would forget the session at every reconnect.

The page only accepts messages posted on its own window by its own
origin, posts to its own origin only, and sends nothing before the
extension asks. It reaches the window through app.util.globals, whose
stand-in for the unit tests accepts postMessage.

* ✨ Keep Penpot tabs connected to MCP from being frozen

Chrome freezes some background tabs, for instance busy ones while
Battery Saver is on, and a frozen page cannot run the MCP plugin.
Chrome does not freeze a page that holds a Web Lock, so the page holds
a lock named penpot-mcp-session from the moment the user asks to
connect until the connection stops: on disconnect, logout, MCP being
turned off or the MCP runtime starting again. The lock is shared, so
several tabs can hold it at once. Browsers without Web Locks skip it,
and when the lock request fails, the next connection request tries
again.

Firefox does not freeze background pages, so the lock changes nothing
there.

* ✨ Add a browser extension that keeps MCP tabs from being discarded

A browser may discard a background tab to save memory. A discarded
Penpot tab reloads when the user returns to it and loses its MCP
session.

Add mcp/packages/browser-extension, a Manifest V3 extension for Chrome
and Firefox, built from one source into dist/chrome and dist/firefox.
It is written in TypeScript, bundled with esbuild and imports the
protocol types from mcp-common. The package joins the mcp workspace, so
the build, test and fmt scripts of mcp/ cover it; the server release
build skips it.

Its content script runs on design.penpot.app, on the test environments
under penpot.dev and on local development instances. It asks the page
for its MCP session on load, again when the page says it is ready and
when the page returns from the back/forward cache, and forwards each
valid session announcement to the background. It accepts only messages
the page posted on its own window from its own origin, and posts its
hello to that origin only. It also tells the background when the page
is hidden.

The background records which tab holds each MCP session. A tab holds
the session it last announced as connected, until its page announces a
disconnect, is hidden, or the tab is closed or replaced. The map lives
in chrome.storage.session, so it survives restarts of the background
and clears when the browser closes. While a tab holds a session, the
browser may not discard it automatically.

The extension asks only for the storage permission. Chrome runs the
background as a service worker; Firefox runs it as an event page and
needs the add-on ID, mcp-extension@penpot.app, and Firefox 140. The
toolbar popup lists the connected tabs. The icon is the Penpot logo,
white on a black rounded square, with the SVG source next to the PNGs.
A README explains how the extension works and how to build and load
it.

* 🐛 Ask MCP plugins for heartbeats so hidden tabs do not look frozen

The server refuses to send a task to a plugin whose last heartbeat is
older than 30 s, taking its tab for frozen. The plugin sent heartbeats
only from a 10 s timer, and Chrome runs the timers of a hidden page as
rarely as once a minute: without Energy Saver after about a minute
hidden, and at once with it. A Penpot tab hidden for a minute therefore
failed that check about half the time, although it could run tasks.

Chrome handles socket messages of a hidden page at once, and a frozen
page cannot handle them at all. The server now sends each plugin a
heartbeat request every 10 s, and the plugin answers it with a
heartbeat, so the heartbeat age again tells a frozen tab from a hidden
one. The plugin keeps its timer, so it still sends heartbeats to a
server that does not ask. A plugin that does not know the request
passes it to the plugin code, which ignores it.
2026-11-03 10:07:50 +01:00
Alonso Torres
2f57d380db
✨ Allow send custom binary to the agents (#12770) 2026-11-03 09:31:59 +01:00
Alexis Morin
ed59b878d9
🌐 Add translations for: French (Canada)
Currently translated at 96.2% (2189 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr_CA/
2026-11-03 02:51:20 +00:00
Andrey Antukh
93d36d04d8
✨ Stabilize fragmented file data across snapshots and GC (#12420)
* ✨ Stabilize fragmented file data across snapshots and GC

Snapshots, automatic and user, now store pointer-based rows that share
fragments with the live file instead of duplicating page content.
`get-file-snapshot` resolves those pointers on the server so version
previews keep rendering plain data, and `restore!` reuses the shared
rows. Export still materializes (the package carries no fragments) and
import regenerates them.

`file-gc` treats the main file and every unpurged snapshot as liveness
roots, reads the file unrealized, and reattaches the original pointer
wherever the cleaned value is unchanged, so a pass over an untouched
file writes no fragments and a pass over an edited file keeps the ids
of the pages it did not touch. Snapshot creation and the restore
backup take the file row `FOR SHARE`, which serializes them with the
pass and closes the race that let a new snapshot reference fragments
the pass was about to mark.

All backend pointer bindings go through the `fdata/with-pointers`,
`with-tracked`, `with-tracked-pointers` and `with-untracked` macros.
The loader reads fragments a previous pass marked until `objects-gc`
purges them, and the frontend fails loudly with retry instead of
dropping an unresolved fragment.

Review fixes in this branch: the automatic snapshot attaches before
pending migrations, `realize-pointers` is the single verb,
`reattach-unchanged` takes the identity fast path, the restore guard
takes its revn from the backup read, and `PointerMap.withMeta`
generates a fresh id when metadata changes.

Tests cover both snapshot writers (red without the lock), snapshot
sharing and pin release through the real delete and purge paths,
reattach against orphan marking, preview after offload, the pointer
binding contracts and the with-meta id change.

Closes #12410

AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: deepseek-v4.1-flash

♻️ Pin no-churn behavior for untouched pages

Pin the behavior the churn verification measured: a save keeps the
fragment id of every page it does not touch, with a page of eleven keys
(PersistentHashMap), a 46-shape objects map and a nine-page index (hash
map). Also pin the mixed case: a data-level objects-map flip rewrites
the old pointer pages once and the next save keeps the new ids.

The memory records the identity rule behind it: re-assoc with the same
value returns the same map in array and hash maps alike.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

⚡ Stop loading untouched pages when enabling objects-map

`enable-objects-map` wrapped every page on every save, and reading a
page's objects forced its PointerMap to load the fragment. A save of
one page in a seven-page file therefore made seven fragment reads.

Skip unloaded pointers: their fragment was already wrapped by the
encode-file that wrote it, so loading it only to wrap it again is a
no-op. The save now loads only the pages it edits. A page with a plain
fragment (a data-level feature flip) keeps its plain fragment until it
is first loaded or edited; a migration that enables the feature must
rewrite the data itself.

Cover the change with load-count tests for save and restore, update the
flip test to the new contract, and drop the stale restore comment.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

⚡ Keep page fragment ids through pending migrations

Applying a pending migration on read realized the file, ran the
migration and persisted it again; the persist wrapped every plain page
in a new PointerMap, so every page got a new fragment id even when the
migration did not change it. That leaves the fragment cache cold and
rewrites every fragment on the first read after a deploy.

Capture the original pointers before realizing, reattach the unchanged
positions after migrating (same value comparison file-gc already uses),
persist the reattached file and return the realized one: the read
contract is plain data. Read-only connections skip the persist and the
reattach. A migration that adds objects-map skips the reattach so the
pages get wrapped.

Move the three helpers to fdata so file-gc and the migration path share
them, and cover the path with a real 0031 migration test: the unchanged
page keeps its id, the changed page gets a new one, the caller gets
realized data and a second read does not migrate again.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Realize library pointers with each library's own loader

Fragment rows are keyed by file id, and every caller binds the
fragment loader of the file it works on. get-resolved-file-libraries
returned libraries with their pages still as pointers, so the first
read of a library page under fdata/pointer-map asked the consuming
file for the fragment and failed with :fragment-not-found. That
broke migrations and file validation for every file using a
pointer-mapped library.

The weak-map loader now realizes each library's pointers with the
library's own id. The test fails without the change.

AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>

* 🐛 Restore the pointer-map require the repair path still needs

The branch dropped `app.util.pointer-map` from the namespace while the
auto-repair path (added to develop before this branch) still rebinds
`pmap/*tracked*` around validation and repair in
`repair-preexisting-damage` and `process-changes-and-validate`. The
namespace did not compile, so the whole backend failed to load.

Those toggles do not fit the fdata macros: `with-untracked` binds nil
and `with-tracked` creates a fresh atom, while the repair path has to
turn tracking off and restore the existing atom (and read it before the
repair starts). Restore the require and keep the bindings explicit.

Also pin the feature-off path with a test: with `fdata/pointer-map` off
(the default), a save creates no fragments, file-gc writes nothing and
the snapshot preview still returns plain data. Drop a stray `\n` char
literal left at the end of one deftest.

Update the memories with the exception to the macro rule, the snapshot
FOR SHARE locking and the pre-migration automatic snapshot.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Validate debug files with their own pointer loader

The `/dbg/actions` file validation passed the file id where
`get-resolved-file-libraries` expects the file map, so the library map
was seeded with a nil key. It also validated a file read without
realizing it, so the first page lookup of a pointer-mapped file raised
"load is not supported when *load-fn* is not bind" and the endpoint
could not validate exactly the files the feature touches.

Read the file realized (validation walks every page and component
anyway) and build the library map from the file. The library loader
already resolves each library's pointers with its own id, so the
realized file and its libraries validate as plain data. Covered by a
handler-level test that fails before the change.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Bind the snapshot loader through the fdata helper

`reduce-snapshots` hand-rolled the same `binding` that
`fdata/with-pointers` builds (it also loaded the function's own
pointer loader before entering the reduction). Use the macro so the
rule holds there too and the namespace no longer needs
`app.util.pointer-map`; the loader is identical (same cfg and file-id)
and snapshot walking behaves the same.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Note the pointer-binding exceptions and snapshot ordering

Record what the feature review settled:

- Two direct `binding`s stay on purpose: the fine-grained `*tracked*`
  toggles of `app.rpc.commands.files-update` (validation/repair turn
  tracking off and restore the existing atom) and
  `binfile.common/encode-file` (fresh tracked set inheriting the
  current one). Atom-scoped macros would only be worth it if the set
  grows.
- The automatic snapshot is taken before the pending migration, and
  the pre-migration window is narrow: the `get-file` RPC migrates on
  read and its ETag hashes the available migrations, so a post-deploy
  read cannot be a 304 from the pre-deploy state. The old-format
  snapshot only appears when the save is the first request to touch
  the file after the deploy, and `get-file-snapshot` previews it as
  stored.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Apply pending migrations when previewing a version

Automatic snapshots are attached before the save applies pending file
migrations, so a version can store the old format. The preview RPC
handed that data to the UI as stored, and the viewport renders it with
the current code.

Realize the snapshot with the live file id (fragments are keyed by
file, not by snapshot), normalize its migrations to a set (the decoder
keeps the pgarray as a vector, and `set/difference` would compare
indexes) and apply the pending migrations with the pure
`fmg/migrate-file` and the live libraries. Nothing is persisted: the
stored version keeps its data and restoring it re-migrates on the next
read. The response now carries the snapshot's migrations.

Use `:features` the way `get-file` does for `fdata/objects-map`: a
client that does not declare it gets plain objects. The historical
feature set is not checked against the enabled one on purpose: a
version may predate a feature change.

Cover the preview of a snapshot with a pending migration and the
objects realization; both pin that the stored row is untouched. Update
the file-data memory with the preview contract and the vector/set
trap.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Decode snapshot migrations as an ordered set

`file-snapshots/decode-snapshot` handed the `file_change.migrations`
pgarray as a vector. `set/difference` falls back to index lookups when
the first collection is shorter than the second, so the pending set was
right only by accident: a stored subset happens to work, while a
stored set larger than `available-migrations` (a snapshot taken by a
newer build) marks every available migration as pending and re-applies
it to already migrated data.

Decode it as a `d/ordered-set`, the type the rest of the code and
`schema:file` expect. This also covers `restore!` and the snapshot
list, and it lets `get-file-snapshot` drop the `(into #{} ...)`
normalization it was carrying. Re-wrap the union in `fmg/migrate` so
the invariant holds even when a caller hands a plain set.

A snapshot with NULL `migrations` predates SQL migration 0140 and
keeps the format version of its time. Pin that legacy path: the
preview regenerates the applied set from that version, migrates in
memory and leaves the stored row untouched. Update the file-data
memory with the invariant and the legacy path.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Fix flaky waiting-gauge assertion in climit test

The throwable-while-waiting test asserted the exact list of
waiting-gauge publications, but the component sampler republishes
the same count when its first tick lands inside the recorded
window under load. Assert only the last value, which is the
contract the helper documents.

AI-assisted-by: muse-spark-1.3

---------

Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Co-authored-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
2026-11-02 15:02:55 +01:00
Belén Albeza
79aebcca49
🐛 Free the worker's copy of a page when leaving it (#12751)
* 🐛 Stop worker index updates from adding nil-keyed entries

When a commit touched a page the worker had not indexed, or deleted the
indexed page, `:index/update` ran the selection and snap updates with a
nil page. That left entries under a nil key (and a stale selection index
for a deleted page) for the rest of the session.

Skip updates for pages the worker does not hold, since a later
`:index/initialize` sends the whole page again, and drop the selection
and snap indexes when the changes delete the page.

AI-assisted-by: claude-opus-5-5

* 🐛 Free the worker's copy of a page when leaving it

The worker kept every visited page, with its selection and snap
indexes, for the whole session. On large files each visited page added
hundreds of MB to the worker heap.

Add an `:index/finalize` worker command that drops a page and its
indexes, and send it from `finalize-page`, which runs on every page
switch and when leaving the workspace. Coming back to a page indexes it
again from the full page, as before.

Closes #12738

AI-assisted-by: claude-opus-5-5
2026-11-02 12:12:57 +01:00
Andrey Antukh
29373fff9f
✨ Rework the progress of the import and export dialogs (#12192)
* ✨ Rework the import dialog

The import wizard paints a Check/Upload/Import stepper derived from the
dialog status, plus a phase title, a description and a progress bar per
phase, and its rows follow the new design: folder icon, file name and a
size subtitle. While checking, the dialog shows no rows and no spinner,
because the phase header already carries the message.

The wizard writes the job milestone once, in the header
(`jp/current-milestone-text`), since every file of a container reports
the same one. A row keeps its name and its size, says "Ready" once its
container finished uploading, and shows the error text when it fails;
the button that drops a file only appears while the wizard still offers
its import step. The `jobs.queued` label now reads "Waiting job slot".

The bar measures the whole import: determinate while the containers are
uploaded, weighted by their size (`imp/upload-pct`), and animated
otherwise, because the stages of a job are not comparable. The phase
title counts the files, and `imp/imported-count` counts them done with
the job's own counter, so the number moves while a container rebuilds
its files one by one.

The analysis reads the entries of a version 3 package in one pass: a
file shows what it occupies inside its container, its own entries plus
the storage objects its media and thumbnails reference, while what the
container weighs stays in `:container-size`, the size the upload bar is
weighed on. The ready step waits for the end of the analysis
(`imp/ready-to-import?`), not for every known file, so Continue never
leaves out the files that arrive later.

Tests cover the file sizes, the upload percentage, the done counter,
the wizard step and the milestone in course. The change also adds
`-XX:NativeMemoryTracking=summary` to the backend devenv so `jcmd`
shows native memory, and documents the wizard, the promise call styles
(`mem:clojure/promesa`) and two CSS pitfalls: a logical `inline-size`
wins over an inline physical `width`, and Rumext leaves a plain
`:style` map raw, so inline styles leave CLJS as `#js` objects.

AI-assisted-by: glm-5.3-flash
AI-assisted-by: deepseek-v4.1-flash

* ✨ Rework the export dialog

Every exported file is its own job, so each row reports its own state:
folder icon, name and a size-plus-`.penpot` subtitle; waiting for a
worker; running with the milestone of the job and an animated bar; or
exported with a tick. A failed row paints red, takes the close icon and
shows the hint of the job, or a generic label, through
`files-export-modal.export-error`.

Success, failure and cancellation replace each other, so a row never
reports two outcomes at once. The ids of the messages are canonicalized
to string before they match a row, because they cross serialization
boundaries. The export result now carries `:size`, so the subtitle says
the format until the job produces the artifact and the size once it
exists; a transport failure builds a synthetic error, so a nil
`ex-data` no longer leaves the row hanging.

The header counts the files and says whether the export runs or ended;
since every file has its own job, it paints no global bar.

Closing changes too: the Close button is gone. While files are in
flight the footer offers Cancel export and the X cancels as well, which
marks the loading rows cancelled and leaves them in place, so their
outcome can be read; once every row reached a terminal state the footer
offers Done, which dismisses the dialog.

The retry button of the finished design is left out, because the client
has no retry mechanism yet. Styles come along: the rows scroll while
the header and the footer stay put, and the new phase header, row and
bar follow the tokens of the import wizard. The title key becomes
`files-export-modal.title`, and the new keys cover the header, the row
states and the buttons.

AI-assisted-by: muse-spark-1.3

* ♻️ Inline the binfile job adapters into their task handlers

The wrappers in app.binfile.jobs only forwarded to the core, so each
task handler now composes its own run: frozen cfg, active check,
progress bridge and format dispatch. export-cfg freezes the whole
export (file-ids, export-type, output) and v3/export-files! takes it
as its single argument; ::bfc/ids becomes ::bfc/file-ids. The bridge
keeps one job: bfj/run forwards the cfg to its callback, and
check-active answers with it to thread the run.

AI-assisted-by: muse-spark-1.3

* 🐛 Migrate binfile tests to the single-arg export-files!

The refactor made v3/export-files! take a single cfg carrying
::bfc/file-ids and ::bfc/output, but six calls in binfile_test
(added by the tokens handling change) still used the old two-arg
form with ::bfc/ids. Move them to the new form, as the rest of
the file already does.

AI-assisted-by: muse-spark-1.3

* 🐛 Fix export artifact expectations to include file size

The export flow now reports the artifact size, but the stale
expectations dropped it and failed the frontend suite. Include the
size in both assertions so they match the emitted message.

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Migrate remaining export-files! test callers to the single-arg form

rpc_binfile_test and offload_file_data_test still passed the output apart
and used ::bfc/ids; move both into the cfg as ::bfc/file-ids/::bfc/output.

* 🐛 Hide the downloads hint when no file was exported

After Cancel or when every job failed, the export header said
"0 files exported · Check your downloads folder."; drop the hint then.

* ♻️ Read the object of a container entry once

The index of a container asked twice for the object of the same zip
entry: once to test it and once as the key of the update. Resolve it
once, in the branch that owns it, so the entries of a file never pay
for a question that does not apply to them.

Addresses a review note on #12192.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Never size an import row with its container's weight

A row of a version 3 package took the size of the container whenever the
container held no entry for its file, while the comment beside it
promised the opposite. An unknown size now stays nil, which the row
already understands: it paints no subtitle instead of a number that
belongs to the whole package.

The test carries a file the manifest lists and the container does not
hold, and the memory records what the row does then.

Addresses a review note on #12192.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Retry pool stats read until snapshot is consistent

Hikari exposes each pool counter through a separate MXBean call, so a
borrow or return between the reads can leave one snapshot off by one.
The test read once and failed in CI with total 2 vs active + idle 3.

Read up to 20 times with a 10ms pause until total equals active +
idle, then assert on the stable snapshot. The invariant stays strict;
only the transient race is tolerated.

AI-assisted-by: muse-spark-1.3-contributor

---------

Co-authored-by: Elena Torro <elenatorro@gmail.com>
2026-11-02 12:12:39 +01:00
penpot-renovate[bot]
ecd0eb2c26
⬆️ Update DHI base images (#12762)
- dhi.io/nginx 1.31.8-debian13-dev@f8c15ec (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@3083ddf (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@ce582f8 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@3083ddf (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@3083ddf (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-02 10:57:24 +01:00
penpot-renovate[bot]
bd7239a05c
⬆️ update GitHub Actions (#12715)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-renovate.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker-devenv.yml)
- docker/login-action v4.7.0 (.github/workflows/build-docker-devenv.yml)
- node 26.13.0 (.github/workflows/plugins-deploy-styles-doc.yml)
- node 26.13.0 (.github/workflows/plugins-deploy-api-doc.yml)
- pnpm/setup v3.1.2 (.github/workflows/plugins-deploy-styles-doc.yml)
- pnpm/setup v3.1.2 (.github/workflows/plugins-deploy-api-doc.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-01 19:27:53 +01:00
penpot-renovate[bot]
31e5fb53d0
⬆️ Update DHI base images (#12725)
- dhi.io/debian-base trixie-debian13-dev@9e479c6 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@9e479c6 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@fefc620 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e479c6 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9e479c6 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-alpine3.24@7a4e902 (docker/images/Dockerfile.storybook)
- dhi.io/nginx 1.31.8-debian13-dev@cdfb6c6 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2-debian13-dev@3f37250 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@2b299a5 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@3f37250 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@3f37250 (docker/images/Dockerfile.exporter)
- node >=20.20.2 (mcp/packages/server/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-01 19:27:24 +01:00
𝐧𝐞𝐫𝐬
b0c1d9acda
🐛 Fix the Cache-Control max-age of redirected assets (#12758) 2026-11-01 19:26:14 +01:00
AntonPalmqvist
39c49fd725
🌐 Add translations for: Swedish
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/sv/
2026-10-31 16:51:35 +01:00
Ingrid Pigueron
8e4fea07a3
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-31 16:51:33 +01:00
Pablo Alba
57f4fc19ff
🐛 Fix variant reparent generates referential-integrity problem (#12706)
* 🐛 Fix variant reparent generates referential-integrity problem

* ✨ Mr changes
2026-10-30 21:42:14 +01:00
Miguel de Benito Delgado
723267cfdd
🐳 Expose GPU in devenv (#12726)
* ✨ Add GPU support to the devenv

AI-assisted-by: gpt-6.1-sol

* 📚 Fix broken link in dev docs
2026-10-30 15:13:29 +01:00
Pablo Alba
11c2292ac6
🐛 Fix variant changed from another page (#12693) 2026-10-30 14:45:19 +01:00
Eva Marco
a69d5d15d2
🐛 Wrap long shortcut key lists in the shortcuts panel (#12735)
* 🐛 Wrap long shortcut key lists in the shortcuts panel

The Next board and Previous board rows of the viewer have five key
combinations. Their keys ran past the panel edge and covered the label.

Since #11081 the label sits inside a span with no flex rules, and the
key list never wrapped. Now each combination is a group that never
breaks, and the list wraps between groups. The comma stays at the end
of its group and "or" leads the last one, so no line starts with a
separator. The label keeps its width and the keys take the rest.

Closes #12641

AI-assisted-by: claude-opus-5-5

* ♻️ Drop unused deprecated imports from the shortcuts styles

shortcuts.scss no longer uses anything from refactor/ or ds/mixins.
Remove both imports and use the typography module without a
namespace. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-30 14:12:49 +01:00
Alejandro Alonso
94836cb8c8 Merge remote-tracking branch 'origin/staging' into develop 2026-10-30 13:22:09 +01:00
Alejandro Alonso
759a2b301c
⚡ Speed up restore of deleted files via storage unmark (#12731)
Rewrite sql:restore-storage-objects so referenced storage ids are
collected as UUIDs first and then joined by primary key. The previous
form cast storage_object.id to text and OR-ed that with id IN, which
forced a scan of deleted storage rows and made restore-deleted-team-
files time out in production.

Closes #12727
2026-10-30 09:41:59 +01:00
Elena Torró
b48d163f5f
⚡ Fix slow drop shadow spread in CPU exports (#12718)
* ⚡ Fix slow drop shadow spread in CPU exports

* 🐛 Fix text opacity in board drop shadows on CPU exports
2026-10-29 18:27:19 +01:00
Eva Marco
0fa1077b9a
🐛 Render no type attribute on DS buttons used as links (#12709)
* 🐛 Render no type attribute on DS buttons used as links

With `to`, button* renders an <a href> but still passed type="button". On
an anchor, type is a MIME type hint, so "button" is invalid HTML. The
type now applies only when the element is a <button> (still "button" by
default).

New "As Link" story with two tests: the link has its href and no type,
and a plain button keeps type="button".

AI-assisted-by: claude-opus-5-5

* 📚 Document the DS button as a link

The Semantics section only said that button* accepts `to`. It now says
when to use it (the control goes to another URL), what it renders (an
<a href> with the button style and no type), how to keep the in-app
navigation on a plain click, and shows the "As Link" story.

AI-assisted-by: claude-opus-5-5
2026-10-29 18:10:16 +01:00
Eva Marco
12dd4d19d0
♻️ Use a DS icon button for Share in the workspace header (#12681)
* ♻️ Use a DS icon button for Share in the workspace header

The Share link of the right header becomes an icon-button* with the DS
share icon, like its neighbours (comments, history, viewer). It gets
keyboard focus, a name and the DS tooltip and hover.

It also makes Share visible again on develop: since the share icon
moved to currentColor, the legacy icon drew with the transparent color
of its svg.

The button is 32px wide instead of 28px, so the zoom, comments and
history controls move 4px to the left.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove the share link styles from the workspace header

Delete .share-btn and its svg rule, which no markup uses since Share is
a DS icon button. refactor/ stays only for the menu placeholders.

No visual change (0 px).

AI-assisted-by: claude-opus-5-5
2026-10-29 18:10:02 +01:00
Eva Marco
755369276d
🐛 Fix stack overflow when applying a library update (#12661)
* 🐛 Fix stack overflow when applying a library update

Each shape updated by a component sync added its reg-objects undo change
with (update :undo-changes concat [...]). That wrapped the whole undo list
in one more lazy concat per shape, with the old list at the head. After a
few thousand shapes, walking the list overflowed the stack, which matches
the nested LazySeq frames reported when a library update is applied.

add-update-attr-changes now conjs the reg-objects before the mod-obj, so
undo restores each shape and then recalculates its parents, as redo does.

The trailing reg-objects also recalculated groups after undoing shapes
that the sync added or removed. To keep that, add-shape-to-instance,
add-shape-to-main and remove-shape now add their own undo reg-objects,
conjed before their other undo changes so it runs after them.

Tests cover many calls to add-update-attr-changes, a sync and undo of
many shapes, and undoing a sync that moves, adds or removes a child of a
group in the copy.

Closes #12576

AI-assisted-by: claude-opus-5-5

* 🐛 Test undo of an inverse sync that adds a shape

The previous commit added an undo :reg-objects to add-shape-to-main, so
undoing an "update main component" recalculates the main's groups after
removing the added shapes. No test covered it.

The new test adds a shape at the start of a group in the copy and changes
a sibling's fill, runs the inverse sync and undoes it. Without the undo
reg, the main group keeps the bounds of the removed shape.

AI-assisted-by: claude-opus-5-5
2026-10-29 18:09:46 +01:00
Eva Marco
66f1f689f8
♻️ Use the DS select for the gradient type in the colorpicker (#12653)
* ♻️ Use the DS select for the gradient type in the colorpicker

The Linear / Radial select of the gradient options used the legacy
select. It now uses select* with string ids, converted back to the
gradient type keyword in the handler.

Visible changes: the DS select look (label 1px higher, DS list with
taller rows and the DS check mark, opening 2px lower). The list still
opens inside the colorpicker, so it needs no portal.

The "Create a RADIAL gradient" e2e test picks the type by role instead of
a DOM position that only matched the legacy markup. New ds visual test
"Gradient type select": closed, hover, open and changed.

AI-assisted-by: claude-opus-5-5

* ♻️ Move colorpicker gradient styles off the deprecated styles

gradients.scss now uses DS spacing, sizes, borders, typography and
logical properties, and resolves the --menu-shadow-color alias. The
deprecated import stays only for %input-element: the stop offset input
moves to the DS with the numeric inputs group.

The offset input uses body-small (line-height 1.3 instead of 1.4); the
input keeps its height, so nothing moves. 0px difference against the
previous commit, including a new shot of the whole gradient panel.

gradients.cljs uses an icon-text class that has no rule in this file.

AI-assisted-by: claude-opus-5-5
2026-10-29 18:09:35 +01:00
Alonso Torres
cb8f563bb2
🐛 Keep swapped and moved children in place when a component syncs (#12716)
* 🐛 Keep swapped and moved children in place when a component syncs

A sync resets an untouched shape of a copy to the main's geometry and the
layout then moves it back. Children the sync keeps, because they are
swapped or the user moved them, did not take the first move but did take
the second, so each sync pushed them away from their parent by the gap
between the copy and the main (e.g. a swapped icon in a copy that fills a
wider layout).

The sync now moves those children by the offset it applies to their
parent. Flex-child attributes still sync as before, so swapped children
keep their absolute position.

Closes #12657

* 🐛 Place swapped children where the swap puts them on sync and switch

A swapped child that the user never moved now goes to the top-left of the
shape it replaced, as a swap does: on a sync, the place of the main's slot
occupant; on a variant switch, the place of the new variant's occupant.
A swapped child the user moved keeps its offset inside its parent.

This stops the variant switch from leaving the swapped child at its old
position while the new variant is created at the main's size, and puts
back children that drifted before the sync fix the next time their copy
syncs.

* 🐛 Make kept children follow their parent's constraints on component sync

When a sync resets the size of a copy's shape, for example a copy root
that fills a wider layout, the layout then grows it back and moves its
children by their constraints. Children the sync keeps (moved by the
user, or swapped) only followed the parent's top-left, so with a right,
center or scale constraint they drifted on every sync, and a swapped
child with a scale constraint grew on every sync.

The sync now passes the parent's geometry before and after down the
tree, and kept children follow that change through their constraints.
Untouched swapped children also undo the change through their
constraints before they go to the swap place, so they keep their size.

The default constraint modifier returns empty modifiers instead of an
empty vector, which add-modifiers could not take.

* 🐛 Make swapped children follow their parent's constraints on switch

A variant switch only moved a swapped child by the parent's position
change. When the new variant's root had a different size, a user-moved
child with a right constraint drifted, and a scale-constrained one kept
its grown size.

The switch now applies the parent change from the old parent to the new
one through the child's constraints. An untouched swapped child then
takes the place of the shape it replaces, as a swap does.

* 🐛 Sync nested swapped copies before the copies that hold them

A swapped child is a copy of its own component, so one sync handles it
twice: once on its own, and once when the copy that holds it places it.
Each step read the shape from the data as it was before the sync, so
the step that ran last won, and the order came from the order of shape
ids. When the swapped child came last, it wrote its old size and
position back over the placement, leaving its x and selrect apart, and
a swapped child with a scale constraint grew on every sync.

The sync now visits the most deeply nested shapes first, and each step
reads the shapes as the earlier steps left them, so the placement
always comes last and builds on the child's own sync.

* 🐛 Keep moved children with a scale constraint at their size on switch

A variant switch carries the size of a child the user moved as an
override. In a copy that fills a wider layout, a child with a scale
constraint already has the size the layout gave it, so the switch
carried that size onto the new variant, created at the main's size, and
the layout then scaled it again: the child grew on every switch.

Before comparing, the switch now resizes the previous child the way its
constraints follow the change from its old parent to its new one. A
child that only grew with the layout then matches the main and is not
carried as an override.

* 🐛 Keep the content of swapped children at its size on sync and switch

When a sync or a variant switch undid the parent change on a swapped
child through its constraints, it applied the child's resize to every
shape inside it. The layout then grows the child back through its own
constraint, but its content follows its own constraints (left and top
by default), so it stayed shrunk: the shape inside a swapped icon with
a scale constraint lost most of its width on every switch.

The change of the swapped child now goes down its subtree the way a
layout update does, each shape through its own constraints.

* 🐛 Accept mixed and empty values in the layout size constraints menu

With several shapes or a group selected, the sidebar merges the layout
item attributes and tokens of the shapes: nil when a shape lacks one,
:multiple when they differ. The props schema of layout-size-constraints*
only accepted the values of a single shape, so selecting such shapes
with fill sizing raised "invalid props".

The schema now accepts nil and :multiple, as the numeric inputs those
values go to do.

* ♻️ Share the swapped child follow-parent step between sync and switch

The component sync and the variant switch both make a swapped child and
its children follow the parent change through their constraints and then
move them to the swap place. Both now use generate-follow-parent, so the
two paths stay the same. The helper that writes geometry without marking
it touched is renamed to update-geometry-untouched.

Adds a test for a swapped child without absolute position in a slot
without layout, and fixes the test comments on why constraints are set
again after a swap.
2026-10-29 17:30:16 +01:00
Andrey Antukh
f8d97ba2ce Merge remote-tracking branch 'origin/develop' into develop 2026-10-29 16:33:02 +01:00
Andrey Antukh
65a76a21cd Merge remote-tracking branch 'origin/staging' into develop 2026-10-29 16:31:26 +01:00
Alejandro Alonso
fc56c02f49 Merge remote-tracking branch 'origin/staging' into develop 2026-10-29 16:28:17 +01:00
Andrey Antukh
cf5600f944 Merge remote-tracking branch 'origin/main' into staging 2026-10-29 16:16:32 +01:00
Andrey Antukh
af2e5921fe 📚 Document the CI build task in the memory index and AGENTS
`critical-info` is the first memory every agent reads, and its
`scripts/ci` entry still described the orchestrator as lint + tests +
format over eight modules. `AGENTS.md` repeated the same incomplete
description in its scripts section.

Both now mention the opt-in production build and list `media-processor`
among the modules.

AI-assisted-by: deepseek-v4.1-flash
2026-10-29 15:15:34 +00:00
Andrey Antukh
04de86f939 ✨ Add a build task to the local CI orchestrator
`./scripts/ci --build` compiles each module the way production does,
without going through `manage.sh build-bundle`, which wipes node_modules
and reinstalls dependencies. It runs the release commands already
defined per module and reuses existing caches, so a broken release build
(shadow-cljs optimizations, the wasm targets, the backend uberjar,
tsc/typedoc) shows up before a bundle is built.

The task is opt-in and runs last, after lint, test and fmt, so the
default flow keeps its duration. `common` has no build of its own and
reports as skipped. `media-processor` joins as the ninth module, with
its lint skipped and `tsc --noEmit` folded into its build.

All build outputs are gitignored. `scripts/test_ci.py` covers the flag
wiring and one command per module through `--dry-run`; it never runs a
real build.

AI-assisted-by: deepseek-v4.1-flash
2026-10-29 15:06:43 +00:00
Andrey Antukh
8491cdaf63 Merge branch 'main-2.18.x' 2026-10-29 15:43:06 +01:00
Andrey Antukh
41be5bfb5f ⬆️ Bump docker base images 2026-10-29 15:28:19 +01:00
penpot-renovate[bot]
936371d9a6
⬆️ Update dependency pnpm to v12.16.1 (#12714)
- pnpm 12.16.1 (docker/images/Dockerfile.media-processor)
- pnpm 12.16.1 (docker/images/Dockerfile.mcp)
- pnpm 12.16.1 (docker/images/Dockerfile.exporter)
- pnpm 12.16.1 (docker/devenv/Dockerfile)
- pnpm 12.16.1 (render-wasm/package.json)
- pnpm 12.16.1 (plugins/package.json)
- pnpm 12.16.1 (plugins/libs/plugins-styles/package.json)
- pnpm 12.16.1 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.16.1 (plugins/libs/plugin-types/package.json)
- pnpm 12.16.1 (plugins/apps/table-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.16.1 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/icons-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/example-styles/package.json)
- pnpm 12.16.1 (plugins/apps/e2e/package.json)
- pnpm 12.16.1 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.16.1 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.16.1 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.16.1 (package.json)
- pnpm 12.16.1 (media-processor/package.json)
- pnpm 12.16.1 (mcp/packages/server/package.json)
- pnpm 12.16.1 (mcp/packages/plugin/package.json)
- pnpm 12.16.1 (mcp/packages/common/package.json)
- pnpm 12.16.1 (mcp/package.json)
- pnpm 12.16.1 (library/package.json)
- pnpm 12.16.1 (frontend/text-editor/package.json)
- pnpm 12.16.1 (frontend/packages/ui/package.json)
- pnpm 12.16.1 (frontend/packages/mousetrap/package.json)
- pnpm 12.16.1 (frontend/packages/draft-js/package.json)
- pnpm 12.16.1 (frontend/package.json)
- pnpm 12.16.1 (exporter/package.json)
- pnpm 12.16.1 (docs/package.json)
- pnpm 12.16.1 (common/package.json)
- pnpm 12.16.1 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-10-29 15:26:20 +01:00
penpot-renovate[bot]
c0af48f873
⬆️ Update DHI base images (#12713)
- dhi.io/debian-base trixie-debian13-dev@cc4e05f (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@cc4e05f (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@2664ef2 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@cc4e05f (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@cc4e05f (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.8-alpine3.24@83489ce (docker/images/Dockerfile.storybook)
- dhi.io/nginx 1.31.8-debian13-dev@fbff190 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.2@ (docker/devenv/Dockerfile)
- dhi.io/node 24.22.2@ (.nvmrc)
- dhi.io/node 24.22.2-debian13-dev@53af56f (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.2-debian13@24fa625 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@53af56f (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.2-debian13-dev@53af56f (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-29 15:24:22 +01:00
Andrey Antukh
857c666022 ⬆️ Update pnpm to 12.16.1 and node to 24.22.2
pnpm 12.5.1 -> 12.16.1: the tarball pin of docker/devenv and of the
media-processor, exporter and mcp images, the packageManager field of
every workspace manifest, and the lockfile metadata of the 11
workspaces. The new pnpm also normalizes one engine range per lockfile
(>=18.* to >=18).

node 24.21.0 -> 24.22.2: devenv NODE_VERSION and both NODE_SHA256,
.nvmrc, and the runtime pin of the two plugins-deploy workflows. The
dhi.io image tags stay as they are until their digests are available.

AI-assisted-by: deepseek-v4.1-flash
2026-10-29 14:09:54 +00:00
Andrey Antukh
c70cc56299 🔧 Skip build output when stamping the pnpm version
The script walks the filesystem, so it stamped generated copies too, such
as media-processor/target/package.json: --check counted files git does
not track and failed whenever one of those copies was stale. Prune
target/ like the other generated trees, with a test for it.

Memory: the JDK and ImageMagick image pins have traps the code cannot
show, now in workflow/docker-images; memory-maintenance gains the rule
that a memory must not restate what the code already says; updating-pnpm
notes the lockfile-only refresh.

AI-assisted-by: deepseek-v4.1-flash
2026-10-29 14:09:43 +00:00
Eva Marco
c373e8da88
🔧 Run the DS visual specs on the dark theme (#12707)
Since accounts without a theme follow the system one (#12480), the ds
Playwright project renders the light theme: headless Chromium reports a
light color scheme. Its screenshots stopped matching the ones taken
before, and the app is reviewed on the dark theme. Set colorScheme to
dark for the ds project only; the default and render-wasm projects keep
their behavior.

AI-assisted-by: claude-opus-5-5
2026-10-29 14:50:20 +01:00
Belén Albeza
d07a54562f
🐛 Add feature flag text-editor-wasm/v1-html-paste (#12689) 2026-10-29 13:46:21 +01:00
Andrey Antukh
76d403cfdb ⬆️ Update npm dependencies on mcp and exporter 2026-10-29 13:34:32 +01:00
Eva Marco
755bb175cd
🐛 Remove wrong flag (#12698) 2026-10-29 13:23:36 +01:00
Eva Marco
2607ec89fa
♻️ Move the profile page to the DS and make the Update overlay a button (#12651)
* ♻️ Make the profile photo Update overlay a button

The "Update" overlay on the profile photo was a span with an on-click,
so the keyboard could not reach it. It is now a button: it shows on
keyboard focus as well as on hover.

The button centers its line in the circle, so a 4px bottom padding keeps
the label where the span drew it (0px difference).

New ds visual test "Profile page controls": the page at rest, the
overlay on hover and on focus, Save enabled, and the Change email link
on hover.

AI-assisted-by: claude-opus-5-5

* ♻️ Move profile page styles off the deprecated styles

profile.scss imported the whole deprecated module and kept a long copy
of shared settings rules. It now uses the DS tokens, sizes, borders,
z-index and typography. The deprecated import stays only for
%button-primary: the save button is the shared forms submit button.

Deleted selectors that match nothing in profile.cljs: .btn-secondary
(both), .field, .field-title, .field-text, .custom-input,
.custom-select, .two-columns, .subtitle, .notification-icon,
.notification-text, .notification-text-email, .newsletter-subs, the h1,
h3, p and hr rules, the form rule (both forms set their own flex) and the
link hover underline (the global link rule already does it).

Typographies: the page title uses title-large (line-height 1.4 instead
of 1.15), the Change email link title-small (line-height 1.2 instead of
the default), and the Update overlay title-large with its own
line-height.

Visible change: the taller title line moves the title 3px down and the
rest of the page 6px down. Nothing else changes.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the profile page links, title and photo classes

The links and the profile photo get their own classes instead of bare
a and img selectors, and the page title uses the DS heading with
title-large (it already used that typography). The link wrapper drops
a class with no styles, and the rule for the file input goes: the
shared uploader hides that input inline, so the rule did nothing. No
visual change.

AI-assisted-by: claude-opus-5-5

* 💄 Use one DS typography and the accent color for the profile links

"Change email" took title-small from its container, overridden by the
global link styles (14px, weight 500), and "Want to remove your account?"
had no typography and inherited 16px. Both links now use body-medium and
--color-accent-primary, as design asked in the review.

The "Change email" line is taller with the 1.4 line height, so Save
settings and the link below it sit 3px lower.

AI-assisted-by: claude-opus-5-5
2026-10-29 12:09:56 +01:00
Dr. Dominik Jain
c0f0b14540
🎉 Add direct MCP feedback channel via dedicated tool (#12551)
* ♻️ Move devenv-only tools to a separate folder

* 🎉 Add an MCP feedback channel

Add a user-approved feedback tool and an MCP-token-authenticated backend
endpoint that queues plain-text email. Enable the tool through the
configured backend URI in both local and multi-user mode.

Validate subject and feedback limits, enforce per-profile rate limits,
and document configuration. Cover authentication, queueing, tool
availability, and failure handling with backend and MCP tests.

Closes #12490

AI-assisted-by: gpt-6

* ♻️ Move MCP feedback submission into RPC

Use the standard RPC wrappers for feedback and remove the dedicated
HTTP endpoint, route wiring, and custom rate limiter.

Restrict MCP tokens to opted-in routes and commands. Add a shared RPC
call helper to the MCP backend client and adapt the existing tests.

AI-assisted-by: gpt-6

* 🐛 Harden MCP feedback auth, validation and messages

Accept MCP tokens when either the mcp or the access-tokens flag is on, so
instances that enable MCP without personal access tokens can send
feedback. Personal tokens still need access-tokens. Keep the issuer in one
constant, decode tokens without raising to reject unknown issuers, and
name the issuer and type checks.

Reject control characters in the subject, and in the feedback all but
tabs and line breaks, in both the command and the tool, so a NUL answers
400 instead of failing when the mail job is stored. Answer 403 with
profile-blocked or profile-not-active for unusable profiles, record only
who sent feedback in the audit event, and add doc metadata.

Replace on OAuth approval an MCP token issued before MCP tokens had their
own issuer, or one this instance cannot decrypt, since the backend rejects
both.

In the MCP server, mention disabled access tokens on 401, drop the unused
503 case, and say that delivery happens in the background. Document the
flags in the README.

AI-assisted-by: claude-opus-5-5

* 📚 Document the MCP feedback tool and MCP token rules

List submit_feedback among the MCP tools and add an MCP feedback section to
the configuration guide covering PENPOT_MCP_BACKEND_URI,
PENPOT_MCP_FEEDBACK_EMAIL, the flags and the email settings. Add commented
examples of both variables to the compose file.

In the agent memories, record when OAuth approval replaces an MCP token,
how the MCP server reaches the backend, and which flags, routes and
commands accept MCP tokens.

AI-assisted-by: claude-opus-5-5
2026-10-29 10:35:46 +01:00
Pablo Alba
9dc222c642
🐛 Fix bad characters allowed on variants properties (#12688) 2026-10-29 10:13:33 +01:00
Eva Marco
a6c225d510
🐛 Move the CSS @charset to the start of the bundled stylesheet (#12678)
Sass adds @charset "UTF-8" to every module with non-ASCII characters,
and concatSass joined the modules as they came, so main.css carried it
in the middle of the file. Browsers only honor @charset as the first
rule, and the server sends text/css with no charset. When the browser
did not take the page encoding for the stylesheet, it read main.css as
Latin-1 and CSS content such as the bullet of the library element
counts showed as "•".

Drop every @charset while joining the modules and put one at the start.

Closes #12677

AI-assisted-by: claude-opus-5-5
2026-10-29 09:28:53 +01:00
Eva Marco
705ea42622
🐛 Keep the default cursor in the token dropdown of numeric inputs (#12680)
A numeric input that can be scrubbed sets the ew-resize cursor on its
wrapper, and the token dropdown renders inside that wrapper, so the
whole list inherited the scrub cursor. Give the options dropdown the
default cursor, so it never takes the cursor of the control that opens
it.

AI-assisted-by: claude-opus-5-5
2026-10-29 09:04:05 +01:00
Eva Marco
1962860f04
🐛 Reduce thumbnail re-renders and report throttled update loops (#12610)
* 🐛 Stop thumbnail subscribers re-rendering on every store change

A thumbnail loaded from the server keeps a media id as :uri, and
workspace-thumbnail-by-id resolves it into a new map on each read. The
ref compared values with identical?, so every root frame, every
component in the assets panel and the interactions layer re-rendered
on every store change, mouse moves included. Compare by value instead.

This does not loop by itself, but it adds work to every commit and
brings other update loops closer to React's limit.

Relates to #12580

AI-assisted-by: claude-opus-5-5

* ✨ Report update loops throttled by the update scheduler

When the scheduler throttles an update loop, it only logs a console
warning, so a loop that no longer crashes leaves no trace. Send an
error report too, with the events in flight and the last events, at
most once per report window.

Relates to #12580

AI-assisted-by: claude-opus-5-5

* 🐛 Report throttled update loops as unhandled exceptions

The throttled update loop report used the handled-exception event, which
only lands in the audit log: error reports and alerts read
unhandled-exception and exception-page. A throttled loop is an
application defect (its cause has :type :internal), and the point of the
report is to find those loops, so it is now an unhandled exception.

AI-assisted-by: claude-opus-5-5
2026-10-29 09:02:55 +01:00
Eva Marco
a3fe43c1ec
🐛 Report raw fetch failures as connectivity errors (#12624)
* 🐛 Report raw fetch failures as connectivity errors

A fetch that fails outside app.util.http (zip.js, plugins, schema
loads) rejects with a bare TypeError such as "Failed to fetch". It
reached the global handlers and was reported as an unhandled crash,
with no stack and no URL.

The global handlers now wrap the browser's network TypeErrors as
:network (or :offline when the browser says so), so they get the
connection toast and a compact handled report. Response body reads
in app.util.http also wrap a TypeError as :network with the request
uri, as fetch itself already did.

Relates to #12581

AI-assisted-by: claude-opus-5-5

* 🐛 Treat only network TypeErrors on body read as connectivity errors

process-response-type turned every TypeError from a body read into a
:network error, so a bug such as "body stream already read" showed a
connection toast. It now uses network-failure?, which moves to
app.util.http so that both the http client and the global error
handlers share it. Add Firefox's "Error in body stream" message.

Tests now check :offline and :network separately by setting
navigator.onLine, and read-error builds on async/->promise.

AI-assisted-by: claude-opus-5-5
2026-10-29 09:02:45 +01:00
Andrey Antukh
3cc91e8ba2
♻️ Take the command limits before the root ones in the RPC limiter (#12686)
* ♻️ Take the command limits before the root ones in the RPC limiter

The RPC concurrency limiter took the root limiters (the ones every request
passes through) before the ones a command declares, so a request waiting for
the limits of its own command held the root permits of its profile while it
waited: a profile with a pile-up of heavy requests saw its other requests
queue behind them and be rejected with 429.

That order was forced by two commands that entered the limiter from inside
their handler: a command that declares no limits of its own can only take
them deeper than the root ones, so two paths of one profile could take the
same limiters in opposite order and park for good.

The commands declare their limits in their metadata now and their handlers
call the work directly, so no command enters the limiter from inside its
handler and every path takes the limiters in the same order. The root
limiters are appended to the sequence first, so the limits a command
declares end up outermost and are taken first: a request waiting for them
does not hold the root permits of its profile. The photo of a team keeps its
limits, because it declares them now that the helper does not.

A call from inside a handler can only reenter the limits that thread
already holds, and it is refused when it asks for a new one, because that
would take it after the root ones and invert the order against the other
paths. The key of a limit is resolved the same way in every path, so both
paths point at the same limiter and the reentrancy check sees it.

Every entry of the shipped climit.edn declares a queue, so the number of
requests waiting for the same limit is bounded instead of piling up until
the timeout. The production file lives outside the repository and has to be
mirrored to pick them up.

The tests cover the acquisition order of a command with and without limits
of its own, the order of the limits as a whole (a ring of any size fails,
not only a pair in opposite order), the nested call that asks for a limit
the thread does not hold, and the queues and timeouts of the shipped
configuration. The import of a binfile declares its request as multipart in
its test, the way the commands that take an upload map do.

AI-assisted-by: minimax-m3.1-flash
AI-assisted-by: deepseek-v4.1-flash

* 🔧 Keep the commands' limits and the shipped config in sync

`profile-plugin-ops` guarded only an UPDATE on the caller's own profile row,
which the row lock already serializes, so its global cap could only reject a
burst of different profiles toggling plugins. Both commands declare the
by-profile limit alone now, and the example config drops the entry.

A declared id without an entry does not limit anything: `wrap` logs a warning
and runs the handler with no limiter at all, which is how a command ends up
unbounded in production. A test fails now when a command declares an id the
config does not list, and the header of the example says the production file
mirrors the shape and lists every declared id, not the values.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Record that background work is not waited for with a sleep

A long command moved to the background reports its own completion, so
polling it with a sleep inside a foreground command only blocks the session
and the notification arrives anyway. The rule says to keep working on
something else, or to end the response and wait for it.

AI-assisted-by: minimax-m3.1-flash
2026-10-28 18:44:36 +01:00
Dr. Dominik Jain
e20a0d748e
✨ Add OAuth support for MCP server (#12480)
* 🎉 Add OAuth support for remote MCP connections

Add MCP-specific discovery, client registration, and approval endpoints.
Exchange single-use authorization codes with PKCE for the fixed MCP user
token, enabling MCP and creating the token on approval when needed.

Use self-contained client IDs without new database tables or stored client
records. Keep temporary authorization state in Redis/Valkey.

Accept bearer credentials while preserving query-parameter connections.
Enable OAuth through PENPOT_MCP_AUTH_URI in multi-user mode. Document the
environment variable in the README and implementation details in Serena.

Explain protocol and concurrency decisions in the Clojure code. Cover token
provisioning, OAuth state, client registration, HTTP flows, and the approval
page with tests.

AI-assisted-by: gpt-6

* 🐛 Wait for devenv initialization before starting builds

Docker reports a container as Running as soon as its main process starts,
while the entrypoint may still be preparing the development environment.
manage.sh used that status to launch tmux jobs, allowing frontend builds
to race with ownership changes to /opt/rustup and Cargo setup.

On a fresh container, rustup could fail with Permission denied before the
frontend watcher started. Nginx would later serve the existing JavaScript,
leaving source changes absent from the running app despite restarting
devenv.

Wait for nginx's local health endpoint before launching the jobs. Nginx
starts after entrypoint setup, so its response confirms that setup has
finished. Allow up to 120 seconds and report a timeout without starting
the jobs if initialization does not complete.

Validation: shell syntax and whitespace checks pass.

AI-assisted-by: gpt-6

* ♻️ Share the login modal with MCP authorization

Extract the viewer login modal and its styles into a shared auth
component, retaining the viewer wrapper and its close behavior.

Use the modal on the MCP approval page and preserve the pending request
through login. Closing it clears the saved return URL and shows a
cancellation message without further actions.

Cover return URL handling and cancellation with frontend tests.

AI-assisted-by: gpt-6

* ✨ Show client icons in MCP authorization

Preserve HTTPS logo URLs from client registration and published metadata
and show the icon beside the application name on the approval page.
Ignore unusable URLs and hide failed images without blocking approval.

Update the access description to refer to MCP connections established
from Penpot. Cover logo metadata and approval responses in backend tests.

AI-assisted-by: gpt-6

* 🐛 Follow the system theme when no preference is configured

Default to the system theme in the shared resolver used by the app and
plugins. Preserve explicit light and dark choices, including the legacy
default value that means dark.

Reuse the reactive theme hook in progress widgets and show the system
preference in settings for accounts without a configured theme.

Verify preference resolution with regression tests and browser checks.
All 1,474 frontend tests pass.

AI-assisted-by: gpt-6

* 🐛 Replace expired MCP tokens during OAuth approval

Settings can give an MCP token an expiry date. OAuth approval reused
the newest MCP token without checking it, so an expired token reached
the client and never matched a plugin connection.

Approval now treats an expired token as missing and creates a new one.
Token exchange rejects a code whose token expired after approval with
invalid_grant.

AI-assisted-by: claude-opus-5-5

* 🐛 Log unexpected errors on MCP OAuth endpoints

The OAuth routes sit outside the shared HTTP middleware, so the shared
error handler never saw them, and their own handler turned unexpected
failures into a 503 without logging anything. Failures in the session
middleware of the approval endpoints had no handler at all.

Unexpected failures now also go through the shared error handler for
its logging, while clients still get an OAuth error body. The OAuth
error handler also wraps the session middleware of the approval
endpoints.

AI-assisted-by: claude-opus-5-5

* 🔧 Add devenv script to run the MCP server with OAuth

start-mcp-devenv-oauth starts the MCP server in multi-user mode with
PENPOT_MCP_AUTH_URI derived from PENPOT_PUBLIC_URI, to test OAuth in
the devenv. Agentic tooling still uses the single-user
start-mcp-devenv.

AI-assisted-by: claude-opus-5-5

* 🐛 Fix failing test case

---------

Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
2026-10-28 18:40:04 +01:00
Eva Marco
8951c57ee6
♻️ Use DS buttons and icons in the dashboard fonts page (#12533)
* ♻️ Use DS buttons and icons in the dashboard fonts page

The fonts page moves off the deprecated icons and raw buttons:

- "Add custom font", "Upload all", "Upload" and "Save" become DS primary
  buttons and "Dismiss all" a secondary one.
- The remove, cancel and menu controls of the font rows become ghost DS
  icon buttons named "Remove", "Cancel" and "Options"; the menu one
  says whether its menu is open. Two of them were spans.
- The delete control of a font style was a span; it is now a button
  named "Delete font style" with the DS close icon, still shown only
  on hover.
- The warning and empty list icons use the DS icons. The loading
  placeholder keeps the legacy loader: there is no DS one.

The font menu offset grows by the 6px the taller menu button added, so
the menu opens where it did. New key: dashboard.fonts.delete-variant
(en, es). A visual test covers a font ready to upload, the style hover,
the font menu and the edition row.

AI-assisted-by: claude-opus-5-5

* ♻️ Move dashboard fonts styles to logical properties

Physical sizes, offsets, margins and paddings become logical properties
and the DS typography module is used without a namespace. The element
selectors (section title, inputs, hero text) stay, with a comment: they
have no class.

The shared dashboard header styles (common-dashboard) still come from
refactor/: ten dashboard modules use them, so they move in their own
change. No visual change.

AI-assisted-by: claude-opus-5-5

* ♻️ Use the DS loader and heading in the dashboard fonts page

The fonts loading placeholder used the legacy loader icon, drawn as an
outline in a 32px box. It now uses the DS loader*, the same animated
pencil as the other dashboard loading screens. Its box is 9px tall,
so the label moves 6px up.

The section title h3 is now a heading* with the title-small typography
and its own class, so the SCSS no longer styles the bare element.

The legacy loader icon has no other use, so its SVG and def are gone.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the dashboard fonts inputs and hero text classes

The font name and search inputs and the hero text get their own
classes instead of bare input and p selectors, and the hero title uses
the DS heading with title-large instead of the global h2 rule. The
title keeps its 24px size and weight; the DS line height (1.4 instead
of 1.15) moves the hero content about 6px down.

AI-assisted-by: claude-opus-5-5

* 🐛 Give the fonts placeholder icon its color back

Removing the legacy loader also removed the .icon svg rule, which gave
the empty placeholder text icon its color, so the icon had none. The
icon now has its own class with the secondary foreground color, and the
wrapper div without styles goes.

AI-assisted-by: claude-opus-5-5
2026-10-28 16:11:52 +01:00
Eva Marco
66c67660f9
♻️ Use DS typographies in the debug shape panel and drop a dead selector (#12650)
* 🔥 Remove a commented-out selector from the debug shape panel

.shape-info-title was left commented out with a "check by hand" TODO in
#12484. Nothing uses it, so delete it instead of keeping it as a comment.

AI-assisted-by: claude-opus-5-5

* ♻️ Use DS typographies in the debug shape panel

The panel text and the shape title only set a font size, with a "no exact
DS typography match" TODO. body-small and body-medium differ from them
only in the line-height, so use them directly and drop the TODOs.

Visible change: the panel lines get the DS line-height (1.3 for the text,
1.4 for the shape title) instead of the browser default, so rows grow a
little taller. The panel is debug-only.

AI-assisted-by: claude-opus-5-5
2026-10-28 14:27:51 +01:00
Eva Marco
e308c2eca6
♻️ Use the DS stroke width in the migrated DS icons (#12673)
* ♻️ Use the DS stroke width in the migrated DS icons

The DS icons in the areas already moved to the DS set their own
stroke width (1px, 1.33px or 2.25px) to match the legacy icons. They
now keep the DS stroke width, so some icon lines get slightly thinner
or thicker: copy button, context menu, layer row, comments filter,
assets section, share link, dashboard sidebar, team pages, templates,
team switcher and subscription icons.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the share link page rows a class

The page rows of the share link permissions get their own class
instead of a bare li selector. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-28 14:27:38 +01:00
Eva Marco
da36e36836
✨ Add the share, handlers and snap icons to the DS (#12659)
* ✨ Add the share, handlers and snap icons to the DS

The share button and the path editing bar could not move to the DS
buttons because their icons were missing from the DS icon set. As agreed
with design:

- handlers-equal, handlers-independent, handlers-mirror and snap join the
  DS as they are (their SVGs already follow the DS format).
- share keeps its drawing, moved onto the 16x16 grid as a single path
  with no fixed color.
- snap-nodes is deleted: nothing used it, and snap is the icon of the
  snap nodes button.

The legacy share button in the workspace header uses the same SVG: its
icon is about 6% smaller and now takes the hover color, which the fixed
stroke color blocked before.

AI-assisted-by: claude-opus-5-5

* 🐛 Show the share icon lines outside the app

The share SVG had no fill or stroke, so plain SVG viewers used the
defaults (black fill, no stroke) and drew only three dots. Set
fill="none" and stroke="currentColor", like settings.svg. The app
already sets both in CSS, so nothing changes there.

AI-assisted-by: claude-opus-5-5

* 🐛 Show the handlers and snap icons right outside the app

Like share, these SVGs had no fill or stroke, so plain SVG viewers
filled the shapes black and drew no lines. Set fill="none" and
stroke="currentColor". The app sets both in CSS, so nothing changes
there.

AI-assisted-by: claude-opus-5-5
2026-10-28 14:26:43 +01:00
Eva Marco
980ed0a719
♻️ Use DS icon buttons to close the subscription dialogs (#12539)
* ♻️ Use DS icon buttons to close the subscription dialogs

The close buttons of the five subscription dialogs (plan management,
success, Nitrate subscription, contact sales and manual cancel) were
raw buttons with no name. They become ghost DS icon buttons named
"Close"; the class that places them in the corner goes on the tooltip
wrapper, so they stay where they were and only show the DS hover.

The plans page link-style buttons stay raw: they have no DS equivalent.
A visual test covers the plans page and the plan management dialog.

AI-assisted-by: claude-opus-5-5

* ♻️ Move settings subscription styles closer to the DS

The settings subscription stylesheet inlines the deprecated
button-style mixin and %button-icon placeholder, resolves its legacy
color aliases and font weight, uses logical properties and the DS
typography module without a namespace. The rules on icon svgs that have
no class keep a comment; two that match nothing (the dropdown arrow svg
and .radio-btns) are commented out with a TODO.

refactor/ stays only for the modal placeholders (%modal-*), which move
with the modals refactor. No visual change.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the settings subscription

The SCSS refactor of the settings subscription left rules commented out
with a TODO because the module does not use them, or because they target
an svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.icon-dropdown svg, .radio-btns, .radio-btns label. No visual change.

AI-assisted-by: claude-opus-5-5

* ♻️ Give the subscription icons classes and drop unused props

Review fixes for the subscription styles:

- The plan icon span sets the letter color with color, not stroke on a
  bare svg, and drops fill, stroke-width and color: transparent, which
  do nothing on a span.
- The plan icon border uses $b-2 (was 1.75px), so it is 0.25px thicker.
- The letter gets its own class for its 2.25px stroke. With the default
  stroke it looks too thin next to the border.
- The open link and editors arrow icons get a cta-icon class instead of
  the bare svg selector, and take the button color. The arrow used an
  icon-dropdown class with no rule.

AI-assisted-by: claude-opus-5-5

* ♻️ Use the DS stroke width and color in the subscription icons

The plan letter and the member icons keep the DS stroke width (the plan
letter line gets thinner), the member icons set their color instead of
the stroke, and the props copied from the button icon placeholder go.
The success dialog logo gets its own class instead of a bare svg
selector.

AI-assisted-by: claude-opus-5-5
2026-10-28 12:11:06 +01:00
Eva Marco
1109e38852
♻️ Use DS icons in the dashboard lists and placeholders (#12536)
* ♻️ Use DS icons in the dashboard lists and placeholders

The trash project options button becomes a ghost DS icon button, as in
the projects list, and says whether its menu is open. The search
placeholders, the empty project "create new file" button, the comments
placeholder and the legacy import library tag use the DS icons. The
legacy search icon had no fill rule, so its lens was filled black; the
DS one is not.

Two fixtures make the trash testable: a deleted project, and deleted
files due in 2100 (the existing ones expired in 2024, so the trash
dropped them). A visual test covers the trash row, the empty project
button, the project rename field and the comments placeholder.

AI-assisted-by: claude-opus-5-5

* 🐛 Remove the hidden close control from the dashboard rename field

The rename field of dashboard projects and files had a close control
next to the input that never showed: its legacy icon is drawn with a
stroke, but the styles only set a fill. It could not be reached with
the keyboard or a screen reader, and a click on it did what leaving
the field does: the blur saves the name first, and its handler saved
it too. It only left a pointer cursor over an empty spot. Enter, Escape
and leaving the field keep working as before. No visual change.

AI-assisted-by: claude-opus-5-5

* ♻️ Move dashboard trash, comments and rename styles to the DS

The trash, comments panel and rename field stylesheets resolve their
legacy color aliases, sizes, font sizes, mixins and the menu shadow to
DS tokens and values, and use logical properties. The comments panel
and the rename field no longer import refactor/; the trash keeps only
the shared dashboard header styles (common-dashboard), used by ten
dashboard modules.

Four selectors that the modules do not use are commented out with a
TODO. The comments dropdown z-index (4) has no DS token and keeps a
TODO. No visual change.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the dashboard trash and comments

The SCSS refactor of the dashboard trash and comments left rules
commented out with a TODO because the module does not use them, or
because they target an svg inside a DS icon that is the svg itself. CSS
modules scope the classes to this module and no other source uses them,
so they go: .thread-group, .comments-icon-small, .add-file-btn,
.add-icon. No visual change.

AI-assisted-by: claude-opus-5-5

* 🐛 Use px2rem for the unread dot size in dashboard comments

The unread dot used $sz-8, which the DS sizes do not define, so the
integration bundle build failed. It now uses px2rem(8), the same 8px.

AI-assisted-by: claude-opus-5-5

* ♻️ Use DS typography and z-index in the dashboard comments

Review fixes for the dashboard lists styles:

- The comments dropdown title uses the headline-small typography. It
  was 11px regular, so it is now 12px medium (larger and bolder).
- The comments dropdown uses --z-index-dropdown instead of a raw 4.
- The rename field selector is the class alone; the input type is not
  needed to win over the input defaults (0px change).

The visual test now also captures the whole comments dropdown, so the
title change shows.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove dead rules from the dashboard comments styles

The comments dropdown has no separators, so its hr rule matched
nothing, and the comments icon set the default stroke width. No visual
change.

AI-assisted-by: claude-opus-5-5
2026-10-28 12:10:47 +01:00
Alejandro Alonso
4dc684dd9c Merge remote-tracking branch 'origin/staging' into develop 2026-10-28 11:50:56 +01:00
María Valderrama
ff9506c551
✨ Add member profile support for organization attributes (#12652) 2026-10-28 11:02:42 +01:00
Eva Marco
4d416bd3f2
♻️ Use DS icons and buttons in the settings sidebar and email modal (#12541)
* ♻️ Use DS icons and buttons in the settings sidebar and email modal

- The settings sidebar back arrow and "Contact us" icon use the DS
  icons. The arrow keeps its place; the small DS icon draws 12px inside
  a 16px box, so the feedback icon sits 2px to the right and its margin
  shrinks to keep the label in place.
- The change email modal close button becomes a ghost DS icon button
  named "Close"; the class that places it goes on the tooltip wrapper.

The form submit buttons of the settings pages stay: they are the shared
forms submit button. A visual test covers the settings sidebar and the
change email modal.

AI-assisted-by: claude-opus-5-5

* ♻️ Move settings sidebar and email modal styles to the DS

The settings sidebar no longer imports refactor/: its legacy mixins,
sizes and color aliases resolve to DS values and tokens, with logical
properties. Its z-index (1) has no DS token and keeps a TODO.

The change email modal does the same and keeps refactor/ only for the
modal placeholders (%modal-*), which move with the modals refactor. Its
legacy uppercase title becomes the DS headline-small typography, which
differs only in its size (12px instead of 11px): the modal grows 2px.
Two unused selectors are commented out with a TODO.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the change email modal

The SCSS refactor of the change email modal left rules commented out
with a TODO because the module does not use them, or because they target
an svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.select-title, .cancel-button. No visual change.

AI-assisted-by: claude-opus-5-5

* ♻️ Tidy the settings sidebar and change email styles

The sidebar uses --z-index-panels instead of auto, the change email
submit button gets its own class instead of a bare button selector, and
the sidebar icons keep the DS stroke width (only the feedback icon line
gets thinner).

AI-assisted-by: claude-opus-5-5
2026-10-28 10:39:35 +01:00
Eva Marco
b1018c77b7
♻️ Use DS icons in the legacy context notification (#12542)
* ♻️ Use DS icons in the legacy context notification

The legacy context notification (still used by the global context
notifications, the fonts page, the team, change email, delete account
and export modals) draws its level icon with the DS icons instead of
the deprecated icon set, colored through the color property. The icons
are the same drawings, so nothing changes on screen.

The component stays: the DS context notification has no links, which
the global context notifications use. A visual test covers the info and
warning levels and a warning with html content.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the legacy context notification styles to the DS

The legacy context notification no longer imports refactor/: its sizes,
spacing, borders and alert color aliases resolve to DS tokens, with
logical properties. The legacy body-small typography becomes the DS
one, which differs only in its line height (1.3 instead of 1.4): one
line notifications do not change, and multi-line ones are about 1.5px
shorter per extra line. The unused .default level is commented out with
a TODO.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the legacy context notification

The SCSS refactor of the legacy context notification left rules
commented out with a TODO because the module does not use them, or
because they target an svg inside a DS icon that is the svg itself. CSS
modules scope the classes to this module and no other source uses them,
so they go: .default. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-28 10:39:20 +01:00
Alejandro Alonso
a8e8a54a2b Merge remote-tracking branch 'origin/staging' into develop 2026-10-28 10:35:31 +01:00
Eva Marco
08228324db
♻️ Use DS icons in the dashboard templates section (#12535)
The arrow and add icons of the dashboard libraries and templates
section use the DS icons and get their color from the color property.
The collapse button says whether the section is expanded, and the left
scroll button, which had no name, is now named "Previous".

The scroll buttons stay raw: they are round 36px buttons with no DS
equivalent. The template cards are anchors without href; they go to the
anchors backlog. A new fixture lists eight builtin templates, and a
visual test covers the cards, a card hover, the scroll buttons and the
collapsed title. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-28 09:21:00 +01:00
Eva Marco
c05b2e2ead
♻️ Use DS buttons and icons in the share link modal (#12562)
* ♻️ Use DS buttons and icons in the share link modal

- The close button becomes a ghost DS icon button named "Close"; the
  class that places it goes on the tooltip wrapper.
- "Get link", "Destroy link" and the confirmation buttons were raw
  inputs; they become DS primary, destructive and secondary buttons.
- The copy link button stays a raw segment of the link field; its icon
  becomes the DS clipboard icon, a bit larger than the legacy drawing.
- "Manage permissions" stays a link-style toggle; its arrow becomes the
  DS icon (the rotation still shows the state) and it says whether the
  permissions are expanded.

The page checkboxes and the permission selects stay for their own
groups. A new create share link fixture and a visual test cover the
modal before and after getting a link, with the permissions expanded,
and the destroy confirmation.

AI-assisted-by: claude-opus-5-5

* ♻️ Move share link modal styles closer to the DS

The share link stylesheet resolves its legacy sizes, spacing, mixins,
z-index and color aliases to DS values and tokens, with logical
properties. The copy button inlines %button-secondary with local color
variables, so it looks the same at rest, on hover, on focus and while
pressed; the visual test now also covers its hover.

The legacy typographies become the DS ones, each differing in one
detail: the title and "Manage permissions" use headline-small (12px
instead of 11px) and the texts body-small (line height 1.3 instead of
1.4), so the modal changes height by about 1px. refactor/ stays only
for the modal container, the link input and the page checkboxes.

AI-assisted-by: claude-opus-5-5

* ♻️ Use a DS icon button for the share link copy button

The copy button next to the link was a raw button with the secondary
button styles inlined in share_link.scss. It is now a secondary
icon-button* named "Copy link", so the .copy-button block goes away
(review feedback).

Visible changes: the DS secondary look (grey icon at rest, no lighter
background on hover) and a 32px wide button instead of 28px, so the
link text ends 4px earlier.

AI-assisted-by: claude-opus-5-5
2026-10-28 09:20:41 +01:00
Eva Marco
ac52c41903
♻️ Use DS icons in the comments panel filter (#12561)
* ♻️ Use DS icons in the comments panel filter

The comments panel filter (workspace and viewer sidebar) draws its
trigger arrow and menu ticks with the DS icons, as the viewer comments
filter does. The trigger button says whether its menu is open
(aria-expanded, aria-haspopup). The menu items stay: they move with the
menus group.

The dropdown item placeholder styles any svg inside a span with a 12px
box and a stroke color, so the tick icon class sets its 16px box and
currentColor stroke. Nothing changes on screen. A visual test covers
the trigger and the open menu.

AI-assisted-by: claude-opus-5-5

* ♻️ Move comments panel styles closer to the DS

The comments panel stylesheet resolves its legacy sizes, spacing,
mixins and color aliases to DS values and tokens, with logical
properties, and inlines the asset element placeholder of the filter
trigger. refactor/ stays only for the dropdown placeholders, which move
with the menus group.

The legacy body-small typography becomes the DS one, which differs only
in its line height (1.3 instead of 1.4): the menu labels sit 1px
higher. The trigger looks the same at rest and on hover; the visual
test now also covers the hover.

AI-assisted-by: claude-opus-5-5
2026-10-28 09:20:22 +01:00
Eva Marco
f6f9df9a1a
♻️ Use DS icons in the layer row buttons (#12477)
* ♻️ Use DS icons in the layer row buttons

The expand, visibility and lock buttons of the layer rows drew their
icons with the deprecated icon set (arrow, shown, hide, unlock, lock).
Use the DS icon instead. The buttons stay raw: they are segments of the
layer row, with backgrounds that follow the row state.

The icons get a class each, so their colors move from svg rules to
those classes, and they keep the old stroke weight. The DS icon draws
12px inside a 16px box:

- The expand button start padding goes from 8px to 6px, so the arrow
  stays in place.
- The visibility and lock buttons leave 12px for the icon, so their
  icons do not shrink (flex-shrink: 0) and stay centered, uncut.

The rows look the same as before, pixel for pixel. Add a visual test
with a selected, hidden and blocked layer and a hovered row.

AI-assisted-by: claude-opus-5-5

* ♻️ Move the layer name styles off the deprecated styles

The layer name and its rename input used the deprecated body-small
typography, mixins, spacing and token aliases. They now use the DS
body-small typography, sizes, borders and px2rem(), and the layer row
color aliases resolve to their DS tokens, in layer_item.scss too.

The DS body-small has a 1.3 line-height instead of 1.4, so the layer
names sit 1px higher. The name color is one local variable for every
state; the rename input sets it too, since it also has .element-name.
A visual test now covers the rename input.

AI-assisted-by: claude-opus-5-5
2026-10-27 16:01:53 +01:00
Dr. Dominik Jain
9d5dad3de7
⬆️ Upgrade to Serena v2.1.0 (#12482)
* ⬆️ Upgrade to Serena v2.1.0

Switch to REPL interface (configured in project)

Additional changes:
* Remove additional Serena project in subfolder mcp/

* 🐛 Fix outdated memories

---------

Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
2026-10-27 14:59:22 +01:00
Eva Marco
c26a67a8dd
♻️ Use a DS button for previous in the onboarding questions (#12550)
* ♻️ Use a DS button for previous in the onboarding questions

The "Previous" button of the onboarding questions becomes a DS
secondary button; it was a raw button with the deprecated modal cancel
styles. It is a bit narrower and its label uses the DS typography.

"Next" stays: it is the shared forms submit button. The "Other" option
icon stays too: it belongs to the forms image radio buttons, which move
with the radio buttons group. A visual test covers the step 2 buttons
at rest and with "Previous" hovered.

AI-assisted-by: claude-opus-5-5

* ♻️ Move onboarding questions styles closer to the DS

The onboarding questions stylesheet resolves its legacy modal color
aliases to DS tokens, uses logical properties and the DS typography
module without a namespace. refactor/ stays only for the modal
placeholders (%modal-*), which move with the modals refactor. No visual
change (checked on steps 1 and 2; only the shuffled options differ
between runs).

AI-assisted-by: claude-opus-5-5
2026-10-27 14:07:42 +01:00
Eva Marco
29e79edc66
♻️ Use DS buttons and icons in the dashboard team pages (#12531)
* ♻️ Use DS buttons and icons in the dashboard team pages

The team pages (members, invitations, webhooks and settings) and their
modals move off the deprecated icons and raw buttons:

- The member and webhook row menus become ghost DS icon buttons named
  "Options", with aria-expanded.
- The member and invitation role selectors become buttons with
  aria-haspopup and aria-expanded, and use the DS arrow icon. They were
  divs; one carried a role and an id repeated on every row.
- The four modal close buttons become ghost DS icon buttons named
  "Close"; the class that places them goes on the tooltip wrapper.
- "Create webhook" becomes a DS primary button.
- The webhook status, team image and settings icons use the DS icons.

The invitation checkbox tick stays until the checkbox migration. A
visual test covers the role and row menus, the resend and webhook
modals and the team image overlay.

AI-assisted-by: claude-opus-5-5

* ♻️ Move dashboard team styles to logical properties and DS tokens

The remaining legacy color aliases resolve to their DS tokens, physical
offsets and margins become logical properties, and the DS typography
module is used without a namespace. Four selectors that team.cljs does
not use are commented out with a TODO.

refactor/ stays only for component-family placeholders: %input-checkbox
and %checkbox-icon (checkboxes) and %modal-* (modals). The shared
dashboard header styles (common-dashboard) are used by ten dashboard
modules and move in their own change. No visual change.

AI-assisted-by: claude-opus-5-5

* 🐛 Open the team row menus below their button

The member and webhook row menus were placed from the bottom of their
cell with a fixed offset, so their position depended on the menu
height and the button height. The two-item webhook menu covered its
button, the member menu covered the lower part of it, and the taller DS
icon button moved both 3px down. They now open 4px below the button.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the dashboard team pages

The SCSS refactor left four rules commented out with a TODO because
team.cljs does not use them. CSS modules scope the classes to this
module and no other source uses them, so they go: .dashboard-top-cta,
.error-notification, .empty-invitations-buttons and
.invitations-dropdown. No visual change.

AI-assisted-by: claude-opus-5-5

* 🐛 Find the member role selector as a button in the dashboard test

The "Bug 9443" test looked for the member role selector as a combobox.
The selector is now a button with aria-haspopup, so the test failed in
CI. It now looks for a button named "Admin", and checks that the owner
has no such button.

AI-assisted-by: claude-opus-5-5
2026-10-27 12:19:47 +01:00
Alejandro Alonso
deb5771248
✨ Use WASM for SVG markup in code inspect and copy (#12567) (#12608)
When render-wasm is active, generate SVG for the Code inspector and
Copy as SVG from the WASM export path instead of classic SVG codegen.
Add multi-root render_shapes_svg so multi-select stays one document.

Closes #12567
2026-10-27 12:05:30 +01:00
Eva Marco
112e6dedd7
♻️ Use DS icons in the copy button (#12558)
* ♻️ Use DS icons in the copy button

The shared copy button (Inspect attribute rows and code blocks, share
link, integrations) draws its clipboard and tick icons with the DS
icons at the small size, colored through the color property. The
button itself stays: it wraps whole rows. The icons are the same
drawings, so nothing changes on screen.

A visual test covers an Inspect computed attribute row and the code
block copy button, at rest, on hover and after copying.

AI-assisted-by: claude-opus-5-5

* ♻️ Move copy button styles off the deprecated styles

The copy button stylesheet no longer imports refactor/: the
button-style, flex-center and copy-wrapper-base mixins are inlined with
DS values, the tertiary button and icon color aliases resolve to their
DS tokens, and physical properties become logical ones. No visual
change (whole viewer visual spec compared; the code tab syntax
highlighting loads late on a cold first run, with either stylesheet).

AI-assisted-by: claude-opus-5-5
2026-10-27 11:41:36 +01:00
Eva Marco
8800ff4bc9
♻️ Use DS buttons and icons in the assets tab (#12559)
* ♻️ Use DS buttons and icons in the assets tab

- "Add library" and "Manage library" become DS primary and secondary
  buttons, full width with the label centered; they keep their test id.
- The filter button stays a raw segment of the search bar; its legacy
  icon becomes the DS filter icon. Its opened state comes before the
  focus and hover rules, as the legacy placeholder did, so the focus
  border still shows.
- The create/rename group modal gets a ghost DS icon button named
  "Close", placed on the tooltip wrapper, and DS cancel (secondary) and
  create/rename (primary) buttons instead of raw inputs.

A visual test covers the add and manage library buttons, the filter at
rest, on hover and opened, and the create group modal.

AI-assisted-by: claude-opus-5-5

* ♻️ Move assets tab styles off the deprecated styles

assets.scss no longer imports refactor/: its mixins, sizes, spacing and
input / button icon color aliases resolve to DS values and tokens, with
logical properties. The unused .sections-container, .section-item and
.section-btn rules are deleted (the filter menu is the shared context
menu).

assets/groups.scss does the same and keeps refactor/ only for the modal
placeholders (%modal-*) and %input-with-label. Its legacy uppercase
title becomes the DS headline-small typography, which differs only in
its size (12px instead of 11px): the group modal grows 2px.

AI-assisted-by: claude-opus-5-5
2026-10-27 11:41:23 +01:00
Eva Marco
33d304c0af
♻️ Use DS icons in the dashboard sidebar (#12534)
* ♻️ Use DS icons in the dashboard sidebar

The search, clear search, pinned projects placeholder and logout icons
of the dashboard sidebar use the DS icons, and they get their color
from the color property. The two search buttons had untranslated test
names as their labels ("dashboard-search"); they are now named "Search"
and "Clear search". The pin keeps its place: the DS small icon draws
12px inside a 16px box, so its margins shrink by 2px.

New key: dashboard.clear-search (en, es). The dashboard page object
looks the search button up by its new name. A visual test covers the
search box with and without a term, the pinned projects placeholder and
the profile menu.

AI-assisted-by: claude-opus-5-5

* ♻️ Move dashboard sidebar styles off the deprecated styles

Legacy color aliases resolve to their DS tokens, deprecated mixins,
sizes and font variables are inlined with DS values, physical
properties become logical ones and the DS typography module is used
without a namespace. An invalid max-width with two values, which the
browser ignored, is gone.

Two rules that match nothing are commented out with a TODO: the profile
menu arrow svg rule (the arrow is the DS icon svg itself) and the
upgrade plan section, which sidebar.cljs does not use.

refactor/ stays only for the profile menu placeholders (%menu-dropdown,
%menu-item-base) and the shared dashboard header styles
(common-dashboard). No visual change.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the dashboard sidebar

The SCSS refactor of the dashboard sidebar left rules commented out with
a TODO because the module does not use them, or because they target an
svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.profile-dropdown-item .open-arrow svg, .upgrade-plan-section,
.penpot-free, .no-limits, .power-up. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-27 11:41:19 +01:00
Eva Marco
afd838ea5b
♻️ Use DS buttons in the dashboard team modals (#12537)
* ♻️ Use DS buttons in the dashboard team modals

The close buttons of the create/rename team modal, the team permission
notice and the two "before you leave" modals become ghost DS icon
buttons named "Close"; the class that places them goes on the tooltip
wrapper. The cancel and "promote and leave" inputs of the leave modals
become DS buttons: secondary, and destructive once a member is picked
(primary and disabled before).

Two fixtures make the leave and reassign modal testable (the logged-in
user owns the second team, which has another member). A visual test
covers the create team modal and the leave and reassign modal.

AI-assisted-by: claude-opus-5-5

* ♻️ Move dashboard team modal styles closer to the DS

The change owner and team form stylesheets resolve their legacy color
aliases, spacing and typography mixins to DS tokens and use the DS
typography module without a namespace. The legacy uppercase title
becomes the DS headline-small typography, which differs only in its
size (12px instead of 11px): the "before you leave" title grows 1px and
the modal 2px. Unused selectors are commented out with a TODO.

refactor/ stays only for the modal placeholders (%modal-*), which move
with the modals refactor, and the input placeholders.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the dashboard team modals

The SCSS refactor of the dashboard team modals left rules commented out
with a TODO because the module does not use them, or because they target
an svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.input-wrapper, .cancel-button, &.danger. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-27 10:58:13 +01:00
Eva Marco
19fc9ebce8
♻️ Use DS icons in the viewer flows and interactions menus (#12520)
* ♻️ Use DS icons in the viewer flows and interactions menus

The flows and interactions triggers in the viewer header were divs with
a click handler, and their menus lived inside them. They are now
buttons with aria-expanded and aria-haspopup, and each menu sits next
to its button in a wrapper, since a button cannot hold the menu
entries. The play, arrow and tick icons use DS icons. The menu entries
stay as they are.

A visual test covers the interactions menu closed and open; the test
files have no flows, and the flows trigger shares its styles.

AI-assisted-by: claude-opus-5-5

* ♻️ Move viewer interactions styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties. The DS icons take their color from the
text and no longer shrink, so the old icon placeholders go away. The
DS body-small line-height is more compact, so the trigger labels sit
1px higher. The refactor/ import stays only for the shared menu
placeholders.

AI-assisted-by: claude-opus-5-5
2026-10-27 10:47:07 +01:00
Eva Marco
9854f8e54f
♻️ Move dashboard subscription styles off the deprecated styles (#12538)
* ♻️ Move dashboard subscription styles off the deprecated styles

The subscription stylesheet inlines the deprecated button-style and
flex-center mixins, resolves its legacy color aliases and sizes to DS
tokens, uses logical properties and the DS typography module without a
namespace. Three rules that match nothing (the arrow svg, .cta-link and
.open-arrow) are commented out with a TODO.

The subscription controls need no DS change: the plan card, the current
plan card and the manage subscription link have no DS equivalent, and
they use no legacy icons. refactor/ stays only for the power up menu
item placeholder and the shared dashboard header styles.

A visual test covers the sidebar plan card (closed and open), the team
settings plan and the members seats warnings. No visual change.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the dashboard subscription

The SCSS refactor of the dashboard subscription left rules commented out
with a TODO because the module does not use them, or because they target
an svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.icon-dropdown svg, .cta-link, .open-arrow. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-27 10:23:55 +01:00
Eva Marco
411f661bdb
♻️ Use DS buttons in the delete shared libraries modal (#12554)
* ♻️ Use DS buttons in the delete shared libraries modal

The delete shared libraries modal (unpublish a library, delete a shared
file or project) gets a ghost DS icon button named "Close", placed on
the tooltip wrapper, and DS buttons for cancel (secondary) and accept
(destructive for the danger style, primary otherwise) instead of raw
inputs. The buttons are narrower and the accept button takes the DS
destructive red.

A new empty library references fixture and a visual test cover the
unpublish library modal from the workspace main menu.

AI-assisted-by: claude-opus-5-5

* ♻️ Move delete shared libraries modal styles closer to the DS

The delete shared libraries stylesheet resolves its legacy modal color
aliases to DS tokens, uses a logical property and the DS typography
module without a namespace. refactor/ stays only for the modal
placeholders (%modal-*), which move with the modals refactor. Two
unused selectors are commented out with a TODO. No visual change.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the delete shared libraries modal

The SCSS refactor of the delete shared libraries modal left rules
commented out with a TODO because the module does not use them, or
because they target an svg inside a DS icon that is the svg itself. CSS
modules scope the classes to this module and no other source uses them,
so they go: &.transparent, .list-wrapper. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-27 10:20:11 +01:00
Pablo Alba
e4c1b154e3
✨ Add profile name and email to nitrate audit log (#12636)
* ✨ Add profile name and email to nitrate audit log

* ✨ Mr changes
2026-10-27 09:30:28 +01:00
David Barragán Merino
4b13562d02 👷 Use only the closest tag in the build key
The full `git describe` output is not stable across checkouts: the
length of its -g<sha> suffix depends on the local object count, so
build-bundle and build-docker could compute different keys for the same
commit on different runner workspaces. The sha already pins the commit,
so the closest tag is enough to tell versions apart.

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-26 20:30:08 +01:00
David Barragán Merino
cf19d7f870 👷 Include the version in the bundle and image build key
The bundle stamps the version into version.txt, so a commit that gets a
new tag (2.18.1-RC1 -> 2.18.1) needs a new bundle and image set. Keying
them by commit sha alone reused the RC artifacts for the final release.

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-26 19:55:11 +01:00
Eva Marco
f07452e36b
♻️ Use DS buttons and icons in the inspect code tab (#12521)
* ♻️ Use DS buttons and icons in the inspect code tab

"Copy all code" becomes a DS secondary button, and the expand panel
buttons ghost DS icon buttons labelled "Expand panel" (new key, en and
es). The collapse toggles of the CSS and markup blocks stay raw
buttons, since they sit in a 12px column; their arrow uses DS icons and
they get a name and aria-expanded. The copy buttons belong to the
shared copy button and move with it.

A visual test covers the code tab buttons, expanded and collapsed.

AI-assisted-by: claude-opus-5-5

* ♻️ Move inspect code tab styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties. The 11px uppercase titles become the
12px DS headline-small, the only difference. The code typography keeps
its values with a TODO: the DS code-font names "Roboto Mono", while
the app loads "robotomono". The full-width "Copy all code" DS button
centers its label explicitly. The refactor/ import stays only for the
copy buttons, styled here until the shared copy button moves to the DS.

AI-assisted-by: claude-opus-5-5
2026-10-26 17:55:54 +01:00
Eva Marco
96b62f6475
♻️ Use DS buttons in the delete account modal (#12540)
* ♻️ Use DS buttons in the delete account modal

The close button of the delete account modal becomes a ghost DS icon
button named "Close"; the class that places it goes on the tooltip
wrapper. The cancel and delete buttons become DS secondary and
destructive buttons, so their labels no longer wrap and the delete
button takes the DS destructive red. The delete button keeps its test
id.

A visual test covers the modal at rest and with the close and delete
buttons hovered.

AI-assisted-by: claude-opus-5-5

* ♻️ Move delete account modal styles closer to the DS

The delete account stylesheet resolves its legacy sizes, spacing,
mixins and color aliases to DS tokens and uses logical properties and
the DS typography module without a namespace. The legacy uppercase
title becomes the DS headline-small typography, which differs only in
its size (12px instead of 11px), so the title and the content below sit
1px lower. The organization counts used a fixed dark-theme gray; they
now use the themed DS secondary foreground. Two unused selectors are
commented out with a TODO.

refactor/ stays only for the modal placeholders (%modal-*), which move
with the modals refactor.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the delete account modal

The SCSS refactor of the delete account modal left rules commented out
with a TODO because the module does not use them, or because they target
an svg inside a DS icon that is the svg itself. CSS modules scope the
classes to this module and no other source uses them, so they go:
.fields-row, .select-title. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-26 17:17:46 +01:00
Eva Marco
376ecda94e
♻️ Move viewport top bar styles off the deprecated styles (#12488)
* ♻️ Move viewport top bar styles off the deprecated styles

The viewport top bars (view only bar of the inspect tab, path edition
bar) already use DS buttons. Move top_bar.scss to DS sizes, borders,
spacing tokens and logical properties, resolve the panel and shadow
color aliases, and drop the refactor import. The unused done-btn class,
the last user of the deprecated button placeholders here, goes away.
The two z-index values have no DS token, so they stay with a TODO.

Add a visual test of the view only bar; it looks the same as before,
pixel for pixel.

AI-assisted-by: claude-opus-5-5

* ♻️ Use the body-small typography in the viewport top bar title

The title only set a 12px font size, which body-small already has.
Use the DS typography instead of the inline size and its TODO. The
view only bar looks the same, pixel for pixel.

AI-assisted-by: claude-opus-5-5
2026-10-26 15:53:05 +01:00
Pablo Alba
f2e5b7e20e
🐛 Fix swapped nested component loses absolute position after a lib update (#12556) 2026-10-26 15:24:18 +01:00
Filip Sajdak
576bf4ed3d
✨ Close search and shortcuts panels with Escape (#12374)
* ✨ Close search and shortcuts panels with Escape

The workspace search and shortcuts panels could only be dismissed with
the mouse, which interrupts keyboard-first workflows.

A new use-escape-to-close hook gives each panel an on-key-down handler
that closes it on Escape while the user is working in it: the latest
pointer press or focus change must have landed inside the panel, or the
panel must hold focus when it opens (a panel that autofocuses its search
box, such as Shortcuts, is closed by the first Escape). Focus left
behind in a panel after the user moved on to the canvas does not count,
so Escape keeps its usual meaning there.

The handler never stops the event. The global Escape shortcut still
runs and finishes in-progress operations such as path drawing; an
earlier version stopped propagation and left a half-drawn path that
failed shape validation on the next commit.

In the search panel an open filters menu takes the first Escape and
the panel closes on the second.

The history panel needs no handler: Escape already closes it through
the :interrupt it emits.

Closes #9540

AI-assisted-by: claude-opus-5-5
AI-assisted-by: claude-sonnet-5-5
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>

* 📎 Add e2e test for closing panels with Escape

Cover the Escape handling added for the layers search and shortcuts
panels with four Playwright cases on the mocked workspace:

- Escape closes the layers search panel.
- With the search filters menu open, the first Escape closes only the
  menu and the second closes the panel.
- Escape closes the shortcuts panel right after it opens with `?`. The
  panel autofocuses its search box, so the first Escape must close it
  instead of only blurring the box.
- A path drawn while focus sits in the shortcuts panel is finished by
  Escape, which proves the panel does not swallow the global shortcut.

The first three cases fail on develop. The shortcuts case also fails
when the hook does not seed its tracker from the focused element.

AI-assisted-by: claude-sonnet-5-5
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>

* 🐛 Close History with Escape when opened by shortcut or menu

Only the header button opened History through initialize-history, which
removes the :document-history flag on the next :interrupt. The
Ctrl/Cmd+Alt+H shortcut and the "Version history" menu item toggled the
flag directly, so the global Escape left the panel open. Both now emit
initialize-history, like the header, and keep their own event origin.

Two e2e cases cover the shortcut and the menu; both fail without this
change.

AI-assisted-by: claude-sonnet-5-5
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>

* ♻️ Apply review suggestions to Escape-to-close

Use dom/child? instead of .contains in the Escape hook and drop the
redundant nil checks. Drive the path test through clickAt so its clicks
wait for the path tool. Add a negative Search case: Escape reaches the
panel after the last click landed on the viewport, and the panel stays
open. That case fails when the containment check is removed.

AI-assisted-by: claude-sonnet-5-5
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>

---------

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
2026-10-26 14:42:43 +01:00
Eva Marco
e24e9dc747
♻️ Use DS buttons and icons in the viewer header (#12523)
* ♻️ Use DS buttons and icons in the viewer header

The section buttons, full screen and edit become ghost DS icon buttons
(the last two were spans), with aria-pressed where they toggle. Share
and "Log in or sign up" become DS buttons. The zoom widget becomes a
button that toggles its menu, like the workspace one, with DS zoom
buttons inside. The breadcrumb and the current board name become
buttons with aria-expanded, and the page menu moves next to its
button. The arrow and tick icons use DS icons. The logo stays: it is
brand art.

A visual test covers the header in two sections and both menus.

AI-assisted-by: claude-opus-5-5

* ♻️ Move viewer header styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties, and the button placeholders go away.
The section buttons column goes from a fixed 92px to auto, since the
three 32px DS buttons no longer fit; the 1fr side columns keep it
centered. The zoom widget follows the workspace header one. The 11px
project name becomes the 12px DS headline-small. The global .selected
rule also highlighted the current page in the page menu: that row
keeps its highlight through its own rule. The refactor/ import stays
only for the shared menu placeholders; the options z-index (10) has no
DS token and keeps a TODO.

AI-assisted-by: claude-opus-5-5
2026-10-26 14:39:06 +01:00
Eva Marco
48a5875a71
♻️ Use DS icon buttons in the viewer board navigation (#12527)
* ♻️ Use DS icon buttons in the viewer board navigation

The previous, next and first board buttons of the viewer become
secondary DS icon buttons with the DS arrow and reload icons. They keep
their size and the viewer background at rest; on hover they take the DS
secondary look. The first board button had no accessible name; it is
now labelled. The position classes move to the tooltip wrappers, so the
fullscreen and comments sidebar offsets still apply.

New key: viewer.go-to-first-board (en, es). A visual test covers the
buttons at rest and on hover.

AI-assisted-by: claude-opus-5-5

* ♻️ Move viewer styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties, and the file no longer imports
refactor/. The section scrollbar uses the DS custom-scrollbar mixin; the
text selection and placeholder styles that %new-scrollbar gave to the
comments layer stay, with DS tokens of the same value. An invalid
padding-right with four values, which the browser ignored, is gone.

The DS body-small typography differs from the old one only in a more
compact line-height, so the board counter sits 1px higher. The z-index
values (10, 2) have no DS token and keep a TODO. The .viewer-header,
.not-fixed, .fixed and .frame-children rules are not used by this module
and are commented out with a TODO.

AI-assisted-by: claude-opus-5-5

* 🔥 Remove unused styles from the viewer

The SCSS refactor of the viewer left rules commented out with a TODO
because the module does not use them, or because they target an svg
inside a DS icon that is the svg itself. CSS modules scope the classes
to this module and no other source uses them, so they go:
.viewer-header, .not-fixed, .fixed, .frame-children g. No visual change.

AI-assisted-by: claude-opus-5-5
2026-10-26 12:48:43 +01:00
Belén Albeza
577cbe43e4
🐛 Validate the text payload before pasting it (#12612)
Text copied in Penpot carries its content in a data attribute of the
clipboard HTML, which plugins and other apps can rewrite. The v3 editor
and canvas paste used it as is, so bad data could reach WASM and the
file and break the shape.

The payload is now dropped, and the paste falls back to plain text in
the caret style, unless its content passes the text shape schema, its
span geometry holds only safe numbers and its file and team ids are
uuids. Payloads in HTML over the paste size limit are not decoded.

Copied text shapes pasted into the editor get the same checks, plus a
numeric position; when they fail, nothing is pasted.

Closes #12167

AI-assisted-by: claude-opus-5-5
2026-10-26 12:36:13 +01:00
Eva Marco
a5fd2df1f0
♻️ Make the grid track direction icon and the swap groups buttons (#12476)
* ♻️ Make the grid track direction icon a button

The direction icon at the start of each grid track value field selected
that column or row from a div with a click handler: no button role, no
name and no keyboard access. Make it a button labeled "Select column" or
"Select row", keeping its class and handler, and reset the browser
border so it looks the same. It stays a raw button: it is a segment of
the track field, not a DS button.

Add the workspace.layout-grid.editor.select-column and select-row keys
(en, es).

AI-assisted-by: claude-opus-5-5

* ♻️ Make the component swap panel groups buttons

The groups of the component swap panel opened from a div with a click
handler: no button role and no keyboard access. Make each group a
button, keeping its class, title and handler, and reset the browser
button styles so it looks the same. Its name is the visible group name.

Add a visual test with a fixture whose components sit in two groups, so
the swap panel lists them.

AI-assisted-by: claude-opus-5-5
2026-10-26 11:38:33 +01:00
Eva Marco
bac62a27bf
♻️ Use DS icons in the viewer comments filter (#12519)
* ♻️ Use DS icons in the viewer comments filter

The "Comments" filter trigger in the viewer header was a div with a
click handler, and its menu lived inside it. It is now a button with
aria-expanded and aria-haspopup, and the menu sits next to it in a
wrapper, since a button cannot hold the menu entries. The arrow and the
ticks use DS icons. The menu entries stay as they are.

A visual test covers the filter closed and open.

AI-assisted-by: claude-opus-5-5

* ♻️ Move viewer comments styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties. The DS icons take their color from the
text and no longer shrink, so the old icon placeholders go away. The
DS body-small line-height is more compact, so "Comments" sits 1px
higher. The refactor/ import stays only for the shared menu
placeholders. The z-index values (1 and 10) have no DS token and keep a
TODO.

AI-assisted-by: claude-opus-5-5
2026-10-26 11:36:46 +01:00
Eva Marco
ef17aea39e
♻️ Use DS icon buttons in the viewer thumbnails panel (#12518)
* ♻️ Use DS icon buttons in the viewer thumbnails panel

The expand, close and scroll buttons of the viewer thumbnails panel
become ghost DS icon buttons. They had no accessible name; they are now
labelled, and the expand button says whether the panel is expanded. The
rotated legacy arrow gives way to the DS arrow icons. The thumbnails
stay raw buttons, since they are list items.

New keys: labels.expand and viewer.thumbnails.scroll-left/right (en,
es). A visual test covers the panel collapsed, on hover and expanded.

AI-assisted-by: claude-opus-5-5

* ♻️ Move viewer thumbnails panel styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties, and the file no longer imports
refactor/. The scroll buttons keep their 60px height through
--button-height and still show only on hover; their tooltip wrappers
take the grid cells. The DS body-small typography differs from the old
one only in a more compact line-height, so the counter and the board
names sit 1px higher. The panel z-index (10) has no DS token and keeps
a TODO.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:33:11 +01:00
Eva Marco
9911f82087
♻️ Use a DS icon in the grid editor track options button (#12486)
* ♻️ Use a DS icon in the grid editor track options button

The options button of each grid track on the canvas drew a deprecated
icon and had no accessible name. Use the DS menu icon and label the
button "Track options". The button stays raw: it lives on the canvas,
where the track editor keeps a fixed screen size at any zoom.

The DS icon has a fixed 16px size, so its class scales it down with the
zoom (transform: scale(1 / zoom)), as the old svg rule did with its
width. It looks the same, but for a few antialiasing pixels.

Add the workspace.layout-grid.editor.track-options key (en, es) and a
visual test of a track at two zoom levels.

AI-assisted-by: claude-opus-5-5

* ♻️ Move grid editor styles off the deprecated styles

Move grid_layout_editor.scss to DS sizes, borders, spacing tokens,
colors and logical properties, and drop the refactor import. The canvas
sizes keep their division by the zoom. The grid editor color aliases
resolve to their DS tokens, and the unused close-btn class, the last
user of the deprecated button placeholders here, goes away.

The track editor and the editing bar look the same as before.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:32:06 +01:00
Eva Marco
7ff8df6ffb
♻️ Name the grid track expand buttons and make the annotation toggle a button (#12489)
* ♻️ Name the grid track expand buttons

The menu button that expands the columns or rows of a grid layout had no
accessible name. Label it "Columns" or "Rows" and expose its state with
aria-expanded. The track title next to it also toggles on click; it
stays a div, as a larger pointer target for the same action, so the
keyboard does not stop twice on it.

Add the workspace.layout-grid.editor.columns and rows keys (en, es).

AI-assisted-by: claude-opus-5-5

* ♻️ Make the annotation title toggle a button

The annotation title of a component toggled from a div with a click
handler, and it holds the edit, delete, save and discard buttons, so it
cannot become a single button. Move the toggle to a button with the
arrow and the "Annotation" name, with aria-expanded, that takes the
whole free width of the title. While editing or creating, the title
stays text and no longer collapses the annotation on click, as its
cursor already suggested.

Extend the annotation visual test with the collapsed state; it looks the
same as before, pixel for pixel.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:31:51 +01:00
Eva Marco
ac08e38c1a
♻️ Use DS buttons and icons in the export assets modal (#12517)
* ♻️ Use DS buttons and icons in the export assets modal

The export modal close button becomes a ghost DS icon button and the
progress toast close button an action one, which fits its 24px grid
column; both are labelled "Close". Cancel and Export become DS buttons.
The checkbox ticks and the toast icons use DS icons. The selection rows
stay raw buttons, since they are list rows, and so does the link-styled
retry/cancel button of the toast.

The export frames e2e spec now finds the Export button by role, and a
visual test covers the modal with all, some and no rows selected.

AI-assisted-by: claude-opus-5-5

* ♻️ Make the export progress ring a button

The export progress ring in the header opens the export details, but it
was a div with a click handler. It is now a button named after the
export status, with aria-expanded, and keeps its look.

AI-assisted-by: claude-opus-5-5

* ♻️ Move export assets modal styles off the deprecated styles

Spacing, sizes, borders, typographies and color aliases move to DS
tokens and logical properties, and the button placeholders go away.
The classes left without use are removed: the old modal buttons, the
old close icon, and the file export entries, which live in files.scss.

The refactor/ import stays only for the shared modal and checkbox
placeholders, which move to the DS with the modals and the checkboxes.
The checkbox placeholder shrinks a ticked icon to 12px, which pushed
the DS tick off center: the tick keeps its 16px box now, and does not
shrink inside the 14px flex box either.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:31:34 +01:00
Eva Marco
29b00f8535
♻️ Use DS icons in the text palette and its menu (#12452)
* ♻️ Use DS icons in the text palette and its menu

The scroll arrows of the text palette and the tick of the selected
palette in its menu used deprecated icons; use the DS arrow and tick
icons instead. The arrows are full-height scroll buttons with a fade, not
DS button sizes, so they stay raw buttons.

Add a visual test that opens a file with 16 local typographies, so the
arrows show, and captures the palette at rest, with the pointer over the
right arrow, after scrolling and with its menu open. Its fixture is the
library fixture with more typographies.

AI-assisted-by: claude-opus-5-5

* ♻️ Move text palette styles off the deprecated styles

Inline the button, flex and ellipsis mixins, move spacing, sizes and
borders to DS tokens and logical properties, and resolve the color
aliases to their DS tokens. The arrow, item and menu colors that change
on hover, disable or selection go through local variables, and the new
icons take them through currentColor instead of svg selectors. The three
typography lines share their ellipsis rule. The .disabled ::after rule
of the left arrow never matched and goes away; :disabled::after already
covers it.

The items, the empty message and the menu use the DS body-small
typography, whose line-height is 1.3 instead of 1.4. The arrows keep
their z-index, now commented, and the menu its legacy context menu one.

text_palette.scss and text_palette_ctx_menu.scss no longer import
refactor/.

AI-assisted-by: claude-opus-5-5

* 🐛 Show the text palette scroll arrows at rest

The enabled scroll arrows of the text palette had no color at rest, so
they only showed on hover, while the color palette arrows show in the
secondary color. Give them the same rest color.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:31:14 +01:00
Eva Marco
904f456c61
♻️ Use DS icons in the color palette and its menu (#12444)
* ♻️ Use DS icons in the color palette and its menu

The scroll arrows of the color palette and the tick of the selected
palette in its menu used deprecated icons; use the DS arrow and tick
icons instead. The arrows are full-height scroll buttons with a fade, not
DS button sizes, so they stay raw buttons. The color cells are swatches
and stay as they are.

Add a visual test that opens a file with 24 local colors, so the arrows
show, and captures the palette before and after scrolling and with its
menu open. Its fixture is the library fixture with more colors.

AI-assisted-by: claude-opus-5-5

* ♻️ Move color palette styles off the deprecated styles

Inline the button, flex and ellipsis mixins, move spacing, sizes and
borders to DS tokens and logical properties, and resolve the color
aliases to their DS tokens. The arrow and menu item colors that change
on hover, disable or selection go through local variables, and the new
icons take them through currentColor instead of svg selectors.

The empty palette message and the menu items use the DS body-small
typography, whose line-height is 1.3 instead of 1.4. The arrows keep
their z-index, now commented: the left arrow comes before the
positioned color list, so without it the colors would cover its fade.
The menu keeps its legacy context menu z-index, also commented.

color_palette.scss and color_palette_ctx_menu.scss no longer import
refactor/.

AI-assisted-by: claude-opus-5-5
2026-10-26 10:27:13 +01:00
David Barragán Merino
14449d9f3d Merge branch 'staging' into develop, keeping the subpath support 2026-10-26 10:03:57 +01:00
Alejandro Alonso
e764b17c31 Merge remote-tracking branch 'origin/staging' into develop 2026-10-26 09:53:59 +01:00
Danial Shirali
bf580e5aa7
🌐 Add translations for: Persian
Currently translated at 98.5% (2242 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fa/
2026-10-25 19:51:23 +00:00
imagoiq
8a15b0a916
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-24 08:51:42 +02:00
Andrey Antukh
74add05226 Merge remote-tracking branch 'origin/staging' into develop 2026-10-23 13:40:44 +02:00
Belén Albeza
2045fc4900
🐛 Keep text image fills in place when pasting (#12552)
A text image fill was always laid over the whole text box, so a pasted
image span showed another part of the picture in a box of another size.

Text image fills now honour the image transform, as shape fills do: the
image covers the transform rect instead of the whole box. Paste gives
each pasted image span a transform that keeps the image at the same
pixel offset from the span, like the gradients.

The SVG export emits one linked image per span transform, rather than
one per image id, and the style panel keeps the transform it reads back.

AI-assisted-by: claude-opus-5-5
2026-10-23 13:00:53 +02:00
Andrey Antukh
76a83387d9
⬆️ Update JVM, Node, pnpm and devenv dependencies (#12600)
Move the backend JDK to its latest maintenance build within Java 26 and
refresh the dependency versions pinned by hand in the development and
production images.

- Backend: Zulu JDK 26.0.2 to 26.0.2.1. The jlink module list keeps
  jdk.management, so the prometheus process metrics stay exported.
- Node.js 24.22.1 in .nvmrc, the development image tarball and the two
  plugins deploy workflows that pin it.
- pnpm 12.15.0 in the four Dockerfiles, the 34 packageManager fields and
  the eleven lockfiles.
- Development tools: clj-kondo 2026.10.20, Babashka 1.13.226, Pixi
  0.83.0, GitHub CLI 2.103.0, AWS CLI 2.38.5 and uv 0.13.5.
- Infra services: RustFS 1.0.3, Mailpit v1.32.0, and Valkey pinned to
  8.1.10 instead of the floating 8.1 tag.

Fix SERENA_UPDATE_VERSION in the compose file. A value there makes the
start script reinstall that Serena release on every start, overriding
the one baked into the image; it was replacing 1.7.0 with 1.6.1.

The hardened base image updates are left to Renovate.

Closes #12599

AI-assisted-by: deepseek-v4.1-flash

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-10-23 12:36:11 +02:00
penpot-renovate[bot]
033b9a3362
⬆️ update GitHub Actions (#12598)
- actions/create-github-app-token v3.3.0 (.github/workflows/auto-label.yml)
- pnpm/setup v3.1.1 (.github/workflows/plugins-deploy-styles-doc.yml)
- pnpm/setup v3.1.1 (.github/workflows/plugins-deploy-api-doc.yml)
- zizmor 1.31.1 (.github/workflows/lint-workflows.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-23 11:53:02 +02:00
penpot-renovate[bot]
c3a23174a1
⬆️ Update DHI base images (#12597)
- dhi.io/debian-base trixie-debian13-dev@9b02576 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@9b02576 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@de80b46 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9b02576 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@9b02576 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.7-debian13-dev@940ee35 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.1@ (docker/devenv/Dockerfile)
- dhi.io/node 24.22.1@ (.nvmrc)
- dhi.io/node 24.22.1-debian13-dev@20cf36e (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.1-debian13@32a2fa6 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.1-debian13-dev@20cf36e (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.1-debian13-dev@20cf36e (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-23 11:50:52 +02:00
Alejandro Alonso
e27c65c77a
✨ Run WASM asset exports locally in the browser (#12515)
When render-wasm is active, shape exports (single file, ZIP, and Export
boards multi-page PDF) run on the shared worker WASM module so the UI
stays responsive. A progressive PDF session (begin/add-page/finish)
updates progress per page; cancel settles on the client. The server
exporter path remains for the SVG renderer.
2026-10-23 11:02:23 +02:00
Florian Schrödl
9aa78a536b
✨ Resolve tokens with the tokenscript-interpreter package (#12348)
Replace the @penpot/tokenscript npm package (the TypeScript
interpreter) with penpot/tokenscript-interpreter, a Clojure and
ClojureScript interpreter that now lives in its own repository and
compiles into the frontend build.

The frontend adapter (app.main.data.tokenscript) resolves tokens with
it behind the existing :tokenscript flag and maps its results to the
values and errors the workspace uses:

- Token values are validated by the interpreter's token types, passed
  by their DTCG names: `1foo` is not a dimension, a number has no
  unit, an opacity is between 0 and 1. Penpot only adds its own
  limits (font weight names, text decoration keywords).
- Division by zero, incompatible units, unknown functions and
  function arguments out of range get messages of their own, and
  token pills show the message of their error.
- Font family names are quoted before resolving, as in the adapter
  this replaces (#11750): `Red Hat` stays one name.

The token form's value input passed the token name to the tokenscript
resolver in place of the typed value, and no name at all for a new
token. It now resolves the typed value under the form's token name, as
the StyleDictionary path does.

StyleDictionary stays the default until the interpreter is stable.

frontend/deps.edn pins the interpreter by tag and commit, v0.1.0.

Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
2026-10-22 17:32:52 +02:00
Andrey Antukh
ba78ccb8c9
✨ Harden file data offload job with lock, summary and metrics (#12403)
Take the file row with FOR UPDATE SKIP LOCKED when the offload job
starts and skip the run when the row is locked or gone, so the job
never runs while the file is being edited. Every live hot-storage
row of the file moves to cold storage inside one all-or-nothing
transaction, and the handler answers a summary map with the
offloaded rows, bytes and per-type stats.

Add bounded metrics for the job: rows and bytes counters labeled by
row type on successful runs only, and a timing histogram with the
ok, skip and error outcomes. The job definition now requires
metrics, the recorders live in app.jobs.metrics, and the existing
ephemeral job timing histogram pins its buckets.

Cover the job with a new test namespace: the happy path, the locked
row, missing and deleted files, the rows and bytes summary with its
no-op rerun, the whole-file abort on storage failure, the metric
definitions and emissions, and the submit-time params validation.
An end-to-end suite reads, snapshots, writes, exports, imports and
deletes offloaded files; concurrency tests pin the stale-revn skip,
the blocked-edit retry and the reclaimable orphans of a partial
failure. The file-gc scheduler test pins one job per eligible file,
and the scheduler private helper drops its bang.

Document the lock invariant, the deliberate no-pacing decision and
the metric label sets in the backend storage memory and the
configuration guide.

Related to #12398.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-22 15:31:16 +02:00
Andrey Antukh
f8f1a668f7
♻️ Use skip-if-active for cron no-overlap in backend worker (#12412)
The cron tick kept its own active-job count beside the shared
submit option. Delegate the no-overlap check to
::jobs/skip-if-active so both paths share one queue-qualified
check served by the partial index.

A tick now skips only when a live job sits on the :cron queue,
and a skip records both the cron counter and the generic skipped
counter.

Closes #12411

AI-assisted-by: muse-spark-1.3-contributor
2026-10-22 15:29:37 +02:00
Belén Albeza
0505eb3b1d
🐛 Fix pasted gradient fills in text editor v3 (#12516)
* 🐛 Keep text gradients in place when pasting at the caret

Text gradients are stored as fractions of the whole text box. Paste
copied those fractions unchanged, so in a box of another size the
handles and colours moved away from the pasted text.

Copy from the v3 editor now records where each span with a gradient or
image fill starts in its text box, and the box size, in the clipboard
payload. A new WASM export, text_editor_get_range_origin, gives the
start of any text range.

Paste at the caret finds where those spans start in the target and
rewrites their gradients so the handles keep the same pixel offset from
the span. Auto-sized texts use the box size they get when the editor
closes.

AI-assisted-by: claude-opus-5-5

* 🐛 Rebuild text gradient shaders when the text box resizes

Text gradient and image shaders are built into the cached Skia
paragraphs against the text box size. A resize with no content change
reused those paragraphs, so the colours stayed matched to the old box
until a reload.

The text now remembers the box size its shaders were built for and lays
itself out again when that size changes and a span has a gradient or
image fill. Text with only solid fills keeps its cache.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep text gradients in place when pasting on the canvas

A text selection pasted on the canvas becomes a new text whose box
differs from the source, so its gradients moved like they did for caret
paste.

After adding the new text, lay it out, find where its gradient spans
start and move their gradients to keep the same pixel offset from the
span, in the same undo step. The range origin export now takes a shape
id, so it works on texts that are not being edited.

AI-assisted-by: claude-opus-5-5

* 🐛 Keep text gradients in place when pasting copied texts at the caret

Text shapes copied whole, rather than as an editor selection, reached the
caret paste without any geometry, so their gradients still moved away from
the pasted text.

A shape copy now records where the gradient and image spans of each
selected text start, and the box size, by shape id. Paste joins them in
the order of the joined paragraphs, with no origin for texts copied
without one.

Copied texts each have their own box, so every span origin now carries
its own box size instead of one box for the whole payload.

AI-assisted-by: claude-opus-5-5
2026-10-22 13:39:26 +02:00
Alejandro Alonso
347a52d514
🐛 Keep Path out of update-team-photo audit props (#12530)
Nitrate encodes audit chunks as JSON and cannot serialize java.nio.file.Path;
mirror update-profile-photo and replace the upload map with safe props.
2026-10-22 11:07:07 +02:00
Eva Marco
6acb360f4a
♻️ Use DS icons and an icon button in the undo history panel (#12472)
* ♻️ Use DS icons and an icon button in the undo history panel

The undo history panel (History > Actions) drew its entry type icons
with the deprecated icon set, and the arrow that shows the details of an
entry was a div with a click handler: no button role, no keyboard
access and no accessible name.

- The entry type icons use the DS icon (drop-icon is i/drop in the DS).
- The arrow becomes a DS action icon button labeled "Show details", with
  aria-expanded for its state. It keeps its data attribute, its classes
  and its click handler, and it also shows on keyboard focus.

The DS icon draws 12px inside a 16px box, so the entry icon gets a
negative margin, and the button wrapper a negative end margin: icon,
arrow and text stay where they were. The arrow turns through its own
class instead of an svg rule.

Add the workspace.undo.entry.show-details key (en, es) and a visual test
that draws a rectangle, opens the actions list and expands the entry.

AI-assisted-by: claude-opus-5-5

* ♻️ Move undo history panel styles off the deprecated styles

Move history.scss to DS tokens, sizes, borders and logical properties,
and drop the refactor import. The entry color aliases become local
variables of the entry, set by its disabled and hover states, and the
arrow opacity goes through one as well.

AI-assisted-by: claude-opus-5-5

* 💄 Place the undo history icon and arrow without negative margins

Drop the -2px margin of the entry icon and the -6px end margin of the
details button wrapper. Keep text and arrow in place with token spacing
instead: the text after the 16px icon box starts with --sp-xs instead
of --sp-s, and the entry end padding is --sp-xxs, since the 24px button
leaves 6px around its 12px arrow. The icon now sits 2px to the right,
centered in its box.

AI-assisted-by: claude-opus-5-5
2026-10-21 15:59:31 +02:00
Eva Marco
d6e850e917
♻️ Use DS icon buttons for the bottom palette toggles (#12443)
* ♻️ Use DS icon buttons for the bottom palette toggles

The color palette and typographies toggles of the bottom palette were raw
buttons with deprecated icons. Replace them with DS ghost icon buttons
with small icons, as before, that show the open palette through
aria-pressed. The layout flags are keywords, so they are coerced to a
boolean for aria-pressed. Their title moves to the label, shown in the DS
tooltip with the shortcut.

The palette actions button spans the palette height, not a DS button
size, so it stays raw; only its deprecated icon moves to the DS icon.

Add a visual test that compares the palette with each toggle selected
with no tolerance.

AI-assisted-by: claude-opus-5-5

* ♻️ Move bottom palette styles off the deprecated styles

The palette toggles are DS icon buttons, so their placeholders go away.
The actions button keeps the tertiary button look through local state
variables, and its icon takes the color from the button. Spacing, sizes,
borders, opacity and the color aliases move to DS tokens and logical
properties, except right and width of the palette, which are the
transitioned properties. The grid template is split into areas, rows and
columns, since Sass reads its slash as a division with a size variable.

The z-index values came from the legacy global scale. The resize area,
the toggles and the actions button had their own (4, 2 and 2), plus 0
overrides while collapsed, but they sit in separate grid areas and the
palette content does not reach them: removing them changes no pixel with
the text or color palette, the actions menu open, the 80px size, or the
palette collapsed and hidden. Only the palette's own z-index stays, with
a comment: it keeps the palette above the viewport overlays (z-index 1),
and drops to their level while collapsed.

The help button styles are not used in palette.cljs, but the external
chat widget injected into the palette wrapper may use them, so they stay,
inlined on DS tokens.

palette.scss no longer imports refactor/.

AI-assisted-by: claude-opus-5-5

* ♻️ Use a DS icon button for the bottom palette actions

The palette actions button (⋮) stayed a raw button because it spans the
palette height, not a DS button size. The DS icon button takes its
height from --button-height, so a class now stretches it to the palette
and the hand-made colors and states go away. It uses the ghost variant,
like the palette toggles next to it.

The button had no accessible name and did not say that it opens a menu:
it now has the "Palette options" label (new en and es keys) with its
tooltip, aria-expanded and aria-haspopup. The class given to its icon
did not exist in palette.scss and is gone.

Only the button corners change in the palette screenshots: the raw
button painted its own rounded background, the ghost button is
transparent at rest.

AI-assisted-by: claude-opus-5-5
2026-10-21 14:35:12 +02:00
David Barragán Merino
95bc48a08a 🐛 Fix Admin Console URLs escaped twice
The Admin Console location built its proxy_pass target from a capture of
$request_uri, which nginx escapes again when the path has escaped
characters: %XX became %25XX and the query string ended up in the path,
so Admin Console links returned 404. Strip the prefix with a rewrite and
let proxy_pass forward the rewritten URI instead.

AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-21 14:22:53 +02:00
Eva Marco
1d54d170cf
♻️ Move layers panel styles off the deprecated button styles (#12478)
The layers panel already uses DS icons and buttons; its two raw buttons
(the layer filter pill and the focus mode title) stay raw, since they are
custom widgets. Their styles still used the deprecated %button-tag
placeholder and color aliases.

- Inline %button-tag in the filter pill: border, cursor, centering, and
  its focus and pressed states through local variables. Its hover rule
  set the icons to the color they already had, so it goes away.
- Resolve the icon, title, pill and menu color aliases to DS tokens. The
  filter menu item texts are the same color in every state, so only the
  background changes, through a local variable.

The refactor import stays for the shared %menu-dropdown placeholder.
Add a visual test of the focus mode title, a filter pill and the filter
menu; the panel looks the same as before, pixel for pixel.

AI-assisted-by: claude-opus-5-5
2026-10-21 14:11:15 +02:00
Eva Marco
ce84fce0dc
💄 Remove the border from the boolean operation buttons (#12510)
The boolean operation buttons at the top of the Design tab showed the
2px border of the legacy radio buttons. The design team is reworking
this section; for now only the border goes away, in normal and selected
state, by making the border color variables transparent there.

AI-assisted-by: claude-opus-5-5
2026-10-21 14:10:53 +02:00
Eva Marco
5259b4b2d5
♻️ Use DS buttons in the workspace right header (#12454)
* ♻️ Use DS buttons in the workspace right header

The comments and history buttons were raw buttons with deprecated icons;
they become DS ghost icon buttons that show their state through
aria-pressed, with the unread dot as a child of the comments button. The
view mode link had a click handler and no href; it becomes a DS icon
button too, so it gets a button role and keyboard activation. Their
tooltips open below and to the left, at the top right corner of the
screen. The zoom menu's Reset is a DS ghost button.

The share link stays raw with its deprecated icon until the DS has a
share icon; it now uses its own class instead of the view mode one.

Add a visual test that captures the header at rest, with comments
active, and with the zoom menu open.

AI-assisted-by: claude-opus-5-5

* ♻️ Move workspace right header styles off the deprecated styles

The comments, history and view mode buttons are DS icon buttons, so their
placeholders go away; the comments class only holds the unread dot. The
zoom widget color goes through a local variable, and spacing, sizes,
borders and the color aliases move to DS tokens and logical properties.

Delete the save status rules: the status widget moved to the left header
in #5773 and nothing here uses them anymore.

refactor/ stays only for the shared menu placeholders and for the
tertiary button and icon placeholders of the share link.

AI-assisted-by: claude-opus-5-5

* 🐛 Give the action icon button a color while pressed or active

The action variant of the DS icon button set the active background but
not the active foreground, so it kept the base value, initial. A custom
property set to initial is invalid at computed time, so the icon took
the color of its container while the button was pressed (:active) or
toggled on (aria-pressed): yellow in the workspace header, for example.

Use the accent color, as the ghost and secondary variants do.

AI-assisted-by: claude-opus-5-5

* 💄 Use the action variant for the right header icon buttons

The comments, history and view mode buttons had no background at rest,
on hover or while active. The DS action variant keeps that: no
background at rest and on hover, accent icon on hover. It is 24px
instead of the ghost 32px, and shows a small background while comments
or history is on.

The unread dot moves to 4px from the button corner, so it stays over the
top right corner of the icon in the smaller button.

AI-assisted-by: claude-opus-5-5

* 💄 Drop the background of a pressed action icon button

A toggled-on action icon button (aria-pressed) showed the quaternary
background next to the accent icon. Keep only the accent icon, as the
workspace header buttons looked before the DS migration. The background
stays for the click feedback (:active).

Only the right header comments and history buttons use the action
variant with aria-pressed today.

AI-assisted-by: claude-opus-5-5

* 💄 Widen the gap between the right header actions

The action icon buttons are 24px wide, smaller than the old ones, so
the comments, history and view mode buttons sat closer together than
before the DS migration. Raise the header gap from --sp-s to --sp-m to
keep the old spacing between the icons without resizing the buttons.
The separator absorbs the extra space, so the header width stays the
same.

AI-assisted-by: claude-opus-5-5

* ♻️ Make the right header zoom widget a button

The zoom percentage of the right header opened its menu from a div with
a click handler: no button role and no keyboard access. Make it a
button with aria-expanded, keeping its classes, title and handler; its
visible percentage is its accessible name. Reset the browser padding of
the button so the percentage stays in place.

AI-assisted-by: claude-opus-5-5

* ✨ Toggle the zoom dropdown from the zoom widget

Clicking the zoom widget while its dropdown was open left it open: the
handler always set the open state, and it stops the click before the
dropdown sees it. The widget now toggles the dropdown, as its
aria-expanded already suggests. Closing from outside or with Escape
works as before.

AI-assisted-by: claude-opus-5-5
2026-10-21 14:10:34 +02:00
Eva Marco
5dbf0731e5
♻️ Use DS buttons and icons in the plugins dialogs (#12470)
* ♻️ Use DS buttons and icons in the plugins dialogs

The plugins manager and the permissions, permissions update and try-out
dialogs used raw buttons, input elements styled as buttons and
deprecated icons. Replace them with the DS components:

- The four close buttons become ghost icon buttons labeled "Close", shown
  in their DS tooltip. The class that places them in the corner goes on
  the tooltip wrapper, so the tooltip anchors to the button.
- Install, and the cancel and accept buttons of the three dialogs, become
  DS primary and secondary buttons.
- The empty list puzzle and the external link icons use the DS icon.

The permission icons (oauth-*) stay: they are brand art.

Add a visual test with fixtures for a team with the plugins feature and
a profile with one installed plugin. It shows the empty manager, the
manager with a plugin and the permissions dialog.

AI-assisted-by: claude-opus-5-5

* ♻️ Move plugins dialog styles off the deprecated styles

Remove the styles the DS buttons and icons no longer need (close-icon,
primary-button, cancel-button, search-icon and the svg rules of the
migrated icons) and move the rest of plugins.scss to DS tokens, DS
typographies and logical properties.

- The close button wrapper keeps only its position; the deprecated
  tertiary button style was landing on the tooltip wrapper.
- The full-width cancel and accept buttons center their label again: the
  DS button does not center it when it grows.
- The external link icon gets a class instead of an svg rule.
- The deprecated body-small used line-height 1.4 and the DS one 1.3, so
  the permissions dialog is a few pixels shorter.

The refactor import stays for the modal placeholders until the modals
move to the DS modal.

AI-assisted-by: claude-opus-5-5
2026-10-21 13:11:04 +02:00
Eva Marco
858d5dad9c
♻️ Use DS buttons in the debug shape panel (#12484)
* ♻️ Use DS buttons in the debug shape panel

The object and tree dump buttons of the debug shape panel were raw
buttons with the browser default look. Use DS secondary buttons. The
DS buttons are taller, so the title row centers the shape name.

Add a visual test that turns on the shape-panel debug option and shows
the title row of the selected shape.

AI-assisted-by: claude-opus-5-5

* ♻️ Move debug shape panel styles off the deprecated styles

Move debug_shape_info.scss to DS spacing tokens, borders and logical
properties, resolve the panel background alias, and drop the refactor
import. The two font sizes match no DS typography, so they stay with a
TODO, and the unused shape-info-title class is commented out for a
manual check. The panel looks the same as with the previous commit.

AI-assisted-by: claude-opus-5-5

* 🐛 Let the debugging tools options scroll

The debugging tools panel lists 32 options in the left sidebar, whose
content area hides what overflows. On a short window the last options
were cut off with no way to reach them. Make the options list fill the
space under the title and scroll on its own.

AI-assisted-by: claude-opus-5-5

* ♻️ Move debugging tools panel styles to DS tokens

Move debug.scss spacing, size and the panel background alias to DS
tokens and logical properties. The refactor import stays only for the
checkbox placeholders, until the checkboxes move to the DS. The panel
looks the same as before.

AI-assisted-by: claude-opus-5-5
2026-10-21 12:32:09 +02:00
Stas Haas
22fe9984f1
🌐 Add translations for: German
Currently translated at 83.6% (1904 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/de/
2026-10-21 11:52:44 +02:00
Andrey Antukh
fcf9c27635 Merge remote-tracking branch 'origin/staging' into develop 2026-10-21 10:14:55 +02:00
Pablo Alba
a5dac03073
✨ Add organization SSO and invitation audit events for nitrate (#12461) 2026-10-20 14:08:40 +02:00
Shreyash Agare
ecfa350470
✨ Auto-scroll sortable lists while dragging near their edges (#11595)
* ✨ Auto-scroll sortable lists while dragging near their edges

Native drag and drop swallows wheel events, so a long list could only be
scrolled mid-drag by hitting the browser's narrow auto-scroll strip at the
very edge of the container.

use-sortable now scrolls the item's nearest scrollable ancestor on every
animation frame while the pointer sits within 56px of its top or bottom
edge, ramping the speed up to 18px per frame as the pointer gets closer.
Every list built on the hook (layers, sitemap, option rows, gradients,
token sets) gets it without any change on their side.

* ✨ Cap auto-scroll zones and test the scroll loop

Each edge zone now takes at most a quarter of the container, so the
middle half of a short list, like the pages panel, never scrolls.

The scroll container is found once when the drag starts. The speed is
scaled by the time between frames, so 120 Hz screens no longer scroll
twice as fast. Starting a drag also stops any loop left over from a
drag whose end event was lost.

auto-scroll-speed is now private. New tests cover the running loop:
only one loop starts, it stops on drag end, when the speed goes back
to zero and after 500ms without dragover, and the frame time scaling.

AI-assisted-by: claude-opus-5-5

---------

Co-authored-by: Shreyash Agare <agareshreyash26@gmail.com>
Co-authored-by: Shreyash Agare <264953665+ShreyashAgare26@users.noreply.github.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-10-20 11:27:31 +02:00
Eva Marco
47d7ad2de1
♻️ Explain layout_container's deprecated import and format visual spec (#12449)
* ♻️ Explain why layout_container.scss keeps the deprecated import

The TODO at the top said to remove the refactor/ import once the buttons
were replaced, and sat above the DS imports instead of that one. The
buttons are now DS buttons (#12428, #12431), but the import is still
needed by the numeric inputs (%input-element, %disabled-input) and by the
two raw track header buttons, which keep the secondary and tertiary
placeholders. Replace the TODO with a comment on the import that says so.

AI-assisted-by: claude-opus-5-5

* 📎 Format the workspace visual spec with Prettier

Three expectations added in #12423, #12428 and #12431 broke a call chain
in a way Prettier rewrites. check-fmt:js did not catch it: its
playwright/**/*.js glob is expanded by the shell without globstar, so it
does not reach visual-specs/.

AI-assisted-by: claude-opus-5-5
2026-10-20 09:19:17 +02:00
Eva Marco
9904e50889
♻️ Use DS icon buttons in the component options (#12394)
* ♻️ Use a DS icon button for the component pill menu

Replace the raw three dots button of the component pill with the DS
secondary icon button. It had no label, so it reuses the "Options" one,
shown in the DS tooltip above and to the left (it sits at the right edge
of the sidebar), and aria-pressed while the menu is open instead of the
selected class.

Its class drops the button placeholders and only keeps the pill segment
shape: full height and square corners on the left. The tooltip wrapper
fills the segment too.

Add a visual test that selects a component copy so the pill shows.

AI-assisted-by: claude-opus-5-5

* ♻️ Use DS icon buttons for the component annotation actions

The save/create, discard, edit and delete actions of a component
annotation were divs with a click handler: no keyboard focus and no button
role. Replace them with DS ghost icon buttons. Their title moves to the
label, shown in the DS tooltip above and to the left, and save is disabled
while the text is blank, which it already ignored.

The class that hides them until the title is hovered, or while editing or
creating, now goes on the tooltip wrapper. The green and red hover colors
and the fixed size go away with the DS look. While editing or creating,
the actions get a small padding so their hover background does not touch
the annotation border; the title row grows from 32 to 36px there.

Add a visual test that creates an annotation on a main component and
shows both sets of actions.

AI-assisted-by: claude-opus-5-5

* 💄 Open component options tooltips above and to the left

The variants help, create variant, add property and remove property icon
buttons sit at the right edge of the sidebar: open their tooltips above
and to the left, like the other actions of the component options.

AI-assisted-by: claude-opus-5-5

* ♻️ Move component options styles off the deprecated styles

Inline the dropdown placeholders of the component pill menu with DS
tokens and a local variable for the item colors that change on hover, and
give the item label its own class instead of styling the span. Resolve
the annotation title border alias to its DS token and delete the unused
.component-title-actions rule.

The item label uses the DS body-small typography; its line-height is 1.3
instead of 1.4, with no visible change since the item centers it.

component.scss no longer imports refactor/.

AI-assisted-by: claude-opus-5-5
2026-10-20 09:18:49 +02:00
Eva Marco
7e5e589744
♻️ Use a DS icon button for the guide defaults menu (#12391)
* ♻️ Use a DS icon button for the guide defaults menu

Replace the raw three dots button that opens the use default / set as
default menu of a guide with the DS ghost icon button. It had no label, so
it gets a new "Guide defaults" one, shown in the DS tooltip, and
aria-pressed while the menu is open instead of the selected class.

The add, show/hide and remove guide buttons, and the new one, sit at the
right edge of the sidebar: open their tooltips above and to the left.

The advanced options toggle stays a raw button: it is one segment of the
guide type control and an icon button would break that shape.

Add a visual test that adds a guide to a board and opens its advanced
options.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove guide defaults button styles

The three dots guide defaults button is a DS icon button now: delete the
.show-more-options rule, its state variables and its svg rule. The rest of
frame_grid.scss already uses DS tokens.

AI-assisted-by: claude-opus-5-5
2026-10-20 09:18:24 +02:00
Andrey Antukh
be2e117f67
♻️ Rename audit-log archivers to consumers and use base Nexus URI (#12459)
* ♻️ Rename audit-log archivers to consumers

The pending-queue owners are no longer only external archives: a
local, non-archiving drain is about to join them, so "archiver" no
longer fits. Rename the column, identifiers, jobs, metrics and docs
to "consumer" in one release; migration 0157 never reached
production, so a single RENAME COLUMN is enough, with no backfill.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Use base Nexus URI with appended ingest path

Mirror the admin-console setup: PENPOT_NEXUS_URI holds the base
URI and the consumer appends api/audit itself, instead of a full
endpoint URI. Keeps the explicit :uri job param as override and the
:task-not-configured guard when no base is set.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-19 23:25:44 +02:00
Andrey Antukh
ccf99c7dcf
♻️ Archive audit log from a pending queue with two jobs (#12373)
* ♻️ Archive audit log from a pending queue with two jobs

Replace the mark-and-sweep audit archive with a pending queue
(audit_log_pending, migration 0157): every row is born pending
for the archivers enabled at insert time, in the same
transaction, so a rollback takes both. Two jobs share one
namespace, one queue each: :audit-log-archive-nexus and
:audit-log-archive-nitrate, each with its own closed params
schema, deps and every-5m cron entry gated by its send flag.

Each job drains its queue oldest-first in 128-row chunks (one
transaction per chunk, same total order, no SKIP LOCKED so
arrival stays chronological) and deletes its pendings on
completion; whoever deletes the last pending deletes the row.
Housekeeping lives in both jobs: reap of disabled pendings
plus sweep of orphans, both idempotent. A refused chunk raises
:archive-chunk-refused and aborts the job; the cron retries.
No separate GC task.

Metrics with bounded labels only: archived/failed/skipped per
destination, discarded per reason (reaped pendings, swept rows:
the silent R3 loss, now dashboard-visible). Timing comes from
the jobs substrate; queue depth is read off the replica DB.
No gauge, no custom histogram. Destination hosts log once at
job-def init; per-run logs carry counts plus frozen elapsed
(log props evaluate lazily in the logger thread).

Pre-existing rows are swept unsent (no backfill, accepted
loss); rows born with no archiver on share that fate (R3).
Memory mem:audit-log documents the design.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Address review comments on audit log archive jobs

Rename the second archive destination from nitrate to admin-console
(jobs, metric and log labels, error code), reusing the :admin-console
flag. Unify the pending-queue insert into a single helper used by
both producers. Stagger the archive crons (Nexus at second 0, admin
console at second 30). Add per-call dbg logs to the admin-console
send. Update the audit-log memory and the affected tests.

Closes #12372

AI-assisted-by: muse-spark-1.3-contributor
2026-10-19 22:35:58 +02:00
Andrey Antukh
fffc3ee222 Merge remote-tracking branch 'origin/staging' into develop 2026-10-19 22:18:27 +02:00
Eva Marco
cdb7dc2977
♻️ Use a DS button for edit grid in the grid cell options (#12385)
* ♻️ Use a DS button for edit grid in the grid cell options

Replace the raw "Edit grid" button of the grid cell options with the DS
secondary button. Drop its alt attribute, which is not valid on a button;
the label already names it.

The column and row icons label the numeric inputs, so they move with the
numeric inputs migration.

Add a visual test that selects a cell in grid edit mode so the options
show.

AI-assisted-by: claude-opus-5-5

* ♻️ Move grid cell options styles to DS tokens

The edit grid button is a DS button now, so its button placeholder and
typography mixin go away; the class only stretches it and centers its
label. Merge the two .grid-cell-menu-container rules, inline the flex
mixins, and move spacing, borders and color aliases to DS tokens and
logical properties.

Use the DS body-small typography in the inputs; its line-height is 1.3
instead of 1.4.

The refactor/ import stays only for %input-element and %button-icon, used
by the inputs and their legacy icons, which move with the numeric inputs
migration.

AI-assisted-by: claude-opus-5-5
2026-10-19 17:05:22 +02:00
Eva Marco
1c70b0ce2d
♻️ Use the DS icon in the workspace left header (#12432)
* ♻️ Use the DS icon in the workspace left header

The shared library badge of the left header used a deprecated icon; use
the DS library icon instead.

Drop the deprecated icons inside the save status dot. They rendered as
500px SVGs with no stroke, so they never showed: the dot is only its
background color, and its title keeps the status text. The Penpot logo is
brand art and stays.

Add a visual test that opens a shared file and shows the header with the
badge, then the saved status after drawing a shape.

AI-assisted-by: claude-opus-5-5

* ♻️ Move workspace left header styles off the deprecated styles

Inline the flex and ellipsis mixins, move spacing, sizes and borders to
DS tokens and logical properties, and resolve the color aliases to their
DS tokens. The status dot takes its color from a local variable. The
badge icon gets its color through a class instead of an svg selector.

The project name uses the DS headline-small typography: 12px instead of
11px. The file name matches title-small exactly.

Left with a TODO: the rename input font size and z-index have no DS
match, and the pending status orange (#f5a91b) has no DS color.

left_header.scss no longer imports refactor/.

AI-assisted-by: claude-opus-5-5

* ♻️ Resolve the TODOs left in the workspace left header styles

The status dot used the legacy warning orange #f5a91b, which drifted from
the DS one when both replaced #fe4811 a few days apart in early 2025. Use
the DS --color-accent-warning (#fe9c07), the orange of every other DS
warning, so the dot looks a little more orange while saving.

The rename input had z-index 20, which did nothing: the input is neither
positioned nor a flex or grid item. Remove it.

The rename input only set its font size and got the rest from the global
reset; it now uses the DS title-small typography, like the file name. Its
line-height goes from normal to 1.2, so it grows from 16 to 16.8px.

AI-assisted-by: claude-opus-5-5
2026-10-19 15:52:48 +02:00
David Barragán Merino
d6e1cb23e0
🐳 Serve Penpot and the Admin Console from a subpath (#12358)
* 🐳 Serve the frontend under the path of PENPOT_PUBLIC_URI

Requests below the path are rewritten to the root, so a reverse proxy can
forward them unchanged. The Admin Console keeps receiving the original path.

AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>

* 🐳 Strip the public path before proxying to the Admin Console

The Admin Console is served at the root of its own server and applies the
public path at runtime, so it needs a matching Admin Console image.

AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>

* 📚 Document subpath deployments in docker-compose.yaml

AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>

* 📚 Describe subpath deployments without proxy rewrites

The frontend now serves the subpath itself, so the reverse proxy must
forward it unchanged.

AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>

---------

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-19 15:49:41 +02:00
Eva Marco
e4da0527fc
♻️ Use DS buttons and icons in the grid layout options (#12431)
* ♻️ Use DS buttons and icons in the grid layout options

The "Edit grid" button of a grid layout and the "Exit" button of the grid
editor were raw buttons with the deprecated secondary style. Replace them
with DS secondary buttons; their text and actions stay the same. Drop the
"alt" attribute of "Edit grid": it is not valid on a button.

The expand and add buttons of the column and row headers are 52px parts of
a custom header, not DS buttons, so they stay raw; only their deprecated
icons move to the DS icon, as does the direction icon of each track.

Add a visual test that shows the grid layout options and the grid editor
with the column tracks expanded.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove deprecated button styles from the grid layout options

"Edit grid" and "Exit" are now DS buttons, so their secondary placeholder
and typography go away; the classes only place them in the grid and center
their label. The track header and direction icons drop the deprecated icon
placeholder and get their colors through classes. The header buttons keep
their secondary and tertiary placeholders, since they stay raw.

AI-assisted-by: claude-opus-5-5
2026-10-19 15:49:14 +02:00
Eva Marco
3d6823eacd
♻️ Use DS icon buttons for the flex wrap and padding mode (#12428)
* ♻️ Use DS icon buttons for the flex wrap and padding mode

The wrap button of a flex layout and the button that shows the four
padding sides (flex and grid layouts) were raw buttons with deprecated
icons. Replace them with DS ghost icon buttons that show their state
through aria-pressed, with the tooltip above and to the left since they
sit at the right edge of the sidebar.

The wrap label is now always "Wrap": its title switched to "No wrap"
while pressed, which conflicts with the pressed state.

Add a visual test that selects a flex board and shows both buttons at
rest and pressed.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove deprecated wrap and padding toggle styles

The wrap and padding mode buttons are now DS icon buttons, so their
tertiary button, icon and selected placeholders go away. The tooltip
wrappers take the same grid cells the buttons had.

AI-assisted-by: claude-opus-5-5
2026-10-19 15:48:59 +02:00
Eva Marco
398678e54c
♻️ Use a DS icon button for the layout element margin mode (#12423)
* ♻️ Use a DS icon button for the layout element margin mode

The button that switches a flex or grid element between simple and
multiple margins was a raw button with a deprecated icon. Replace it with
a DS ghost icon button that shows the multiple state through aria-pressed.
Its title moves to the label, shown in the DS tooltip above and to the
left since the button sits at the right edge of the sidebar. The grid
column class goes on the tooltip wrapper.

The legacy margin icons label the numeric inputs and stay for the numeric
inputs group; the radio buttons are out of scope.

Add a visual test that selects a flex element and shows both margin modes.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove deprecated button styles from layout element options

The margin mode button is now a DS icon button, so its tertiary button,
icon and selected placeholders and the deprecated 32px height go away.
The class only places the tooltip wrapper in the third grid column.

The refactor/ import stays for %input-element, still used by the legacy
numeric inputs.

AI-assisted-by: claude-opus-5-5
2026-10-19 15:48:43 +02:00
Eva Marco
c345c7cbaa
♻️ Use a DS icon button to remove imported SVG attributes (#12382)
* ♻️ Use a DS icon button to remove imported SVG attributes

Replace the raw remove button of each imported SVG attribute with the DS
ghost icon button and the DS remove icon. The button had no title, so it
gets a new "Remove attribute" label, which the DS tooltip shows. Drop the
wrapper div that only held the button.

Add a visual test that selects a group imported from SVG so the section
shows.

AI-assisted-by: claude-opus-5-5

* ♻️ Move imported SVG attributes styles to DS tokens

The remove button is a DS icon button now, so .attr-actions and
.attr-action-btn with their button placeholders go away. Inline the flex
and ellipsis mixins, move spacing to DS tokens and logical properties,
sizes to px2rem and the title color alias to its DS token.

Use the DS body-small typography; its line-height is 1.3 instead of 1.4.
Drop the line clamp rules of the attribute name: its later display:
inline-block already overrode the -webkit-box they need.

The refactor/ import stays only for %input-element on the value input,
which moves with the inputs migration.

AI-assisted-by: claude-opus-5-5
2026-10-19 15:35:44 +02:00
Eva Marco
18ca25518e
♻️ Use a DS icon button to close the nudge amount modal (#12439)
* ♻️ Use a DS icon button to close the nudge amount modal

The close button of the nudge amount modal (main menu, Preferences) was a
raw button with a deprecated icon and no accessible name. Replace it with
a DS ghost icon button labeled "Close", shown in its DS tooltip. The class
that places it in the corner goes on the tooltip wrapper, so the tooltip
anchors to the button. The numeric inputs stay for the inputs group.

Add a visual test that opens the modal from the main menu.

AI-assisted-by: claude-opus-5-5

* ♻️ Move nudge modal styles off the deprecated styles

The close button is a DS icon button, so its placeholder goes away; it is
now 32px wide instead of 28px, and sits 4px from the edge instead of 6px
so the icon stays in the same place. Spacing, sizes and the color aliases
move to DS tokens and logical properties, the flex mixin is inlined, and
the label rule targets its class instead of the label element.

The title matches the DS headline-medium typography exactly. The content
and labels use body-large and the wrapper body-small, whose line-heights
differ by 0.1 from the legacy ones; the labels keep their 1.5 override,
so nothing visible changes.

refactor/ stays only for the shared modal and labeled input placeholders.

AI-assisted-by: claude-opus-5-5
2026-10-19 14:28:03 +02:00
Stephen Sagarino Jr.
8787937b7c
🐛 Truncate long labels in the shadow options panel (#12414)
Long translations of "Blur" and "Spread" ran over the number input in the
advanced shadow options. The label inherited display: flex from the legacy
%input-label placeholder, where text-overflow has no effect, and it could not
shrink below its text.

Make the label a block with the DS text-ellipsis mixin, size it to its text,
and cap it at the field width minus room for the number. Labels that fit look
the same as before; longer ones end in an ellipsis and keep their tooltip.

Closes #4456
2026-10-19 10:24:11 +02:00
Rafael X.
addc5bc31e
📚 Fix typo in incomplete user info hint (#12413)
Signed-off-by: Rafael X. <chocoladisco@gmail.com>
2026-10-18 22:29:18 +02:00
Andrey Antukh
49aeeb053b 🐛 Use multipart helper in jobs upload-chunk test
The multipart-only upload guard rejects the :content map sent by
upload-chunked! over plain command!. Send it through
multipart-command! like the other upload tests; forged-path tests
stay on plain command!.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-17 21:53:29 +00:00
Andrey Antukh
b633093336 Merge remote-tracking branch 'origin/staging' into develop 2026-10-17 16:28:59 +02:00
Andrey Antukh
fcbc4f1769 🐛 Remove duplicate declarations in the typography sample
`.typography-sample-input` declared `font-size` and `line-height`
twice with the same values, so stylelint failed the frontend lint
with `declaration-block-no-duplicate-properties`.

The duplication came from an earlier staging into develop merge: one
side moved the pair next to its comment, the other kept it below
`block-size`, and the merge kept both. Drop the second pair.

No behaviour change: both pairs held the same values, and only the
last declaration of a block takes effect.

AI-assisted-by: deepseek-v4.1-flash
2026-10-17 12:53:13 +00:00
Andrey Antukh
8c5113b815 Merge remote-tracking branch 'origin/staging' into develop 2026-10-17 14:44:44 +02:00
penpot-renovate[bot]
1a68928aca
⬆️ Update DHI base images (#12399)
- dhi.io/debian-base trixie-debian13-dev@7b1d652 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@7b1d652 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@7b1d652 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@7b1d652 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.7-alpine3.24@6d3811f (docker/images/Dockerfile.storybook)
- dhi.io/nginx 1.31.7-debian13-dev@10c0037 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.22.0@ (docker/devenv/Dockerfile)
- dhi.io/node 24.22.0@ (.nvmrc)
- dhi.io/node 24.22.0-debian13-dev@39f5547 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.22.0-debian13@c6ddacf (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.0-debian13-dev@39f5547 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.22.0-debian13-dev@39f5547 (docker/images/Dockerfile.exporter)
2026-10-17 14:06:09 +02:00
penpot-renovate[bot]
53f83c98cf
⬆️ Update GitHub Actions (#12401)
- actions/checkout v7 (.github/workflows/commit-checker.yml)
- actions/download-artifact v8.0.2 (.github/workflows/tests-e2e.yml)
- actions/upload-artifact v7.0.2 (.github/workflows/tests-e2e.yml)
- actions/upload-artifact v7.0.2 (.github/workflows/tests-e2e.yml)
- actions/upload-artifact v7.0.2 (.github/workflows/tests-e2e.yml)
- actions/upload-artifact v7.0.2 (.github/workflows/tests-e2e.yml)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
Co-authored-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-17 14:01:55 +02:00
Stas Haas
a776050bb6
🌐 Add translations for: German
Currently translated at 77.0% (1753 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/de/
2026-10-16 10:51:29 +00:00
Andrey Antukh
472ba8dde5 Merge remote-tracking branch 'origin/staging' into develop 2026-10-16 12:49:04 +02:00
Andrey Antukh
dfc55a7192 Merge remote-tracking branch 'origin/staging' into develop 2026-10-16 12:48:49 +02:00
Andrey Antukh
122d4ebcd3 🔥 Remove unused requires flagged by the Clojure linter
The dashboard required app.common.exceptions without ever using it,
and the plugin shape namespace still referred to
token-attr-plugin->token-attr after the per-side stroke width work
replaced its only use with expand-token-attrs. Both are leftovers of
earlier refactors; dropping them leaves clj-kondo with no warnings.

AI-assisted-by: deepseek-v4.1-flash
2026-10-15 20:23:25 +02:00
Eva Marco
ab7a994d6c
🐛 Fix libraries modal visual test (#12367)
The libraries modal now opens on the "This file" tab, so the visual test
timed out looking for the shared library. Open the "Libraries" tab first,
as the functional test does, mock the file with the library linked, and
check the "Added" label instead of the old "File library" text.

AI-assisted-by: claude-opus-5-5
2026-10-15 20:08:48 +02:00
Eva Marco
539983db2f
♻️ Use a DS icon button for flatten in the boolean options (#12368)
* ♻️ Use a DS icon button for flatten in the boolean options

Replace the raw flatten button and its deprecated icon with icon-button*
and the DS boolean-flatten icon. The title text moves to aria-label, so
the DS tooltip shows it. The grid column goes on the tooltip wrapper
through tooltip-class.

Add a visual test that selects two shapes so the boolean options show.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove deprecated button styles from the boolean options

The flatten button is a DS icon button now, so its placeholders, sizes
and icon colors go away; only the grid column stays, on the tooltip
wrapper. Use block-size and drop the refactor/ import.

AI-assisted-by: claude-opus-5-5
2026-10-15 20:07:04 +02:00
Eva Marco
d26a401637
♻️ Use DS buttons for more colors in the selected colors (#12369)
* ♻️ Use DS buttons for more colors in the selected colors

Replace the three raw "more colors" buttons (library colors, colors and
color tokens) with the DS secondary button.

Add a visual test that selects a board with more than three colors so
the button shows.

AI-assisted-by: claude-opus-5-5

* ♻️ Move selected colors styles to DS tokens

The "more colors" buttons are DS buttons now, so their placeholder and
typography mixin go away; the class only centers the label, since the
column stretches the button. Delete the unused .add-fill rule, move
spacing to DS tokens and logical properties, and drop the refactor/
import.

AI-assisted-by: claude-opus-5-5
2026-10-15 20:06:29 +02:00
Eva Marco
47f0e15744
♻️ Use DS icon buttons in the align options (#12366)
* ♻️ Use DS icon buttons in the align options

Replace the eight raw align and distribute buttons with icon-button*
and DS icons. The title text moves to aria-label, so the DS tooltip
shows the same text and shortcut. Disabled distribute buttons now use
the DS disabled color.

Add a visual test that selects two shapes so the align options show.

AI-assisted-by: claude-opus-5-5

* ♻️ Remove deprecated button styles from the align options

The align buttons are DS icon buttons now, so the .align-button rules
and their placeholders go away. Move the row height to a DS size token
and drop the refactor/ import.

AI-assisted-by: claude-opus-5-5
2026-10-15 20:05:58 +02:00
Andrey Antukh
57fa280209
🐛 Validate and decode a job result with its declared schema (#12350)
* 🐛 Decode job results with the schema of the job-def

A job result is stored as plain JSON, so a uuid comes back as the text
it is on disk and a uuid used as a map key comes back as a keyword. The
import resolution is a map keyed by file id, and the wizard lost that
identity: choosing a library in the link step created no link at all,
with no error and with the modal closing as if it had worked.

A job-def can now declare a result schema and its precompiled decoder,
the pair the params already have, and the substrate applies them when the
row is read. A def that declares neither answers with the JSON the row
holds, as before; a row whose job-def is gone is read untouched instead
of raising.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Type the import result so its resolution keeps uuid keys

The resolution of a version 3 import is a map keyed by the new file id,
and the result column is plain JSON: without a schema of its own every
key came back as a keyword while the :id of the entry came back as text,
so the dashboard wizard could not match the file it was linking and
created no library link at all.

Declare the shape of the resolution (files, done and pending entries and
their candidates) and give the job-def its result schema and decoder.
The check before the result is stored keeps validating the same schema.

Closes #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Take the imported file id from the entry, not the map key

The wizard looked a file up in the selection with the key of the
resolution map while it had indexed the selection by the :id of the
entry. The key is keywordized on its way through the job result, so the
lookup missed and the confirm button created no library link at all,
with no error and with the modal closing as if it had worked.

Take the id from the entry, the source the rest of the wizard already
uses, and move the decision to a pure function covered by tests of both
shapes of the resolution.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Test that an import result keeps its ids as uuids

The regression this branch fixes is only visible through the read path
of the substrate: the resolution of an import is a map keyed by the new
file id, and a test that works with the value in memory never sees the
keys coming back as keywords from the plain JSON column.

Import a package whose library needs a replacement chosen by the user,
read the result with `jobs/get-job` as the client does, and check the
keys, the file ids and the ids of the candidates. Without the result
decoder of the import job-def the test answers with text ids.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📚 Document the result schema of a job-def

The memories and the docstring of the substrate said a job result has no
schema of its own and comes back as the JSON the row holds. A job-def can
now declare its result schema and decoder, and the substrate applies them
when the row is read.

Keep the rule next to it: a result map keyed by uuid needs its schema
declared, or every key comes back as a keyword, and the link-later step
of the import wizard takes the file id from the entry and never from the
key of the resolution map.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📚 Fix the documented contract of a job result

The docstring of get-job said a result has no schema of its own and comes
back as the JSON the row holds, which is what this branch changes: the
result is decoded with the schema its job-def declares, and a result that
does not conform never reaches the column because the write validates it.

Say the same in the management command that stores the result of an
external worker, the other writer of the column.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Check the result of a job against its schema before storing it

The result of a job crosses a plain JSON column, so a uuid comes back as
text and a uuid used as a map key as a keyword. The job-def now declares
the schema of its result and the substrate compiles the decoder from it,
which is what the read path uses to get the types back.

The same schema is the contract of the write path: complete refuses a
result that does not conform, with the malli explanation, so nothing
invalid reaches the column, and the answer is the same for the runner and
for the external worker that completes a job through the management RPC.
A job that is no longer completable is skipped without checking, and the
read path decodes only: a row written by another version stays readable.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Check the result of an import inside its transaction

The check ran after the transaction of the import had committed, so a
result the schema does not accept failed the job with the files already
created in the destination project: a half imported project and an error.
The core decides whether the whole import is kept or rolled back, so the
check belongs inside its transaction, which is what turns a contract
violation into a rollback.

The substrate checks the result again when the job is completed, which is
the only net for a job that does not validate its own result.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Cover the edges of an import result read by the client

The shape of a result with no libraries to resolve, and the one of a
version 1 import, is the most common in production and neither was covered
by the tests that read a result the way the client does.

Cover both through jobs/get-job, and the case of a registry that no longer
knows the kind of the row: it answers with the JSON of the column,
keywordized keys and uuid as text, instead of raising. Say in the two tests
that depend on the decoder what implementation without it makes them fail.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📚 Fix the prose and the fixtures of the result contract

The result schema of a job-def is what the read decodes with and what the
write checks: say that in the memories, with the rule that any uuid of a
result needs it, whether a value or a map key.

The motive of the wizard helper was a fact this branch makes false: what a
key of the resolution map holds depends on the result schema of the job-def
that wrote it, so the rule is to read the id from the entry. Same
correction in its memory and in the test that pins the shape.

The candidates of a pending entry match by slugified name, not by name, and
every test row now states the kind of its job, because the name is what
resolves its job-def when the result is read or checked.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* ♻️ Keep the wizard link helper private and cover its shape

The helper that decides which links the wizard creates belongs to its
namespace, as its neighbours do: reach it from the test with @#', the
pattern the frontend tests already use for a private helper.

Cover the shapes it had no test for: a file with two libraries pending
resolution, a file whose libraries were auto-linked, and a selection where
only one of two entries was resolved.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* ♻️ Share one job-row helper across the job tests

Three test namespaces had their own copy of the same insert, differing only
in the kind of the job and in whether the row holds the resource it
consumes. Move it to the test helpers as create-job*, with the kind of the
job as its first argument: it is what resolves the job-def of the row when
its result is read or checked, so every test states it.

The bang of the name goes away with it: the design rules keep it for the
mutating API of app.db, and this is a fixture of the tests. The three
namespaces lose the requires that only their removed copy used.

Relates to #12339

AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Andrey Antukh <niwi@niwi.nz>

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-10-15 19:18:04 +02:00
Philippe Accorsi
c92b5f25da
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-15 10:51:30 +02:00
imagoiq
754e9dd7f6
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-15 10:51:29 +02:00
moeenio
1211212fd2
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-15 10:51:28 +02:00
andrés gonzález
7383e08683
✨ Focus the search field when opening the color tokens tab (#12345)
The color tokens tab of the color picker did not focus its search
field, so users had to click it before typing a token name. The token
dropdown of numeric inputs already focuses its search on open.

Closes #12344

AI-assisted-by: claude-opus-5-5
2026-10-15 10:10:11 +02:00
Alonso Torres
e3b8b006db
🐛 Fix font family space in tokens (#11750)
* 🐛 Fix font family space in tokens

* 🐛 Resolve token merge conflicts

Keep the font-family token fix and the newer
background-plugin change together in the
unreleased changelog and plugin API test suite.

AI-assisted-by: gpt-6-luna
---------

Co-authored-by: niwinz <843689+niwinz@users.noreply.github.com>
2026-10-15 10:05:44 +02:00
Andrey Antukh
9dc61b1c18 Merge remote-tracking branch 'origin/staging' into develop 2026-10-14 18:36:25 +02:00
Alonso Torres
286a6d5c9a
✨ Add new dashboard methods to the API (#12169)
* ✨ Add new dashboard methods to the API

* 🐛 Fix some issues with the new methods

* 🐛 Fix problem with mcp test
2026-10-14 15:24:53 +02:00
Andrey Antukh
d0f82b9979 Merge remote-tracking branch 'origin/staging' into develop 2026-10-14 14:37:58 +02:00
Pablo Alba
ba81f2f8bb
🐛 Fix audit-log-archive cron submit and nitrate wiring (#12321)
After the unified jobs migration, cron submitted empty params that
failed validation because the job schema still required Integrant
system keys. Narrow the schema to optional REPL overrides, wire
:app.nitrate/client onto the archive job-def, and drop the unused
::jobs/defs ref from nitrate that created an Integrant cycle.

AI-assisted-by: composer
2026-10-14 14:21:18 +02:00
Alejandro Alonso
1ebb4f3866
🔧 Fail CI when render-wasm JS glue exports invoke_*V (#12243)
* 🔧 Fail CI when render-wasm JS glue exports invoke_*V

Add check-no-invoke-v (path from _build_env) and run it in CI: WASM
after a production frontend build so JS longjmp + SIMD ABI
regressions are caught in the pipeline.

* 🐛 Fail closed in render-wasm invoke_* glue check

* 🔧 Check exporter wasm glue for any invoke_* wrapper

---------

Co-authored-by: Elena Torro <elenatorro@gmail.com>
2026-10-14 10:15:53 +02:00
Andrey Antukh
4023642acb
🌐 Complete Russian translations to 100% (#11956)
Complete ru.po from 1460 to 2275 keys (100%),
add words.ru.txt catalog, fix two dropped %s
placeholders and normalize board/layout/shortcut
terms. Documents Russian conventions in the
translations memory.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-14 09:58:09 +02:00
Andrey Antukh
0c0336da42 Merge remote-tracking branch 'origin/staging' into develop 2026-10-13 21:04:58 +02:00
Elena Torró
a44c8aba88
🔧 Remove wasm warnings (#12291)
* 🔧 Drop emscripten verbose linker output from debug builds

* 🔧 Fix render-wasm warnings from Rust 1.99
2026-10-13 16:33:51 +02:00
andrés gonzález
89c6bad8cd
✨ Open files and projects in a new tab with Ctrl/Cmd+click (#12230)
Ctrl+click (Cmd+click on macOS) on a dashboard file card, a project
title, the "Show all files" button, or a project or Drafts item in the
sidebar now opens it in a new browser tab. It reuses the same
`::rt/new-window` navigation as the "Open in new tab" file menu option.

On file cards the modifier click does not change the selection, and a
modifier double click does not navigate the current tab. Files in the
trash are not opened.

Closes #12228

AI-assisted-by: claude-opus-5-5
2026-10-13 12:56:48 +02:00
Miguel de Benito Delgado
6f34794f54
🐛 Fix render-wasm build arguments and artifact selection (#12254)
Forward Cargo arguments as an array and honor explicit profiles.
Build and copy each renderer target once under the repository lock,
with watch rebuilds using the same wrapper.

Select this checkout's compiler artifact and matching generated enum
table from Cargo JSON. Record actual features, profile and output paths,
and allow callers to override publication destinations.

Run the existing Cargo parser tests before the renderer's native tests
and stop on parser failures.

AI-assisted-by: gpt-6.1-sol
2026-10-13 12:54:48 +02:00
Pablo Alba
2965a4c043
✨ Remove duplicated nitrate log type (#12275) 2026-10-13 09:33:06 +02:00
Andrey Antukh
5eb33add7e
✨ Add skip-if-active option to jobs submit (#12266)
* ✨ Add skip-if-active option to jobs submit

Producers that need at most one live job per key can now pass
::jobs/skip-if-active: the submit counts live jobs for the same
tenant, name, queue and label in the same transaction and skips
the insert (returning nil) when one lives. The option requires an
explicit label and documents the same best-effort race caveat as
the dedupe it complements.

Closes #12265

AI-assisted-by: muse-spark-1.3-contributor

* ✨ Record skipped job submissions as a metric

Skip-if-active submissions withheld nothing observable: the skip
now records penpot_jobs_skipped_total (name, queue) instead of the
submitted counter, so dashboards can tell withheld work apart from
new work. The insert path is untouched.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Drop jobs counters duplicated by histogram counts

The completed and requests counters counted exactly the same events
as the _count of their timing histograms, so they go away: terminal
totals now come from penpot_jobs_total_ms_count by outcome and
request totals from penpot_jobs_request_ms_count by outcome. One
event, one family from here on: counters only cover events without
latency, and the dispatched vs queue-wait pair stays because pushes
and claims are different events.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-11 18:48:02 +02:00
Surfoo
d33b59b0f7
🌐 Add translations for: French
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-10-10 18:51:24 +02:00
Filip Sajdak
10955f17e2
✨ Add get-file-tokens RPC to read a file's design tokens (#12172)
Clients that only need a file's design tokens, such as a CI job that
syncs them to a git repository after an update-file webhook, had to
download the whole file with get-file.

get-file-tokens returns the file's tokens as a DTCG document, the same
format the token export produces, or nil when the file has none. Active
themes and sets come from the file's tokens status, so they reflect
what is currently active.

The document is returned as a plain map. A bare TokensLib is not a
collection, so the response middleware would hand it to ring unencoded.

Relates to #7091

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-10-09 20:08:14 +02:00
breken
fa5e5b6176
🐛 Log errors readably in MCP execute_code console output (#11930)
* 🐛 Log errors readably in MCP execute_code console output

The console captured during execute_code rendered every object argument
with JSON.stringify. An Error has no enumerable fields, so logging a
caught error printed "{}" and the agent lost the message. A circular
object or a BigInt made JSON.stringify throw, so a console.log call
aborted the executed code with an unrelated serialization error.

Format console arguments in a small pure module: errors by their string
form (name and message), other objects as indented JSON, falling back to
the string form when JSON cannot serialize the value.

AI-assisted-by: claude-opus-5.5
Signed-off-by: breken-ai <312387581+breken-ai@users.noreply.github.com>

* ♻️ Keep console formatting within ExecuteCodeTaskConsole

---------

Signed-off-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
Co-authored-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
2026-10-09 19:39:40 +02:00
Andrey Antukh
6f33d8d9d8 Merge remote-tracking branch 'origin/staging' into develop 2026-10-09 19:16:03 +02:00
ump45nose
1125f72d81
🐛 Remove third-party Work Sans import from plugin styles (#12087)
@penpot/plugin-styles loaded Work Sans from fonts.bunny.net via a remote
@import in core/fonts.css, which every plugin built with the package
fetched at runtime. That broke air-gapped deployments, leaked user IPs
to a third party, and forced deployments to widen the CSP style-src
and font-src directives introduced in #11374.

Drop the remote import and use the system-ui stack instead of
'Work Sans' in the same package (both in core/fonts.css for code
blocks and in core/generic.css for the base typography, which relied
on the same webfont), so plugins no longer reach any external domain
and 'self' is enough for both CSP directives.

Verified by bundling the package entry point with esbuild using the
same options as tools/scripts/build-css.mjs: the produced styles.css
is balanced, contains system-ui, and contains no external import.

Closes #12023

Signed-off-by: ump45nose <1729065730@qq.com>
2026-10-09 18:38:05 +02:00
Filip Sajdak
d7ccc0f3a6
✨ Add get-file-plugin-data RPC to read file plugin data (#10171)
The Plugin API can write arbitrary key/value plugin data onto the file
root, but it could only be read back by downloading and decoding the
whole file. Add a `get-file-plugin-data` read command that returns the
file-level plugin data map, optionally narrowed to a single namespace,
so external tooling can fetch metadata such as issue or commit
references cheaply.

Read-only: reuses `bfc/get-file` and `check-read-permissions!`, and
adds no data-model or change-type changes. Object/page level reads are
intentionally left out of this first command.

Relates to #10169

Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-10-09 18:15:07 +02:00
Francis Santiago
e463e7c95f
Merge pull request #12236 from penpot/fc-subpath-documentation
📚 Document subpath configuration and migration
2026-10-09 12:47:17 +02:00
Francis Santiago
6839b16735 📚 Document subpath configuration and migration 2026-10-09 12:26:57 +02:00
Alejandro Alonso
cda19e0d51
⬆️ Use Wasm EH and SjLj for render-wasm with Rust 1.99 (#12235)
Rust 1.91 links emscripten with JS exception handling, which cannot
mix with SUPPORT_LONGJMP=wasm. Bump the devenv to Rust 1.99 (native
Wasm EH) and align EMCC_CFLAGS / SKIA_BINARIES_URL with a Skia
-ljmp-wasm prebuild so +simd128 no longer hits invoke_*V TypeErrors.
2026-10-09 11:53:28 +02:00
Miguel de Benito Delgado
5afdeffa5f
♻️ Move frame type constants to webgl.cljs (#11949)
* ♻️ Move some constants out of app.render-wasm.api

- Moved to aapp.render-wasm.wasm where they can be imported
  without pulling half the editor with them
- Renamed one for consistency

* 🔥 Delete unused function, update docstring
2026-10-09 11:22:19 +02:00
Pablo Alba
76e9c132ef
✨ Send more enriched events to nitrate log (#12210)
* ✨ Send more enriched events to nitrate log

* ✨ Fix mr
2026-10-09 10:58:05 +02:00
María Valderrama
7745c1f052
✨ Make the whole current plan card clickable (#12238) 2026-10-09 10:46:33 +02:00
Alejandro Alonso
4eec950b96 Merge remote-tracking branch 'origin/staging' into develop 2026-10-09 10:00:13 +02:00
Andrey Antukh
bcb7a8329c Merge remote-tracking branch 'origin/develop' into develop 2026-10-09 09:01:39 +02:00
Andrey Antukh
473a59b134 Merge remote-tracking branch 'origin/staging' into develop 2026-10-09 09:01:12 +02:00
María Valderrama
d0351a3ee2
✨ Add event name for create attributes (#12165)
* ✨ Add event name for create attributes

* 📎 Code review
2026-10-09 08:50:35 +02:00
Belén Albeza
2c08a065bb
🔧 Auto-detect architecture to set the right skia binaries for test scripts (#12224) 2026-10-08 16:50:00 +02:00
Marina López
28c576e1e6
🐛 Preserve admin console URL path segments (#12196) 2026-10-08 11:38:13 +02:00
Belén Albeza
cb1118f904
🎉 Preserve Penpot's internal formatting when pasting (v3) (#12137)
* 🎉 Copy v3 editor text with its styles in a payload

Copy and cut in the v3 text editor put the styled selection in a
hidden span of text/html. Add its codec and the clean-up for pasting
into another file or team.

Refs #12047

AI-assisted-by: claude-opus-5-5

* 🎉 Paste text copied in Penpot at the v3 editor caret

Text copied in Penpot keeps its styles when pasted at the caret. The
first paragraph keeps the target's attrs; later ones bring their own.
The context menu paste now shares the keyboard paste path.

Refs #12047

AI-assisted-by: claude-opus-5-5

* 🎉 Paste text copied in Penpot on the canvas with its styles

Refs #12047

AI-assisted-by: claude-opus-5-5

* 🎉 Paste copied text shapes at the v3 editor caret with styles

Copied shapes now record their team. Shapes that are not texts paste
nothing into the editor.

Refs #12047

AI-assisted-by: claude-opus-5-5

* ✨ Replace custom fonts in shapes pasted across teams

Refs #12047

AI-assisted-by: claude-opus-5-5
2026-10-07 18:20:03 +02:00
Miguel de Benito Delgado
f252439d84
♻️ Factor GL context initialization out of canvas init (#11931)
* ♻️ Factor GL context initialization out of canvas init

- fixes setting of gl-context-handle to be after _init so an exception
  does not leave invalid state behind.
- api.init-canvas-context uses a new webgl.init-context!
- this decouples config from init, so we can reuse it in benchmarks
- some helpers move to the webgl ns

AI-assisted-by: muse-spark-1.3, deepseek-flash, glm-5.3

* 🐛 Release acquired GL context when renderer init fails

- Properly release resources if init fails
- Rethrown exceptions with the original cause
- Fix vacuous check in clear-canvas

AI-assisted-by: muse-spark-1.3, deepseek-flash, glm-5.3

* ✨ Test webgl context init and release

AI-assisted-by: muse-spark, glm-5.3

* 💄 Address review comments

* 💄 Drop `!` from some function names
2026-10-07 18:19:34 +02:00
Eva Marco
dcf0b197eb
🐛 Fix inner shadow covering the shape in the SVG renderer (#12141)
* 🐛 Fix inner shadow covering the shape in the SVG renderer

The svg filter components moved to the `*` syntax (#9448), but
filter-entry* still built their props as a raw JS object with
kebab-case keys. Raw JS props skip the kebab to camel conversion, so
the components read nil for filterId and filterIn, and no filter
primitive got its result or in2. With the chain broken, an inner
shadow was blended over the empty flood instead of the shape, hiding
the fill (and a board's children) under the shadow color. Exports
using these filters broke the same way. Pass the props in camelCase.

Closes #11967

AI-assisted-by: claude-opus-5-5

* ♻️ Build filter entry props with mf/props

Use mf/props instead of a raw JS object so the props keep the
kebab-case keys used across the codebase. The macro converts them to
camelCase at compile time, so there is no runtime cost, and the
comment about the manual conversion is no longer needed.

AI-assisted-by: claude-opus-5-5
2026-10-07 15:45:10 +02:00
Alonso Torres
f62f4024be
Merge pull request #12090 from oraios/mcp-global-plugin
🎉 Add global plugins and enable MCP across workspaces
2026-10-07 15:38:17 +02:00
alonso.torres
9517f2ac94 ♻️ Type the MCP plugin with the repo plugin types
Link the MCP plugin to plugins/libs/plugin-types instead of the
published 1.5.0 package, which lacks PenpotMgmt. The repo types
declare the penpotMgmt global, so the inline import in index.d.ts
goes away and the plugin checks against the API it runs with.

Type Context.management as PenpotMgmt without on/off, so new
management methods need no extra listing.

Mark the background plugin mock in load-plugin.spec.ts as
NonNullable: createPlugin can return undefined, and the build
type-check failed on it.
2026-10-07 15:21:16 +02:00
alonso.torres
a125977fce 🐛 Keep the MCP plugin working without penpotMgmt
Penpot only exposes penpotMgmt to plugins running with global
scope. Without it, the MCP plugin failed on load when registering
its workspacechange listener.

When penpotMgmt is missing, the plugin now builds the workspace
metadata from the open file and skips the listener. execute_code
receives penpotMgmt as null and waits for layout updates as before.
The tool description and instructions tell agents that only the
open file is available when penpotMgmt is null.
2026-10-07 14:58:32 +02:00
alonso.torres
dfff188409 ✨ Ask users to allow global plugins on install
A plugin with "scope": "global" keeps running across the dashboard
and files and can find and open the user's projects and files. Any
manifest could ask for this without the user being told.

Add the allow:global permission. The plugin registry adds it to
every manifest with global scope, so the install dialog lists it,
and a plugin that becomes global in a later version shows the
permissions update dialog again. The permission is valid in the
shared schema, which the backend checks when saving, and in the
runtime manifest schema. The built-in MCP plugin declares it too.

Document the scope property and the new permission for plugin
developers.
2026-10-07 14:58:32 +02:00
alonso.torres
a381761aa4 ✨ Add project and file creation to penpotMgmt
Add penpotMgmt.createProject({name, teamId?}) and
penpotMgmt.createFile({projectId, name}). Both require content:write,
trim the name and require 1 to 250 characters, and return the same
metadata as listProjects and listFiles without changing the current
view. When the new project or file belongs to the current team, the
dashboard state is updated so it shows up right away.

listProjects, listFiles and the creation methods share the summary
builders, so fileCount is now 0 instead of missing for new projects.

Update the plugin types, the MCP API docs and the MCP instructions.
2026-10-07 14:58:32 +02:00
alonso.torres
cf1fbb4d7c 🐛 Close plugins whose load is cancelled before they run
Unloading or replacing a plugin while its code was still being
fetched could leave a late instance running, or block reopening a
plugin that closed during startup.

Each load now carries an AbortController. Unloading, a newer load
of the same plugin or logout aborts it, and the plugin manager
closes itself on abort, also while it fetches its code.
createPlugin returns nothing for a plugin closed before its
sandbox exists, and failed loads clear their pending state so the
plugin can be loaded again.
2026-10-07 14:58:32 +02:00
alonso.torres
c4d2ca9cd1 ✨ Move the dashboard MCP menu to the page headers
Show the MCP menu next to the layout toggle in the header of the
projects, files and deleted pages, instead of in the sidebar. It
uses the secondary button style there and a visible dot colour
while disconnected.

Clicking the MCP button now toggles the menu instead of only
opening it. Dashboard events report "dashboard:header" as their
origin.
2026-10-07 14:58:32 +02:00
Dominik Jain
036f59266b 🐛 Discard cancelled plugin loads after unloading
Cause:
- Disabling MCP during script loading could leave a late plugin instance
  attached to a stopped controller, blocking connection after re-enabling.

Fix:
- Track pending loads and discard instances cancelled by unload.
- Prevent duplicate global loads and preserve newer replacement instances.
- Clear pending state after failures so a subsequent load can retry.

Validation:
- Pass 84 runtime tests, the runtime build, lint and formatting checks.
- Reproduce the race with a delayed script in the browser, then verify
  re-enabling restores the session ID, themed indicator and MCP execution.

AI-assisted-by: gpt-6
Signed-off-by: Dominik Jain <dominik.jain@oraios-ai.de>
2026-10-07 14:58:32 +02:00
Dominik Jain
2bb6cd557e ✨ Add project and file discovery to penpotMgmt
Discovery:
- Add listProjects and project-scoped listFiles with content:read checks.
- Return metadata from existing RPCs in dashboard and workspace contexts.
- Reject invalid IDs and preserve backend access errors.

Documentation:
- Keep usage in API type comments and include regenerated MCP API docs.
- Remove README implementation details and consolidate the changelog.
- Add usage information to the MCP server's initial instructions

Validation:
- Pass 8 frontend tests, 81 runtime tests and the runtime build.
- Verify both APIs through live MCP calls in dashboard and workspace.
- Pass type, lint, formatting and diff checks.

AI-assisted-by: gpt-6
Signed-off-by: Dominik Jain <dominik.jain@oraios-ai.de>
2026-10-07 14:58:32 +02:00
Dominik Jain
54cce66c13 ✨ Share the MCP connection menu with the dashboard
Connection status:
- Replace the dashboard's action-only link with an explicit connected
  label and indicator, using the theme's existing accent colour.
- Reflect the actual connection state independently of connection intent.

Shared controls:
- Extract the toolbar's MCP menu and styles into a shared component used
  by both the dashboard and workspace.
- Provide connection status, session ID, copy, connect and disconnect
  actions in both locations, preserving hover and keyboard behaviour.
- Keep dashboard and toolbar event origins distinct and update the
  dashboard connection instructions.

Validation:
- Confirm dashboard disconnect/reconnect, menu opening, keyboard
  navigation and Escape dismissal in the running app.
- Pass live compilation, targeted ClojureScript and SCSS lint,
  formatting checks and git diff checks.

AI-assisted-by: gpt-6
Signed-off-by: Dominik Jain <dominik.jain@oraios-ai.de>
2026-10-07 14:58:32 +02:00
Dominik Jain
be17fabc6a 🎉 Introduce global plugins and move MCP to application scope
Global plugin lifecycle:
- Add optional manifest scope "global" for plugins that remain active
  across dashboard and workspace navigation within one browser tab.
  Existing plugins retain workspace scope by default.
- Preserve global sandboxes when a workspace ends or another plugin
  loads, including when that other plugin fails to initialize.
- Avoid duplicate instances of an already-running global plugin.
- Close global plugins on explicit closure or logout, with idempotent
  cleanup of listeners, timers and UI resources. Global scope does not
  imply automatic startup or persistence across a browser reload.

Plugin API and navigation:
- Keep a permanently live penpot facade whose getters follow the current
  workspace, while user, theme and UI facilities remain available on
  the dashboard.
- Expose penpotMgmt to global plugins with workspace metadata,
  none/loading/ready status and workspacechange subscriptions.
- Add awaitable openFile(fileId, {teamId}) navigation in the current tab,
  defaulting to the current team and resolving only once the requested
  file and its active page are ready.
- Reject invalid identifiers, failed or superseded navigation and waits
  exceeding 30 seconds; release navigation watchers on completion.
- Guard workspace-bound operations while no workspace is ready and
  return null from root, currentFile and currentPage in that state.
- Require content:read for the initial management API. Callers must
  reacquire file, page and shape references after changing files.

MCP lifecycle and connection controls:
- Make MCP a global plugin and initialize its integrated controller per
  authenticated application session instead of per workspace.
- Load the enabled integration with a valid token on the dashboard as
  well as in a file, retaining its existing hidden-iframe implementation.
- Keep enabling and connecting separate: an enabled plugin starts idle,
  and each tab connects explicitly through dashboard or workspace
  controls.
- Preserve the iframe, WebSocket, heartbeat and reconnect machinery
  during navigation; disconnect on request and close on disable/logout.

MCP sessions and execution:
- Derive the short MCP session ID solely from currentUser.sessionId,
  excluding the file ID. Retain the SHA-256-based, 50-bit Base32 encoding
  to produce a stable, copyable 10-character identifier.
- Allow sessions without an open file and update workspace metadata over
  the existing connection without replacing its routing identity.
- Refresh context after the connection handshake so initialization does
  not leave stale file metadata.
- Expose penpotMgmt to execute_code, preserve execution storage across
  navigation and skip layout waits when no workspace is ready.
- Update API types, tool guidance, documentation and the changelog.

Validation:
- Cover authenticated startup, workspace survival, logout cleanup,
  readiness, navigation failure, duplicate loads and session routing.
- Verify that short IDs remain stable across files and reconnects and
  that disconnect cancels pending asynchronous ID initialization.
- Pass 16 focused frontend tests, 73 runtime tests, 30 MCP plugin tests,
  builds, TypeScript checks and frontend/runtime lint.
- Pass all 73 MCP server tests with --test-force-exit; the normal runner
  still stays alive after assertions finish.
- Verify a live dashboard-to-file-to-dashboard round trip without a
  document or iframe reload, preserving facade identity and storage.

AI-assisted-by: gpt-6
2026-10-07 14:58:32 +02:00
Alejandro Alonso
31000c3055
⚡ Enable rustc wasm SIMD for render-wasm builds (#12163)
Set RUSTFLAGS target-feature=+simd128 on the emscripten
target so our Rust code can autovec; Skia SIMD remains
via the -simd prebuild and EMCC -msimd128.
2026-10-07 14:52:46 +02:00
David Barragán Merino
6473dc85ab 👷 Notify new devenv images in penpotdev
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-07 14:11:21 +02:00
Andrey Antukh
c4c1c5a955 🐛 Fix media-processor upload-storage tests
Update media-processor dependencies (multer 2.4, vitest 5
and others) and fix mockFile to emit Buffer chunks so
multer memory storage Buffer.concat works. String chunks
caused TypeError and 5s timeouts in two tests.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-07 11:52:46 +00:00
David Barragán Merino
27ccb32ea1 👷 Fix the devenv image build workflow
AI-assisted-by: Claude
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-07 13:17:13 +02:00
Marina López
d0cf399807
🐛 Show trial status on subscription page (#12158) 2026-10-07 13:14:58 +02:00
Marina López
65b248eec3
🐛 Preserve team ID in Nitrate membership URL (#12157) 2026-10-07 13:08:02 +02:00
David Barragán Merino
edd500302c ⬆️ Update ImageMagick to 7.1.2-32
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-07 13:02:27 +02:00
Belén Albeza
1d7c775f72
🐛 Fix not setting mimetype in image fills (#12136) 2026-10-07 12:26:45 +02:00
Alejandro Alonso
cc5f2f57ad
⬆️ Upgrade skia-safe to 0.153.3 with wasm SIMD (#12148)
Point SKIA_BINARIES_URL at the 0.153.3 simd prebuild and adapt
fonts, path stroke expansion, and gradients to the new API.
2026-10-07 12:08:33 +02:00
Andrey Antukh
9abcfebf60
✨ Run binfile import and export as durable jobs (#12056)
* ✨ Run binfile import and export as durable jobs

Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.

Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.

How:
- The substrate gains the pieces a user-facing job needs: optional
  job-def metadata (family, resource-role), an expiry on submit, the
  owner in the runner context, and job resources in their own bucket,
  written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
  business params, the registry checks the family, the job-def decodes
  and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
  vocabulary with self-contained counters, throttled heartbeats, and an
  interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
  a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
  the package descriptor; the import re-checks the edition permission,
  runs the core in a transaction so a cancellation rolls it back, and
  releases the consumed package and its temporary copy.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Handle externally-cancelled binfile jobs end to end

The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.

- The export emits `{:cancelled true}` per file and the modal paints a
  neutral cancelled row instead of success; a message without an
  artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
  through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
  they replace.

Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Pre-translate the cancelled import message

The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Move cancelled strings under generic labels keys

The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.

AI-assisted-by: muse-spark-1.3-contributor

* ✨ Add minor improvements

* ♻️ Rename export job command to create-export-binfile-job

Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.

submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.

The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.

AI-assisted-by: deepseek-v4.1-flash

* ✨ Polish the import and export status rows

The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.

In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Show the cancelled export and fix the import error

The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).

The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.

The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Reflect the real job params in the creation commands

The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.

Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.

A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Stop the test system from running job runners

The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.

That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.

Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.

AI-assisted-by: deepseek-v4.1-flash

* ✨ Upload the package one chunk at a time and as a percentage

The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.

The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.

AI-assisted-by: deepseek-v4.1-flash

* 🌐 Complete the Spanish catalogue

es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).

sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Move the job status keys into the jobs namespace

The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.

Rename them so the whole job vocabulary is one namespace:

  labels.queued           -> jobs.queued
  labels.export-cancelled -> jobs.export-cancelled
  labels.import-cancelled -> jobs.import-cancelled

No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.

AI-assisted-by: deepseek-v4.1-flash

* 🔥 Remove the translation keys nothing references

Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:

  labels.uploading-file    a duplicate of jobs.progress.stage.upload
                           since the upload stage says "Uploading file…"
  labels.downloading-file  dead since the durable jobs replaced the old
                           import and export progress rows
  inspect.attributes.stroke.alignment.center
  inspect.attributes.stroke.alignment.inner
  inspect.attributes.stroke.alignment.outer
  workspace.toolbar.mcp-connect-here

The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.

AI-assisted-by: deepseek-v4.1-flash
2026-10-07 11:52:48 +02:00
Alejandro Alonso
accea1abde
⚡ Enable wasm SIMD for Skia render-wasm builds (#12145)
Link with -msimd128 and point SKIA_BINARIES_URL at the pdf+simd prebuilt archive.
2026-10-07 11:30:40 +02:00
Danny Shirely
051f01d559
🐛 Keep per-side stroke widths out of inspect geometry (#12055)
Remove per-side stroke widths from both Inspect geometry panels so
Stroke is the only section that lists them. Keep dimensions, position,
corner radii, and transforms in Size and position.

Add rendered-component regressions for boards, rectangles, multiple
selection, uniform strokes, and CSS declarations.

Closes #12026

AI-assisted-by: GPT-6

Signed-off-by: Danial Shirali <danialshirali16@gmail.com>
Signed-off-by: Eva Marco <eva.marco@kaleidos.net>
Co-authored-by: Eva Marco <eva.marco@kaleidos.net>
2026-10-07 11:18:46 +02:00
Marina López
a9bf53adbf
Revert "🐛 Preserve path segments when joining backend URIs (#12152)" (#12156)
This reverts commit 2a6e92a7d9a5520365276f7b9a6ae59d478159ee.
2026-10-07 11:10:02 +02:00
Marina López
2a6e92a7d9
🐛 Preserve path segments when joining backend URIs (#12152) 2026-10-07 10:45:17 +02:00
Andrey Antukh
a5df69f60f Merge remote-tracking branch 'origin/staging' into develop 2026-10-07 10:25:01 +02:00
Andrey Antukh
fc8a126048 Merge remote-tracking branch 'origin/staging' into develop 2026-10-06 18:59:56 +02:00
David Barragán Merino
1afa8b1afa 🔧 Skip triage for Renovate's dashboard and GitHub Actions PRs
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 18:09:57 +02:00
David Barragán Merino
8204e97a98 🔧 Manage GitHub Actions with Renovate
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 18:09:57 +02:00
David Barragán Merino
b6a3ab67ef 🔥 Remove Dependabot configuration
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 18:09:57 +02:00
Belén Albeza
7a37cefe4c
🐛 Pass team-id in binfile SVG import tests (#12139)
AI-assisted-by: claude-opus-5-5
2026-10-06 17:33:43 +02:00
David Barragán Merino
472973533e 🐳 Verify the Node.js tarball checksum in the devenv
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 17:14:55 +02:00
David Barragán Merino
f38cb4dc07 🔥 Remove unused package-lock.json from plugins-runtime
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 16:30:30 +02:00
Andrey Antukh
746184caf9 Merge remote-tracking branch 'origin/staging' into develop 2026-10-06 15:58:01 +02:00
penpot-renovate[bot]
a61a4fceb1 ⬆️ Update dependency pnpm to v12.8.1
- pnpm 12.8.1 (docker/images/Dockerfile.media-processor)
- pnpm 12.8.1 (docker/images/Dockerfile.mcp)
- pnpm 12.8.1 (docker/images/Dockerfile.exporter)
- pnpm 12.8.1 (render-wasm/package.json)
- pnpm 12.8.1 (plugins/package.json)
- pnpm 12.8.1 (plugins/libs/plugins-styles/package.json)
- pnpm 12.8.1 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.8.1 (plugins/libs/plugin-types/package.json)
- pnpm 12.8.1 (plugins/apps/table-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.8.1 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/icons-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/example-styles/package.json)
- pnpm 12.8.1 (plugins/apps/e2e/package.json)
- pnpm 12.8.1 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.8.1 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.8.1 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.8.1 (package.json)
- pnpm 12.8.1 (media-processor/package.json)
- pnpm 12.8.1 (mcp/packages/server/package.json)
- pnpm 12.8.1 (mcp/packages/plugin/package.json)
- pnpm 12.8.1 (mcp/packages/common/package.json)
- pnpm 12.8.1 (mcp/package.json)
- pnpm 12.8.1 (library/package.json)
- pnpm 12.8.1 (frontend/text-editor/package.json)
- pnpm 12.8.1 (frontend/packages/ui/package.json)
- pnpm 12.8.1 (frontend/packages/tokenscript/package.json)
- pnpm 12.8.1 (frontend/packages/mousetrap/package.json)
- pnpm 12.8.1 (frontend/packages/draft-js/package.json)
- pnpm 12.8.1 (frontend/package.json)
- pnpm 12.8.1 (exporter/package.json)
- pnpm 12.8.1 (docs/package.json)
- pnpm 12.8.1 (common/package.json)
- pnpm 12.8.1 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-06 15:12:15 +02:00
penpot-renovate[bot]
e17c3c0c8f ⬆️ Pin DHI base image digests
- dhi.io/debian-base trixie-debian13-dev@61dc022 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13-dev@61dc022 (docker/images/Dockerfile.backend)
- dhi.io/debian-base trixie-debian13@20079b5 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@61dc022 (docker/imagemagick/Dockerfile)
- dhi.io/debian-base trixie-debian13-dev@61dc022 (docker/devenv/Dockerfile)
- dhi.io/nginx 1.31.6-alpine3.24@9465275 (docker/images/Dockerfile.storybook)
- dhi.io/nginx 1.31.6-debian13-dev@32e4340 (docker/images/Dockerfile.frontend)
- dhi.io/node 24.21.0-debian13-dev@2e51e83 (docker/images/Dockerfile.media-processor)
- dhi.io/node 24.21.0-debian13@f3575cf (docker/images/Dockerfile.mcp)
- dhi.io/node 24.21.0-debian13-dev@2e51e83 (docker/images/Dockerfile.mcp)
- dhi.io/node 24.21.0-debian13-dev@2e51e83 (docker/images/Dockerfile.exporter)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-10-06 13:58:17 +02:00
David Barragán Merino
bcd4631082 🔧 Add Renovate configuration
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 13:19:13 +02:00
David Barragán Merino
d47753dbb3 🔧 Build Docker images on Renovate branches
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-06 13:19:13 +02:00
Belén Albeza
8d8dff8de4
🐛 Fix dropping caret styles when pasting html text (#12108) 2026-10-06 10:29:53 +02:00
Andrey Antukh
d3ca9a0b83
✨ Add unified jobs substrate replacing legacy one-shot task system (#11542)
* 🎉 Move backend jobs to the unified job table substrate

Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.

New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.

New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.

New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.

All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.

Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.

AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5

* ✨ Address review comments on jobs substrate

Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.

The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.

AI-assisted-by: muse-spark-1.3-contributor
2026-10-06 07:33:12 +02:00
David Barragán Merino
0eb1e8ba95 🐛 Request pull-requests permission in auto-label token
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-05 14:32:04 +02:00
David Barragán Merino
b2ea46a71a 🔧 Use client-id instead of app-id in auto-label workflow
actions/create-github-app-token v3 deprecates the app-id input in
favour of client-id. Switch to the new TRIAGE_APP_CLIENT_ID secret.

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-10-05 13:11:08 +02:00
dependabot[bot]
66c5e6fdfd ⬆️ Bump the github-actions group across 1 directory with 8 updates
Bumps the github-actions group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [actions/create-github-app-token](https://github.com/actions/create-github-app-token) | `1.12.0` | `3.2.0` |
| [actions/github-script](https://github.com/actions/github-script) | `7.1.0` | `9.0.0` |
| [actions/checkout](https://github.com/actions/checkout) | `6.1.0` | `7.0.1` |
| [pnpm/setup](https://github.com/pnpm/setup) | `2.0.2` | `3.0.0` |
| [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) | `4.3.1` | `6.3.0` |
| [actions/cache/restore](https://github.com/actions/cache) | `5.1.0` | `6.1.0` |
| [actions/cache/save](https://github.com/actions/cache) | `5.1.0` | `6.1.0` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.1` |



Updates `actions/create-github-app-token` from 1.12.0 to 3.2.0
- [Release notes](https://github.com/actions/create-github-app-token/releases)
- [Changelog](https://github.com/actions/create-github-app-token/blob/main/CHANGELOG.md)
- [Commits](d72941d797...bcd2ba4921)

Updates `actions/github-script` from 7.1.0 to 9.0.0
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](f28e40c7f3...3a2844b7e9)

Updates `actions/checkout` from 6.1.0 to 7.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](d23441a48e...3d3c42e5aa)

Updates `pnpm/setup` from 2.0.2 to 3.0.0
- [Release notes](https://github.com/pnpm/setup/releases)
- [Commits](84cb39b217...fbda4c85fc)

Updates `aws-actions/configure-aws-credentials` from 4.3.1 to 6.3.0
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases)
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md)
- [Commits](7474bc4690...e1253824e5)

Updates `actions/cache/restore` from 5.1.0 to 6.1.0
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](caa2961268...55cc834586)

Updates `actions/cache/save` from 5.1.0 to 6.1.0
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](caa2961268...55cc834586)

Updates `actions/download-artifact` from 7.0.0 to 8.0.1
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](37930b1c2a...3e5f45b2cf)

---
updated-dependencies:
- dependency-name: actions/create-github-app-token
  dependency-version: 3.2.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/github-script
  dependency-version: 9.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/checkout
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: pnpm/setup
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: aws-actions/configure-aws-credentials
  dependency-version: 6.3.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/cache/restore
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/cache/save
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
- dependency-name: actions/download-artifact
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-05 12:35:56 +02:00
Andrey Antukh
341e5d1027 Merge remote-tracking branch 'origin/staging' into develop 2026-10-05 11:59:05 +02:00
Elena Torró
aed4e12c63
⚡ Avoid decoding all pages when building file thumbnail data (#12054) 2026-10-05 11:34:10 +02:00
Eva Marco
b06a43404b
🎉 Add detach to typography and shadow tokens (#11969)
* 🎉 Add detach to typography tokens

* 🎉 Add detach to shadow tokens

* 🎉 Add tests
2026-10-05 11:32:29 +02:00
Belén Albeza
7c039231f7
🎉 Implement paste of basic HTML formatting (v3) (#12050)
* ✨ Add rich HTML paste to the v3 text editor

Pasting HTML into the v3 text editor keeps bold, italic, decoration and
text-transform, and drops fonts, colors and links. Gated behind
text-editor-wasm/v1-html-paste.

Refs #10465

AI-assisted-by: claude-opus-5-5

* ✨ Paste HTML on the canvas as a styled text shape

With v3 HTML paste on, pasting HTML on the canvas creates a text shape
with the same emphasis. Clipboard options are now read at paste time.

Refs #10465

AI-assisted-by: claude-opus-5-5
2026-10-02 15:07:12 +02:00
Dr. Dominik Jain
8efbd9b5d3
🎉 Allow MCP to work with multiple files in parallel (#11927)
* 🎉 Add multi-file session routing to the MCP server

Session routing:
- Track each user's Penpot connections by session ID.
- Dispatch explicit sessions directly; otherwise discover the sole session.
- Return a retryable error when discovery fails or is incomplete.

Plugin and tools:
- Initialize connections with a fresh session ID and file metadata.
- Share the optional sessionId schema and place it last in tool inputs.

Validation:
- 84 tests, server type checking, and MCP formatting checks pass.
- Disable test logging to avoid the logging worker shutdown hang.

AI-assisted-by: gpt-6

* 🎉 Support independent MCP connections

- Require explicit connection intent for each integrated workspace.
- Derive short session IDs from the Penpot app instance and file.
- Show and copy session IDs in the MCP menu and standalone plugin UI.
- Stop stale callbacks and reconnect attempts after disconnect.

Validate with frontend and plugin tests, type checking, live multi-tab
checks, and standalone UI checks with a simulated connection. Server
tests pass with --test-force-exit; the normal runner can hang on shutdown.

AI-assisted-by: gpt-6

* 📚 Document multi-file MCP sessions

- Explain independent connections, session selection, and copying IDs.
- Describe session ID lifetimes for integrated and standalone plugins.
- Correct browser focus, connection, and recovery guidance.

Validate with the documentation site build and diff checks.

AI-assisted-by: gpt-6

* ✨ Change behavior on session ID duplication

* ✨ Open MCP toolbar menu on hover and match menu style

---------

Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Co-authored-by: elhombretecla <delacruzgarciajuan@gmail.com>
2026-10-02 14:22:52 +02:00
Elena Torró
d531c0050b
⚡ Skip builder local apply on plain translations (#12036)
Translations that touch no group-like shape or ancestor do not need
the builder objects updated, since only parent resizing reads them.

AI-assisted-by: claude-opus-5-5
2026-10-02 12:39:18 +02:00
Elena Torró
da55d6723c
⚡ Reduce per-frame CLJS work while dragging (#12027)
Memoize snap points and frame titles on the base objects and compute
the drag target frames once per gesture.
2026-10-02 12:24:04 +02:00
Elena Torró
822dbd6517
⚡ Speed up sidebar menus for large selections (#12028)
Reuse extracted colors for unchanged shapes and cache the text attrs
read from each text content.
2026-10-02 12:05:25 +02:00
Elena Torró
c148ba5267
⚡ Rebuild drag tiles per moved subtree (#12030)
WASM receives only the transforms that differ from the parent (bool
operands and masks are kept), rebuilds tiles once per moved subtree
and invalidates only the previous frame coverage.
2026-10-02 11:39:24 +02:00
Elena Torró
13a9da3bc3
⚡ Reduce commit cost when dropping large selections (#12032)
Collect children ids into one transient vector and track WASM shape
changes in a JS Map instead of a volatile persistent map.

AI-assisted-by: claude-opus-5-5
2026-10-02 11:22:19 +02:00
Dr. Dominik Jain
3a0ed93360
📚 Improve MCP README (#12022)
* Make the scope of the usage instructions clear (local usage)
* Provide overview of the server's different modes of operation
* Improve npm release instructions
* Remove deprecated multi-user mode documentation
2026-10-02 11:08:53 +02:00
Elena Torró
1554847d40
⚡ Compute the drop ignore tree in a single pass (#12033)
Walk only from the top-most transformed shapes, carry the component
root down the walk and memoize each shape transform so the update
step reuses it.

AI-assisted-by: claude-opus-5-5
2026-10-02 10:51:21 +02:00
Elena Torró
09ffcb1fbc
⚡ Reduce render-wasm GPU memory and move cost on large pages (#11980) 2026-10-02 10:24:05 +02:00
Stephan Paternotte
a3ab8bf6fd
🌐 Add translations for: Dutch
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/nl/
2026-10-02 07:51:22 +02:00
Nicola Bortoletto
4a5da94db2
🌐 Add translations for: Italian
Currently translated at 82.9% (1888 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/it/
2026-10-02 07:51:20 +02:00
Andrey Antukh
84c794c5b8 Merge remote-tracking branch 'origin/staging' into develop 2026-10-01 19:25:59 +02:00
Pablo Alba
03e24d18ce
✨ Dual-send audit log archive to Nitrate and Nexus (#12037)
Run the audit archive cron when :nexus or :admin-console is on. Ship
allowlisted events to Admin Console first (with row ids for
idempotency), then the full chunk to Nexus when :nexus is set. Mark
archived_at for the whole chunk on success, including nitrate-only
mode. Rename the gate flag from :audit-log-archive to :nexus.

AI-assisted-by: Composer

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-01 17:17:11 +02:00
JiMyung Lee
cc2aff6ce8
🐛 Fix typography sample overflow in non-Latin locales (#11487)
* 🐛 Fix typography sample overflow in non-Latin locales

The typography sample glyph sits in a fixed 1.5rem grid column, sized
for the English sample "Ag". Locales that translate it to something
wider wrap it onto several lines and overflow the row: ko translates
the sample as "가나다" and renders it as three stacked characters, ar
as "أسلوب خط النص" and renders it as three stacked words. This affects
the workspace assets panel (list row, detail panel and rename row) and
the dashboard library card.

Let the sample column size to its content and keep the sample on a
single line. The 1.5rem minimum is preserved, so locales whose sample
already fits — English and every locale that keeps "Ag" — render
exactly as before.

Signed-off-by: JiMyung Lee <lee.ji.myung@gmail.com>

* 🐛 Keep spacing between typography sample and name

Follow-up to review on the typography sample overflow fix. Letting
the sample grow removed the fixed-width slack that used to separate it
from the name text, so:

- Dashboard library card: `.library-name-block` no longer assumes a
  24px sample via a hard-coded calc; it flexes to the remaining space
  and the sample does not shrink.
- Typography detail panel, list row and rename/advanced-edit row: add
  an explicit `var(--sp-xs)` gap between the sample and the name, and
  let the name input shrink instead of pushing the action buttons.
- Libraries "Updates" tab: the sample div had no class and still
  wrapped per character; give it a `nowrap` class.

Signed-off-by: JiMyung Lee <lee.ji.myung@gmail.com>

---------

Signed-off-by: JiMyung Lee <lee.ji.myung@gmail.com>
Signed-off-by: Eva Marco <eva.marco@kaleidos.net>
Co-authored-by: Eva Marco <eva.marco@kaleidos.net>
2026-10-01 16:02:21 +02:00
María Valderrama
9111ebb3bf
✨ Add same-subscription option to move-teams permission (#11995) 2026-10-01 13:03:55 +02:00
Andrey Antukh
d82f2e1005 Merge remote-tracking branch 'origin/staging' into develop 2026-10-01 12:10:40 +02:00
Andrey Antukh
c561a0baff Merge remote-tracking branch 'origin/staging' into develop 2026-10-01 10:59:50 +02:00
Shreyash Agare
970948159e
🐛 Fix token display for multi-selected text layers (#11944)
* 🐛 Fix token display for multi-selected text layers

When multiple text layers share the same fill token,
the design panel showed the hex value instead of the
token name. type->token-attrs derived token keys from
type->editable-attrs, which returns empty for text
shapes in the fill group. Fall back to the group's
own attrs when editable-attrs is empty.

Closes #11924

AI-assisted-by: claude-opus-4-6

* 🐛 Take text token attrs from the group attrs

The :text read mode reads values from the group attrs, so its
token attrs now come from those attrs too, instead of falling
back when the editable attrs are empty. type->token-attrs is
restored to its original form.

Add regression tests for fill tokens on multiple selections of
text shapes, and of a rect mixed with a text.

AI-assisted-by: claude-opus-5-5

---------

Co-authored-by: Shreyash Agare <264953665+ShreyashAgare26@users.noreply.github.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-10-01 10:13:42 +02:00
Stephan Paternotte
b31bcd95f4
🌐 Add translations for: Dutch
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/nl/
2026-09-30 16:51:25 +00:00
imagoiq
193ca4b0d2
🌐 Add translations for: French
Currently translated at 98.1% (2234 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/fr/
2026-09-30 16:51:22 +00:00
Andrey Antukh
8b2ec216e4 Merge remote-tracking branch 'origin/staging' into develop 2026-09-30 14:53:40 +02:00
Andrey Antukh
34f4c8ee28 Merge remote-tracking branch 'origin/staging' into develop 2026-09-30 08:42:19 +02:00
Andrey Antukh
49f1936bfc Merge remote-tracking branch 'origin/staging' into develop 2026-09-29 23:07:47 +02:00
Marina López
90ab142f00
🐛 Add nitrate management methods (#11962) 2026-09-29 16:05:27 +02:00
Oğuz Ersen
f2c472762d
🌐 Add translations for: Turkish
Currently translated at 100.0% (2275 of 2275 strings)

Translation: Penpot/frontend
Translate-URL: https://hosted.weblate.org/projects/penpot/frontend/tr/
2026-09-29 08:51:23 +00:00
David Barragán Merino
eb2576b8c0 👷 Run plugins package deploy inside devenv container
Parallel deploy jobs on the shared self-hosted host raced on
/home/runner/setup-pnpm (ENOTEMPTY / missing tarball errors).
Use penpotapp/devenv, which already ships the Node and pnpm versions
pinned in develop, and mount a persistent pnpm store from the host
instead of pnpm/setup + actions/cache.

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-09-28 22:01:22 +02:00
Miguel de Benito Delgado
efbb554af1
♻️ Move use-shape into app.common.render_wasm (#11917)
- Introduces a new ns since there wasn't a suitable one
- Simplifies serialize-shapes-batch! and allows usage outside the frontend
2026-09-28 18:19:55 +02:00
David Barragán Merino
494d60a671 🐛 Fix script injection in GitHub Actions workflows
Values coming from `${{ }}` expressions were interpolated directly into
`run:` scripts, so GitHub substituted them into the shell source before
bash parsed it. A commit title containing a double quote broke the
"Write step summary" step of the bundle build with a syntax error, and
the same pattern allowed arbitrary command execution on the
self-hosted runners.

Pass every expression used inside `run:` through step/job `env:` and
reference it as a quoted shell variable instead. Use the runner's
default variables (GITHUB_RUN_ID, GITHUB_REPOSITORY, ...) where the
value comes from the `github` context.

Also validate `plugin_name` in plugins-deploy-package.yml against
`^[a-z0-9][a-z0-9-]*$`, since it is free-form and reaches paths,
worker names, GITHUB_ENV and action inputs.

Affected workflows: build-bundle, build-docker,
build-docker-admin-console, plugins-deploy-package,
plugins-deploy-api-doc, plugins-deploy-styles-doc, release, tests-e2e.

Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-09-28 16:59:03 +02:00
1467 changed files with 180603 additions and 63727 deletions

View File

@ -13,19 +13,17 @@ Multi-dimensional code review with quality gates. Every change gets reviewed bef
## When to Use ## When to Use
- The reviewer subagent of the `review-code` flow loads this skill to perform - The reviewer subagent of the `review-code` flow loads this skill to perform the review of a code change.
the review of a code change. - To review code, always go through the `review-code` flow — never load this skill directly for that. This is the criteria reference, not the flow.
- To review code, always go through the `review-code` flow — never load this
skill directly for that. This is the criteria reference, not the flow.
## Core Principles ## Core Principles
These principles underpin every axis. When in doubt, default to them. These principles underpin every axis. When in doubt, default to them.
- **DRY (Don't Repeat Yourself):** Every piece of knowledge has one authoritative representation. If the same logic appears in two places, extract it into a shared helper, model, or type. Reviewers: flag duplicated logic as a required change — it's not "just similar," it's drift that will diverge. - **DRY (Don't Repeat Yourself):** Every piece of knowledge has one authoritative representation. Flag duplicated logic as a required change — it is not "just similar", it is drift that will diverge.
- **KISS (Keep It Simple, Stupid):** The simplest solution that works is the best solution. Complexity must earn its place. Reviewers: if you need more than one sentence to explain what a piece of code does, it's too complex — push for simplification before merge. - **KISS (Keep It Simple, Stupid):** The simplest solution that works is the best solution. Complexity must earn its place: if it needs more than a sentence to explain, push for simplification before merge.
- **YAGNI (You Aren't Gonna Need It):** Don't add abstractions, hooks, or generalizations for hypothetical future use cases. Generalize on the third occurrence, not the first. Reviewers: delete speculative generality. - **YAGNI (You Aren't Gonna Need It):** No abstractions, hooks or generalizations for hypothetical future use cases. Generalize on the third occurrence, not the first.
- **Don't invent problems:** Do not manufacture issues to produce more feedback. Every finding must be a real risk, a real readability barrier, or a real architectural concern — not a hypothetical or a stylistic preference disguised as a problem. - **Don't invent problems:** Every finding must be a real risk, a real readability barrier, or a real architectural concern — not a hypothetical or a stylistic preference disguised as a problem.
## The Five-Axis Review ## The Five-Axis Review
@ -38,7 +36,7 @@ Does the code do what it claims to do?
- Does it match the spec or task requirements? - Does it match the spec or task requirements?
- Are edge cases handled (null, empty, boundary values)? - Are edge cases handled (null, empty, boundary values)?
- Are error paths handled (not just the happy path)? - Are error paths handled (not just the happy path)?
- Does it pass all tests? Are the tests actually testing the right things? - Do the tests cover the changed behavior, including its edge cases?
- Are there off-by-one errors, race conditions, or state inconsistencies? - Are there off-by-one errors, race conditions, or state inconsistencies?
### 2. Readability & Simplicity ### 2. Readability & Simplicity
@ -47,10 +45,7 @@ Can another engineer (or agent) understand this code without the author explaini
- Are names descriptive and consistent with project conventions? (No `temp`, `data`, `result` without context) - Are names descriptive and consistent with project conventions? (No `temp`, `data`, `result` without context)
- Is the control flow straightforward (avoid nested ternaries, deep callbacks)? - Is the control flow straightforward (avoid nested ternaries, deep callbacks)?
- Are there any "clever" tricks that should be simplified? - Are there "clever" tricks that should be simplified? Could this be done in fewer lines? (1000 lines where 100 suffice is a failure)
- **KISS check:** Is this the simplest approach that solves the problem? A 20-line straightforward function beats a 5-line clever one that requires a comment to explain.
- Could this be done in fewer lines? (1000 lines where 100 suffice is a failure)
- Are abstractions earning their complexity? (Don't generalize until the third use case)
- Is a new conditional bolted onto an unrelated flow? Push the logic into its own helper, state, or policy. - Is a new conditional bolted onto an unrelated flow? Push the logic into its own helper, state, or policy.
- Do repeated conditionals on the same shape appear? They signal a missing model or dispatcher. - Do repeated conditionals on the same shape appear? They signal a missing model or dispatcher.
- Are there dead code artifacts: no-op variables, backwards-compat shims, or `// removed` comments? - Are there dead code artifacts: no-op variables, backwards-compat shims, or `// removed` comments?
@ -60,14 +55,13 @@ Can another engineer (or agent) understand this code without the author explaini
Does the change fit the system's design? Does the change fit the system's design?
- Does it follow existing patterns or introduce a new one? If new, is it justified? - Does it follow existing patterns or introduce a new one? If new, is it justified?
- Does it maintain clean module boundaries? - Does it keep clean module boundaries, with dependencies flowing the right way (no cycles)?
- **DRY check:** Is there existing code that does the same thing? Reuse the canonical helper instead of writing a near-duplicate. If two branches do nearly the same thing, collapse them.
- Are dependencies flowing in the right direction (no circular dependencies)?
- Is the abstraction level appropriate (not over-engineered, not too coupled)? - Is the abstraction level appropriate (not over-engineered, not too coupled)?
- Does this refactor reduce complexity or just relocate it? Count the concepts a reader must hold. Prefer the restructuring that makes whole branches disappear over one that re-centralizes the same logic. Prefer deleting an abstraction to polishing it. - Is there existing code that already does this? Reuse the canonical helper instead of a near-duplicate, and collapse branches that do nearly the same thing.
- Is feature-specific logic leaking into a shared or general-purpose module? - Is feature-specific logic leaking into a shared or general-purpose module?
- Are type boundaries explicit? Question gratuitous `any`/`unknown`/optional/casts and silent fallbacks. - Are type boundaries explicit? Question gratuitous `any`/`unknown`/optional/casts and silent fallbacks.
- **Structural remedies:** When you flag a problem, propose the move — not just the problem. Replace conditionals with dispatchers, collapse duplicate branches, separate orchestration from business logic, extract helpers, split large files. Prefer the remedy that removes moving pieces over one that spreads the same complexity around. - Does the change reduce complexity or relocate it? Count the concepts a reader must hold: prefer the restructuring that makes whole branches disappear, and deleting an abstraction over polishing it.
- **Give the remedy, not only the problem:** replace conditionals with dispatchers, separate orchestration from business logic, extract helpers, split large files. Prefer the remedy that removes moving pieces over one that spreads the same complexity around.
### 4. Security ### 4. Security
@ -111,7 +105,17 @@ For each finding, describe the circumstances under which it could fail: specific
Lead with what matters: correctness and security first, then structural issues, then everything else. A few high-conviction comments beat a long list. Lead with what matters: correctness and security first, then structural issues, then everything else. A few high-conviction comments beat a long list.
5. **Verify the verification** — What tests were run? Did the build pass? Was the change tested manually? Screenshots for UI changes? 5. **Check the evidence, do not re-run it** — Which tests cover the change? What did the author run, and what did CI report? Never re-run the full suite or a build yourself: the author's run and CI already cover that, and repeating it only lengthens the review. Run at most the single test file that settles a specific doubt. When no checks have been run at all (a change you did not see tested), run the module's checks before the verdict.
## Change Sizing
Small, focused changes are easier to review, faster to merge, and safer to deploy. Flag the size when a diff exceeds ~300 lines, or when it grows a file past ~1000 total lines, and name the remedy: split it, and separate refactoring from feature work. This is a finding about the change, not work to do inside the review.
## Dependencies
Before accepting a new dependency: does the existing stack already solve it, how large is the bundle impact, is it maintained, does `npm audit` report anything, is the license compatible. Prefer the standard library and existing utilities — every dependency is a liability.
For upgrades: read the changelog, not just the version number; one dependency per change; a green suite before and after; review the lockfile diff and never hand-edit it. Supply-chain triage follows `security-and-hardening`.
## Review Output ## Review Output
@ -151,96 +155,18 @@ Choose one:
List the finding IDs the verdict depends on (e.g. "Request changes: R1, R2"). List the finding IDs the verdict depends on (e.g. "Request changes: R1, R2").
## Change Sizing
Small, focused changes are easier to review, faster to merge, and safer to deploy.
```
~100 lines changed → Good. Reviewable in one sitting.
~300 lines changed → Acceptable if it's a single logical change.
~1000 lines changed → Too large. Split it.
```
**Watch file size, not just diff size.** Around 1000 *total* lines in a single file is a common inspection signal. When a change materially grows an already-large file, decompose first.
**Splitting strategies:**
| Strategy | How | When |
|----------|-----|------|
| **Stack** | Submit a small change, start the next one based on it | Sequential dependencies |
| **By file group** | Separate changes for groups needing different reviewers | Cross-cutting concerns |
| **Horizontal** | Create shared code/stubs first, then consumers | Layered architecture |
| **Vertical** | Break into smaller full-stack slices of the feature | Feature work |
**Separate refactoring from feature work.** A change that refactors and adds new behavior is two changes — submit them separately.
## Change Descriptions
- **First line:** Short, imperative, standalone. "Delete the FizzBuzz RPC" not "Deleting the FizzBuzz RPC."
- **Body:** What is changing and why. Include context and reasoning not visible in the code itself.
- **Anti-patterns:** "Fix bug," "Fix build," "Add patch," "Phase 1."
## Dependencies
Before adding any dependency:
1. Does the existing stack solve this? (Often it does.)
2. How large is the dependency? (Check bundle impact.)
3. Is it actively maintained? (Check last commit, open issues.)
4. Does it have known vulnerabilities? (`npm audit`)
5. What's the license? (Must be compatible with the project.)
**Rule:** Prefer standard library and existing utilities over new dependencies. Every dependency is a liability.
**Upgrading dependencies:**
- Read the changelog, not just the version number. Semver is a promise the maintainer may not have kept.
- One dependency per change. When a bulk bump breaks the build, you've lost which package did it.
- Let the tests decide — a green suite before *and* after, not just "it installed."
- Review the lockfile diff, not just `package.json`. Commit it and never hand-edit it.
For supply-chain risk triage, follow the `security-and-hardening` skill.
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "It works, that's good enough" | Working code that's unreadable, insecure, or architecturally wrong creates debt that compounds. |
| "I wrote it, so I know it's correct" | Authors are blind to their own assumptions. Every change benefits from another set of eyes. |
| "We'll clean it up later" | Later never comes. The review is the quality gate — use it. |
| "AI-generated code is probably fine" | AI code needs more scrutiny, not less. It's confident and plausible, even when wrong. |
| "The tests pass, so it's good" | Tests are necessary but not sufficient. They don't catch architecture, security, or readability problems. |
| "The refactor makes it cleaner" | Relocating complexity isn't reducing it. If the reader still holds the same number of concepts, the structure didn't improve. |
| "It's only a small addition to this file" | Small diffs still push files past healthy size and bolt branches onto unrelated flows. |
| "It's just a version bump" | A bump is a behavior change you didn't write. Read the changelog. |
| "I'll upgrade everything in one PR" | A bulk bump hides which package broke the build. One per change. |
| "It's duplicated but it's only two places" | Two becomes three becomes five. Extract now, before the copies diverge. |
| "The abstraction is future-proof" | YAGNI. Delete speculative generality — generalize on the third occurrence, not the first. |
| "It's clever but efficient" | Cleverness is a readability tax. If it needs a comment to understand, simplify it. |
## Red Flags ## Red Flags
- PRs merged without any review - A change merged without any review.
- Review that only checks if tests pass (ignoring other axes) - A review that only checks whether the tests pass.
- "LGTM" without evidence of actual review - Security-sensitive changes without a security-focused review.
- Security-sensitive changes without security-focused review - A change too large to review properly: split it.
- Large PRs that are "too big to review properly" (split them)
- No regression tests with bug fix PRs
- Accepting "I'll fix it later" — it never happens
- A refactor that moves code around without reducing the number of concepts a reader must hold
- New conditionals scattered into unrelated code paths (a missing abstraction)
- A bespoke helper that duplicates an existing canonical one
- A bulk "bump dependencies" PR with no changelog review
## Verification ## Verification
Before emitting the verdict, verify the change as it stands. This is the reviewer's own due diligence — it covers the state of the code at review time, not the later resolution of findings (fixing findings is the author's job; confirming them is a new review): Before emitting the verdict, confirm the evidence the review rests on. This covers the state of the code at review time, not the later fixing of findings:
- [ ] Tests pass — run them yourself, don't trust the claim - [ ] The suite is green, and the evidence is the author's run or CI — not your own re-run. Say which one you relied on.
- [ ] Build succeeds - [ ] New behavior has tests that cover the edge cases in the diff.
- [ ] The verification story is documented (what changed, how it was verified) - [ ] The verification story is stated: what changed and how it was verified.
- [ ] Dependency upgrades reviewed against changelog, isolated per package, verified by green suite - [ ] Dependency changes were reviewed against the changelog and the lockfile diff.
## See Also
- For detailed security review guidance, see `security-and-hardening`

View File

@ -63,6 +63,8 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
- Do not pass `--author`. Author identity comes from the local git config. - Do not pass `--author`. Author identity comes from the local git config.
- Do not amend a commit you did not create in this session, unless explicitly asked. - Do not amend a commit you did not create in this session, unless explicitly asked.
- Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked. - Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
- Never add a `Co-authored-by:` trailer. It is forbidden and fails `./scripts/check-commit`. - Add `Co-authored-by:` only for a human co-author. AI providers, agents and
bots are rejected by `./scripts/check-commit`; for AI participation use the
`AI-assisted-by:` trailer.
- Do not add untracked files that were not created in this session. - Do not add untracked files that were not created in this session.
- Do not skip the `scripts/check-commit` verification step (Iron Rule 4). - Do not skip the `scripts/check-commit` verification step (Iron Rule 4).

View File

@ -1,6 +1,6 @@
--- ---
name: local-ci name: local-ci
description: Run local CI-style checks with ./scripts/ci (lint, tests, format) per monorepo module. Use when verifying changes before declaring work done, running lint or tests locally, fixing formatting, or repairing Clojure delimiter errors. description: Run local CI-style checks with ./scripts/ci (lint, tests, format, and on request production builds) per monorepo module. Use when verifying changes before declaring work done, running lint or tests locally, fixing formatting, repairing Clojure delimiter errors, or checking that a module still compiles.
--- ---
# Local CI # Local CI
@ -29,10 +29,12 @@ Run from the repo root:
``` ```
Modules: `frontend` `backend` `common` `render-wasm` `exporter` `mcp` Modules: `frontend` `backend` `common` `render-wasm` `exporter` `mcp`
`plugins` `library`, or `--all` for every module. `plugins` `library` `media-processor`, or `--all` for every module.
With no task flags it runs three tasks per module, in order: **lint**, With no task flags it runs three tasks per module, in order: **lint**,
**test**, **fmt** (format check; `--fix` formats files instead). **test**, **fmt** (format check; `--fix` formats files instead). **build**
is opt-in: it compiles the module the way production does and is the slow
one, so it never runs unless you ask for it.
| Flag | Effect | | Flag | Effect |
|------|--------| |------|--------|
@ -41,6 +43,7 @@ With no task flags it runs three tasks per module, in order: **lint**,
| `--lint` / `--no-lint` | Run only lint / drop lint | | `--lint` / `--no-lint` | Run only lint / drop lint |
| `--test` / `--no-test` | Run only tests / drop tests | | `--test` / `--no-test` | Run only tests / drop tests |
| `--fmt` / `--no-fmt` | Run only format check / drop it | | `--fmt` / `--no-fmt` | Run only format check / drop it |
| `--build` / `--no-build` | Run only the production build / drop it |
| `--fix` | Format files instead of checking (other tasks unaffected) | | `--fix` | Format files instead of checking (other tasks unaffected) |
| `--paren-repair` | Fix delimiter errors in Clojure/CLJS files | | `--paren-repair` | Fix delimiter errors in Clojure/CLJS files |
| `--fail-fast` | Stop at the first failure | | `--fail-fast` | Stop at the first failure |
@ -72,6 +75,12 @@ log path.
# Format the module without running the test suite # Format the module without running the test suite
./scripts/ci --fix --no-test frontend ./scripts/ci --fix --no-test frontend
# Check that the module still compiles (slow: minutes)
./scripts/ci --build frontend
# Compile every module (8 builds, ~4 min with warm caches)
./scripts/ci --build --all
# Broke delimiters in Clojure/CLJS files: repair first, then lint # Broke delimiters in Clojure/CLJS files: repair first, then lint
./scripts/ci --paren-repair frontend ./scripts/ci --paren-repair frontend
./scripts/ci --lint frontend ./scripts/ci --lint frontend
@ -89,7 +98,12 @@ log path.
- Test tasks are long-running (backend runs `clojure -M:dev:test`); give - Test tasks are long-running (backend runs `clojure -M:dev:test`); give
the bash call a generous timeout (10–20 minutes) instead of letting it the bash call a generous timeout (10–20 minutes) instead of letting it
time out mid-run. time out mid-run.
- `mcp` has no lint task — it shows as skipped, not failed. - `mcp` and `media-processor` have no lint task — they show as skipped,
not failed. `common` has no build task for the same reason.
- Build runs are the slow ones (frontend ~2 min even with warm caches;
cold caches roughly double it). Give the shell a generous timeout and
do not run two builds at once in the same checkout: `render-wasm` takes
a build lock.
- `--paren-repair` only fixes delimiters; run lint afterwards to catch - `--paren-repair` only fixes delimiters; run lint afterwards to catch
what remains. See `mem:scripts/paren-repair`. what remains. See `mem:scripts/paren-repair`.
- What to run and how to read test results: `mem:testing`. - What to run and how to read test results: `mem:testing`.

View File

@ -0,0 +1,330 @@
---
name: migrate-ds-area
description: Migrate one Penpot UI area to the Design System components for one group of the DS migration epic (#12375): buttons and icons (icon-button*, button*, icon*), selects (select*) and later groups. Moves the area's SCSS off the deprecated styles, proves every change with a zero-tolerance visual test, then opens the draft PR and updates the group issue (PRs table row, QA comment). Use when the user asks to migrate an area or module to the DS, continue a DS migration group, or names an area of a group issue.
---
# Skill: migrate-ds-area
Move one UI area (one `.cljs` + `.scss` pair, or a few related modules)
from legacy components and raw HTML to the Design System components,
prove with pixels what changed, and ship it as a draft PR that QA can
test.
The epic is #12375. Each control type is a group with its own issue
(buttons and icons #12376, selects #12377, radios #12378, …). The group
issue holds the plan (order, waiting areas), the changes QA should
expect, and the PRs table where people claim areas. Read it first.
## Input
- The group issue (e.g. `#12376`) and the area: a module path (e.g.
`dashboard/fonts`) or an area named in the group issue.
- No area given: take the next unclaimed area in the group issue's plan
order, and say which one.
## Ground rules
- Never `git push`. The user pushes; then you open the PR.
- PRs go to `develop`; branches start from `origin/develop`.
- **Stay in the group.** Only the controls of this group change. In the
buttons/icons group, selects, radios, numeric/text inputs, checkboxes,
menus (`%menu-*`, `%dropdown-*`) and modal structure (`%modal-*`) stay;
a checkbox tick or an input label icon stays legacy.
- **Accept the DS look.** Do not add CSS to imitate legacy sizes or
colors. Do keep positions: icons and labels must not move unless the
DS size forces it, and then say so.
- No negative margins to compensate; use positive DS spacing or
`px2rem()`.
- **Gaps in the DS wait.** An area that needs an icon or a feature the
DS lacks keeps its legacy control; say so in the PR.
- Skip legacy shared components that another group will replace
(`context_menu_a11y`, `forms`, `select`, `dropdown`, `radio_buttons`,
`search_bar`, `title_bar`, …): they wait for that group's cleanup,
where they are updated if still used or deleted if not.
- Debug-only UI gets no QA comment and no compare image.
## Devenv
Run every test, lint and `pnpm` command inside the workspace container,
never on the host (a host `pnpm` relinks `node_modules` and breaks the
container's watcher). The container is `penpot-devenv-ws<N>-main`, where
`<N>` is the workspace number (`ws0` is the main repo; `wsN` lives in
`~/.penpot/penpot_workspaces/wsN`). Write output to a file, then read it:
```bash
C=penpot-devenv-ws1-main # your workspace
docker exec -u penpot -w /home/penpot/penpot/frontend $C \
bash -lc '<command>' > <scratch>/out.txt 2>&1
```
When the app does not behave as the code says, check the build first:
- **Stale build after a branch switch** ("is not a function", or a page
with empty `stl/css` classes): `touch` the stale `.cljs`/`.scss`, wait
for the compiled file under `frontend/resources/public/` to update,
hard-reload.
- **A whole area unstyled:** a new `.scss` added after the watcher
started is missing from `main.css` (grep `main.css` for the module
prefix), or `main.css` was written corrupted after many `.scss`
changed at once. Touch a `.scss`; if the module is still missing,
restart the watcher in the workspace's tmux session (`penpot:0`,
`./scripts/watch app`).
- **Blank app after switching between `develop` and `staging`:**
dependencies differ. Run `pnpm install --frozen-lockfile` in the
container and restart the watcher.
## Workflow
### 1. Claim the area and branch
1. Read the group issue: plan, waiting areas, expected changes, PRs table.
2. Add a row to the PRs table with no PR number, the user's @login as
Owner and the state `Claimed` (`gh issue view/edit <n> --body-file`).
Read the body right before editing, and read it again after saving:
when two people edit at the same time, GitHub keeps only the last
edit, so a row can vanish. Put yours back if it is gone.
Skip an area someone else already claimed, unless the claim has had
no PR for 7 days and its owner did not answer a comment asking.
3. `git switch -c <user>-ds-<group>-<area> origin/develop`.
4. Survey the area: every control of the group in the `.cljs`, the
legacy icons, the placeholders and mixins the `.scss` uses, the
`div`/`span` with `:on-click`.
### 2. Visual baseline
Add a `test.describe("<Area> controls", …)` block to the matching spec
in `frontend/playwright/ui/visual-specs/` (project `ds`, dark theme).
Put it next to the blocks of the same screen, not always at the end of
the file: two PRs that both append at the end conflict.
- Every shot: `toHaveScreenshot({ maxDiffPixelRatio: 0, threshold: 0 })`.
- Show every changed control at rest, on hover and open or pressed.
Capture the element or the smallest stable container.
- Locators must work before and after the change: text, role, testid,
`[class*='…']`, or `a.or(b)` when the element type changes (raw
`[class*='menu-btn']` before, `getByRole("button", { name: "Options" })`
after). Scope `.or()` to the area so it cannot match elsewhere.
- Missing data? Add a fixture under `frontend/playwright/data/` and mock
it (`dashboardPage.mockRPC(...)`, `DashboardPage.mockConfigFlags` for
flags). Watch for expired or inconsistent fixtures (dates in the past,
wrong file for a mock).
- Capture with `--update-snapshots=all`:
```bash
npx playwright test --project=ds <spec> -g "<describe>" --update-snapshots=all
```
Plain `--update-snapshots` keeps a snapshot within the project's 0.5%
tolerance, so a real diff looks like 0.
- Copy the snapshots (`<spec>.spec.js-snapshots/<Describe>-*`) to
`<scratch>/diffs/<area>/before/` and look at them: the state you
wanted must really be on screen.
### 3. Controls commit
#### Buttons and icons
| Legacy | DS |
|---|---|
| icon-only raw button | `[:> icon-button* {:variant "ghost" :aria-label (tr …) :icon i/x :on-click f}]` (always pass `:variant`; `"action"` for 24px icon toggles) |
| pressed/selected | `:aria-pressed (boolean x)` |
| text button | `[:> button* {:variant "primary"/"secondary"/"ghost"/"destructive" :type "button"} label]` |
| `[:input {:type "button"}]` | `button*` |
| modal close | `icon-button*` ghost, `:aria-label (tr "labels.close")`, `:tooltip-class` carrying the corner position |
| legacy icon outside a button | `[:> icon* {:icon-id i/x :class …}]` |
- Not every raw button changes: rows, tabs, swatches, avatar widgets,
full-height segments, round custom buttons and link-style text buttons
stay raw (their legacy icon still changes). List the kept ones in the
PR.
- A `div`/`span` with `:on-click` in the area becomes a `[:button]`
(with `aria-expanded`/`aria-haspopup` when it opens something). An
`[:a]` without `href` stays; list it in the PR.
- **Tooltip wrapper.** `icon-button*` renders inside a tooltip trigger
wrapper. Position (`position: absolute`, grid placement, margins
against siblings) goes on `:tooltip-class`; look (`--button-height`,
colors) goes on `:class`.
- **Tooltip at the right edge.** Small icon buttons at the right end of
a sidebar row or title bar get `:tooltip-placement "top-left"`; list
it as an expected change.
- **Sizes through custom properties:** `--button-height: #{px2rem(64)}` /
`--button-width`. Sass does not evaluate functions inside custom
properties without `#{}`.
- **A grown `button*` keeps its label at the start**: add
`justify-content: center` when it is wider than its content.
- **Icon colors:** DS icons use `stroke: currentColor`. Set `color` on
the icon class, not `stroke`/`fill`. A legacy `color: transparent` left
on the button or wrapper makes the DS icon invisible: drop it. A class
that redefines the color variable on the icon breaks the hover that
changes it on the button: define the variable on the button.
- **Every DS icon needs a color of its own** (on its class or a parent):
one that inherits none shows in the devenv debug yellow.
- **Icon sizes:** `"s"` draws 12px inside a 16px box (compensate the
margin by 2px per side to keep the drawing in place); `"m"` 16px; `"l"`
24px; `"xl"` 32px.
- **Stroke width:** DS icons keep the DS stroke width. Never set
`stroke-width` on a DS icon to match the legacy one; a thinner or
thicker line is part of the DS look.
- **Legacy placeholders that style `svg` descendants** (e.g.
`%dropdown-element-base` → `span svg { 12px box; stroke }`) override a
DS icon placed inside them: give the icon class its own box
(`inline-size`/`block-size` 16px) and `stroke: currentcolor` with more
specificity.
- **Flex shrink:** a DS svg inside a small flex button can shrink and be
cut; add `flex-shrink: 0` to the icon class.
- **Rotated legacy arrows:** prefer `i/arrow-left`/`-right`/`-up`/`-down`
over rotating `i/arrow` (same path), unless a rotation class drives the
open/closed state.
- **Menus anchored by the button size** move when the button grows; open
them from the top (`inset-block-start: calc(100% + var(--sp-xs))`) or
fix the context menu offset, and say so.
#### Selects
`[:& select {:default-value v :options [{:value :label}] :on-change f}]`
→ `[:> select* {:default-selected <id> :options <memoized vector of {:id :label}> :on-change f}]`
(`app.main.ui.ds.controls.select`).
- **Ids are non-blank strings.** A blank id shows a dimmed "--", and an
id that matches no option falls back to the first one. Give mixed,
none and auto their own ids (`"multiple"`, `"none"`, `"auto"`) and map
them back in the handler. Use `""` only where "--" is the wanted look.
- **Convert the value.** The legacy select ran `d/read-string`, so
handlers got keywords, numbers or uuids; `select*` passes the `:id`
string. Convert with `keyword`, `d/parse-integer` or `uuid/parse`.
- **Memoized options, live selection.** `select*` resets its selection
when `:default-selected` or the `:options` identity changes. Build
options with `mf/with-memo` and `mapv` (the schema wants a non-empty
vector) and pass `:default-selected` from the live value, never a
constant.
- **Never pass nil.** In dev builds a present prop with value nil
(`:class`, `:wrapper-class`, `:disabled`, `:default-selected`) throws
`invalid props on component select*`. Use `(boolean x)`, `d/nilv`, or
add the prop only when it has a value.
- **Props the legacy select ignored stay out**: `select*` passes rest
props to its button, so they would reach the DOM.
- **Classes:** `:wrapper-class` for layout, `:class` for look.
- **Portal:** add `:has-portal true` only where the visual test shows the
list cut at the bottom of a panel, and say so in the QA comment.
- **Locators:** the button is a `combobox` with no accessible name; use
`getByRole("combobox").filter({ hasText: "Linear" })` (it matches the
legacy select too) and `getByRole("option", { name: … })`.
- **Keyboard:** the arrows no longer change the value while the list is
closed. It is an expected change, already listed in #12377.
#### All groups
- New labels: add the key to `en.po` and `es.po` by hand, in sorted
position, with a `#:` reference to the cljs line. Never run the
translations sync.
- The `.scss` only gets what the new controls need in this commit.
- Some e2e specs are not Prettier-clean on develop: never run
`prettier --write` on a whole spec; format only your block.
Then wait for the watcher (grep the compiled JS/CSS under
`frontend/resources/public/` for the change), recapture with `=all`,
copy to `after/`, diff:
```bash
SK=.agents/skills/migrate-ds-area/scripts
python3 -I $SK/diff.py <before> <after> # per-file pixel count + bbox
python3 -I $SK/trip.py <b> <a> <out.png> <scale> # before | after | red diff
```
`trip.py` pads images of different sizes, so a taller "after" still
renders.
Explain every diff. Fix what is not a DS-look change (moved icons, cut
icons, lost hover color). Rerun a shot that changes on unchanged code
before trusting it; flaky spots seen so far: the invite modal textarea
border, the search input caret, the user account page, the path handler
menu trigger.
Lint, then commit (read `mem:workflow/creating-commits` first):
```bash
npx prettier --check <spec> <fixtures>; npx stylelint <scss>; clj-kondo --lint <cljs>; cljfmt check <cljs>
git add -f <cljs> <scss> <po> <spec> <fixtures> # -f: visual-specs/ is gitignored
./scripts/check-commit
```
A plain `git add` with an ignored path in the list adds nothing; always
`-f`, and check `git show --stat`. Never commit the screenshots.
Subject: `:recycle: Use DS <controls> in <area>`. A pre-existing bug
found and fixed on the way goes in its own `:bug:` commit.
### 4. SCSS commit
Run the `refactor-scss` skill on every `.scss` the PR touched (only
those), in its own commit. Keep `refactor/` only for component-family
placeholders (`%modal-*`, `%menu-*`, `%input-*`, `%checkbox-*`) and say
why in a comment above the `@use`. Unused selectors are deleted, not
commented out (list them in the commit body). Typography without an
exact match uses the closest DS typography. Run the skill's
**Self-review** before committing: no bare element selectors (add
classes in the `.cljs`), no copied properties that do nothing, z-index
tokens by purpose. Compare against the controls commit: it should be
0 px; explain anything else (typically a typography or border change).
For files that several modules share or that are huge, stop and propose
a separate PR instead.
### 5. Hand off, PR and tracking
1. Report to the user: commits, visible changes, kept raw controls,
pre-existing bugs, and the compare image path
(`<scratch>/diffs/<area>/compare.png`, before | after | diff). Give
the push command: `git push -u origin <branch>` (the branch tracks
`origin/develop`, so a bare `git push` is refused).
2. After the user pushes, check `git rev-parse HEAD origin/<branch>`,
then (read `mem:workflow/creating-prs` first):
```bash
gh pr create --repo penpot/penpot --draft --base develop --head <branch> \
--title "<commit subject>" --body-file <body> \
--label "WG adoption" --label "no issue required" \
--milestone <group issue milestone> --project Main
gh pr edit <n> --repo penpot/penpot --remove-label "needs triage"
```
Never add `no changelog` to a PR: that label goes on issues.
Body: AI note, `Part of #<group> (DS migration epic #12375).`, a
short What / How, the kept raw controls, an empty `## Screenshots`
when something is visible (the user attaches the image). Never
`Closes #<group>`. Keep it short: reviewers find long PRs hard to read.
3. In the group issue's PRs table, fill your row:
`| #<n> | <area>: <controls> (<visible changes>) | @<owner> | Draft |`.
4. Post a QA comment on the group issue: `**#<n>: <area>**`, *Where* (how
to reach it), *Expected changes* (every visible change; QA treats
anything not listed as a bug), *No other change* (what still works
the same).
Do not mark the draft ready or ask for review: the user decides when.
### 6. Review fixes
When the reviewer comments on a PR:
1. Fix each comment in a new commit on the branch (never amend pushed
commits). A comment you do not apply gets a short reason the user can
post, with an image when it is visual.
2. Rerun the area's visual spec before and after the fix with `=all`.
Build a compare image for every visible change and give the user its
full path, saying which PR it goes to.
3. Update everything the fix makes stale, in the same turn: the QA
comment (*Expected changes* must list the new visible changes), the
PRs table row, and the PR body.
4. When working on several PRs in one go, report per PR: branch, new
commit, what changed, image path. Never mix two branches in one list.
5. Give the push command per branch. If a rebase is coming soon, say
that pushing now costs one extra CI run; the user decides.
6. After a round of review fixes, check whether the same issue exists in
the user's other open PRs of the group, and fix it there too.
### 7. After merges
Whenever PRs merge (or go from draft to ready), sync the state column of
the group's PRs table with GitHub for **every** row, not only your own:
read each row's PR (`gh pr view <n> --json state,isDraft`) and set
Merged / In review / Draft. The table drifts when other people merge.
## Output
- Branch with 2 commits (controls, SCSS) plus any `:bug:` commit.
- Visual spec block with zero-tolerance shots.
- Compare image in the scratch folder.
- After the push: draft PR, row and QA comment in the group issue.

View File

@ -0,0 +1,25 @@
"""Pixel diff of two folders of PNG snapshots with the same file names.
Usage: python3 -I diff.py <before-dir> <after-dir>
Prints, per file, the number of differing pixels and their bounding box,
or the two sizes when they differ.
"""
import os
import sys
from PIL import Image, ImageChops
before, after = sys.argv[1], sys.argv[2]
for name in sorted(os.listdir(before)):
other = os.path.join(after, name)
if not os.path.exists(other):
print(name, "missing in", after)
continue
a = Image.open(os.path.join(before, name)).convert("RGB")
b = Image.open(other).convert("RGB")
if a.size != b.size:
print(name, "size", a.size, b.size)
continue
diff = ImageChops.difference(a, b)
count = sum(1 for p in diff.getdata() if p != (0, 0, 0))
print(name, count, diff.getbbox())

View File

@ -0,0 +1,33 @@
"""Before | after | red diff image, one row per snapshot.
Usage: python3 -I trip.py <before-dir> <after-dir> <out.png> <scale>
Both folders hold PNGs with the same names (crop them first if needed).
"""
import os
import sys
from PIL import Image, ImageChops
before, after, out, scale = sys.argv[1], sys.argv[2], sys.argv[3], int(sys.argv[4])
rows = []
for name in sorted(os.listdir(before)):
a = Image.open(os.path.join(before, name)).convert("RGB")
b = Image.open(os.path.join(after, name)).convert("RGB")
w, h = max(a.width, b.width), max(a.height, b.height)
pad_a = Image.new("RGB", (w, h), (200, 200, 200)); pad_a.paste(a, (0, 0))
pad_b = Image.new("RGB", (w, h), (200, 200, 200)); pad_b.paste(b, (0, 0))
mask = ImageChops.difference(pad_a, pad_b).convert("L").point(lambda v: 255 if v else 0)
red = Image.new("RGB", (w, h), (255, 255, 255))
red.paste((255, 0, 0), mask=mask)
row = Image.new("RGB", (w * 3 + 20, h), (200, 200, 200))
for i, im in enumerate([pad_a, pad_b, red]):
row.paste(im, (i * (w + 10), 0))
rows.append(row.resize((row.width * scale, row.height * scale), Image.NEAREST))
width = max(r.width for r in rows)
height = sum(r.height + 10 for r in rows)
canvas = Image.new("RGB", (width, height), (255, 255, 255))
y = 0
for r in rows:
canvas.paste(r, (0, y))
y += r.height + 10
canvas.save(out)

View File

@ -0,0 +1,421 @@
---
name: refactor-scss
description: Migrate one Penpot SCSS module off the deprecated styles under frontend/resources/styles/common/refactor/ to the current Design System (ds/ tokens, mixins and px2rem). Pure refactor, no visual change. Use when the user asks to refactor, migrate or clean up a .scss file to the DS.
---
# Skill: refactor-scss
Refactor a single SCSS file so that it no longer depends on the deprecated
styling system, using the current Design System instead.
This is a **pure refactor**. Do NOT introduce visual or behavioral changes.
Preserve the generated CSS behavior exactly unless a rule below requires
otherwise. The rules that allow a small visible change (closest DS
typography, DS border width) want it shown in the visual check and listed
in the report and the commit body.
Reviewers read every line of the result. A line that is
copied but does nothing, a bare element selector or a raw z-index comes
back as a review comment, so the rules below are strict about them.
## Input
- The path to one `.scss` file (e.g. `frontend/src/app/main/ui/foo/bar.scss`).
- If no path is given, ask for it. Do not pick a file yourself.
## Workflow
1. Read the target `.scss` file and the `.cljs` file with the same name next
to it.
2. Read every deprecated source the file uses, so you inline real values,
never remembered ones:
- `frontend/resources/styles/common/refactor/mixins.scss`
- `frontend/resources/styles/common/refactor/design-tokens.scss`
- `frontend/resources/styles/common/refactor/spacing.scss`
- any other file under `frontend/resources/styles/common/refactor/` that
the stylesheet pulls in (placeholders, borders, z-index, fonts...).
3. Read the DS modules you will map to:
- `frontend/src/app/main/ui/ds/_utils.scss` (`px2rem()`)
- `frontend/src/app/main/ui/ds/spacing.scss`
- `frontend/src/app/main/ui/ds/_sizes.scss`
- `frontend/src/app/main/ui/ds/typography.scss` (`use-typography()`)
- `frontend/src/app/main/ui/ds/_borders.scss`
- `frontend/src/app/main/ui/ds/z-index.scss`
4. Apply the rules below and edit the file in place.
5. Verify (see **Verification** and **Visual check**).
6. Report (see **Output**). Do not commit. When the refactor goes with other
changes, it belongs in its own commit, after the task's commits.
## Primary goal
After the refactor, the SCSS file must not depend on anything under
`frontend/resources/styles/common/refactor/`. This includes imports, mixins,
placeholder selectors, functions, deprecated variables and design-token
aliases.
### Partial refactor
Some placeholders stand for a whole component family that will move to the DS
in its own PR: `%input-element`, `%input-with-label`, `%input-checkbox`,
`%modal-*`, `%menu-dropdown`, `%menu-item-base`, `%shortcut-*` and similar.
Inlining them in a small task produces big, risky diffs. In that case:
- migrate everything else (tokens, sizes, spacing, small mixins);
- keep the `refactor/` import only for those placeholders;
- list each remaining dependency and the selectors that use it in the report,
so the user can plan it as a separate PR.
## General rules
- Keep the resulting SCSS as small and readable as possible.
- Avoid duplicated declarations and duplicated selectors.
- Do not change class names.
- Do not change DOM structure.
- Do not change specificity unless required.
- Follow the project's SCSS formatting (Prettier, see **Verification**).
## Imports
Use the DS modules. Import them with `as *` when they expose mixins,
variables or functions:
```scss
@use "ds/_utils.scss" as *;
```
not
```scss
@use "ds/_utils.scss" as utils;
```
Do not introduce namespaces such as `utils.`, `typography.`, `borders.`,
`spacing.` or `sizes.` unless a module explicitly requires one.
Remove every unused import.
## Deprecated mixins
If the stylesheet uses mixins from
`frontend/resources/styles/common/refactor/mixins.scss`:
- locate the mixin;
- inline its generated CSS (then apply the rest of these rules to it);
- remove the `@include`;
- remove the import.
Do not leave deprecated mixins.
## Placeholders
Replace every `@extend %placeholder` with the generated CSS. Remove every
deprecated placeholder, except the component-family ones described in
**Partial refactor**.
Inline only what does something on that element. Do not paste a
placeholder (or mixin) body as it is. For each property, check:
- **It applies to the element.** `fill`, `stroke` and `stroke-width` only
act on SVG shapes: drop them from a `span`, `div` or `button`.
`color: transparent` on a wrapper that only holds an icon does nothing
useful: drop it and set the real color.
- **It is not the default or already given.** `stroke-width: 1px` is the
SVG default; `fill: none` and `stroke: currentColor` are already set by
the DS `icon*` (`.icon`). Drop them.
- **A later rule does not override it.**
- **It is not a stroke width on a DS icon.** DS icons keep the DS
stroke width; drop any `stroke-width` that only imitates the legacy
icon (1.33px, 2.25px...).
`%button-icon` is the usual case: on a wrapper it leaves `color:
transparent; fill: none; stroke-width: 1px`, and none of the three is
needed.
## Element selectors
Do not style HTML elements directly (`svg {}`, `h3 {}`, `button {}`,
`input.x {}`, or `.component { svg {} }`).
When no class exists, **add one in the `.cljs`**: a DS component takes
`:class (stl/css :x)` (`icon*`, `heading*`, `button*`…), and a plain
element takes `{:class (stl/css :x)}`. Touching the `.cljs` for this is
part of the refactor. A comment like "the icon has no class" is not an
answer.
Prefer the DS component when one fits: `heading*` for `h1`–`h6`
(`:level`, `:typography`), `text*` for text, `icon*` for icons.
`element.class` (e.g. `input.element-title`) is not needed to beat the
input defaults: try the class alone and check it in the visual check.
An element selector stays only for markup you cannot reach (injected
HTML, third-party widgets), with a comment saying so.
**Icon colors.** A DS icon draws with `stroke: currentColor`. Set
`color` on the icon class or its parent, never `stroke`/`fill` on an
`svg` selector.
## Color variables
Whenever a color changes across states, introduce a component-local CSS
variable.
Instead of
```scss
.button {
color: var(--color-foreground-primary);
&:hover {
color: var(--color-accent-primary);
}
&:disabled {
color: var(--color-foreground-disabled);
}
}
```
write
```scss
.button {
--button-foreground-color: var(--color-foreground-primary);
color: var(--button-foreground-color);
&:hover {
--button-foreground-color: var(--color-accent-primary);
}
&:disabled {
--button-foreground-color: var(--color-foreground-disabled);
}
}
```
Use this pattern for `color`, `background-color`, `border-color`,
`outline-color`, `fill`, `stroke` and any similar property.
## Design token aliases
Remove every alias defined in
`frontend/resources/styles/common/refactor/design-tokens.scss`. Resolve each
alias to its final DS token. Do not keep intermediate aliases.
Instead of `color: var(--button-foreground-hover);` write
`color: var(--color-accent-primary);`.
## Spacing
Replace deprecated spacing variables using
`frontend/src/app/main/ui/ds/spacing.scss` for `margin`, `padding`, `gap` and
`inset`.
If no exact spacing token exists, use `px2rem()`. Do not approximate with the
closest token.
## Sizes
Replace deprecated size variables using
`frontend/src/app/main/ui/ds/_sizes.scss` for `inline-size`, `block-size`,
`width`, `height` and min/max sizes.
If no exact size exists, use `px2rem()` from
`frontend/src/app/main/ui/ds/_utils.scss`. Do not leave hardcoded pixel values
unless absolutely necessary.
## Typography
When any of `font-size`, `font-weight`, `line-height`, `font-family` or
`letter-spacing` appear, check whether together they match one of the
definitions in `frontend/src/app/main/ui/ds/typography.scss`.
If they match, replace all the individual declarations with:
```scss
@include use-typography("<typography-name>");
```
If there is no exact match, use the **closest DS typography** (same role:
a panel title in uppercase → `headline-small`, body text → `body-*`),
even when more than one detail differs. Show the change in the visual
check and state it in the report and in the commit body (e.g. "11px
regular → 12px medium").
Keep the original declarations only when no DS typography fits the role
(e.g. a 32px display number in a 14px-only area); then leave a TODO
naming the closest one and ask the user.
## Borders
Replace deprecated border radius and border width values using
`frontend/src/app/main/ui/ds/_borders.scss`. A raw width close to a token
(`1.75px`) takes the token (`$b-2`); say so in the report. If nothing is
close, use `px2rem()`.
## Logical properties
Prefer logical properties whenever they preserve the existing behavior:
- `width` → `inline-size`
- `height` → `block-size`
- `max-width` → `max-inline-size`
- `margin-left` → `margin-inline-start`
- `padding-right` → `padding-inline-end`
- `top` → `inset-block-start`
- `left` → `inset-inline-start`
## Z-index
Never leave magic z-index values. Use tokens from
`frontend/src/app/main/ui/ds/z-index.scss`, and pick the token **by what
the element is, not by its number**: a dropdown, menu or select →
`--z-index-dropdown`; a toolbar, palette, sidebar or panel →
`--z-index-panels`; and so on (read the comments in that file). A raw `4`
on a dropdown becomes `var(--z-index-dropdown)` even though the number
changes. Check the visual check for overlap.
Leave a TODO only when the element fits no token's purpose.
Legacy z-index values between sibling elements that do not overlap can be
removed, but only when the **Visual check** shows no pixel change in every
state that matters (open menus, collapsed, hidden). Say so in the report.
## Duplicated properties
Within the same selector, avoid duplicated properties, duplicated CSS
variables and duplicated declarations introduced by the refactor. Allow
duplicates only when they are intentional browser fallbacks.
## Unused classes
Open the `.cljs` file with the same name as the SCSS file and check whether
every CSS class is referenced (as `(stl/css :class-name)`,
`(stl/css-case :class-name ...)` or similar).
CSS modules prefix every class with its module
(`main_ui_<path>__<class>`), so only the `.cljs` of the same module can use
it. Before calling a selector unused, also search the whole `frontend/src`
for the class name as a string, a `:global` use, or a dynamic class built
at runtime.
If a selector is unused after that check:
- delete it (do not leave it commented out, do not add a TODO);
- list every deleted selector in the report and in the commit body.
A rule that matches nothing because the markup changed (e.g. `.icon svg`
when the DS icon is the `svg` itself) is deleted the same way. Before
deleting a descendant rule, check every element it matched, not only
the one you changed: `.placeholder .icon svg` can also color a second
icon, which then has no color (in the devenv it shows in the debug
yellow).
Every DS icon needs a color of its own (`color` on its class or on a
parent with a defined color): an icon that inherits nothing shows in
the debug yellow in the devenv.
Classes used by markup injected from outside (e.g. an external widget)
stay, with a comment saying so.
Also check the other direction: every class the `.cljs` passes to
`stl/css` / `stl/css-case` must exist in the `.scss`. A missing one silently
does nothing; report it.
## Sass and CSS pitfalls
- Sass does not evaluate functions inside CSS custom properties. Write
`--button-height: calc(var(--height) - #{px2rem(16)});`, not
`... - px2rem(16))`, or the browser gets the literal `px2rem(16)`.
- A `/` between Sass size variables is read as a division
(`grid-template: $sz-32 / auto` breaks). Split it into
`grid-template-rows` / `grid-template-columns` (or areas).
- A CSS-wide keyword cannot be part of a shorthand: `inset-inline: unset 0`
is invalid and stylelint does not catch it. Use the longhands.
- After editing, check the compiled CSS for any of these
(`frontend/resources/public/css/main.css`, search the module class).
## Uncertainty
If any transformation is uncertain, never guess. Leave a TODO in the code
and explain why in the report.
## Self-review
Before the report, read the final diff as the reviewer would and search
the file for these; each hit must be fixed or have a real reason:
```bash
rg -n 'svg \{|^\s*(h[1-6]|p|hr|input|button|span|a)\b[^-_]|[a-z]\.[a-z-]+ \{' <file>
rg -n 'stroke-width|color: transparent|fill: none|z-index: [0-9]|TODO' <file>
rg -n '[0-9]\.[0-9]+px|[^(]\b[0-9]+px' <file>
```
- an element selector → add a class (see **Element selectors**);
- a property that does not apply or is a default → drop it (see
**Placeholders**);
- a raw z-index → token by purpose;
- a typography TODO → closest DS typography;
- a raw px value → token or `px2rem()`.
## Verification
Run the project scripts from `frontend/` inside the devenv container
(`penpot-devenv-ws<N>-main` for workspace `wsN`; never run `pnpm` on the
host), writing output to a file first (never pipe lint output into
filters):
```bash
docker exec -u penpot -w /home/penpot/penpot/frontend penpot-devenv-ws<N>-main \
bash -lc 'pnpm run lint:scss; echo LINT=$?; pnpm run check-fmt:scss; echo FMT=$?' \
> <scratch>/scss-check.txt 2>&1
```
If the format check fails, run `pnpm run fmt:scss` and check again.
**Dependency check.** A grep for `refactor/` is not enough. For the final
file, check:
- `@use` / `@import` sources: all must be `ds/...` (or the sidebar common
module, which is DS-only);
- every `@include`, `@extend` and `%placeholder`;
- every `$variable`: which imported module defines it (`$br-*` exists in both
`ds/_borders.scss` and `refactor/borders.scss`; the `@use` decides);
- every `var(--token)`: defined in the DS (colors, spacing, sizes) or only as a
`refactor/` alias.
Report any failure as it is; do not hide it.
## Visual check
"No visual change" must be shown, not assumed. When a visual spec covers the
component (`frontend/playwright/ui/visual-specs/`, project `ds`), or one can be
added:
1. Capture before (base branch styles) and after with
`pnpm exec playwright test --project=ds -g "<test>" --update-snapshots=all`
in the container. `=all` is required: plain `--update-snapshots` only
rewrites past the 0.5% tolerance, so a real diff can look like 0.
2. Before each capture, wait until the compiled CSS/JS shows the expected
version (search `main.css` for the changed rule).
3. Compare with zero tolerance. If pixels change, build the before | after |
diff (red) image and explain every change in the report.
## Output
1. The diff of the file (`git diff -- <file>`).
2. A summary with:
- removed imports;
- removed deprecated mixins;
- removed placeholders;
- typography replacements;
- design-token aliases removed;
- logical properties introduced;
- deleted unused selectors;
- TODOs requiring manual review, each with its reason;
- remaining `refactor/` dependencies (partial refactor) and where they are
used;
- classes missing from the `.scss`, if any.
3. The verification results, including the visual check (pixel count, or why
it was not possible).

View File

@ -22,8 +22,20 @@ Act as a senior software engineer and perform a thorough code review.
diff, a commit range, a branch, a PR (number or URL), or specific files. If diff, a commit range, a branch, a PR (number or URL), or specific files. If
the target is ambiguous, ask before reviewing. the target is ambiguous, ask before reviewing.
2. Delegate the review to the `general` subagent (via the task tool), unless the 2. Delegate the review to the `general` subagent (via the task tool), unless the
user specifies another agent. Include in the prompt the user specifies another agent. The prompt carries the **`code-review-criteria`**
**`code-review-criteria`** skill name and all user context. skill name, all user context, and these requirements:
- Load `code-review-criteria` and follow its process and output format.
- Read `AGENTS.md` (if present) and the module memories it points to for the
code being reviewed.
- Return the COMPLETE review, verbatim, exactly as the skill instructs it to
be produced. Never summarize it.
- Read-only: modify no file and create no commit. Reviewing never writes.
- Do not invent problems: every finding must be real and actionable.
- Missing tests are an issue, not a suggestion — a severity-tagged finding.
- Prioritize by impact: one structural issue outweighs ten nits, and "this
could be better" is not a finding — say why and how.
- Skip generated files, lockfile-only changes and unrelated modifications
unless they introduce security risks.
3. When the subagent returns, output the review to the user verbatim. Do not 3. When the subagent returns, output the review to the user verbatim. Do not
summarize it and do not act on its findings. summarize it and do not act on its findings.
4. Right after the review, suggest how to proceed based on the findings. These 4. Right after the review, suggest how to proceed based on the findings. These
@ -44,43 +56,18 @@ the findings. Once the user explicitly states the review is done (or moves on to
a different task), this rule no longer applies and you act as a normal build a different task), this rule no longer applies and you act as a normal build
agent again. agent again.
## Instructions for the subagent
1. Load the **`code-review-criteria`** skill and follow its process and output
format.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and
reading all related testing documentation from memories before reviewing.
3. Return in your final message the COMPLETE review, verbatim, exactly as the
skill instructs it to be produced. Do not summarize it — include the full
structured review.
### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing
never writes.
3. Be specific and constructive. "This could be better" is not helpful — explain
why and how.
4. Prioritize by impact. One structural issue outweighs ten nits.
5. Missing tests are an issue, not a suggestion. Report as a severity-tagged
finding — never as a recommendation.
6. Skip generated files, lockfile-only changes, and unrelated modifications
unless they introduce security risks.
## Posting as a GitHub comment ## Posting as a GitHub comment
Only post the review as a comment when the user asks for it explicitly. Keep Post only when the user asks for it explicitly; otherwise keep the review in the
it in the session otherwise. session. PR comments go in English, with this minimal structure:
When posting: PR comments go in English, with this minimal structure:
1. **Intro line** — one sentence: what was found and the overall tone. No 1. **Intro line** — one sentence: what was found and the overall tone. No
greetings. greetings.
2. **`## Main issues`** — one subsection per required finding: what breaks, 2. **`## Main issues`** — one subsection per required finding: what breaks, the
the failure circumstances, and the minimal fix (code snippet when useful). failure circumstances, and the minimal fix (code snippet when useful). A
A missing test belongs here, not under minor suggestions. missing test belongs here, not under minor suggestions.
3. **`## Minor suggestions`** — one bullet per minor finding or suggestion: 3. **`## Minor suggestions`** — one bullet per minor finding: file/function,
file/function, problem, concrete remedy. problem, concrete remedy.
4. **Closing** — optional and short. Cut it when it adds nothing. 4. **Closing** — optional and short. Cut it when it adds nothing.
Reference every finding by its ID (`R1`, `M1`, `S1`, … from Reference every finding by its ID (`R1`, `M1`, `S1`, … from

View File

@ -6,64 +6,40 @@ slash: true
# Review Plan # Review Plan
Act as a senior software engineer and perform a thorough review of an Act as a senior software engineer and perform a thorough review of an implementation plan.
implementation plan.
## When to use ## When to use
- The user asks to review a plan, in any phrasing: "review this plan", - The user asks to review a plan, in any phrasing: "review this plan", "does this plan look right?", "second opinion on the plan" — or runs `/review-plan`.
"does this plan look right?", "second opinion on the plan" — or runs - A plan was just produced (typically by `/make-a-plan`) and the user wants it evaluated before executing it.
`/review-plan`.
- A plan was just produced (typically by `/make-a-plan`) and the user
wants it evaluated before executing it.
## Instructions ## Instructions
1. **Determine the plan under review** from the session context (for example, a 1. **Determine the plan under review** from the session context (for example, a plan just produced by `/make-a-plan`) or from a plan file path given by the user (typically under `.agents/plans/`). If a file path is given, read the file first so the complete plan is in context.
plan just produced by `/make-a-plan`) or from a plan file path given by the 2. Delegate the review to the `general` subagent (via the task tool), unless the user specifies another agent. Include in the prompt the **`plan-review-criteria`** skill name and all user context.
user (typically under `.agents/plans/`). If a file path is given, read the 3. When the subagent returns, output the review to the user verbatim. Do not summarize it and do not act on its findings.
file first so the complete plan is in context. 4. Right after the review, suggest the next step based on the verdict. These are suggestions — the user decides, and any instruction overrides them:
2. Delegate the review to the `general` subagent (via the task tool), unless the
user specifies another agent. Include in the prompt the
**`plan-review-criteria`** skill name and all user context.
3. When the subagent returns, output the review to the user verbatim. Do not
summarize it and do not act on its findings.
4. Right after the review, suggest the next step based on the verdict. These
are suggestions — the user decides, and any instruction overrides them:
- **Approve** → suggest `/implement-plan` to execute it. - **Approve** → suggest `/implement-plan` to execute it.
- **Request changes** → suggest `/make-a-plan` to make a plan to address the - **Request changes** → suggest `/make-a-plan` to make a plan to address the findings.
findings.
### Hard rule — read-only while reviewing ### Hard rule — read-only while reviewing
This flow is read-only **for the duration of the review**: from the moment it This flow is read-only **for the duration of the review**: from the moment it starts until the user considers the review finished (including any feedback, questions, or clarifications about it). During that period, never fix, implement, edit files or create commits — not even "obvious" fixes derived from the findings. Once the user explicitly states the review is done (or moves on to a different task), this rule no longer applies and you act as a normal build agent again.
starts until the user considers the review finished (including any feedback,
questions, or clarifications about it). During that period, never fix,
implement, edit files or create commits — not even "obvious" fixes derived from
the findings. Once the user explicitly states the review is done (or moves on to
a different task), this rule no longer applies and you act as a normal build
agent again.
## Instructions for the subagent ## Instructions for the subagent
1. Load the **`plan-review-criteria`** skill and follow its process and output 1. Load the **`plan-review-criteria`** skill and follow its process and output format.
format. 2. Read `AGENTS.md` (if present) and follow its instructions for finding and reading all related documentation and testing memories before reviewing.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and 3. Cada hallazgo debe tener un identificador único y estable, con el formato `F1`, `F2`, etc. El identificador debe aparecer en el título del hallazgo y no debe reutilizarse dentro de la misma revisión. Esto permite que el usuario pueda responder sobre un hallazgo concreto.
reading all related documentation and testing memories before reviewing. 4. Return in your final message the COMPLETE review, verbatim, exactly as the skill instructs it to be produced. Do not summarize it — include the full structured review.
3. Return in your final message the COMPLETE review, verbatim, exactly as the
skill instructs it to be produced. Do not summarize it — include the full
structured review.
### Strong rules for the subagent ### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable. 1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing 2. Read-only: do not modify any file and do not create a commit — reviewing never writes.
never writes. 3. Be specific and constructive. "This could be better" is not helpful — explain why and how.
3. Be specific and constructive. "This could be better" is not helpful — explain
why and how.
4. Prioritize by impact. One structural issue outweighs ten nits. 4. Prioritize by impact. One structural issue outweighs ten nits.
5. Judge the plan as the implementer would: every task executable without 5. Judge the plan as the implementer would: every task executable without guessing, ordering follows the dependency graph, risks named.
guessing, ordering follows the dependency graph, risks named.
## User input, overrides and additional context ## User input, overrides and additional context

View File

@ -5,16 +5,18 @@ description: Update the project CHANGES.md with issues from a given GitHub miles
# Skill: update-changelog # Skill: update-changelog
Update `CHANGES.md` with entries for all issues and PRs in a given GitHub Entry point for all changelog work. All rules — sections, exclusions, entry
milestone. Each entry references the user-facing issue (not the PR) as the format, the `:boom:`, `:rocket:` and `:lock:` contracts, pre-flight, and the
primary link, with the fix PR inline on the same line. anomaly report — live in `mem:workflow/updating-changelog` (file:
`.serena/memories/workflow/updating-changelog.md`). This skill routes to that
memory; it does not repeat it.
## When to Use ## When to Use
- Before a new release, to populate the changelog with all fixed issues - Before a new release, to populate the changelog with all fixed issues
- When new issues are added to an existing milestone and the changelog needs - When new issues are added to an existing milestone and the changelog needs
to be refreshed to be refreshed
- To ensure every entry follows the correct format for the changelog - To audit or repair an existing version section
## Prerequisites ## Prerequisites
@ -23,289 +25,37 @@ primary link, with the fix PR inline on the same line.
- `scripts/gh.py` for GitHub queries, `scripts/changelog.py` for changelog - `scripts/gh.py` for GitHub queries, `scripts/changelog.py` for changelog
checks (run `python3 scripts/changelog.py <command> --help` for usage) checks (run `python3 scripts/changelog.py <command> --help` for usage)
## Workflow ## What to Read
### 1. Determine the target version 1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth,
layout, entry rules, the `## Breaking change note` comment contract, the
advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
`dev-only-bugs`, `report`.
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
The version is typically a semver string like `2.15.3`. Confirm with the user ## Commands
if not specified.
### 2. Fetch all issues in the milestone
```bash ```bash
# All closed issues (default) python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"
python3 scripts/gh.py issues "2.16.0" python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog" # closed non-bug residue to review by hand
python3 scripts/gh.py advisories --patched-version "<VERSION>"
# Include open issues too python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
python3 scripts/gh.py issues "2.16.0" --state all python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
python3 scripts/changelog.py check-merged <PR numbers...>
# Exclude entries that should not go in the changelog python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py security "<VERSION>" # print the :lock: section
python3 scripts/changelog.py security "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py cross-ref "<VERSION>"
python3 scripts/changelog.py report "<VERSION>"
``` ```
**Exclusion rules (issue-level):** ## Hard Rules
- `no changelog` label — chore/refactor work, no entry needed
- `release blocker` label — blocked issues not yet ready for changelog
- `Task` issue type — internal chores, not user-facing; excluded by `gh.py`
(use `--include-tasks` to override)
- **Rejected project status** — issues with "Rejected" status on the "Main"
project board are excluded by `gh.py` (use `--include-rejected` to override).
This status is independent of the GitHub issue `state`.
**Exclusion rules (PR-level):** PRs with these labels stay out regardless of - `CHANGES.md` is edited only on explicit request or during the release process.
their linked issue's labels: `release blocker`, `no issue required`. - Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
- `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
Each entry carries `number`, `title`, `state`, `issue_type`, `labels`, - Verify PR merge status before writing; only merged PRs are referenced.
`closing_prs`, and `project_status`. - Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.
### 3. Identify missing entries (optional)
```bash
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --compare CHANGES.md
```
Returns only milestone issues not yet referenced in the changelog. Note: it
compares **issues** only. For unreferenced merged **PRs**, use the
cross-reference in step 8.
### 4. Fetch additional PR details when needed
```bash
# One or more PR numbers (also: --file prs.txt, or --stdin)
python3 scripts/gh.py prs 9179 9204 9311
# All merged PRs in a milestone (default); --state all/open/closed for others
python3 scripts/gh.py prs --milestone "2.16.0"
```
Returns `number`, `title`, `body`, `state`, `merged_at`, `author`, `labels`,
and `closing_issues`. Milestone mode uses paginated GraphQL (100 per page).
### 5. Categorize entries — strictly by issue type, never by labels or emoji
Use the **Issue Type** field (`issue_type` in the `gh.py` output). No separate
query is needed.
> **⚠️ CRITICAL: Never use labels or title emoji prefixes for categorization.**
> Labels like `bug`/`enhancement` and prefixes like `:bug:`/`:sparkles:` are
> often wrong or missing. `issue_type` is the single source of truth.
| `issue_type` value | Changelog section |
|--------------------|-------------------|
| `Bug` | `### :bug: Bugs fixed` |
| `Feature` or `Enhancement` | `### :sparkles: New features & Enhancements` |
| `Task` | **Exclude** — internal chores, not user-facing |
| `null` (not set) | Fallback to labels: `bug` label → bugs, otherwise enhancements |
**Breaking changes override everything:** an issue with the `breaking change`
label goes under `### :boom: Breaking changes & Deprecations`, no matter its
issue type. This and `release highlight` below are the only label-based
rules — explicit exceptions to the "never by labels" principle above. The
`:boom:` subsection goes first in the version, before `:rocket:` (matching
the existing precedent).
**Highlights come from the `release highlight` label:** an issue with that
label also gets an entry under `### :rocket: Epics and highlights`, on top of
its regular entry (usually `:sparkles:`). In the section being refreshed,
`:rocket:` lists exactly the labelled issues: never pick highlights yourself
and never keep an unlabelled entry there. Product decides the highlights by
labelling issues on GitHub, so a regeneration cannot undo that choice. Older
version sections are left as they are.
**Community attribution:** if the issue or its fix PR has the
`community contribution` label, add `(by @<github_username>)` on the entry
line, **before** the issue/PR references. Use the **PR author** (the `author`
field from step 4), not the issue author:
```markdown
- Fix description of the bug (by @username) [#<ISSUE>](...) (PR: [#<PR>](...))
```
**Only closed issues are included**, even if open ones are tracked in the
milestone.
**Pairing rules:**
| Pattern | Changelog format |
|---------|-----------------|
| Closed issue + one or more fix PRs | Primary link = issue, PRs inline comma-separated |
| PR with no linked issue | Link the issue if a matching closed one exists in the milestone; otherwise skip (the issue is the changelog unit) |
| Closed issue with no fix PR in milestone | Link the issue directly, no PR reference |
> **False-positive associations:** a PR may wrongly claim to close an issue
> from another context (ancient PR, cross-project reference). If titles are
> clearly unrelated or the PR predates the issue by years, treat it as a data
> glitch and skip it.
### 5a. Verify PR merge status before writing
A closed issue may list closing PRs that were **closed without merging**
(e.g. a superseded community PR). Only **merged** PRs go in the changelog:
```bash
python3 scripts/changelog.py check-merged <ALL_PR_NUMBERS>
# also accepts: --file prs.txt, or numbers via --stdin
```
If a closing PR is closed-unmerged, find the merged PR that superseded it
(other PRs in the issue's closing list, similar titles, or pointers in the
closed PR's timeline) and reference that one instead.
### 5b. Security advisory (GHSA) entries
Advisories fixed in a release go in the changelog even though they are
**neither milestone issues nor PRs**. The GHSA ID and description come from
the user or the release notes — never from the milestone fetch.
```markdown
- Fix <user-facing description> (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX)
```
Rules: place under `### :bug: Bugs fixed` with **no issue or PR link**; do
**not** fetch or verify the URL (it may be draft/unpublished and 404); write
the description in imperative mood from the advisory title. These entries are
invisible to the automation — add them by hand, and in step 9 apply only the
backport/duplicate check to them.
### 6. Read the current CHANGES.md and run pre-flight checks
Newest version goes at the top, right after the `# CHANGELOG` header:
```markdown
## <VERSION>
### :boom: Breaking changes & Deprecations
- <breaking change or deprecation> [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
### :bug: Bugs fixed
- Fix description of the bug [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
- Fix another bug (by @contributor) [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
### :sparkles: New features & Enhancements
- Add new feature description [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
```
Format details: entries start with `- ` plus a short imperative description;
PR refs stay inline (`(PR: [#<N>](<url>))`, comma-separated for several);
`(by @<username>)` goes before the issue link; only include non-empty
sections; blank line between a section's last entry and the next title; never
duplicate an entry from an earlier version section (the earlier version wins
for backports).
**Pre-flight checks — fix violations directly in `CHANGES.md` before writing
the new section.** Reconcile every existing entry (any version section) and
every milestone candidate against the current milestone state (re-fetch, do
not trust cached data):
1. **Duplicate across versions** → remove from the current section.
2. **Stale milestone assignment** (issue moved out of this milestone) →
remove the entry (or drop it if the section does not exist yet).
3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) →
remove the entry.
4. **Issue no longer closed/deleted/Rejected** → remove the entry.
5. **Unmerged or moved PR reference** → fix the reference or remove the
entry (a PR merged in a *different* milestone is a step-9 anomaly, do not
silently remove it).
6. **Issue type changed to `Task`** → remove the entry.
7. **Breaking change misplaced** (issue has the `breaking change` label but
sits in another section) → move the entry to `:boom:`.
8. **Highlight out of sync** (`:rocket:` entry without the `release
highlight` label, or a labelled issue missing from `:rocket:`) → remove
the `:rocket:` entry or add it (step 7b). The regular entry stays.
9. **Missing valid issues** (closed, non-excluded, unreferenced anywhere) →
add them to the current section per step 5.
### 7. Build the description text
Derive it from the **issue title**, not the PR title. Strip leading emoji
prefixes (`:bug:`, `:sparkles:`, `:tada:`) and describe the user-facing
behavior:
| Issue title | Changelog description |
|-------------|----------------------|
| `Plugin API token methods fail with schema validation error on PRO` | `Fix Plugin API token methods failing with schema validation error on PRO` |
| `Comment content is not sanitized before rendering, enabling stored XSS` | `Sanitize comment content on rendering` |
| `Custom uploaded font family names are not sanitized` | `Sanitize font family names on custom uploaded fonts` |
Insert the new version section right after the `# CHANGELOG` header with the
`edit` tool and enough context for a unique match.
### 7b. Populate `:rocket: Epics and highlights` from the label
List the milestone issues that pass step 2 exclusions and carry the
`release highlight` label:
```bash
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" \
| python3 -c "import sys,json; [print(i['number'], i['title']) for i in json.load(sys.stdin) if 'release highlight' in i['labels']]"
```
Create the subsection if missing (place it before `### :sparkles:`) with one
entry per labelled issue, same text and references as its regular entry.
Every `:rocket:` entry MUST carry issue AND PR references. If no issue is
labelled, leave `:rocket:` out and tell the user: the highlights are
product's call, not the agent's. To change them, add or remove the label on
GitHub and re-run; never edit `:rocket:` by hand.
### 8. Cross-reference milestone PRs against the changelog
```bash
python3 scripts/changelog.py cross-ref "<MILESTONE>" [--changes CHANGES.md]
```
Lists merged milestone PRs missing from the changelog section (decide per PR:
add it or confirm its exclusion labels) and warns about CLOSED (unmerged) PRs
in the milestone. GHSA entries never appear here — that absence is expected.
### 9. Generate the anomaly report
```bash
python3 scripts/changelog.py report "<MILESTONE>" [--changes CHANGES.md --output CHANGES-ISSUES.md]
```
Overwrites `CHANGES-ISSUES.md` with the current state. Every number renders as
a full `[#N](https://github.com/penpot/penpot/issues/N)` or
`[#N](https://github.com/penpot/penpot/pull/N)` link.
**An anomaly is a milestone mismatch or a `:boom:` mislabel** (the changelog
pairing is misleading and a human must judge intent):
1. Issue in this milestone, referenced PR in another milestone (or none).
2. PR in this milestone, closed issue in another milestone — except an issue
with *no* milestone, which belongs to another (probably private) project
and is neither anomaly nor changelog candidate.
3. `:boom:` entry whose issue lacks the `breaking change` label. These stay
listed in the report **and** in the changelog: either label the issue or
move the entry to its regular section.
4. `:rocket:` out of sync with the `release highlight` label. Step 6
pre-flight should have fixed it: if it shows up, re-run the workflow.
**Highlight gaps are warnings, not anomalies:** missing `:rocket:` on a
released X.Y.0 (patches never carry highlights); `:rocket:` entry without
issue AND PR references. Gaps never count toward the anomaly total.
**Anything else is a rule violation**, not a report item: fix it in step 6
pre-flight. If one shows up in the report, re-run the workflow.
## Key Principles
- **Issue = changelog unit**, PR = implementation detail inline.
- **Latest version first**, below the `# CHANGELOG` header.
- **Issue Type decides the section — exclusively**, except `breaking change`
label → `:boom:` first, and `release highlight` label → also `:rocket:`.
- **User-facing descriptions** in imperative mood.
- **Community attribution** uses the PR author, placed before the issue link.
- **Only closed issues**; **Rejected** project status excludes.
- **`no changelog` and `Task`** stay out.
- **Multiple fix PRs** go comma-separated inline.
- **Duplicates**: the earlier version section wins.
- **Taiga references**: resolve to the GitHub issue via description text or
PRs mentioning the Taiga URL, then link issue + PR.
- **GHSA entries** live under `:bug:` with only the advisory URL (see 5b).
- **Re-fetch before editing**; prefer `scripts/gh.py` over raw `gh api`.
- **Verify PR merge status** (`check-merged`); PR-level exclusions apply.
- **Cross-reference PRs, not just issues** (`cross-ref`); watch for
false-positive PR-to-issue links.

View File

@ -9,7 +9,8 @@
app.common.data.macros/get-in clojure.core/get-in app.common.data.macros/get-in clojure.core/get-in
app.common.data.macros/with-open clojure.core/with-open app.common.data.macros/with-open clojure.core/with-open
app.common.data.macros/select-keys clojure.core/select-keys app.common.data.macros/select-keys clojure.core/select-keys
app.common.logging/with-context clojure.core/do} app.common.logging/with-context clojure.core/do
benches.render-wasm.errors/attempt clojure.core/do}
:hooks :hooks
{:analyze-call {:analyze-call
@ -24,7 +25,11 @@
shadow.lazy/loadable hooks.export/rumext-lazycomponent shadow.lazy/loadable hooks.export/rumext-lazycomponent
app.util.i18n/tr hooks.i18n/tr-dynamic app.util.i18n/tr hooks.i18n/tr-dynamic
app.common.i18n/tr hooks.i18n/tr-dynamic app.common.i18n/tr hooks.i18n/tr-dynamic
}} benches.render-wasm.declarations/defscene hooks.benches/defscene
benches.render-wasm.declarations/defcase hooks.benches/defcase
benches.render-wasm.runtime.commands/defcommand hooks.benches/defcommand
benches.render-wasm.runtime.protocol/resolve-or-reject hooks.benches/resolve-or-reject
benches.render-wasm.runtime.protocol/resolve-or-reject-with hooks.benches/resolve-or-reject-with}}
:output :output
{:exclude-files {:exclude-files
@ -36,7 +41,7 @@
"^(?:backend|frontend|exporter|common)/deps.edn" "^(?:backend|frontend|exporter|common)/deps.edn"
"^(?:backend|frontend|exporter|common)/scripts/" "^(?:backend|frontend|exporter|common)/scripts/"
"^(?:backend|frontend|exporter|common)/dev/" "^(?:backend|frontend|exporter|common)/dev/"
"^(?:backend|frontend|exporter|common)/test/"] "^(?:backend|frontend|exporter|common)/test/(?!performance_tests/)"]
:linter-name true} :linter-name true}
@ -101,6 +106,4 @@
:unused-binding :unused-binding
{:exclude-destructured-as true {:exclude-destructured-as true
:exclude-destructured-keys-in-fn-args false :exclude-destructured-keys-in-fn-args false}}}
}
}}

View File

@ -0,0 +1,77 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.benches
(:require [clj-kondo.hooks-api :as api]))
(defn defscene
"Analyzes scene options and its compiled builder body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcase
"Analyzes case options and its compiled operation body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and (seq body) argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcommand
"Analyzes command options and both generated functions."
[{:keys [node]}]
(let [[_ command & forms] (:children node)
[doc forms] (if (string? (api/sexpr (first forms)))
[(first forms) (next forms)]
[nil forms])
[options forms] (if (api/map-node? (first forms))
[(first forms) (next forms)]
[nil forms])
[argv & body] forms
[conditions body] (if (and (api/map-node? (first body)) (next body))
[(first body) (next body)]
[nil body])
handlers (:async (some-> options api/sexpr))
core (api/list-node
(into [(api/token-node 'fn)
(api/vector-node (mapv api/token-node handlers))]
body))
starred (api/token-node (symbol (str (api/sexpr command) "*")))
measured (get (some-> options api/sexpr) :measured-variant true)
definition (api/list-node
(into [(api/token-node 'defn) command]
(concat (when doc [doc]) [argv]
(when conditions [conditions]) [core])))]
{:node (api/list-node
(into [(api/token-node 'do)]
(concat (when options [(api/coerce (dissoc (api/sexpr options) :async))])
[definition]
(when measured
[(api/list-node [(api/token-node 'def)
starred command])]))))}))
(defn resolve-or-reject
"Analyzes the body of a Promise-returning form."
[{:keys [node]}]
(let [[_ & body] (:children node)]
{:node (api/list-node (into [(api/token-node (quote do))] body))}))
(defn resolve-or-reject-with
"Analyzes the cause handler and body. Binds `cause` in the handler only."
[{:keys [node]}]
(let [[_ cause on-error & body] (:children node)]
{:node (api/list-node
(into [(api/token-node (quote do))
(api/list-node
[(api/token-node (quote let))
(api/vector-node [cause (api/token-node nil)])
on-error])]
body))}))

View File

@ -1,3 +1,9 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.export (ns hooks.export
(:require [clj-kondo.hooks-api :as api])) (:require [clj-kondo.hooks-api :as api]))

View File

@ -10,5 +10,7 @@
:extra-indents {rumext.v2/fnc [[:inner 0]] :extra-indents {rumext.v2/fnc [[:inner 0]]
cljs.test/async [[:inner 0]] cljs.test/async [[:inner 0]]
promesa.exec/thread [[:inner 0]] promesa.exec/thread [[:inner 0]]
specify! [[:inner 0] [:inner 1]]} specify! [[:inner 0] [:inner 1]]
benches.render-wasm.runtime.commands/defcommand [[:inner 0]]
benches.render-wasm.runtime.protocol/resolve-or-reject-with [[:inner 0]]}
} }

View File

@ -85,6 +85,10 @@ target instance is running and its MCP config has been generated:
# 'main', 'ws1' and similar spellings are rejected. # 'main', 'ws1' and similar spellings are rejected.
./manage.sh start-coding-agent claude --ws 1 ./manage.sh start-coding-agent claude --ws 1
./manage.sh start-coding-agent opencode --ws 2 ./manage.sh start-coding-agent opencode --ws 2
# Run a different executable (e.g. a wrapper script) with the same MCP
# wiring. --ws and --bin may come in any order.
./manage.sh start-coding-agent claude --bin claude-alt
``` ```
Equivalents by hand (run from inside the workspace directory): Equivalents by hand (run from inside the workspace directory):

View File

@ -1,7 +1,7 @@
description: Create a report to help us improve description: An unexpected problem or behavior
name: Bug report
type: Bug
labels: ["needs triage"] labels: ["needs triage"]
name: Bug
type: Bug
body: body:
- type: markdown - type: markdown
@ -15,7 +15,7 @@ body:
- type: textarea - type: textarea
attributes: attributes:
label: Steps To Reproduce label: Steps To Reproduce
description: Steps to reproduce the behavior. description: Steps to reproduce the behavior
placeholder: | placeholder: |
Steps to reproduce the behavior: Steps to reproduce the behavior:
1. Go to '...' 1. Go to '...'
@ -26,22 +26,22 @@ body:
- type: textarea - type: textarea
id: expected id: expected
attributes: attributes:
description: A clear and concise description of what you expected to happen.
label: Expected behavior label: Expected behavior
description: A clear and concise description of what you expected to happen
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: actual id: actual
attributes: attributes:
description: A clear and concise description of what happens instead; what the bug is.
label: Actual behavior label: Actual behavior
description: A clear and concise description of what happens instead; what the bug is
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: screenshots id: screenshots
attributes: attributes:
description: If applicable, add screenshots to help explain your problem.
label: Screenshots or video label: Screenshots or video
description: If applicable, add screenshots to help explain your problem
- type: textarea - type: textarea
id: desktop id: desktop
attributes: attributes:
@ -87,4 +87,4 @@ body:
id: additional-context id: additional-context
attributes: attributes:
label: Additional context label: Additional context
description: Any other context about the problem. description: Add any other context or screenshots about the bug here

View File

@ -1 +1 @@
blank_issues_enabled: true blank_issues_enabled: false

38
.github/ISSUE_TEMPLATE/enhancement.yml vendored Normal file
View File

@ -0,0 +1,38 @@
description: Improvement to existing behavior
labels: ["needs triage"]
name: "Enhancement"
type: Enhancement
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: problem
attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
validations:
required: true
- type: textarea
id: solution
attributes:
label: Describe the solution you'd like
description: A clear and concise description of what you want to happen
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the enhancement here

26
.github/ISSUE_TEMPLATE/epic.yml vendored Normal file
View File

@ -0,0 +1,26 @@
description: An epic is a large user story that cannot be delivered as defined within a single iteration
labels: ["needs triage"]
name: "EPIC"
type: EPIC
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: description
attributes:
label: Epic description
description: Tell us which epic feature do you want to discuss
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the feature request here

View File

@ -1,7 +1,7 @@
description: Suggest an idea for this project. description: Suggest an idea for this project
labels: ["needs triage"] labels: ["needs triage"]
name: "Feature request" name: "Feature request"
type: Enhancement type: Feature
body: body:
- type: markdown - type: markdown
@ -15,24 +15,24 @@ body:
- type: textarea - type: textarea
id: problem id: problem
attributes: attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...) description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
label: Is your feature request related to a problem? Please describe.
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: solution id: solution
attributes: attributes:
description: A clear and concise description of what you want to happen. label: Describe the solution you'd like
label: Describe the solution you'd like. description: A clear and concise description of what you want to happen
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: alternatives id: alternatives
attributes: attributes:
label: Describe alternatives you've considered. label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered. description: A clear and concise description of any alternative solutions or features you've considered
- type: textarea - type: textarea
id: additional-context id: additional-context
attributes: attributes:
label: Additional context label: Additional context
description: Add any other context or screenshots about the feature request here. description: Add any other context or screenshots about the feature request here

26
.github/ISSUE_TEMPLATE/task.yml vendored Normal file
View File

@ -0,0 +1,26 @@
description: A specific piece of work
labels: []
name: "Task"
type: Task
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: task
attributes:
description: A clear and concise description of the task
label: Please describe
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the task here

View File

@ -1,20 +0,0 @@
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
# Actions are pinned to a commit SHA with a `# vX.Y.Z` comment; Dependabot
# bumps both. All updates are grouped into a single monthly PR.
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 5
labels:
- "dependencies"
- "team infra"
- "no issue required"
commit-message:
prefix: ":arrow_up:"
groups:
github-actions:
patterns:
- "*"

View File

@ -5,7 +5,7 @@ name: Auto Label and Add to Project
# code, it only reads ids from the event payload. # code, it only reads ids from the event payload.
on: on:
issues: issues:
types: [opened] types: [opened, closed]
pull_request_target: pull_request_target:
types: [opened] types: [opened]
@ -43,6 +43,9 @@ jobs:
// === 1. CONFIGURATION === // === 1. CONFIGURATION ===
const PROJECT_NUMBER = 8; // <--- Replace with your project board number const PROJECT_NUMBER = 8; // <--- Replace with your project board number
const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization
const STATUS_FIELD_ID = 'PVTSSF_lADOAcyBPM4BV2pqzhRPY48';
const STATUS_BACKLOG = 'f75ad846';
const STATUS_TECH_REVIEW = 'b0f45db9';
const OWNER = context.repo.owner; const OWNER = context.repo.owner;
const REPO = context.repo.repo; const REPO = context.repo.repo;
@ -50,6 +53,100 @@ jobs:
const isPR = !!context.payload.pull_request; const isPR = !!context.payload.pull_request;
const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id; const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id;
// A completed issue is already closed by GitHub. Route its Main
// project status according to whether QA needs to verify it.
if (context.eventName === 'issues' && context.payload.action === 'closed') {
const issue = context.payload.issue;
if (issue.state_reason !== 'completed') {
console.log(`Issue #${issue.number} was closed as ${issue.state_reason}; leaving project status unchanged.`);
return;
}
const targetStatus = issue.labels.some(({ name }) => name === 'team-qa')
? 'Ready for QA'
: 'Product Review';
const projectQuery = `
query($owner: String!, $number: Int!) {
organization(login: $owner) {
projectV2(number: $number) {
id
fields(first: 100) {
nodes {
... on ProjectV2SingleSelectField {
id
name
options { id name }
}
}
}
}
}
}
`;
const projectRes = await github.graphql(projectQuery, {
owner: OWNER,
number: PROJECT_NUMBER
});
const project = projectRes.organization.projectV2;
if (!project) {
throw new Error(`Could not find Main project #${PROJECT_NUMBER}.`);
}
const statusField = project.fields.nodes.find(({ name }) => name === 'Status');
const statusOption = statusField?.options.find(({ name }) => name === targetStatus);
if (!statusField || !statusOption) {
throw new Error(`Could not find Status option "${targetStatus}" in Main.`);
}
const issueProjectItemsQuery = `
query($issueId: ID!) {
node(id: $issueId) {
... on Issue {
projectItems(first: 100) {
nodes {
id
project { id }
}
}
}
}
}
`;
const issueProjectItemsRes = await github.graphql(issueProjectItemsQuery, {
issueId: contentId
});
const projectItems = issueProjectItemsRes.node?.projectItems?.nodes || [];
const projectItem = projectItems.find(
({ project: itemProject }) => itemProject.id === project.id
);
if (!projectItem) {
console.log(`Issue #${issue.number} is not in Main project #${PROJECT_NUMBER}; skipping status update.`);
return;
}
const updateStatusMutation = `
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId
itemId: $itemId
fieldId: $fieldId
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item { id }
}
}
`;
await github.graphql(updateStatusMutation, {
projectId: project.id,
itemId: projectItem.id,
fieldId: statusField.id,
optionId: statusOption.id
});
console.log(`Issue #${issue.number} closed; Main status set to "${targetStatus}".`);
return;
}
// Define your labels here // Define your labels here
const labelToApply = 'needs triage'; const labelToApply = 'needs triage';
@ -88,6 +185,65 @@ jobs:
} }
`; `;
await github.graphql(addToProjectMutation, { projectId, contentId }); let itemId;
try {
const addRes = await github.graphql(addToProjectMutation, { projectId, contentId });
itemId = addRes.addProjectV2ItemById.item.id;
} catch (error) {
const messages = [
error && error.message,
...(Array.isArray(error && error.errors)
? error.errors.map(e => e && e.message)
: [])
].filter(Boolean).join('\n');
if (!messages.includes('Content already exists in this project')) {
throw error;
}
console.log('Item already in project, fetching existing item...');
const existing = await github.graphql(
`query($contentId: ID!) {
node(id: $contentId) {
... on Issue {
projectItems(first: 50) {
nodes { id project { id } }
}
}
... on PullRequest {
projectItems(first: 50) {
nodes { id project { id } }
}
}
}
}`,
{ contentId }
);
const nodes = (existing.node && existing.node.projectItems && existing.node.projectItems.nodes) || [];
const match = nodes.find(n => n.project.id === projectId);
if (!match) {
throw new Error(`Item already in a project, but not found in project ${projectId}`);
}
itemId = match.id;
}
// === 4. SET PROJECT STATUS ===
const statusOptionId = isPR ? STATUS_TECH_REVIEW : STATUS_BACKLOG;
const statusName = isPR ? 'Tech Review' : 'Backlog';
console.log(`Setting status to "${statusName}" for ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
await github.graphql(
`mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId,
itemId: $itemId,
fieldId: $fieldId,
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item {
id
}
}
}`,
{ projectId, itemId, fieldId: STATUS_FIELD_ID, optionId: statusOptionId }
);
console.log("Automation successfully completed!"); console.log("Automation successfully completed!");

View File

@ -38,13 +38,13 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry (push destination) - name: Login to Docker Registry (push destination)
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry (base image pull) - name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -200,7 +200,7 @@ jobs:
ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }} ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }}
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
@ -210,7 +210,7 @@ jobs:
# images from DockerHub for unregistered users. # images from DockerHub for unregistered users.
# https://docs.docker.com/docker-hub/usage/ # https://docs.docker.com/docker-hub/usage/
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
@ -222,7 +222,7 @@ jobs:
# reuses the same PUB_DOCKER_* credentials as the DockerHub # reuses the same PUB_DOCKER_* credentials as the DockerHub
# login above. # login above.
- name: Login to Docker Hardened Images registry (base image pull) - name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -301,7 +301,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
@ -371,7 +371,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}

View File

@ -114,20 +114,20 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to Docker Registry - name: Login to Docker Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: ${{ secrets.DOCKER_REGISTRY }} registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -191,13 +191,13 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to DockerHub Registry - name: Login to DockerHub Registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }} password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -242,7 +242,7 @@ jobs:
persist-credentials: false persist-credentials: false
- name: Login to Docker Hardened Images registry - name: Login to Docker Hardened Images registry
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
with: with:
registry: dhi.io registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }} username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -34,7 +34,7 @@ permissions: {}
env: env:
ACTIONLINT_VERSION: '1.7.12' ACTIONLINT_VERSION: '1.7.12'
ZIZMOR_VERSION: '1.31.1' ZIZMOR_VERSION: '1.32.0'
jobs: jobs:
lint: lint:

View File

@ -48,10 +48,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that # No dependency cache here: a deploy must not restore a cache that
# other workflows can write to. # other workflows can write to.
- name: Setup pnpm + Node.js - name: Setup pnpm + Node.js
uses: pnpm/setup@a0881977cd8ea73b1ed212236c99ca10dcc80cf4 # v3.1.1 uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
with: with:
working-directory: plugins working-directory: plugins
runtime: node@24.22.1 runtime: node@24.22.3
install: false install: false
- name: Install deps - name: Install deps

View File

@ -46,10 +46,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that # No dependency cache here: a deploy must not restore a cache that
# other workflows can write to. # other workflows can write to.
- name: Setup pnpm + Node.js - name: Setup pnpm + Node.js
uses: pnpm/setup@a0881977cd8ea73b1ed212236c99ca10dcc80cf4 # v3.1.1 uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
with: with:
working-directory: plugins working-directory: plugins
runtime: node@24.22.1 runtime: node@24.22.3
install: false install: false
- name: Install deps - name: Install deps

View File

@ -0,0 +1,76 @@
name: Require Milestone on Linked Issues
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- ready_for_review
- converted_to_draft
workflow_dispatch:
permissions:
contents: read
pull-requests: read
issues: read
jobs:
check-linked-issues:
name: Require Milestones on Linked Issues
runs-on: ubuntu-latest
steps:
- name: Check linked issues
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
IS_DRAFT: ${{ github.event.pull_request.draft }}
REPOSITORY: ${{ github.repository }}
run: |
if [ "$IS_DRAFT" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
OWNER="${REPOSITORY%%/*}"
REPO="${REPOSITORY#*/}"
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
ISSUES=$(gh api graphql \
-F owner="$OWNER" \
-F repo="$REPO" \
-F number="$PR_NUMBER" \
-f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
closingIssuesReferences(first: 100) {
nodes {
number
title
url
milestone {
title
}
}
}
}
}
}
')
MISSING=$(echo "$ISSUES" | jq -r '
.data.repository.pullRequest.closingIssuesReferences.nodes[]
| select(.milestone == null)
| "- #\(.number): \(.title) (\(.url))"
')
if [ -n "$MISSING" ]; then
echo "::error::The following linked issues have no milestone:"
echo "$MISSING"
exit 1
fi
echo "All linked issues have a milestone."

45
.github/workflows/require-milestone.yml vendored Normal file
View File

@ -0,0 +1,45 @@
name: Require Milestone
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- milestoned
- demilestoned
- ready_for_review
- converted_to_draft
permissions:
contents: read
pull-requests: read
jobs:
check-milestone:
name: Require Milestone
runs-on: ubuntu-latest
steps:
- name: Check milestone
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
pr_api="repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}"
is_draft=$(gh api "$pr_api" --jq '.draft')
milestone=$(gh api "$pr_api" --jq '.milestone.title // ""')
if [ "$is_draft" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
if [ -z "$milestone" ]; then
echo "::error::This pull request must have a milestone."
exit 1
fi
echo "Milestone assigned: $milestone"

View File

@ -49,7 +49,7 @@ jobs:
- name: Format - name: Format
working-directory: ./render-wasm working-directory: ./render-wasm
run: | run: |
cargo fmt --check ./fmt --check
- name: Lint - name: Lint
working-directory: ./render-wasm working-directory: ./render-wasm

2
.gitignore vendored
View File

@ -121,7 +121,7 @@ opencode.json
/.opencode/reports /.opencode/reports
/.opencode/prompts /.opencode/prompts
/.ci-logs /.ci-logs
/tmp/ /tmp
/.codex/ /.codex/
/scripts/__pycache__/ /scripts/__pycache__/
/tools/__pycache__ /tools/__pycache__

2
.nvmrc
View File

@ -1 +1 @@
v24.22.1 v24.22.3

View File

@ -12,14 +12,15 @@ The system has:
- **Three producers** (they create events): the Penpot backend (RPC layer), the Penpot frontend (browser app), and the Nitrate admin-console, which sends both server-side and client-side events. - **Three producers** (they create events): the Penpot backend (RPC layer), the Penpot frontend (browser app), and the Nitrate admin-console, which sends both server-side and client-side events.
- **One central buffer**: the Postgres `audit_log` table in Penpot. It is temporary: it holds events for minutes/hours until they are archived, then they are deleted. - **One central buffer**: the Postgres `audit_log` table in Penpot. It is temporary: it holds events for minutes/hours until they are archived, then they are deleted.
- **Four consumers** (they read events): webhooks, error reporters, telemetry, and the external archive in Nexus. - **Four consumers** (they read events): webhooks, error reporters, telemetry, and the external archives in Nexus and in the admin console.
- **One long-term archive**: **Nexus**, a separate service that stores events for the long term. Archiving to **Nitrate** is in development and **does not exist on the current branch** (see section 11). - **Two long-term archives**: **Nexus**, a separate service that stores events for the long term, and the **admin console** (Nitrate), which ingests its own allowlisted events through a second archive job (see section 12).
``` ```
Penpot frontend ─ POST push-audit-events (public) ─────────▶┐ Penpot frontend ─ POST push-audit-events (public) ─────────▶┐
Nitrate admin-console (browser) ─ POST push-audit-events ──▶│ Nitrate admin-console (browser) ─ POST push-audit-events ──▶│
Nitrate admin-console (server) ─ POST push-audit-events ──▶│ ─▶ audit_log table (buffer) ── cron every 5m ──▶ Nexus (archive) Nitrate admin-console (server) ─ POST push-audit-events ──▶│ ─▶ audit_log table (buffer) ──┬─ cron 0s every 5m ──▶ Nexus (archive)
Penpot backend (RPC wrap-audit) ─ submit ───────────────────┘ │ ├──▶ webhooks Penpot backend (RPC wrap-audit) ─ submit ───────────────────┘ │ └─ cron 30s every 5m ─▶ admin console (archive)
│ ├──▶ webhooks
│ ├──▶ error reporters │ ├──▶ error reporters
└──▶ telemetry (shadow rows) └──▶ telemetry (shadow rows)
``` ```
@ -44,7 +45,7 @@ Rule of thumb: if you need to know for sure "did this user do X?", look at `sour
## 3. The `audit_log` table: a buffer, not a store ## 3. The `audit_log` table: a buffer, not a store
The Penpot `audit_log` table **is not the final destination**. It is a waiting tray: events sit here until the archive cron sends them to Nexus and the cleanup cron (GC) deletes them. The Penpot `audit_log` table **is not the final destination**. It is a waiting tray: events sit here until every active consumer (Nexus, admin console) confirms them, and whoever finishes last deletes the row.
Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table.sql` and later ones): Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table.sql` and later ones):
@ -56,15 +57,16 @@ Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table
- `source` text: `backend` / `frontend` for full rows, `telemetry:backend` / `telemetry:frontend` for anonymized copies (see section 10). - `source` text: `backend` / `frontend` for full rows, `telemetry:backend` / `telemetry:frontend` for anonymized copies (see section 10).
- `ip_addr` inet: real IP (or `0.0.0.0` on telemetry rows). - `ip_addr` inet: real IP (or `0.0.0.0` on telemetry rows).
- `props` / `context` jsonb holding transit-encoded maps: action data and metadata (versions, browser, request-id...). - `props` / `context` jsonb holding transit-encoded maps: action data and metadata (versions, browser, request-id...).
- `archived_at` timestamptz: NULL until Nexus confirms receipt; only then the row can be deleted. - No `archived_at`: the `audit_log_pending` queue (migration `0157`, consumer `1` = Nexus, `2` = admin console, ids never reused) tracks what is left. Every record is born pending only for the consumers enabled at insert time (never a fixed both): cheaper on single-consumer hosts and true to the design doc. Consequence, accepted on purpose: rows born with no send flag on get no pendings, so enabling archiving later does not pick them up — the sweep deletes them unarchived.
Indexes (each consumer has its own, so a slow consumer never blocks the others): Indexes (each consumer has its own, so a slow consumer never blocks the others):
- Partial `created_at WHERE archived_at IS NULL`: used by the archive scan. - `audit_log_pending` PK `(consumer, audit_log_id)`: each consumer owns its queue at constant cost, even with millions pending.
- Partial `archived_at WHERE archived_at IS NOT NULL`: used by the GC. - `audit_log (created_at, id)` (migration `0157`): the chronological drain order (`ORDER BY created_at ASC, id ASC`; same total order for both consumers, so concurrent drains cannot deadlock).
- `audit_log_pending (audit_log_id)`: serves the "is anything still pending?" check of the final delete.
- `(source, created_at)`: used by the telemetry scan. - `(source, created_at)`: used by the telemetry scan.
Life of a row: `INSERT` → (every 5 min) archive sets `archived_at = now()` **only if Nexus answers 204**, in the same transaction → (every 5 min) GC runs `DELETE WHERE archived_at IS NOT NULL` **with no age filter**. So archive must run before GC, or data is deleted without being sent. Life of a row: `INSERT` (row plus one pending per enabled consumer, same transaction; a rollback takes both) → (every 5 min, Nexus at second 0 and admin console at second 30) each archive job sends its own 128-row batches oldest-first and deletes its pendings in the same transaction → whoever deletes the last pending deletes the row. A batch the destination refuses rolls back and stays for the next tick. Pendings of consumers that are off are reaped at the start of each job run (covers later deactivations), and orphan rows are swept at the end of each run, so there is no separate GC task. The FK is plain `DEFERRABLE` (never `INITIALLY DEFERRED`): every deleter removes pendings before rows, including the telemetry task's own deletes.
## 4. Anatomy of an event ## 4. Anatomy of an event
@ -180,7 +182,7 @@ Who uses it:
Failure policy: every send is best-effort. A failure never rolls back the business action; it is only recorded for debugging. Failure policy: every send is best-effort. A failure never rolls back the business action; it is only recorded for debugging.
Backend side (`backend/src/app/rpc/management/nitrate.clj` `::push-audit-events`): `::audit/skip`, `::rpc/auth false` (trust comes from the shared key, not from a user session). It builds context with `audit/prepare-context-from-request` + `request-id`, takes the given `profile-id` as-is (no coercion here, must already be a uuid), sets `tracked-at = request-at`, defaults `type` to `"action"`, and calls `audit/submit` (which defaults `source` to `"backend"`). So these rows land as **`source = backend`**: they are audited like backend events because a trusted service set the author. Backend side (`backend/src/app/rpc/management/nitrate.clj` `::push-audit-events`): `::audit/skip`, `::rpc/auth false` (trust comes from the shared key, not from a user session; only the `admin-console` key id is allowed). It builds context with `audit/prepare-context-from-request` + `request-id`, takes the given `profile-id` as-is (no coercion here, must already be a uuid), sets `tracked-at = request-at`, defaults `type` to `"action"`, and calls `audit/submit` (which defaults `source` to `"backend"`). So these rows land as **`source = backend`**: they are audited like backend events because a trusted service set the author.
### 7b. Browser channel (best-effort, like frontend events) ### 7b. Browser channel (best-effort, like frontend events)
@ -250,9 +252,9 @@ Result: crash reports only exist if the frontend collector runs and `push-audit-
## 11. Consumers III: telemetry (shadow rows) ## 11. Consumers III: telemetry (shadow rows)
Telemetry reuses the same table with **anonymized shadow rows** (`source LIKE 'telemetry:%'`): day-truncated times, `0.0.0.0` IPs, props reduced to uuid/boolean/number values plus a few allowed fields (`lang`, `auth-backend`, derived `email-domain` — never raw emails), and a minimal context allowlist. Full + shadow rows can exist together per event when both flags are on; that duplication is on purpose. Telemetry reuses the same table with **anonymized shadow rows** (`source LIKE 'telemetry:%'`): day-truncated times, `0.0.0.0` IPs, props reduced to uuid/boolean/number values plus a few allowed fields (`source`, `lang`, `auth-backend`, `activation`, derived `email-domain` — never raw emails), and a minimal context allowlist. Full + shadow rows can exist together per event when both flags are on; that duplication is on purpose.
Filters (`filter-telemetry-props` / `filter-telemetry-context`): by default only uuid/boolean/number with a simple key; exceptions: login/register/update-profile and frontend `identify` keep `lang`/`auth-backend`/`email-domain`; `organization-sso-auth-failed` keeps `failure-reason` if it is a known one; `instance-start` passes whole; frontend `navigate` keeps `[:route :file-id :team-id :page-id]`. Free text drops from telemetry but **stays in the full row and in Nexus**. Filters (`filter-telemetry-props` / `filter-telemetry-context`): by default only uuid/boolean/number with a simple key; exceptions: login/register/update-profile and frontend `identify` keep `lang`/`auth-backend`/`source`/`activation`/`email-domain`; `organization-sso-auth-failed` keeps `failure-reason` if it is a known one; `instance-start` passes whole; frontend `navigate` keeps `[:route :file-id :team-id :page-id]`. Subscription actions have event-specific contracts: backend and frontend `create-subscription` / `create-trial-subscription` preserve the default uuid/boolean/number props plus `type`, `deployment`, `quantity` and `subscription-mode`; backend `update-subscription` preserves only `type`, `deployment`, `previous-status`, `status`, `previous-cancel-at-period-end`, `cancel-at-period-end`, `previous-cancel-at`, `cancel-at` and `changed-fields`; backend `subscription-summary` preserves only `type`, `deployment`, `instance-id`, `snapshot-date`, the per-status `*-count` values, `total-count`, `sync-succeeded`, `sync-failed` and `snapshot-complete`. These subscription exceptions apply only to action events. Other free text drops from telemetry but **stays in the full row and in Nexus**. In `register-profile`, `activation` is a prop (it is what reaches PostHog). `action` exists only in the `context` of the full `register-profile-retry` rows; `filter-telemetry-context` drops it from the anonymised copy, which keeps only `:version`, `:initiator`, `:client-version` and `:client-user-agent`.
Shipping (`backend/src/app/tasks/telemetry.clj`): the cron sends shadows to `:telemetry-uri` as JSON in 10k batches and **deletes** them on success; it purges leftovers older than 7 days. It also sends a legacy report (counters, teams, email domains...). On official hosts nothing is collected or sent (`telemetry-excluded?` covers `penpot.app`/`penpot.dev`). Shipping (`backend/src/app/tasks/telemetry.clj`): the cron sends shadows to `:telemetry-uri` as JSON in 10k batches and **deletes** them on success; it purges leftovers older than 7 days. It also sends a legacy report (counters, teams, email domains...). On official hosts nothing is collected or sent (`telemetry-excluded?` covers `penpot.app`/`penpot.dev`).
@ -271,17 +273,19 @@ Nexus dispatches on `:type`: `"telemetry-events"` batches are coerced and insert
- **No personal data is expected.** Profile ids are random UUIDs with no PII, so the telemetry consumer needs no login guard; grouping is by `instance`, not by person (see posthog-telemetry below). - **No personal data is expected.** Profile ids are random UUIDs with no PII, so the telemetry consumer needs no login guard; grouping is by `instance`, not by person (see posthog-telemetry below).
- **The endpoint always answers 200.** Failures are swallowed and only counted (`nexus_telemetry_errors_total`), so a bad batch never breaks the sending instance; successes are counted (`nexus_telemetry_events_total`, `nexus_telemetry_reports_total`). A body the parser cannot read never reaches the handler: it answers 400 and counts in `nexus_http_malformed_requests_total`. Old instances may send a fressian+zstd base64 blob instead of a plain vector; Nexus decodes both (50 MiB cap against decompression bombs). - **The endpoint always answers 200.** Failures are swallowed and only counted (`nexus_telemetry_errors_total`), so a bad batch never breaks the sending instance; successes are counted (`nexus_telemetry_events_total`, `nexus_telemetry_reports_total`). A body the parser cannot read never reaches the handler: it answers 400 and counts in `nexus_http_malformed_requests_total`. Old instances may send a fressian+zstd base64 blob instead of a plain vector; Nexus decodes both (50 MiB cap against decompression bombs).
## 12. Archive to Nexus (today) and to Nitrate (future) ## 12. Archive to Nexus and to the admin console
### Nexus: how it works today ### How it works today
Nexus is the long-term store. Every 5 min the `:audit-log-archive` cron takes 128-row chunks of unarchived rows (`FOR UPDATE SKIP LOCKED`, ordered by `created_at`), POSTs them as transit `{:events [...]}` with `x-shared-key: "nexus <key>"` (`:nexus-shared-key`, or derived from the instance secret) and `origin` = public-uri, and marks `archived_at = now()` **only on HTTP 204**, in the same transaction. Anything else is retried next run; flag on but no URI raises `:task-not-configured`. It loops with 100 ms pauses until empty. Every 5 min each archive job drains its own queue (`:audit-log-consumer-nexus` on flag `:nexus`, cron `0 */5 * * * ?`; `:audit-log-consumer-admin-console` on flag `:admin-console`, cron `30 */5 * * * ?`, 30s offset so the two drains never start on the same second), in 128-row chunks taken oldest-first (`ORDER BY created_at ASC, id ASC`) with `FOR UPDATE OF audit_log` (`SET LOCAL lock_timeout = '30s'`) inside the same transaction that sends and completes them, with 100 ms pauses between chunks. Batches arrive at Nexus in chronological order (it numbers by arrival for order-sensitive consumers). Nexus receives transit `{:events [...]}` with `x-shared-key: "nexus <key>"` (`:nexus-shared-key`, or derived from the instance secret) and `origin` = public-uri; flag on but no URI raises `:task-not-configured`. The admin console receives only its allowlisted event names (enriched with team names, props emails, and optional top-level `:profile-email` / `:profile-name` from the actor `:profile-id`); anything filtered out counts as done for the admin-console queue. A chunk whose destination refuses it (Nexus non-204 raises `:consumer-chunk-refused`) or throws (transport failure, admin-console error) rolls back and stays for the next tick, and the job aborts as failed: it archives what it can, and the cron retries the rest. Committed chunks stay sent. The send fns return counts or throw carrying the attempted count as `:events`; `run-single-consumer` records the `failed` metric from it, outside every chunk transaction (a local DB failure carries no `:events` and stays uncounted — it is not a destination failure). No `warn` before the throw: the runner reports the aborted job. Each job touches only its own queue, so a fallen Nexus never starves the admin console (nor vice versa).
A chunk is **one POST** and the task advances only on 204, so a single oversized event wedges the whole queue: a crash report whose message embeds a whole payload (a serialized file, for example) can push the request past the proxy body limit and the rejection blocks every later event until it is pruned by hand. Frontend crash reports are therefore size-bounded at the source (`mem:frontend/routing-app-shell-subtleties`): `app.common.exceptions/get-hint` and the CLJS `format-throwable` clamp exception messages (`max-message-length`). A chunk is **one POST** and the task advances only on 204, so a single oversized event wedges the whole queue: a crash report whose message embeds a whole payload (a serialized file, for example) can push the request past the proxy body limit and the rejection blocks every later event until it is pruned by hand. Frontend crash reports are therefore size-bounded at the source (`mem:frontend/routing-app-shell-subtleties`): `app.common.exceptions/get-hint` and the CLJS `format-throwable` clamp exception messages (`max-message-length`).
On the other side, Nexus checks `origin` against its allowlist (403 if not allowed), coerces the events, and inserts them into its own `audit_log` (partitioned by `received_at`, with an incrementing sequence number and `source`). Watch the rename: Penpot's `created-at` is stored as **`received-at`** in Nexus; Nexus's `created-at` is when it was created there. `tracked-at` keeps the origin time. Telemetry batches arriving at `telemetry.penpot.app` land in the same table through the telemetry endpoint above. On the other side, Nexus checks `origin` against its allowlist (403 if not allowed), coerces the events, and inserts them into its own `audit_log` (partitioned by `received_at`, with an incrementing sequence number and `source`). Watch the rename: Penpot's `created-at` is stored as **`received-at`** in Nexus; Nexus's `created-at` is when it was created there. `tracked-at` keeps the origin time. Telemetry batches arriving at `telemetry.penpot.app` land in the same table through the telemetry endpoint above.
Concurrent runs are safe: rows are locked with `FOR UPDATE SKIP LOCKED` inside the same transaction that sends and marks them, so two archivers running at once each take different rows. Only rows Nexus acknowledges (HTTP 204) get marked; anything else is retried on the next run. Concurrent jobs are safe: both lock rows in `created_at ASC, id ASC` order (one total order, deliberately blocking — no `SKIP LOCKED`, so archive arrival stays chronological), so the second waits for the first to commit and then sees whether it was last. Only acknowledged work deletes pendings; an admin-console failure never blocks the Nexus send of the same rows (each job owns its queue).
Observability (`backend/src/app/loggers/audit/consumer_task.clj`, definitions in `backend/src/app/main.clj:default-metrics`): one `dbg` summary per job run (`:consumer` label, sent/skipped/chunks, reaped disabled pendings, swept orphans, `:elapsed` as `"1.5s"` via `ct/format-duration`, frozen in a strict binding first because log props evaluate lazily in the logger thread), one `dbg` per admin-console send call (`ready` with totals, `sent`/`failed` with counts), plus `dbg` when the queue is empty or the job's send flag is off. Queue depth is Grafana's business: it queries the database directly, so the run performs no `GROUP BY` for the logs. No `warn` before a throw: the runner reports the aborted job (refusal details travel in the error data). Metrics, all recorded with bounded labels only: `penpot_audit_consumer_events_total{ destination = nexus|admin-console, outcome = archived|failed|skipped }` (`skipped` = filtered by the admin-console allowlist) and `penpot_audit_consumer_discarded_total{ reason = reaped|swept }` (entries discarded without archiving: `reaped` counts pendings dropped for disabled consumers, `swept` counts rows deleted while nobody waited for them — the silent R3 loss, now dashboard-visible instead of log-only). Timing needs no custom histogram: the jobs substrate already records execution time per job (`jobs-execution-timing`, split by job name), so each archive job is timed as itself. No queue-depth gauge: pending depth is read off the replica database directly. The bulk frontend ingest counts `penpot_audit_ingest_events_total{ kind = full|shadow, outcome = inserted|failed }` (both faces share one transaction, so a failure drops the batch on both). Archive success counts are recorded by `run-single-consumer` on the drain totals: a chunk that returns normally committed its transaction (a rollback always throws instead), so they cannot inflate. Logs never carry event data (`props`, `context`, `profile-id`, `ip-addr`, `team-name`, emails): only counts, statuses and the uri host.
### How Nexus consumes the data (and forwards to PostHog) ### How Nexus consumes the data (and forwards to PostHog)
@ -291,20 +295,16 @@ Stored rows are not the end: a consumer pipeline reads them back in sequence ord
2. **`posthog`** (runs after `local`): forwards our own SaaS archived events to PostHog. It sends an allowlist of backend track events (teams, members, projects, files, comments, Nitrate/SSO, ...) plus frontend events minus a noise exclusion list, grouped by **team**, with person identify data from the local profile mirror and snake_case props. Both paths include the `initiator` property when the stored context has one, so PostHog can split Penpot vs admin-console traffic. 2. **`posthog`** (runs after `local`): forwards our own SaaS archived events to PostHog. It sends an allowlist of backend track events (teams, members, projects, files, comments, Nitrate/SSO, ...) plus frontend events minus a noise exclusion list, grouped by **team**, with person identify data from the local profile mirror and snake_case props. Both paths include the `initiator` property when the stored context has one, so PostHog can split Penpot vs admin-console traffic.
3. **`posthog-telemetry`**: forwards events with `source = telemetry:frontend|telemetry:backend` to a separate PostHog project. Because these rows are already anonymous (no IPs, truncated times, filtered props), grouping is by **instance** (the self-hosted server, from `context.instance-id`), with one `$groupidentify` event per instance per chunk so PostHog learns per-instance properties (version, URI, flags). Each consumer has its own PostHog credentials. 3. **`posthog-telemetry`**: forwards events with `source = telemetry:frontend|telemetry:backend` to a separate PostHog project. Because these rows are already anonymous (no IPs, truncated times, filtered props), grouping is by **instance** (the self-hosted server, from `context.instance-id`), with one `$groupidentify` event per instance per chunk so PostHog learns per-instance properties (version, URI, flags). Each consumer has its own PostHog credentials.
So the full journey is: Penpot buffer → Nexus `audit_log` (via archive cron or telemetry endpoint) → consumers → PostHog (SaaS events grouped by team, telemetry grouped by instance) plus local analytics views and materialized KPIs. So the full journey is: Penpot buffer → Nexus `audit_log` (via either archive cron or the telemetry endpoint) → consumers → PostHog (SaaS events grouped by team, telemetry grouped by instance) plus local analytics views and materialized KPIs.
### Nitrate: what exists and what is planned ### Admin console: the second consumer
Today **only Nexus archives**. Archiving Penpot → Nitrate is in development and not on this branch. Penpot → admin-console archiving lives in its own job (`:audit-log-consumer-admin-console`, consumer `2`, flag `:admin-console`, params closed empty): it drains its own pending queue and ingests allowlisted events through `nitrate/call :ingest-audit-log`, enriched as described above. What also exists (do not confuse with archiving):
What already exists (do not confuse with archiving): - **Nitrate → Penpot**: `POST /api/management/methods/push-audit-events` (`backend/src/app/rpc/management/nitrate.clj`) lets the trusted Nitrate backend inject events into the Penpot audit log (no RPC auth, `::audit/skip`, request context + `submit`; only the `admin-console` shared-key id can call it). Section 7a is the main user.
- **Nitrate → Penpot**: `POST /api/management/methods/push-audit-events` (`backend/src/app/rpc/management/nitrate.clj`) lets the trusted Nitrate backend inject events into the Penpot audit log (no RPC auth, `::audit/skip`, request context + `submit`). Section 7a is the main user.
- **Admin-console browser → collector**: usage events from the admin-console UI traveling through the normal public `push-audit-events` (section 7b). - **Admin-console browser → collector**: usage events from the admin-console UI traveling through the normal public `push-audit-events` (section 7b).
- **Penpot frontend Nitrate bits**: membership/subscription events (`frontend/src/app/main/data/nitrate.cljs`, `nitrate-audit/*`) through the same public endpoint. - **Penpot frontend Nitrate bits**: membership/subscription events (`frontend/src/app/main/data/nitrate.cljs`, `nitrate-audit/*`) through the same public endpoint.
The future Penpot → Nitrate archive will follow the Nexus pattern (cron reading the buffer and POSTing batches), but there is no contract or flag on this branch yet. When it lands, document here: endpoint, auth, format, which `source` values travel, and how it shares the GC.
## 13. Flags and config ## 13. Flags and config
Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=enable-<name>`): Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=enable-<name>`):
@ -312,13 +312,13 @@ Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=ena
| Flag | What it does | | Flag | What it does |
|---|---| |---|---|
| `:audit-log` | Stores full rows; turns on `wrap-audit` and frontend ingest | | `:audit-log` | Stores full rows; turns on `wrap-audit` and frontend ingest |
| `:audit-log-archive` | Turns on the cron that sends to Nexus (needs `:audit-log-archive-uri`) | | `:nexus` | Send flag of the Nexus archive job `:audit-log-consumer-nexus` (cron `0 */5 * * * ?`; needs `:nexus-uri` (ingest path `api/audit` appended by the job)) |
| `:audit-log-gc` | Turns on the cron that deletes archived rows | | `:admin-console` | Send flag of the admin-console archive job `:audit-log-consumer-admin-console` (cron `30 */5 * * * ?`, 30s after Nexus); also the module flag for the admin-console integration |
| `:audit-log-logger` | Structured `app.audit` log per event | | `:audit-log-logger` | Structured `app.audit` log per event |
| `:error-reporting` | Starts the error reporters (off by default); database reporter on flag alone, Mattermost on flag plus webhook URL | | `:error-reporting` | Starts the error reporters (off by default); database reporter on flag alone, Mattermost on flag plus webhook URL |
| `:telemetry` | Stores shadow rows and turns on shipping (auto with `:telemetry-enabled`) | | `:telemetry` | Stores shadow rows and turns on shipping (auto with `:telemetry-enabled`) |
Config (`backend/src/app/config.clj`): `:audit-log-archive-uri`, `:telemetry-uri` (default `https://telemetry.penpot.app/`), `:nexus-shared-key` / instance secret, `:public-uri` (sent as `origin` to Nexus). Config (`backend/src/app/config.clj`): `:nexus-uri`, `:telemetry-uri` (default `https://telemetry.penpot.app/`), `:nexus-shared-key` / instance secret, `:public-uri` (sent as `origin` to Nexus).
## 14. Practical guide: add or change events ## 14. Practical guide: add or change events
@ -337,6 +337,6 @@ Config (`backend/src/app/config.clj`): `:audit-log-archive-uri`, `:telemetry-uri
- Penpot frontend producer: `frontend/src/app/main/data/event.cljs` (collector, buffer, send), `frontend/src/app/main/repo.cljs` + `frontend/src/app/util/http.cljs` (headers), `frontend/src/app/main/data/nitrate.cljs` (Nitrate membership events). - Penpot frontend producer: `frontend/src/app/main/data/event.cljs` (collector, buffer, send), `frontend/src/app/main/repo.cljs` + `frontend/src/app/util/http.cljs` (headers), `frontend/src/app/main/data/nitrate.cljs` (Nitrate membership events).
- Admin-console producer (private repo, no paths cited here): server-side organization/subscription events through the management `push-audit-events`, browser-side UI events through the public `push-audit-events`. - Admin-console producer (private repo, no paths cited here): server-side organization/subscription events through the management `push-audit-events`, browser-side UI events through the public `push-audit-events`.
- Consumers: `backend/src/app/loggers/webhooks.clj`, `backend/src/app/loggers/database.clj`, `backend/src/app/loggers/mattermost.clj`, `backend/src/app/tasks/telemetry.clj`. - Consumers: `backend/src/app/loggers/webhooks.clj`, `backend/src/app/loggers/database.clj`, `backend/src/app/loggers/mattermost.clj`, `backend/src/app/tasks/telemetry.clj`.
- Archive/GC: `backend/src/app/loggers/audit/archive_task.clj`, `backend/src/app/loggers/audit/gc_task.clj`, wiring and crons in `backend/src/app/main.clj`. - Archive (two jobs sharing one namespace + pending queue, no GC): `backend/src/app/loggers/audit/consumer_task.clj` (`:audit-log-consumer-nexus` / `:audit-log-consumer-admin-console` job-defs, shared drain/reap/sweep helpers), producer hook in `backend/src/app/loggers/audit.clj` (`append-audit-entry`, single `queue-pendings` helper) plus the bulk ingest in `backend/src/app/rpc/commands/audit.clj` (`handle-events`), metrics/cron wiring in `backend/src/app/main.clj` (one cron entry per job, Nexus at second 0 and admin console at second 30, each gated by its own send flag), single migration `0157` (queue, drop of `archived_at`, covering `(created_at, id)` index for the chronological scan; no backfill — pre-existing rows are swept unsent, an accepted loss).
- Nexus (private repo, no paths cited here): archive ingest, ordered consumers (local profile mirror, PostHog SaaS events, PostHog telemetry), telemetry endpoint, analytics views. - Nexus (private repo, no paths cited here): archive ingest, ordered consumers (local profile mirror, PostHog SaaS events, PostHog telemetry), telemetry endpoint, analytics views.
- Tests: `backend_tests/rpc-audit-test.clj`, `rpc-team-test/accept-organization-invitation-audit-event`. - Tests: `backend_tests/rpc-audit-test.clj`, `rpc-team-test/accept-organization-invitation-audit-event`.

View File

@ -11,8 +11,14 @@
- Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in. - Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in.
- Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`. - Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`.
- Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI. - Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
- Organization SSO (Nitrate): the RPC `check-nitrate-sso` takes the route's **query string proper** (`:query-string`, e.g. `screen=workspace&team-id=…`), never a URL and never with the `?` separator: the client strips it (`rt/token->query-string`) and the server puts it back. The server builds the destination with `app.auth.oidc/build-dest-url` (`public-uri` + `?` + the params, parsed and rendered back), so the redirect can never leave the public URI; a value whose parsed keys are not param names is not a route: a URL parses into a single key holding the whole URL, and a key carrying `:`, `/`, `?` or `#` is rejected with a `:warn` and the user lands on the public URI (the RPC never fails because of it). Do not reason from `u/map->query-string`: it keeps a valueless key as `key=`, so rendering is not what filters a URL out — the key shape is. A leading `?` is still trimmed server-side, so a client bundled before the trim keeps its destination. Never require `screen`: the root of the app carries its params instead of a screen (`?template=…`), and `check-nitrate-sso` is called from that branch too (`dtm/resolve-login-team-id`, which today reads only `:authorized` and never follows the redirect, so a wrong destination there is a silent `:warn` and not a visible break); the callback re-checks the destination before redirecting, which covers state tokens minted before that. The state token also carries `:profile-id`: only that session is granted the `props :sso` entry (`session-mismatch` otherwise, and the redirect still happens).
- Organization SSO needs `app.nitrate/organization-access?` whenever the RPC receives an `:organization-id` (that id always comes from the client): the profile must be a member of the organization or its owner; otherwise the RPC answers `{:authorized true :reason :no-organization-access}`, the same short-circuit shape as `:no-team-access`.
- Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation. - Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. - When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. SSO registration (OIDC and compatible providers) follows the OIDC callback gate: it requires `:registration` or `:oidc-registration`, or a team invitation token issued for the registering email (an invitation substitutes the flags only when `member-email` matches the OIDC/`prepared-register` email via `clean-email`, as in the password flow, which raises `:email-does-not-match-invitation` otherwise). The OIDC callback verifies the invitation JWT and the email before redirecting to register, and `register-profile` revalidates the invitation under a row lock when no SSO flag is set; it never requires password login. Only OIDC-typed backends (`oidc-provider-types` in `app.auth.oidc`) take the SSO registration gate; any other `:backend` in a prepared-register token is rejected as `:invalid-token`.
- Access tokens (`app.http.access-token`): personal tokens have JWT issuer `access-token` and DB type other than `mcp`; MCP tokens have issuer `urn:penpot:mcp-token` (`actoken/mcp-token-issuer`) and DB type `mcp`. Authentication requires issuer and type to agree, and the claim `uid` to match the row's profile; other issuers and undecryptable tokens are treated as anonymous.
- Management RPC (`/api/management/methods/:name`) is shared-key only. The middleware authenticates `<id> <secret>`; the RPC wrapper `wrap-shared-key-authorization` authorizes the key id against the method's `::rpc/allowed-shared-keys` set. Absent metadata accepts any configured key; all current methods declare one (`exporter`/`media-processor` for `upload-tempfile`, `nexus` for file analysis, `exporter` for jobs, `admin-console` for Nitrate). A valid key outside the set answers 403 `:shared-key-not-allowed`; denied calls do not reach the audit wrapper.
- Personal tokens need flag `:access-tokens`. MCP tokens need `:mcp` or `:access-tokens`, and only on routes whose `actoken/authz` gets `::actoken/allow-mcp true` (the RPC routes); elsewhere (e.g. assets) they are anonymous.
- On RPC, an MCP token reaches only commands with `::rpc/token-type "mcp"` (`wrap-authentication` in `app.rpc`); every other command answers 403 `:token-auth-required`. A command with that metadata also rejects sessions and personal tokens. Currently only `send-mcp-feedback`. Blocking a profile invalidates sessions, not tokens, so such commands check `:is-blocked`/`:is-active` themselves.
- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/subtleties`. - HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/subtleties`.
## Permission model ## Permission model
@ -20,13 +26,16 @@
- `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence. - `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence.
- Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read. - Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read.
- File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2). - File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2).
- The organization-owner fallback (`get-organization-owner-permissions` via Nitrate over HTTP) grants read-only access for a non-member organization owner. It resolves the owning team through `resolve-team-id`, which refuses soft-deleted file/project/team: the file and project lookups filter `deleted_at`, and a final `db/get*` check covers the team row itself — `nil` means no access. `db/get*` only filters soft-deleted rows when `deleted_at` is in the projection. The re-checks of open websocket subscriptions rely on this (`mem:backend/websocket-subscriptions`).
- File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics. - File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics.
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. - Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. RPCs that accept `share-id` enforce the link's page scope before mutating (`share-link-pages` + `check-page-scope`): a thread or page outside the link raises `:not-found`, and that check runs before the ownership checks. `get-profiles-for-file-comments` filters its query by the same pages. `mark-all-threads-as-read` does not take `share-id`.
## Teams, projects, and invitations ## Teams, projects, and invitations
- Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so. - Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so.
- Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token. - Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token.
- The whole invitation family is team-admin only: creating, updating and deleting invitations, minting the invitation token (`get-team-invitation-token`) and listing the pending ones (`get-team-invitations`, which carries the invited email and the offered role). The token of an invitation that grants the `owner` role is restricted to the team owner, because an admin cannot create nor promote an owner invitation (`cant-promote-to-owner`). This family is the exception to the rule above: it answers `:validation` / `:insufficient-permissions` instead of `:not-found`, so the five commands speak one language.
- With the `:admin-console` flag, the organization `send-invitations` policy (role value plus Nitrate `restrictions-met`, through `cto/allowed?`) gates every invitation command that hands out or manages access: create (and resend, which reuses it), `update-team-invitation-role`, `delete-team-invitation` and `get-team-invitation-token`. Only `get-team-invitations` stays on team admin alone, so a blocked admin still sees the list. The shared helper `check-send-invitations-policy` in `teams_invitations.clj` does a strict Nitrate lookup (fails closed; 404 means no organization) and runs outside a DB transaction, which is why the role and delete commands carry no `::db/transaction`.
- Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup. - Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup.
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing. - Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing.

View File

@ -9,6 +9,7 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment` - Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
- Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`. - Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`.
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains` - Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
- WebSocket subscriptions, the connection registry (local per backend, unlike the bus), and how a lost permission gets revoked: `mem:backend/websocket-subscriptions`.
- Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí) - Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí)
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`. - Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
@ -32,29 +33,65 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
RPC commands are defined with `app.util.services/defmethod` and schemas. Use `get-` prefixes for read operations. Command metadata usually includes auth, docs version, params schema, and result schema. Return plain maps/vectors or raise structured exceptions from `app.common.exceptions`. RPC commands are defined with `app.util.services/defmethod` and schemas. Use `get-` prefixes for read operations. Command metadata usually includes auth, docs version, params schema, and result schema. Return plain maps/vectors or raise structured exceptions from `app.common.exceptions`.
Backend RPC command areas without focused memories include access tokens, binfile, demo, feedback, file snapshots, fonts, management, Nitrate, and webhooks beyond the notes in `mem:backend/auth-permissions-product-domains`; inspect nearby command tests and command metadata before changing them. Backend RPC command areas without focused memories include access tokens (auth rules, incl. MCP tokens, in `mem:backend/auth-permissions-product-domains`), binfile, demo, feedback, file snapshots, fonts, management, Nitrate, and webhooks beyond the notes in `mem:backend/auth-permissions-product-domains`; inspect nearby command tests and command metadata before changing them.
## DB conventions ## DB conventions
`app.db` helpers accept cfg, pool, or conn in most places and convert kebab-case to snake_case: `app.db` helpers accept cfg, pool, or conn in most places and convert kebab-case to snake_case:
- `db/get`, `db/get*`, `db/query`, `db/insert!`, `db/update!`, `db/delete!`. - `db/get`, `db/get*`, `db/query`, `db/insert!`, `db/update!`, `db/delete!`.
- `db/get*`/`db/query` signature is `(ds table where-params & {:as opts})`: trailing keywords become an opts MAP (ignored by the select builder — use `{::db/remove-deleted false}` map as the single vararg to also see rows with `deleted_at` set), while a trailing ODD number of keywords crashes with "Don't know how to create ISeq from: Keyword". Extra keywords are NOT a column filter: rows always come back with all columns.
- Next.js/JDBC caveat: where-maps don't support value vectors (no `IN`); use `status = ANY(?)` with `db/create-array` inside `db/tx-run!`.
- Use `db/run!` for multiple operations on one connection. - Use `db/run!` for multiple operations on one connection.
- Use `db/tx-run!` for transactions. - Use `db/tx-run!` for transactions.
- `job`-substrate specifics (unified jobs): the only writers of a job status are `claim`, `retry-job`, `complete`, `fail` and `cancel` in `app.jobs`; each one is a conditional update plus its `job_event` row in the same transaction, so history can never contradict the row. A writer that affects no row writes no event and no metric. They never join the caller's transaction, because a lifecycle write reports on work that already happened: a rolled back caller cannot undo a transition, and an independent connection cannot see the caller's uncommitted job row, so a job must be committed before it can be claimed. GC deletes return resource ids via `RETURNING` inside one transaction.
- Every public function of `app.jobs` takes a cfg map. A bare pool or connection is a caller bug and is not coerced: the substrate has no conversion of its own, and the ones in `app.db` are the db layer's business.
- Every new foreign key of the substrate is `DEFERRABLE`, `job_event.job_id` included.
- Some tables carry a SQL deletion guard (`raise_deletion_protection` + a `deletion_protection__tgr` statement trigger): `profile`, `team`, `team_font_variant`, `file_media_object`, `file_thumbnail`, `file_tagged_object_thumbnail` and `job`. A `DELETE` raises unless the transaction runs `SET LOCAL rules.deletion_protection TO off` (the guard is on when the GUC is unset, empty or `on`); see `app.tasks.objects-gc`, `app.jobs.gc` and `th/db-force-delete`.
Database migrations live in `backend/src/app/migrations/`; pure SQL migrations are under `backend/src/app/migrations/sql/`. SQL filenames conventionally start with a sequence and verb/table description, e.g. `0026-mod-profile-table-add-is-active-field`. Applied migrations are tracked in the `migrations` table. Database migrations live in `backend/src/app/migrations/`; pure SQL migrations are under `backend/src/app/migrations/sql/`. SQL filenames conventionally start with a sequence and verb/table description, e.g. `0026-mod-profile-table-add-is-active-field`. Applied migrations are tracked in the `migrations` table.
Migrations must stay within PostgreSQL 15 syntax: the official docker images (`docker/images/docker-compose.yaml`) pin `postgres:15`, so anything newer breaks those deployments (`SET STORAGE DEFAULT` is PostgreSQL 16, for instance). The documented floor is >=15. Migrations must stay within PostgreSQL 15 syntax: the official docker images (`docker/images/docker-compose.yaml`) pin `postgres:15`, so anything newer breaks those deployments (`SET STORAGE DEFAULT` is PostgreSQL 16, for instance). The documented floor is >=15.
For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/subtleties`. For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/subtleties`.
## Background tasks ## Background tasks (unified jobs)
A task handler is an Integrant component with `ig/assert-key`, `ig/expand-key`, and `ig/init-key`, returning the function run by the worker. New tasks also need wiring in `app.main`: handler config, worker registry entry, and cron entry if scheduled. Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` in its namespace + a malli params schema + an `ig/init-key` that returns the job-def map `{::jobs/name, ::jobs/schema, ::jobs/handler, ::jobs/decoder, ::jobs/validator}` (decoder/validator precompiled at init), optionally with `::jobs/result-schema` when the result carries types plain JSON does not keep (a uuid, whether a value or a map key): the substrate compiles the decoder from that schema on registry init and checks a result against the schema before storing it. The registry is the `::jobs/defs` wiring in `app.main`, which also populates a module-level registry used by `jobs/submit` as fallback — job-def components cannot ig/ref `::jobs/defs` (wiring cycle). Submit from RPC code passes its RPC cfg (it carries `::jobs/defs` via ig/ref).
- `::jobs/handler` is `(fn [context params] ...)`. It is NOT given the runner cfg: a job-def closes over its own dependencies, and handing it the runner cfg would let a handler reach the pool or any other component. A wrapper passes the context to its implementation only when the implementation takes it; most do not, and their implementation stays `[cfg params]`.
- The context is built by `jobs/make-context` and has exactly five keys: `id`, `name`, `label`, `resource-id`, `profile-id`, behind a closed schema. `profile-id` is the owner of a user-facing job and nil for an internal one, and it is there so a handler can revalidate permissions and check who owns its resource without a second query. `queue` is absent on purpose (a routing detail of the dispatcher, not a property of the job) and so are `retry-num`/`max-retries`/`attempt` (retry policy belongs to the runner, and an attempt number cannot be durable because the `noop` strategy re-runs without incrementing the counter).
- `jobs/invoke` is the in-process escape hatch: it delivers a nil context without a row, validates and delivers `::jobs/context` when given, and keeps `::jobs/job-id` independent. `::jobs/context` and `::jobs/job-id` are cfg options, not params.
- Business params live in the `params` column (plain JSON, decoded with the job-def decoder). `submit` takes optional `::jobs/profile-id`, `::jobs/resource-id` (both validated as UUIDs) and `::jobs/expires-at` (an instant: the retention the jobs GC sweeps by), all stored as columns; nothing is inferred from `params`. The binfile creation commands are the callers that pass a profile-id and a resource.
- A job-def may carry optional metadata keys read from the registry by its consumers, never by the substrate: `::jobs/family` (e.g. `:export`) groups the jobs of a family for quotes, `::jobs/resource-role` (`:input` or `:output`) says whether the job's `resource_id` is what it consumes or what it produces, and `::jobs/queue-name` names the queue the creation command routes the job to (`submit-job` falls back to `:binfile` when the def does not set it).
- Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
- The v3 and v1 imports only materialize a library relation when both ids belong to the package and the link would neither be a self link nor close a dependency cycle (the same `bfc/library-link-status` check that `link-file-to-library` uses to raise); relations are processed in sorted `(file_id, library_file_id)` order, which `get-files-rels` also returns, so the first edge wins deterministically, and every edge it ignores is logged with a warning. The v2 import stays out of that guard on purpose: it is only reachable from the developer REPL (`app.srepl.binfile`) and keeps inserting the relations as they come. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
- A version 1 package lists its files in the metadata and the importer creates one file per entry. The v1 export (gone from the product; only tests and the developer REPL use it) unioned the requested files with their libraries without deduplicating, so a package exported from a file inside a library cycle lists that file twice. The importer reads the pair of objects of every entry to keep the stream in sync, imports the first occurrence only, warns about the repeats, and answers with one id per imported file.
- The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
- Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
- The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.
- A user-facing job is created through `app.rpc.commands.jobs`. One command per job type: `create-export-binfile-job` and `create-import-binfile-job` name the job they create and their body carries only `:params`, whose schema the command imports from the job namespace (`::sm/params`), so the RPC validates the business params up front, before the command runs; the job-def is still the place that owns them and validates them again at run time. The commands check the permission and the quote of the family before the expensive part, submit with `max-retries 0`, an expiry from `:jobs-user-ttl` and the queue the job-def routes to through `::jobs/queue-name` (`:binfile` when the def says nothing; read by the command, never by the substrate), and answer with a summary of the row. The import command has its own creation schema (`import-binfile/schema:create-params`, version optional) because the request is not the stored params: the command fills the version (from the caller or the package header) and the manifest metadata kept for audit. It assembles the upload, reads the manifest of a version 3 package for the audit (never its content) and stores the package in `job-resource` as the resource of the job, releasing it when the job never comes to own it.
- `get-job` is the read surface a profile has over its own jobs: it answers with the same summary the creation commands return plus the `result` of a finished job or its public error (through `decode-job-error`, so the keywords come back), and a job of another profile is not distinguishable from one that does not exist: both answer `not-found`. `decode-row` decodes `params`, `result` and `error`, and the result is then decoded with the decoder the substrate compiled from the result schema the job-def the row names declares (`decode-result`; a def without one, or a row whose job-def is gone, keeps the plain JSON the row holds: a uuid as the text it is on disk, a uuid used as a map key as a keyword). The event log still has no read surface: a client that follows a job takes its progress from the websocket.
- `heartbeat` uses named options `[cfg & {:keys [job-id progress] :as options}]` and returns the number of durable writes, so 0 means there was nothing to write: no job context, or the throttle did not allow it. A write that finds the job no longer active (terminal or gone) raises an `:interrupt` (`code :job-interrupted`, carrying the job id and the status it found) through the shared `jobs/check-active` instead of counting zero, so a task in course stops at its next beat; the interrupt is internal and never reaches HTTP (the management progress report answers `:skip` for it). The job id falls back to `::jobs/job-id` on the cfg and then to the runner-bound `*job-id*`. The `modified_at` touch is throttled to ~1s and the progress event to ~200ms, so the check fires at most once per second with no extra query (the read of the row happens only when a write already found nothing). A progress report is a milestone (`schema:progress`): a required `:stage` keyword plus an optional `:counters` map of `scope -> {:current, :total?}`, and no other key. Counters are non-negative, a `total` is positive and never lower than its `current`, and stage and counter keys are names of at most 64 characters. A milestone without counters is valid (a point of the run with no units to count), the stage vocabulary belongs to the worker and not to the substrate, and a client must degrade for a stage it does not know. Read a stored payload back with `jobs/decode-progress`: the database holds the keywords as strings. The 1s beat (down from 60s, so a cancelled heavy job is seen as gone at its next beat) costs one `modified_at` touch per beating job per second: cheap while the only heavy beater runs alone on its queue, revisit if concurrent beating jobs ever grow.
- `job.error` conforms to `app.jobs/schema:job-error` (`type` and `code` keywords, `hint` text, extra details allowed). Read it back with `jobs/decode-job-error`; a plain `db/decode-json-pgobject` leaves the keywords as strings. `complete` takes named options and an optional `resource-id`, which is only ever set, never replaced: it reads the row locked inside its own transaction and applies the rules in a fixed order, skipping a job that is no longer completable before refusing one for its resource, so the answer is a function of the row and not of the order two callers arrive in. It checks the result against the schema the job-def of the row declares before it writes (a violation raises `:invalid-result` and nothing is stored), and only for a transition that is going to write: a terminal job answers `:skip` without validating. `complete` is the single door of a result, so the check covers the runner and the external worker that completes a job through the management RPC (`::complete-job`), which stores a result the substrate would otherwise take blind.
For worker dispatch, cron, retry semantics (`ex/raise :type ::wrk/retry` with `:delay`/`:strategy`), deduplication, job events, and queue internals: `mem:backend/subtleties`.
## Jobs metrics
- `app.jobs.metrics` owns the jobs metric names, bounded labels and the periodic backlog sampler.
- Metrics use only `name`, `queue`, `outcome`, `reason`, `stage` and `kind` labels. Job IDs, profile IDs, params, reply bodies and exception text must never be labels. Every label goes as-is (lowercased where case varies): values come from fixed call sites, so a new value in Prometheus means new code, not noise. Only a missing value falls back (`other`, or `failed` for outcomes).
- The canonical job name is a keyword (`app.jobs/schema:job-name`): `submit` and `invoke` reject strings. The row stores text, so `jobs/get-job-def` still coerces (it is the runner's boundary), cron entries carry the job under `:job` with `d/name` only at the SQL/log boundary, and test helpers keyword-ize row names before invoking.
- The durable lifecycle metrics are submitted, dispatched, completed, retries, orphaned and rescheduled counters; queue-wait, execution and total-time histograms; dispatcher and backlog state; GC, cron and ephemeral request metrics.
- `app.metrics/run!` is safe for recording failures, but metrics components still require a valid metrics instance. Durable submit call sites carry `::mtx/metrics` through RPC or job-def configuration. The jobs substrate takes the **cfg**, not the instance, and resolves it with `app.metrics/instance`, which raises `:missing-metrics` when absent: `app.jobs`, `app.jobs.metrics`, the runner, the dispatcher, cron and the jobs GC all follow that. `app.http`, `app.rpc.climit` and `app.storage.s3.metrics` still take the instance and were left out of scope on purpose, so do not read the codebase as uniform. The six job writers resolve at the top of their body, before writing, because their metric lands in a `db/after-commit` callback that swallows exceptions. The GC, the dispatcher and cron resolve inside that callback instead, so their guarantee rests on the component schema instead. A job that calls `heartbeat` from a component cfg must carry `::mtx/metrics` on it, and the event helper records the counter from the cfg the transaction hands it.
- SQL-backed job metrics are registered with `app.db/after-commit` from INSIDE the transaction that changed the row, so a rollback cannot inflate submitted, terminal, retry, event or dispatcher counters. Every lifecycle writer owns that transaction (`app.jobs/own-transaction` drops `::db/conn`), and `db/transact!` gives a transaction that joins no other its own callback context, so the write and its metric commit or vanish together. `submit` is the exception and joins the caller on purpose: creating a job is part of the caller's business work.
- `penpot_jobs_events_total` counts stored job events with no labels: the interesting split is `kind`, and grouping a growing append-only table to build a metric is not worth the query on the write path.
- The backlog sampler is `::jobs-metrics/sampler`; it runs every 30 seconds on worker-enabled, writable systems, groups by status and publishes the oldest pending age, and does not start on a read-only database.
- `penpot_tasks_timing` remains exported for compatibility while the jobs-specific histograms are adopted.
For worker dispatch, cron, retry semantics, deduplication, and queue internals: `mem:backend/subtleties`.
## REPL ## REPL
@ -101,9 +138,7 @@ IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory.
* **Linting:** `pnpm run lint:clj`. * **Linting:** `pnpm run lint:clj`.
* **Formatting:** `pnpm run check-fmt:clj` to check, `pnpm run fmt:clj` to fix. After running `fmt:clj`, `check-fmt:clj` is redundant. Avoid unrelated whitespace diffs. * **Formatting:** `pnpm run check-fmt:clj` to check, `pnpm run fmt:clj` to fix. After running `fmt:clj`, `check-fmt:clj` is redundant. Avoid unrelated whitespace diffs.
**Before linting:** if delimiter errors are suspected (after LLM edits), run **Before linting:** if delimiter errors are suspected (after LLM edits), run `scripts/paren-repair` on the affected files first. Delimiter errors produce misleading linter/compiler output. See `mem:scripts/paren-repair`.
`scripts/paren-repair` on the affected files first. Delimiter errors produce
misleading linter/compiler output. See `mem:scripts/paren-repair`.
## Testing ## Testing

View File

@ -19,6 +19,12 @@
- Renewal fires when `modified-at` is older than 6h (`default-renewal-max-age`, not configurable). It `UPDATE`s the same row (`modified-at` only) and issues a new token that keeps the original `:exp`. - Renewal fires when `modified-at` is older than 6h (`default-renewal-max-age`, not configurable). It `UPDATE`s the same row (`modified-at` only) and issues a new token that keeps the original `:exp`.
- `read-session` does not check age. Idle expiration is enforced on every request by `wrap-authz` (sessions past the idle window authenticate like missing ones and are never renewed); the GC only deletes the rows afterwards. - `read-session` does not check age. Idle expiration is enforced on every request by `wrap-authz` (sessions past the idle window authenticate like missing ones and are never renewed); the GC only deletes the rows afterwards.
## Revocation paths
- Profile deletion → `session/invalidate-all` (all sessions of the profile die).
- Authenticated password change → `session/invalidate-others` (every session except the caller's dies).
- Password recovery (`recover-profile`, GHSA-3j2p-qr45-q365) → `session/invalidate-all` inside the same transaction as the password `UPDATE`; there is no current session to preserve, and a captured session must not survive recovery. Sibling of GHSA-mj9f-5cwq-7p3q (logout deletes the current session row).
## GC (`::tasks/gc`, cron `session-gc`, daily) ## GC (`::tasks/gc`, cron `session-gc`, daily)
- Deletes from `http_session_v2` where `modified_at < now - :auth-token-cookie-max-age` **or** `created_at < now - :auth-token-cookie-max-age-absolute`. - Deletes from `http_session_v2` where `modified_at < now - :auth-token-cookie-max-age` **or** `created_at < now - :auth-token-cookie-max-age-absolute`.

View File

@ -25,12 +25,19 @@
- Existing object values can remain readable until physical deletion. - Existing object values can remain readable until physical deletion.
- `:expired-at` blocks reads after the expiration time. - `:expired-at` blocks reads after the expiration time.
- `del-object!` sets `deleted_at` on live rows only (`deleted_at IS NULL`): a repeated call returns `false`. It does not remove backend content. - `del-object!` sets `deleted_at` on live rows only (`deleted_at IS NULL`): a repeated call returns `false`. It does not remove backend content.
- `storage-gc-deleted` removes the database row and backend content after the deletion delay, processing the pending objects in chunks of 25 with keyset pagination (`(deleted_at, id)` cursor, matching the `storage_object(deleted_at, id)` index). Only `status = 'valid'` objects are read: pending ones (still being written, see the status lifecycle) stay untouched even when already marked as deleted. Each chunk is one short read-write transaction: `SELECT ... FOR UPDATE SKIP LOCKED` (so two concurrent runs never take the same chunk; no separate read transaction exists and the run does not block the table vacuum) followed by the database delete and the backend delete grouped by backend. The cursor advances to the `(deleted_at, id)` of the last row read regardless of the deletion result, so a chunk that fails is skipped and not visited again in the same run: the error aborts the transaction (nothing of the chunk is deleted, locks are released), the handler catches it inside the transaction function and rolls it back explicitly — the commit that `tx-run!` performs afterwards is a JDBC-driver no-op on the already-rolled-back connection — and the loop continues with the returned rows. Accepted quirks, both benign and self-healing: (1) with rows of several backends in one chunk, a failure after one group's physical delete leaves those rows pending with already-removed content — pending rows are excluded from reads and the next run finishes them, since the physical delete is idempotent (S3 ignores missing keys, FS uses `deleteIfExists`); (2) rows locked by a concurrent run are skipped via `SKIP LOCKED` and deferred to the next scheduled run, and the `deleted_at <= now` cutoff is sampled per chunk. It emits the `penpot_storage_gc_deleted_objects_total{backend}` counter (one increment per deleted backend group, reported only after the chunk commits, so rolled-back chunks are never counted twice); run duration is covered by the generic `penpot_tasks_timing{name="storage-gc-deleted"}` histogram and chunk failures by the log-derived `penpot_log_events_total`. Every 5000 deleted objects the run pauses 5 seconds (throttle between chunk transactions, never holding one open). - `storage-gc-deleted` removes the database row and backend content after the deletion delay, processing the pending objects in chunks of 25 with keyset pagination (`(deleted_at, id)` cursor, matching the `storage_object(deleted_at, id)` index). Only `status = 'valid'` objects are read: pending ones (still being written, see the status lifecycle) stay untouched even when already marked as deleted. Each chunk is one short read-write transaction: `SELECT ... FOR UPDATE SKIP LOCKED` (so two concurrent runs never take the same chunk; no separate read transaction exists and the run does not block the table vacuum) followed by the database delete and the backend delete grouped by backend. The cursor advances to the `(deleted_at, id)` of the last row read regardless of the deletion result, so a chunk that fails is skipped and not visited again in the same run: the error aborts the transaction (nothing of the chunk is deleted, locks are released), the handler catches it inside the transaction function and rolls it back explicitly — the commit that `tx-run!` performs afterwards is a JDBC-driver no-op on the already-rolled-back connection — and the loop continues with the returned rows. Accepted quirks, both benign and self-healing: (1) with rows of several backends in one chunk, a failure after one group's physical delete leaves those rows pending with already-removed content — pending rows are excluded from reads and the next run finishes them, since the physical delete is idempotent (S3 ignores missing keys, FS uses `deleteIfExists`); (2) rows locked by a concurrent run are skipped via `SKIP LOCKED` and deferred to the next scheduled run, and the `deleted_at <= now` cutoff is sampled per chunk. It emits the `penpot_storage_gc_deleted_objects_total{backend}` counter (one increment per deleted backend group, reported only after the chunk commits, so rolled-back chunks are never counted twice); run duration is covered by the generic `penpot_tasks_timing{name="storage-gc-deleted"}` histogram and chunk failures by the log-derived `penpot_log_events_total`. Every 5000 deleted objects the run pauses 5 seconds (throttle between chunk transactions, never holding one open). It runs as the `storage-gc-deleted` job (`::job-def`) and beats `jobs/heartbeat` after each committed chunk, so a long sweep keeps its lease and never holds it during a chunk transaction.
- `storage-gc-touched` finds references before it sets `deleted_at`. - `storage-gc-touched` finds references before it sets `deleted_at`.
- Orphan watch: `SELECT COALESCE(metadata->>'bucket', metadata->>'~:bucket') AS bucket, count(*) FROM storage_object so WHERE touched_at IS NULL AND deleted_at IS NULL AND status = 'valid' AND NOT EXISTS (SELECT 1 FROM upload_session_chunk usc WHERE usc.object_id = so.id) GROUP BY 1;` Only the `upload-session` row of that output is a leak signal (its rows always have a mapping or a pending touch/deleted mark); other buckets use domain-table references, not this mapping. - Orphan watch: `SELECT COALESCE(metadata->>'bucket', metadata->>'~:bucket') AS bucket, count(*) FROM storage_object so WHERE touched_at IS NULL AND deleted_at IS NULL AND status = 'valid' AND NOT EXISTS (SELECT 1 FROM upload_session_chunk usc WHERE usc.object_id = so.id) GROUP BY 1;` Only the `upload-session` row of that output is a leak signal (its rows always have a mapping or a pending touch/deleted mark); other buckets use domain-table references, not this mapping.
- `objects-gc` removes deleted domain rows and touches their storage object IDs. Each proc streams its pending keys with `db/plan` inside a read-only transaction and deletes them in chunks, each chunk in its own transaction with `FOR UPDATE SKIP LOCKED`: a chunk that fails is logged and skipped instead of aborting the run. - `objects-gc` removes deleted domain rows and touches their storage object IDs. Each proc streams its pending keys with `db/plan` inside a read-only transaction and deletes them in chunks, each chunk in its own transaction with `FOR UPDATE SKIP LOCKED`: a chunk that fails is logged and skipped instead of aborting the run.
- Use `::db/reuse-conn true` with `sto/resolve` inside a database transaction. - Use `::db/reuse-conn true` with `sto/resolve` inside a database transaction.
## GC Cost Model
- `sto/touch-object!` is one `UPDATE storage_object SET touched_at`: DB-only, never touches S3/FS backend content.
- Physical FK cascades are avoided by convention; domain deletion is logical (`deleted_at`) everywhere. GC removes small objects first, then parents, then grandparents (see the call order in `app.tasks.objects-gc/execute-objects-gc`). Per-row delete cost is small and bounded.
- `objects-gc` sweeps in chunks of 100 (`::chunk-size`, overridable per job params); `storage-gc-deleted` in chunks of 25. `objects-gc` (and the other `db/plan` sweeps) stream the pending keys in a read-only transaction of their own and delete every chunk in a transaction of its own: the cursor fetch and the deletes are never the same transaction. `storage-gc-deleted` has no separate cursor transaction: it reads, locks and deletes each chunk in the same short transaction (see the keyset pagination above).
- Consequence: per-chunk cost is DB-only and bounded, so these sweeps cannot outrun the 30-minute jobs lease in practice (would need millions of pending rows in one proc). Per-chunk `heartbeat` where present is belt-and-braces, not load-bearing.
## Connection Reuse Details ## Connection Reuse Details
### `app.storage/resolve` patterns: ### `app.storage/resolve` patterns:
@ -71,6 +78,7 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
- Objects can therefore share content across users and files within one bucket. - Objects can therefore share content across users and files within one bucket.
- Deleted objects are not reused. - Deleted objects are not reused.
- `tempfile` objects never use deduplication, even when the caller requests it. - `tempfile` objects never use deduplication, even when the caller requests it.
- `job-resource` objects never use deduplication either, whatever the caller asks: the object belongs to one profile, and a shared blob would serve the artifact of one user to the job of another. `put-object!` excludes the bucket from the dedupable set, so the invariant does not depend on call sites.
- Use `sto/wrap-with-hash` when the caller already calculated the content hash. - Use `sto/wrap-with-hash` when the caller already calculated the content hash.
## Bucket Rules ## Bucket Rules
@ -85,15 +93,16 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
| `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. | | `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
| `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. | | `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. | | `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. |
| `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. | | `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference: the `file_data.storage_object_id` column (FK `NO ACTION DEFERRABLE INITIALLY IMMEDIATE`); the object metadata keeps `file-id` and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
| `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. | | `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
| `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. | | `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
| `job-resource` | Storage objects owned by job rows (`job.resource_id`), written by `app.jobs.storage`. | No | Authentication required, owner-scoped | jobs-GC touch → reference scan. |
- The valid bucket set lives in `app.storage/valid-buckets`. - The valid bucket set is `app.storage.schema/metadata-buckets` (derived from `bucket-requirements`, where a new bucket and its required metadata keys are one entry); `app.storage/valid-buckets` aliases it.
- `file-media-object` is the default bucket for old rows without bucket metadata. - `file-media-object` is the default bucket for old rows without bucket metadata.
- Do not assign a new bucket without adding its access and cleanup behavior. - Do not assign a new bucket without adding its access and cleanup behavior.
- The touched-object collector raises an internal error for an unknown bucket. - The touched-object collector raises an internal error for an unknown bucket.
- It supports `file-media-object`, `team-font-variant`, `file-object-thumbnail`, `file-thumbnail`, `profile`, `file-data`, `tempfile`, `upload-session`, and `organization`. - It supports `file-media-object`, `team-font-variant`, `file-object-thumbnail`, `file-thumbnail`, `profile`, `file-data`, `tempfile`, `upload-session`, `job-resource`, and `organization`.
- It does not support `file-data-fragment` or `file-change`. - It does not support `file-data-fragment` or `file-change`.
## Access Rules ## Access Rules
@ -101,6 +110,7 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
- `app.http.assets` decides direct object authentication from the bucket. - `app.http.assets` decides direct object authentication from the bucket.
- Public buckets are `file-media-object`, `file-object-thumbnail`, `team-font-variant`, `file-data-fragment`, and `organization`. - Public buckets are `file-media-object`, `file-object-thumbnail`, `team-font-variant`, `file-data-fragment`, and `organization`.
- Other valid buckets require a session or access-token profile ID. - Other valid buckets require a session or access-token profile ID.
- `tempfile` and `job-resource` are owner-scoped: the request must come from the profile stored in the object metadata, and a mismatch answers 404 (counted as `unauthorized`). A missing owner is tolerated only in `tempfile`, where legacy objects predate the metadata; `job-resource` always carries one, so an object without it is never served.
- File-media routes also require file read permission. - File-media routes also require file read permission.
- Non-public direct responses set `content-disposition: attachment`. - Non-public direct responses set `content-disposition: attachment`.
- FS responses use `x-accel-redirect` for the configured asset path. - FS responses use `x-accel-redirect` for the configured asset path.
@ -108,13 +118,40 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
## File Data ## File Data
- `file-data-backend` accepts `legacy-db`, `db`, or `storage`. - `file-data-backend` accepts `legacy-db`, `db`, or `storage`; `legacy-db` is read-only and the default is `db`. The configuration reads `legacy-db` as `db` at startup, with a warning (`normalize-file-data-backend`), so a deployment that kept the variable saves instead of failing on every write.
- `legacy-db` stores main data in `file.data` and snapshots in `file_change.data`. - `legacy-db` reads main data from `file.data` and snapshots from `file_change.data`; it no longer writes them (`fdata/upsert!` rejects it).
- `db` stores encoded data in `file_data.data`. - `db` stores encoded data in `file_data.data`.
- `storage` stores encoded data in storage subsystem with `file-data` bucket and keeps `data` nil in `file_data` table. - `storage` stores encoded data in storage subsystem with `file-data` bucket, keeps `data` nil in `file_data` table, and points at the object with `file_data.storage_object_id`.
- The `file_data.metadata.storage-ref-id` value points to the storage object. - `file_data.storage_object_id` is a `NO ACTION DEFERRABLE INITIALLY IMMEDIATE` foreign key to `storage_object(id)`: the database refuses to physically delete a referenced object. The `IMMEDIATE` is load-bearing: the RI violation fires inside the `storage-gc-deleted` chunk transaction, so the chunk rolls back, is logged at error level with a generic hint plus the `constraint` and `table` the database reports (any `NO ACTION` foreign key on `storage_object` can block the delete, not only `file_data`'s) and is retried on the next run; with `INITIALLY DEFERRED` the check would fire at commit, outside the catch, and the sweep would never move past the blocked chunk.
- `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row. - The column is the only source of truth. The release is a coordinated restart (no rolling deploy) when the instance uses the storage backend (`PENPOT_FILE_DATA_BACKEND=storage`) or still has rows carrying `metadata.storage-ref-id`; with the default `db` backend and no such rows the previous and the new code read the same `file_data.data` column, so a rolling deploy is safe. `metadata.storage-ref-id` is no longer written or read, and the `file-data` metadata schema does not declare it. The migration backfills the rows it can resolve (valid uuid with an existing `storage_object`) and removes the key from those rows; a corrupt or orphan pointer keeps the key, which nothing reads. A large instance runs `operations/2.20.0-file-data-storage-object-id.md`: column, index and FK online, then the backfill (only if a row still carries `metadata.storage-ref-id`) with every instance stopped, verifies, and marks the migration as applied (`INSERT INTO migrations`). The backfill must never run while the previous code serves when the storage backend is in use (or rows with the pointer remain), because that code writes and scans the metadata pointer. A rollback to the previous code cannot read the rows written by this release when the storage backend is in use.
- `fdata/upsert!` touches the `storage_object_id` it receives in the params (the pointer the persisted row had) before replacing it, so the replaced blob is reclaimed. Callers pass the column from the file map (main/restore), `nil` for a new row (fragments, snapshots, new files) or the row itself (offload); there is no query in the upsert path, which is why every caller reads the file under the row lock first (the migration-on-read path included, see `mem:backend/subtleties`). It returns `{:written <bool> :storage-object-id <uuid or nil>}`, the pointer of the object the write created (nil for `db`/`legacy-db`), and `binfile.common/update-file!` propagates it so a migrating read returns the map with the pointer of the row it persisted instead of the pre-write one. The write also compares the expected pointer with `IS NOT DISTINCT FROM` (the caller's value, not the new one) and raises `:stale-file-data-pointer` when the row changed under it, so a stale map fails instead of leaking the replaced blob; the check lives in `upsert-in-database` because the `legacy-db` branches do not use the upsert SQL.
- `binfile.common/update-file!` takes the file map read from the database plus the fields to change (`[cfg file & params]`), merges them and builds the `file_data` params from the merged map, so the pointer travels in the read map and a caller cannot lose it by building a partial map; the params cannot override `:storage-object-id`. `insert-file!` always drops it (an INSERT never replaces a persisted pointer; `duplicate-file` hands the source map there). `bfc/get-file` keeps the pointer for the write paths, and every response goes through `bfc/file->response`, which strips it and any residual `storage-ref-id` left in the metadata of the rows the migration does not backfill: each `file-data` object is authentication-only and any logged-in profile could otherwise fetch it from `/assets/by-id`.
- The in-place binfile import was removed: `import-files!` raises `:not-supported` when the cfg carries `::bfc/file-id`, so no caller can overwrite a storage-backed row without carrying its previous pointer.
- File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata. - File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
- Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil and `storage_object_id` points at the new object) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer locks the `file` row first (the update RPC and the snapshot paths with `FOR UPDATE` through `lock-for-update?`, the migration-on-read path with `FOR UPDATE`). A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`.
- The offload normalizes each `SELECT fd.*` row before `upsert!`: the row builder omits NULL columns, jsonb metadata arrives as a `PGobject`, and a JSON `null` decodes to nil and must disappear instead of travelling as `:metadata nil`, which the update schema rejects.
- Offload pacing: none by deliberate decision. Sleeps only make sense when batching multiple transactions; the per-file transaction is short by construction and the scheduler run takes minutes per production metrics, so neither loop sleeps. `jobs/heartbeat` (pool-based liveness, ~1 write/s) stays on both paths.
- Offload metrics (bounded labels only, never ids or error text): `penpot_file_data_offload_rows_total{type}` and `penpot_file_data_offload_bytes_total{type}` (`type`: `main`/`snapshot`/`fragment`, successful runs only) plus `penpot_file_data_offload_timing_ms{result}` (`result`: `ok`/`skip`/`error`). Helpers live in `app.jobs.metrics`; the job-def requires `::mtx/metrics`.
- Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`.
- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/touch-storage-object` with the row's `storage_object_id`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
- `storage-gc-touched` resolves the `file-data` reference with `SELECT EXISTS (SELECT 1 FROM file_data WHERE storage_object_id = ?)`, an indexed lookup that no longer parses the object metadata.
- A row with `backend='storage'` and no readable object makes `resolve-file-data` raise `:missing-storage-object` with the file and row ids instead of failing later in the storage backend.
- The `file-id` and row id kept in the storage object metadata are informational for diagnostics; nothing parses them to resolve the reference (the reference is the column).
## File Change
- `file_change` is partitioned by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition (`enable_partitionwise_join` is set on every pooled connection).
- The automatic migration (`0158-partition-file-change`) refuses to run when `file_change` is above 10 GiB total size or 2M rows (`pg_total_relation_size` + `pg_class.reltuples`, catalog reads). It analyzes the table first, because `reltuples` is -1 without statistics. It raises with the path of `operations/2.20.0-file-change-partition.md`, because the copy happens inside the migration transaction and a big instance would restart in a loop. Below the guard it raises `statement_timeout` to 30 minutes for the copy and the index builds, and analyzes the new partitions at the end, after the copy and the indexes, so the planner does not start with empty statistics.
- Its primary key is `(file_id, id)`: a partitioned table needs the partition key in the primary key, so every access filters by `file_id`. From 2.20 the four snapshot commands (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot`, `unlock-file-snapshot`) require the `file-id` param, because they use it to prune the partition; without it they answer `:params-validation`.
- It has a `kind` column: `1` is a snapshot (it has a label and its data lives in `file_data`; the read-only `legacy-db` backend keeps old payloads in `data`) and `2` is a revision (it has no label and it holds the changes). The values are in `app.features.file-changes`.
- `kind` is a deliberate reserve, not a leftover: today no query filters by it, the queries still tell a snapshot from a revision by its label (`label IS NOT NULL`), and `kind` takes over when they are rewritten. No constraint ties `kind` and `label` together on purpose: the day `kind` is the source of truth, that constraint would have to go. It stays nullable on purpose: the previous version inserts without it, and a rollback to it has to keep writing; the `NOT NULL` comes in a later migration.
- Every write of the application to a row of `file_change` moves `updated_at` to the moment of the write (`lock-by!`, `unlock!`, the snapshot rename and delete, `delete-object`, the restore of a file). The actions of the foreign keys (`ON DELETE SET NULL` on `locked_by` and `profile_id`) change the row without touching it; during the partition both tables carry the same keys, so both receive them. The runbook of the partition uses it as the watermark of its delta passes: a write of the application that leaves it behind is lost at the cutover.
- `file_change_legacy` is a frozen copy of the previous layout of the partition: it keeps the rows and the indexes of the old table. A new instance (empty table) does not keep it. On an upgraded instance the table stays until the operator drops it by hand days later (`DROP TABLE file_change_legacy;`, see `operations/2.20.0-file-change-partition.md`); nothing drops it automatically.
- A revision carries only the delta of a save (the changes). The features of the file are not repeated on it: nobody read them from there.
- A snapshot carries the features and the migrations of the file.
- It keeps the `data` column for reading only: migration `0157-migrate-legacy-file-data` moved the existing blobs to `file_data`, and `legacy-db` remains as a read-only backend for the payloads still in `data`. `fdata/upsert!` rejects `legacy-db` and the `file-data-backend` default is `db`. It has no `data_ref_id` or `data_backend` columns: nothing read them since the file storage refactor.
- It has no index for the system snapshots. The old one filtered them by `data IS NOT NULL`, and with the payload living in `file_data` the column no longer tells snapshots apart. The new one could only filter them by `deleted_at IS NULL`, but the system snapshots always carry a deletion date in the future, so it would never match a row. The query that would use it filters that date with an "or", which no partial index can serve.
- The `objects-gc` purge groups each chunk by `file_id` and locks it with `file_id = ?` and `id = ANY(?::uuid[])`: a tuple `IN ((file_id, id), ...)` gives the planner no constant partition key, so it scans all 16 partitions in every chunk. Each group is one `FOR UPDATE SKIP LOCKED` query inside the chunk transaction.
## Metrics ## Metrics

View File

@ -9,6 +9,7 @@
- Params with `::sm/params` are decoded/conformed through the JSON transformer and successful IObj results get `:encode/json` metadata. Legacy spec conforming only applies when no Malli params schema exists. Client params are stripped of qualified keys (`d/without-qualified`) before merging with the server auth context, so request bodies cannot override `::profile-id`, `::auth-type`, or `::token-perms`. - Params with `::sm/params` are decoded/conformed through the JSON transformer and successful IObj results get `:encode/json` metadata. Legacy spec conforming only applies when no Malli params schema exists. Client params are stripped of qualified keys (`d/without-qualified`) before merging with the server auth context, so request bodies cannot override `::profile-id`, `::auth-type`, or `::token-perms`.
- Params schemas are open by default, so undeclared client keys reach the handler unless the map is `:closed true`. Creation commands (`create-file`, `create-project`, `create-team`, `create-team-with-invitations`, `upload-file-media-object`, `create-file-media-object-from-url`, `assemble-file-media-object`) use closed schemas: a client-provided `:id` fails with `:params-validation`. Their internal creation functions still accept an optional explicit `:id` for imports, duplicates and deterministic test fixtures. - Params schemas are open by default, so undeclared client keys reach the handler unless the map is `:closed true`. Creation commands (`create-file`, `create-project`, `create-team`, `create-team-with-invitations`, `upload-file-media-object`, `create-file-media-object-from-url`, `assemble-file-media-object`) use closed schemas: a client-provided `:id` fails with `:params-validation`. Their internal creation functions still accept an optional explicit `:id` for imports, duplicates and deterministic test fixtures.
- Nil RPC bodies become HTTP 204 unless explicit status metadata is present. Stream bodies default to `application/octet-stream` when no content type is set. - Nil RPC bodies become HTTP 204 unless explicit status metadata is present. Stream bodies default to `application/octet-stream` when no content type is set.
- The upload schema (`media.v/schema:upload`) takes a `:path` from the caller, and a Transit body can tag any string as a `path`, so a command that reads such an upload reads a file the client named. `create-import-binfile-job` avoids that boundary by accepting only a chunked upload (`:upload-id`); the commands that still take a plain file (the legacy `import-binfile`, the media and font uploads) trust it, and the guard belongs where uploads are accepted, not in each command.
## Concurrency limiter (CLIMIT) ## Concurrency limiter (CLIMIT)
@ -17,35 +18,48 @@
- The waiting counters are a field of the `ConcurrencyLimiter` component (a `ConcurrentHashMap` of name to `AtomicLong`), so they start at zero, die with the component, and the sampler that closes over that map republishes only its own names every 15 s; the sampler stops with the component in `halt-key!`. `impl-invoke` pairs one decrement with one increment through a per-call flag: the flag turns a second call into a no-op and the `finally` guarantees that there is one. The increment sits inside the `try` and the decrement lives only in the acquired callback and the `finally`, so a handler that throws, a rejection and a throwable from the publishing path all leave the count where it was and the thread without the limiter marked as held (a leftover mark would skip that limiter for that key for ever). Concurrent `inc`/`dec` publications of two threads can land out of order and the value can read low for a moment; the republish corrects it. - The waiting counters are a field of the `ConcurrencyLimiter` component (a `ConcurrentHashMap` of name to `AtomicLong`), so they start at zero, die with the component, and the sampler that closes over that map republishes only its own names every 15 s; the sampler stops with the component in `halt-key!`. `impl-invoke` pairs one decrement with one increment through a per-call flag: the flag turns a second call into a no-op and the `finally` guarantees that there is one. The increment sits inside the `try` and the decrement lives only in the acquired callback and the `finally`, so a handler that throws, a rejection and a throwable from the publishing path all leave the count where it was and the thread without the limiter marked as held (a leftover mark would skip that limiter for that key for ever). Concurrent `inc`/`dec` publications of two threads can land out of order and the value can read low for a moment; the republish corrects it.
- The `app.rpc.climit` log lines carry `queue`, `queue-max`, `permits`, `elapsed`, `code` and `reason`, and their `id` carries the key (`name/<profile or key>`). That is the only place where the profile behind a queue is visible: a pile-up is chased there, on demand, not in the metrics. The `queue` of the line is the calls inside the limiter minus its permits, so it reads 0 for a lone caller that times out: that line goes out at TRACE instead of WARN, and the rejection count has to come from the metric. - The `app.rpc.climit` log lines carry `queue`, `queue-max`, `permits`, `elapsed`, `code` and `reason`, and their `id` carries the key (`name/<profile or key>`). That is the only place where the profile behind a queue is visible: a pile-up is chased there, on demand, not in the metrics. The `queue` of the line is the calls inside the limiter minus its permits, so it reads 0 for a lone caller that times out: that line goes out at TRACE instead of WARN, and the rejection count has to come from the metric.
- Nothing in the limiter path has a timeout unless the config sets one, and `psm/acquire` then calls `Semaphore.acquire`, which parks the calling thread forever. A parked thread still counts as in-system for every outer limiter of its chain, so the outermost one (the method's own by-profile limiter) stops granting permits and answers 429 to everything. A limiter without `:queue` gets an unlimited queue and therefore never rejects: its pile-up is silent, with no 429 and no log line except the WARN `enqueued`/`acquired` lines of the keyed id. - Nothing in the limiter path has a timeout unless the config sets one, and `psm/acquire` then calls `Semaphore.acquire`, which parks the calling thread forever. A parked thread still counts as in-system for every outer limiter of its chain, so the outermost one (the method's own by-profile limiter) stops granting permits and answers 429 to everything. A limiter without `:queue` gets an unlimited queue and therefore never rejects: its pile-up is silent, with no 429 and no log line except the WARN `enqueued`/`acquired` lines of the keyed id.
- Deadlock shape between a command's own limits and the root ones (fixed): a command declaring its own limits held `process-image/by-profile` before `root/by-profile`, while a command with no limits of its own takes the roots first and then re-enters with the same keyed limits from inside its handler (`create-file-media-object-from-url`, `upload-photo`) and waits for them holding a root permit. Two calls of one profile then held the same two limiters inverted and parked for good: 69 threads at 0 CPU, an audit limiter with 0 acquisitions in 24h and 100% 429s, on an instance that never served a 204. - Deadlock shape between a command's own limits and the root ones (fixed): a command declaring its own limits held `process-image/by-profile` before `root/by-profile`, while a command with no limits of its own takes the roots first and then re-enters with the same keyed limits from inside its handler (`create-file-media-object-from-url`, `upload-photo`) and waits for them holding a root permit. Two calls of one profile then held the same two limiters inverted and parked for good: 69 threads at 0 CPU, an audit limiter with 0 acquisitions in 24h and 100% 429s, on an instance that never served a 204. Both commands declare their limits now, so the shape cannot happen any more: the root limits are taken last and every path takes the limiters in the same order.
- `climit/wrap` now appends the root limiters after the command's own, so `root/global`/`root/by-profile` end up outermost and every path meets them first; the nested `climit/invoke!` chains carry only the command's own limits and run inside them, which puts both paths in the same order. `impl-invoke` also keeps, per thread, the set of limiters that thread is inside of and runs the handler inline when a call asks for one it already holds, instead of queueing behind itself. That check relies on `pbh/invoke!` running the handler on the calling thread; the mark is removed in a `finally`. Commands declare the by-profile limit before the global one, which makes it the outermost of the pair: keep that declaration order, the reverse one would put two paths back in inverted order. The two nested call sites (`create-file-media-object-from-url`, `upload-photo`) still share `process-image` with the enclosing call instead of getting a limiter of their own, so the comment there about keeping the download out of that limit no longer holds. - `climit/wrap` takes the command's own limits before the root ones: the root limiters are appended to the sequence first and the last element of the sequence is the outermost one, so `root/global`/`root/by-profile` end up innermost. A request waiting for its own limits no longer holds the root permits of its profile, which is what kept a pile-up of heavy requests from blocking the other commands of that profile. Every acquisition goes through that wrapper, so every path takes the limiters in the same order: there is no way to enter the limiter from inside a handler, and a command that calls another command internally would take the limiters twice (the second call waits for a permit the first one holds and answers 429 when the timeout runs out). The key of a limit is a function of the params (a keyword is looked up in them, a function is applied to them, any other value is the key itself). Commands declare the by-profile limit before the global one, which makes it the outermost of the pair, and a test fails when the relation between all the limits the commands declare cannot be sorted: two commands that take the same limits in opposite order, or a ring of three, leave the paths of one profile holding them in opposite order.
- The shipped `backend/resources/climit.edn` sets a timeout on every limiter, 60s for operations that can be slow and 30s otherwise, with a test that fails when an entry ships without one, and `init-key` logs a WARN for every entry without a timeout. The production file lives outside the repository: mirror it there, and remember that a parked thread only goes away with a restart or a deploy. - The two commands that used to enter climit from inside their handler declare their limits now, so the whole command runs inside them: the download of an external image takes a `process-image` permit, and that limit has one permit per profile, so a slow download holds the image processing of that profile for as long as it takes. It takes the global permit of that limit too (8 permits, queue 20), so eight slow downloads of eight different profiles leave the image processing of every profile waiting. The copy of the body has no timeout (the 30 s of the request covers the connection and the headers), so a server that stops sending keeps the permits until the operating system gives up. The storage write of the profile photo runs inside its limit too.
- The shipped `backend/resources/climit.edn` sets a timeout and a queue on every limiter, 60s for operations that can be slow and 30s otherwise, with a test that fails when an entry ships without one, and `init-key` logs a WARN for every entry without a timeout. The queue matters because the root limits no longer bound the waiters: a command takes its own limits first, so a pile-up waits in the queue of that limit. The file and the declarations have to list the same ids: a `::climit/id` without an entry makes `wrap` log `no config found for specified queue` and run the handler with no limiter at all, and a test fails when a command declares an id the file does not list. Only key a limit by profile when the work contends on something shared (`update-file`, images, fonts, thumbnails, the profile row): a global cap on a cheap command adds a rejection path nobody needs, which is why `profile-plugin-ops` and `submit-audit-events` declare only `by-profile`. The production file lives outside the repository: mirror it there, and remember that a parked thread only goes away with a restart or a deploy.
## DB helpers ## DB helpers
- Most `app.db` helpers accept a pool, connection, or map containing `::db/pool` / `::db/conn`; preserve that convention in shared code. - Most `app.db` helpers accept a pool, connection, or map containing `::db/pool` / `::db/conn`; preserve that convention in shared code.
- `db/tx-run!` uses `next.jdbc.transaction/*nested-tx* :ignore`: nested transaction calls reuse the outer transaction, not a savepoint. Use explicit savepoints when nested rollback semantics matter. - `db/tx-run!` uses `next.jdbc.transaction/*nested-tx* :ignore`: nested transaction calls reuse the outer transaction, not a savepoint. Use explicit savepoints when nested rollback semantics matter.
- Whether a call is nested is decided by the CONNECTION, not by an ambient flag: `db/transact!` binds `*after-commit-conn*` around the body, and a transactable that is not identical to that connection is an independent unit of work that owns its own after-commit context. A `tx-run!` that passes a pool while a transaction is open therefore commits on its own and drains its own callbacks, instead of leaving them in a context that may roll back.
- `after-commit` callbacks run on commit only. A transaction marked `::db/rollback` (used by `srepl` and `app.tasks/offload-file-data`) commits nothing, so it runs none of them.
- `db/run!` opens/reuses one connection but does not create a transaction. - `db/run!` opens/reuses one connection but does not create a transaction.
- `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Worker task props use Transit and are decoded with `decode-transit-pgobject`. - `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Job params (`job.params`) are plain JSON, decoded with the job-def decoder (`app.jobs/decode-params`); only the legacy `task` props were Transit (`decode-transit-pgobject`). Job results (`job.result`) are plain JSON too: they are read with the result decoder of the job-def (`app.jobs/decode-result`), and a result map keyed by uuid must declare its schema (`[:map-of ::sm/uuid ...]`, the import resolution does) or every key comes back as a keyword and every uuid as the text it is on disk.
- Advisory transaction locks accept UUIDs or ints. UUID locks are hashed using a zero-UUID seeded siphash. - Advisory transaction locks accept UUIDs or ints. UUID locks are hashed using a zero-UUID seeded siphash.
## Workers and cron ## Workers and cron
- Task queues are tenant-prefixed. Submit dedupe only removes not-yet-due `new` tasks with the same name/queue/label; it does not dedupe due, scheduled, retry, running, or completed work. - `job` carries `tenant` (the instance the job belongs to) and a bare `queue`. Several instances share one database and the tenant is what keeps an instance from claiming another environment's jobs, so every query that acts on one instance's work filters on `tenant`: dispatch claim, orphan sweep, lost-job reschedule, cron no-overlap count, submit dedupe. The `job__dispatcher__idx` leads with `tenant` (equality) then `(priority DESC, scheduled_at)`; `queue` does not lead because the dispatcher claims every queue of its tenant. The `<tenant>:<queue>` prefix survives only in the Redis key, composed once by `app.worker/queue-key`.
- The dispatcher selects `new`/`retry` tasks with `FOR UPDATE SKIP LOCKED`, marks them `scheduled`, and publishes Redis payload `[id scheduled-at]`. The runner skips Redis messages whose scheduled timestamp no longer matches DB state. - Submit dedupe only removes not-yet-due `new` jobs of the same tenant with the same name/queue/label; it does not dedupe due, scheduled, retry, running, or completed work. The `::jobs/skip-if-active` submit option goes further: it skips the insert (returning `nil`, recording `penpot_jobs_skipped_total`) when a live job (`new`/`scheduled`/`running`/`retry`) already exists for the same tenant/name/queue/label, requires an explicit non-blank label, and carries the same best-effort cross-backend race caveat as dedupe.
- Lost `scheduled` tasks are rescheduled after about 5 minutes; `running` tasks older than about 24 hours are marked failed as orphans. - Jobs metrics rule (one event, one family): counters exist only for events without latency; when an event has a histogram, its `_count` is the counter (no twin counter). `dispatched` (pushes) vs `queue-wait` (claims) are deliberately different events — do not merge them.
- A task handler that is missing or returns an invalid result currently defaults to completed after warning. Throwing with `ex-data :type ::retry` controls retry behavior; `:strategy ::noop` retries without incrementing retry count. - The dispatcher selects `new`/`retry` jobs with `FOR UPDATE SKIP LOCKED`, marks them `scheduled`, and publishes the JSON payload `[id scheduled-at]` to the `penpot.worker.queue:<tenant>:<queue>` Redis list. The runner skips Redis messages whose scheduled timestamp no longer matches DB state.
- Cron jobs lock their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only. - Lost `scheduled` jobs are rescheduled after 5 minutes; `running` jobs untouched longer than `:jobs-lease` (default 30 min) are marked `aborted` with `app.jobs/orphan-error` (`type :internal`, `code :orphan`). `aborted` is a system-side terminal state like `cancelled` (never retried, swept by retention like other terminals, reported with an error log); long jobs must `heartbeat` per chunk so a healthy backlog is never mistaken for a dead process. The `:progress` report is optional and only feeds the durable log. A beat whose write finds the job no longer active raises an `:interrupt` (`code :job-interrupted`) instead of counting zero: it is the internal stop-the-work signal, not an error, so neither the listener nor the runner logs it as one.
- An orphan gets its `end` event (`outcome "aborted"`) from the sweep itself: one bulk `INSERT ... SELECT` in the same transaction as the bulk `UPDATE`, never one insert per orphan. No msgbus notification goes out for it; orphans alert through the error log and the orphaned counter.
- `job_event` is append-only and deleted by cascade with the job. `progress` is a row there, not a column: no Redis and no mutable column to keep in sync.
- `job` is deletion-protected at the SQL level (`deletion_protection__tgr`): a stray `DELETE FROM job` raises instead of dropping the row and its `job_event` cascade without touching the referenced `storage_object`. The only intended deleters, the jobs GC and the submit dedupe, disable the guard with `SET LOCAL rules.deletion_protection TO off` inside their transaction; tests use `th/db-force-delete`.
- A missing job-def raises (`:no-job-definition`) instead of completing. Throwing with `ex-data :type ::retry` still controls retry behavior; `:strategy ::noop` retries without incrementing retry count.
- Cron entries claim their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, submit one `job` row per entry when no active instance exists (no-overlap), and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only.
- Integrant params schemas name injected components as bare `::ns/key` entries (pool, metrics, msgbus, job defs), and a bare entry is a malli registry lookup, not `any?`: the owning namespace must `sm/register!` it at load, or the backend crashes at boot with `:malli.core/invalid-schema` instead of asserting. Unit tests never build those keys, so `backend-tests.worker-test` compiles the three worker schemas and `backend-tests.http-websocket-test` compiles the revocation-watcher one to catch it. The assertion only runs with `PENPOT_FLAGS=enable-backend-asserts`, so without it a broken schema reaches production and only crashes at boot.
- Whether a bare entry works has nothing to do with its position or with how many of them there are: it is a registry lookup, so it works only if that exact keyword was `sm/register!`'d. `[:map ::db/pool ::metrics/metrics]` is fine, `[:map :app.aaa/unregistered :app.ccc/unregistered]` throws. When a component is **not** registered as a malli type, spell it as an explicit pair instead of registering a type just to satisfy the lookup — `[:app.nitrate/client {:optional true} [:maybe :map]]` is how `auth.oidc/schema:routes-params` and `loggers.audit.archive-task/schema:audit-log-archive-params` declare it. `{:optional true}` matters because `init-key` yields `nil` when the owning feature flag (`:admin-console`) is off, and a required key would then fail the assert on a perfectly normal boot.
- `sm/valid?` catches every `Throwable` and returns `false`, so an uncompilable schema and a genuinely invalid value both surface as a bare `Assert failed:` with no cause. When an `assert-key` fires with no message, compile the schema by hand to see the real exception.
## Config and HTTP/session middleware ## Config and HTTP/session middleware
- `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them. - `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them.
- Duration config values are parsed as ISO-8601 with the `PT` prefix, so a day is written in hours (`:jobs-retention` and `:jobs-user-ttl` default to a week, and an operator writes `168h`): `7d` does not parse, and the value fails validation at boot instead of being silently ignored.
- `app.main` wires `PENPOT_DATABASE_*` into `::db/pool`, and `app.db/create-datasource-config` forwards pgjdbc's own knobs as `dataSourceProperties`, which `Properties.getProperty` silently drops unless the value is a String (hence the `str`). A query parameter of the same name in `PENPOT_DATABASE_URI` (for example `?prepareThreshold=0`) wins over the property: pgjdbc ignores the Properties value when the URL carries it, so the URI is the place to look when a variable seems to do nothing. The prepared statement cache is three optional `PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_*` keys: the threshold is the executions before a statement is server-prepared and accepts `-1` or non-negatives (`{:min -1}`; the driver clamps any other negative to `-1`, so rejecting it only turns a typo into a boot error), its `0` stops server-side prepared statements while the driver's client-side cache of parsed queries stays in use, `..._QUERIES` and `..._SIZE` reject negatives and `0` in either disables that cache, and the size is bytes rounded up to the whole MiB the driver accepts. Do not describe the threshold `0` as disabling the cache: it only stops the server-side prepares.
- `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag. - `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
- The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags. - The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413. - Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413.
- Response formatting negotiates with `Accept` or `_fmt=json`. Transit is the default for collection/boolean bodies; JSON encoding has special pointer-map handling. - Response formatting negotiates with `Accept` or `_fmt=json`. Transit is the default for collection/boolean bodies; JSON encoding has special pointer-map handling.
- Auth prefers the session cookie token before the `Authorization` header. Headers may be `Token` or `Bearer`. Only `kid=1`/`ver=1` tokens are decoded as session tokens; anything else is left unauthenticated (legacy v1 tokens were removed). - Auth prefers the session cookie token before the `Authorization` header. Headers may be `Token` or `Bearer`. Only `kid=1`/`ver=1` tokens are decoded as session tokens; anything else is left unauthenticated (legacy v1 tokens were removed).
- Shared-key auth requires `x-shared-key` as `<key-id> <key>` and stores the lowercased key id on the request. If no shared keys are configured it always rejects. - Shared-key auth requires `x-shared-key` as `<key-id> <key>` and stores the lowercased key id on the request. If no shared keys are configured it always rejects. Management methods declare the key ids they accept with `::rpc/allowed-shared-keys`; a valid key outside that set answers 403 `:shared-key-not-allowed`, and a method without the metadata accepts any configured key.
- Session management uses DB storage unless the DB pool is read-only, then falls back to the in-memory manager. Sessions use only the v2 UUID model (`http_session_v2`); legacy string ids were removed. - Session management uses DB storage unless the DB pool is read-only, then falls back to the in-memory manager. Sessions use only the v2 UUID model (`http_session_v2`); legacy string ids were removed.
- Session cookies are renewed when `modified-at` is older than the 6h renewal interval. SameSite is `none` for CORS, otherwise strict/lax based on config. Session lifetime config and GC: `mem:backend/session-expiration`. - Session cookies are renewed when `modified-at` is older than the 6h renewal interval. SameSite is `none` for CORS, otherwise strict/lax based on config. Session lifetime config and GC: `mem:backend/session-expiration`.
@ -59,7 +73,9 @@
## Metrics recording ## Metrics recording
- `app.metrics/run!` is safe by default: a recording failure never throws (a metrics bug must not change the behavior of the measured operation). The first failure per metric id logs at `warn`, later ones at `debug`. The `instance` precondition is a plain assert (the backend enables `:backend-asserts`), and the collector lookup sits outside the recording guard, so a missing instance fails hard even when asserts are disabled. `::mtx/metrics` is required by the storage, s3-backend, and db-pool schemas; `app.db` wires the prometheus `MetricsTrackerFactory` unconditionally. Storage-specific metric contracts: `mem:backend/storage`. - `app.metrics/run!` is safe by default: a recording failure never throws (a metrics bug must not change the behavior of the measured operation). The first failure per metric id logs at `warn`, later ones at `debug`. The `instance` precondition is a plain assert (the backend enables `:backend-asserts`), and the collector lookup sits outside the recording guard, so a missing instance fails hard even when asserts are disabled. `::mtx/metrics` is required by the storage, s3-backend, db-pool and jobs APIs; jobs helpers never silently skip a missing metrics instance. The resolver is `app.metrics/instance` and it raises, unlike `run!`.
- `app.msgbus` (`pub!`, `sub!`, `purge!`) keeps taking the **instance** on purpose: msgbus is what produces it, and `app.rpc.notifications` already shows the caller passing a cfg. `app.jobs` resolves `::mbus/msgbus` from the cfg internally, and `insert-event` keeps its explicit pre-insert check so a profile job never loses its notification.
- `app.db/after-commit` is the boundary for SQL-backed metrics: callbacks are drained by the commit of the transaction that owns the connection they were registered on, and discarded when it rolls back. Because nesting follows the connection, that transaction is the outermost one only for a real nested call.
- `penpot_http_malformed_requests_total` (counter, no labels) counts requests whose body the parser could not read. `app.http.middleware/wrap-parse-request` emits it where it classifies the error, so the instance travels as a middleware argument (`[mw/parse-request (::mtx/metrics cfg)]`) and each service declares the definition with its own prefix (nexus: `nexus_http_malformed_requests_total`). The per-endpoint request counters of nexus do not count these requests: they sit inside the parse middleware, so the failure happens before them. - `penpot_http_malformed_requests_total` (counter, no labels) counts requests whose body the parser could not read. `app.http.middleware/wrap-parse-request` emits it where it classifies the error, so the instance travels as a middleware argument (`[mw/parse-request (::mtx/metrics cfg)]`) and each service declares the definition with its own prefix (nexus: `nexus_http_malformed_requests_total`). The per-endpoint request counters of nexus do not count these requests: they sit inside the parse middleware, so the failure happens before them.
## Log-derived metrics ## Log-derived metrics
@ -71,21 +87,32 @@
- The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation. - The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation.
- Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`. - Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
- SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing. - SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
- The SVG sanitizer (`app.media.svg`) is a **blocklist**, not an allowlist: it drops dangerous elements and attributes by name and escapes every text and attribute value when it writes the tree back, so a payload hidden in a value cannot become live markup again. The allowlist rework is [#12190](https://github.com/penpot/penpot/issues/12190) and lands in media-processor, not here. `sanitize-svg` also tracks the namespace bindings through the recursion and drops any element that is not in the SVG namespace, so a document whose root is not an `svg` element in the SVG namespace is rejected and a child that rebinds `xmlns` to XHTML (`iframe`/`srcdoc`/`meta`) is dropped with its subtree; it compares the scheme of an URL the way the browser does (dropping tab/CR/LF/spaces), so `java&#x09;script:` does not slip through the `javascript:` filter. The paths that store an uploaded SVG as its final object go through `app.media.svg/sanitize-svg-upload` (media upload, the URL download, both thumbnail writers and the organization logo); the three binfile imports go through `app.binfile.common/sanitize-imported-svg`, which calls `sanitize-svg` directly. The profile picture is not an SVG: `import-profile-picture` ignores an `image/svg+xml` download, because the avatar is always rendered as an image. The caller checks the size before calling it: the helper reads the whole file, and the size limit applies to the upload, not to the sanitized copy, which can be up to six times bigger.
- Upload handlers run the `media.v` checks in a fixed order: `check-upload-path`, then `check-media-type`, then `check-media-size` (or `check-font-size` for fonts). The path check gates every later read: `check-media-type` sniffs the bytes at `:path` and `check-media-size` reads `fs/size`, so a path the client named outside `allowed-upload-prefixes` must be rejected before either touches the disk. `check-media-size` leans on that and answers a file it cannot read with the path code (`:params-validation`), not `:media-max-file-size-reached`. Keep the order when touching or adding a handler; two branches that reorder these lines conflict.
- Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8. - Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
- Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error. - Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error.
- Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts. - Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts.
## File data persistence ## File data persistence
- File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data. - File data backends for writing are `db` and `storage`; `legacy-db` is a read-only alias for old payloads. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores nil data and points at the object with the `storage_object_id` column (FK `NO ACTION DEFERRABLE`).
- `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob. - `fdata/upsert!` touches the storage object the persisted row pointed at before replacing it; the pointer travels in the params (`file->file-data-params` from the read file map, `nil` for a new row, the row for the offload) and the params of `update-file!` cannot override it. `binfile.common/update-file!` merges the read file map with the fields to change, so the pointer survives; `insert-file!` drops it because an INSERT never replaces a persisted object. The column is the only source of truth and the metadata no longer carries `storage-ref-id` (the migration removes it from the backfilled rows and the write path never stores it); the release is a coordinated restart when the storage backend is in use (the previous code cannot read the new rows), while the default `db` backend reads the same data column and tolerates a rolling deploy. Responses built from a file map go through `bfc/file->response`, which strips the pointer and a residual `storage-ref-id` from the metadata.
- Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written. - Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written. A pointer keeps its id while nothing changes; content (`assoc`/`without`) and metadata (`with-meta`) changes produce a fresh id, so a fragment row is never rewritten after it is written. Re-assoc with an identical value returns the same map in array and hash maps alike, so a save keeps the fragment id of every page it does not change.
- `enable-objects-map` skips unloaded pointer pages: their fragment was already wrapped when it was written, and loading it only to wrap it again costs one page read per untouched page on every save. A page with a plain fragment is wrapped when it is first loaded or edited; a data migration that enables `fdata/objects-map` must rewrite the data itself.
- Snapshots share fragment rows with the live file (they are stored with pointers, not materialized). `file-gc` marks every fragment referenced neither by the current main file nor by an unpurged snapshot; the file is read unrealized and unchanged pages keep their ids via reattach, so a pass over an unchanged file writes no new fragments.
- A save attaches its automatic snapshot before applying pending file migrations: the snapshot keeps the pre-migration pointers, data and migration set, and restoring it leaves the next read to migrate the file. This also holds for files without the feature. The preview (`get-file-snapshot`) applies the pending migrations in memory with the live file's libraries and never persists them: the stored version keeps its data, the payload carries the snapshot's `:migrations` (an ordered set) and `fdata/objects-map` is realized for clients that do not declare it. The historical feature set is deliberately not checked against the enabled one: a version may predate a feature change. The window is narrow: the `get-file` RPC reads with `migrate?` on and its ETag hashes `available-migrations`, so a post-deploy read cannot be a 304 from the pre-deploy state and migrates before any save in the normal flow; the pre-migration snapshot only appears when the save is the first request to touch the file after the deploy (an open tab that saves without re-reading, or a non-UI writer). `create-file-snapshot` and the restore backup read with `migrate?` on, so those versions are stored already migrated.
- `:migrations` is always a `d/ordered-set`: `file-migrations/resolve-applied-migrations` builds one, `fmg/migrate` re-wraps the union after applying, and `file-snapshots/decode-snapshot` converts the `file_change` pgarray (as a plain vector, `set/difference` and `contains?` would compare indexes instead of names). A snapshot with NULL `:migrations` and an old `:version` predates the `file_change.migrations` column (SQL migration 0140): the preview regenerates the set from the stored version and applies that full chain, while `restore!` uses a fixed 67 baseline, so preview and restore can disagree for very old snapshots.
- The migration read path (`binfile.common/get-file*`) reads the `file` row once (`sql:get-file`, with its data columns; with the storage backend the blob lives in the object, not in the column) plus an explicit `resolve-applied-migrations`, because a row read does not resolve them. A read with no pending migration is completed by `attach-file-data` on that map: one row read and one migrations resolution. Only when a migration is pending is the row re-read under `FOR UPDATE` on the `file` row, where `need-migration?` is re-checked on the fresh map and the migrated file persisted (`binfile.common/migrate-file`); that re-read resolves the migrations again, and `migrate-file` once more after persisting, to return the applied set. The migration path holds two raw copies of the blob (the first read and the locked re-read) but a single decoded one: the alternative, a row-only probe before the read, avoided the first raw copy at the cost of an extra row read on every open, and was removed. `get-file` runs the read inside `db/tx-run!` (joining the caller's transaction when the cfg carries a connection) so the persist commits with the read; a read-only read (`:read-only?`) takes no lock and writes nothing, so it runs on a plain connection (`db/run!`) instead of opening a transaction of its own, and with a cfg that already carries a connection it still joins the caller's. It accepts a cfg map only and throws an `IllegalArgumentException` otherwise (`assert` is elided without `:backend-asserts`); a data read rejects `:lock-for-share?` (only row-only reads accept it), because the migration branch re-reads with `FOR UPDATE`, and a lock option needs a read-write transaction from the caller because PostgreSQL rejects row locks in a read-only transaction. A row-only read (`load-data?` false) resolves the applied migrations by default too (`:load-migrations?`), because the row can be completed later and the caller may need them; a caller that does not want them passes `:load-migrations? false`. Persisting is an optimization of the read (the read already returns migrated data): a row that changed between the first read and the locked read is not overwritten, and a pending migration discovered on the unlocked read is applied in memory only (the next read persists it under the lock). This was the only writer of the file row outside the lock protocol (save, library absorption, restore, `file-gc` and offload already take it); without it, a save that committed in between was overwritten by the stale map (lost content, `revn` moving back so the next save fails with `:revn-conflict`, and the object the save left as a frozen orphan). `migrate-file` captures `fdata/original-pointers` before realizing, reattaches unchanged positions after applying the migrations, and persists the reattached file; it returns that reattached map when it persists (the pages the migration does not change keep their pointers and fragment ids) and the realized map when it does not, so a read is plain only when the caller asks with `:realize?`, and a migration that adds `fdata/objects-map` skips the reattach so the pages get wrapped.
- `fdata/load-pointer` reads rows that a previous `file-gc` pass marked on purpose (explicit `::db/remove-deleted false`): the fragment is only gone once `objects-gc` purges it. The `get-file-fragment` RPC keeps rejecting marked rows.
- Bind the fragment loader through `fdata/with-pointers` (reads), `fdata/with-tracked-pointers` (writes with a loader) or `fdata/with-tracked` (writes with nothing to load yet, e.g. file creation), and `fdata/with-untracked` (validation passes that must track nothing) instead of hand-rolled `binding` forms. Two direct bindings remain on purpose: the fine-grained tracked toggles of `app.rpc.commands.files-update` (validation/repair turn tracking off, restore the existing atom and read it before repairing), and `binfile.common/encode-file` (a fresh tracked set inheriting the current one). Converge them only if atom-scoped macros (`with-tracking`, `with-inherited-tracking`) are added; today the extra indirection buys nothing.
- Snapshot creation takes `FOR UPDATE` on the file row: both `create-file-snapshot` and the backup read of `restore-file-snapshot` hold it from the read to the insert, so they serialize with a `file-gc` pass (`FOR UPDATE ... SKIP LOCKED`) and with a concurrent save, and never pin fragments the pass is about to mark. The lock is exclusive and not `FOR SHARE` because the migration-on-read path may re-read the same row with `FOR UPDATE`: a shared lock that asks for the exclusive one deadlocks two concurrent readers (one creating a snapshot, the other saving, both seeing a pending migration). `get-file-snapshot` realizes the snapshot pointers with the live file id (fragment rows are keyed by file, not by snapshot).
- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders. - `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
- A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data. - A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data.
- Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision. - Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision.
- File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only written by the `legacy-db` backend, plus the rows written before the 2025-07 file storage refactor. - File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only read by the `legacy-db` backend (it is no longer written), plus the rows written before the 2025-07 file storage refactor.
- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. - Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. `0157-migrate-legacy-file-data` moved the labelled leftovers to `file_data` and deleted the unlabelled payload rows, so from 2.19 on the label is the only marker there is.
- The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again. - The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again.
- Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter. - Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter.
- A deletion date is assigned once and never moved: `restore-files` (`app.rpc.commands.files`) unmarks only durable history, the `:file` method of `app.tasks.delete-object` dates only rows that have no date yet, and `objects-gc` collects only rows that carry one. Unmarking a revision leaves it invisible to `objects-gc` for ever, because nothing sets that date again. - A deletion date is assigned once and never moved: `restore-files` (`app.rpc.commands.files`) unmarks only durable history, which is `label IS NOT NULL` (the unlabelled rows that still carry a legacy payload are removed by `0157-migrate-legacy-file-data`, so restore leaves them dated), the `:file` method of `app.tasks.delete-object` dates only rows that have no date yet, and `objects-gc` collects only rows that carry one. Unmarking a revision leaves it invisible to `objects-gc` for ever, because nothing sets that date again.
- Binfile v1/v3 import and the v3 export `get-file` step run inside one long transaction. They raise `idle_in_transaction_session_timeout` to 30 min (`app.binfile.common/transaction-idle-timeout-ms`) instead of disabling it, so a connection left idle while the ZIP is decoded or the file data is realized is bounded; `statement_timeout` stays at 300 s. - Binfile v1/v3 import and the v3 export `get-file` step run inside one long transaction. They raise `idle_in_transaction_session_timeout` to 30 min (`app.binfile.common/transaction-idle-timeout-ms`) instead of disabling it, so a connection left idle while the ZIP is decoded or the file data is realized is bounded; `statement_timeout` stays at 300 s.
- `update-file` with `:changes-with-metadata` groups that all carry `:commit-id` is partial: fast path applies and validates everything once; on a change failure it bisects the commits for the longest prefix that applies and validates (each attempt applies a prefix from the original data and runs referential integrity + file schema; ~log2 K attempts, none for a single commit, the first attempt's error reused), keeping that prefix (a commit that repairs an earlier one can keep both) and answering 200 with `:rejected {:commit-ids :failed-commit-id :error}`, plus `:preexisting-damage true` when the data the first rejected commit touches already fails validation without it (`preexisting-damage?`: one more referential + schema check of its pages/components on the base file, the accepted prefix or the original; the frontend then says the page data is damaged). Only accepted changes reach `file_change`, persistence and the msgbus; a fully rejected request writes and broadcasts nothing (revn unchanged). Flat `:changes` or groups without ids stay all-or-nothing. Before each attempt (`apply-prefix`, which returns `{:file :tracked :refs}` or the cause) `pmap/*tracked*` and the `cpc/*state*` atom are reset to their pre-attempt values (at the end `apply-commits` sets those of the kept prefix): a pointer map already `modified?` keeps its id when changed again, so the failed version would be persisted. DB/IO/interrupt errors and `:not-found` fragments (`change-failure?`) fail the whole request, never reject a commit. A rejection whose ex-info has no `:type` reports `:internal :unexpected-error`. `app.http.errors` answers 503 (`:server-error` + code) for transient SQLSTATEs (classes 08/53, 57P01–3, 40001, 40P01), `SQLTransientException`, `InterruptedException` and only the `IOException`s that pass on their own (`transient-io-exception?`: `SocketException`, `InterruptedIOException` incl. socket timeouts, `UnknownHostException`, `HttpTimeoutException`), which the client's save retries (and GETs retry by themselves); any other `IOException` (EOF from a corrupt blob, Jackson parse errors, missing files) stays 500 `:io-exception`, as do other SQL errors. With the `:debug-save-faults` flag, a `:set-plugin-data` file change in namespace `:penpot-debug` keyed `save-fault:<kind>` raises that fault in `apply-changes` (used by `debug.failNextSave`/`commitInvalidChange`). With a schema check running (hard schema validation without skip-validate, or soft) and 2+ commits, the touched pages' objects are decoded on the original file first (`decode-objects`; referential validation alone reads few objects, so no pre-decode then): objects-map caches decoded values per instance and a derived map copies the cache, so otherwise every attempt re-decodes the page (~260 ms at 10k shapes). Validation step timings (`with-validation-timing`, upstream `penpot_rpc_update_file_validation_timing` + the `update-file` debug log) come from the first attempt only (retries run with a cfg without metrics and timings); the whole bisection is one `retry` step sample, and the rejection log carries `:attempts`/`:elapsed`.

View File

@ -3,12 +3,34 @@
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`. JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs. - READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
- Test DB/Redis come from the `PENPOT_TEST_*` env vars already set in the shell (`PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`, ...); per instance they point at `penpot_test` / `penpot_test_wsN` and valkey DB `6+N` (`mem:devenv/core`). Do not override them with values copied from CI logs or other sources unless the user asks (e.g. "use a temporary database"); the CI env (`redis://redis/1`, `postgresql://postgres/penpot_test`) only works inside the CI docker network.
- The test database may have migrations applied from newer versions (it is shared dev infra). Reinitializing it — `dropdb`/`createdb` of the `PENPOT_TEST_*` database only — is explicitly permitted when that blocks a test run; never drop the main `penpot` database or another instance's test database.
- All CLI commands must be executed from the `backend/` subdirectory. - All CLI commands must be executed from the `backend/` subdirectory.
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed. - Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
- `clojure.core/assert` is elided at compile time unless `PENPOT_FLAGS` contains `enable-backend-asserts` (`app.config` sets `*assert*` from it). `scripts/ci` and `tests-backend.yml` export the flag; a bare `clojure -M:dev:test` (or `--focus` run) without it silently skips every precondition assert. - **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand.
- **Why the flag matters:** `clojure.core/assert` is elided at compile time when `*assert*` is false, and `app.config` only sets that var from the flags when it loads. The test fixture redefines the flags after that, so they never reach it: without the flag the suite runs with every assertion compiled out, and a test that depends on an assert only means something with it set. `scripts/ci` and `tests-backend.yml` export the flag (GitHub Actions sets it too). When running the suite directly — including a `--focus` run — the command is `PENPOT_FLAGS="enable-backend-asserts" clojure -M:dev:test`.
- Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated. - Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
- Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var. - Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var. Keep the `PENPOT_FLAGS="enable-backend-asserts"` prefix on these too; a focused run without it hides assert-dependent failures exactly like a full one.
- A **whole-suite** crash reported as `Execution error ... Assert failed: ...` with no `FAIL in` line is not a test failure: an `ig/assert-key` ran while the test system was being built, which happens outside `clojure.test`, so kaocha reports it as a load/build error and the summary has no counts. Fix the offending schema (see `mem:backend/subtleties`) rather than hunting for a test.
- Multiple focus symbols are supported by REPEATING the flag (kaocha accumulates them, last-wins does not apply): `clojure -M:dev:test --focus backend-tests.ns-one --focus backend-tests.ns-two` — preferred over a shell loop of isolated runs.
- Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas. - Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas.
- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`. - If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
- RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request. - RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request.
- `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there). - The test system is `main/system-config` + `main/worker-config` with the
components that must not run in the background dropped: HTTP, the srepl,
cron, the dispatcher and **every** `[<profile> :app.worker/runner]`
(`helpers/remove-job-runners`). A live runner of any queue races the tests
that drive the loop by hand (`wdisp/run-batch` + `@#'wrkr/run-worker-loop`):
it takes the job the test just dispatched and completes it behind the test's
back, so the test's own assertions read a state it did not produce. Do not
re-add a runner by editing that list by hand; the removal matches by key
shape on purpose.
- `system-config` and `worker-config` are `def`s: their `(cf/get ...)` calls
are evaluated when the namespace loads, against the real environment, while
test code calling `cf/get` at run time sees the `penpot-test`-prefixed
config. The two can disagree — in the devenv `:tenant` is `devenv-ws0` for
the defs and `default` inside a test — so a component built from
`worker-config` can talk to a different Redis key prefix than a cfg the test
builds itself. The symptom is a component that silently does nothing
locally and races in CI (which sets no `PENPOT_TENANT`).
- `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there). The logger swaps the record on an executor thread, after the call that logged it returns: keep the watch installed until the record arrives (poll for it) and only then remove it, or the record lands on nobody under load.

View File

@ -0,0 +1,44 @@
# Backend WebSocket Subscriptions
## Registry
- `app.http.websocket/state` holds `{:connections {conn-id → wsp} :by-profile {profile-id → #{conn-id}}}` in **one** atom, so a single `swap!` keeps both consistent; `register-connection`/`unregister-connection` maintain them together and drop a profile from the index once its last connection leaves.
- Each connection's own `::ws/state` atom holds its subscriptions (`::file-subscription`, `::team-subscription`). `state` is shadowed by `::ws/state` in every `handle-message` destructuring, so only `on-connect`, `on-disconnect` and the `repl-*` helpers touch the registry — reading the registry from a handler is a bug.
- `::profile-id` and `::session-id` on a connection live in the `app.http.websocket` namespace, while `::ws/id`, `::ws/state`, `::ws/output-ch`, `::ws/session-id` live in `app.util.websocket`. Mixing them up yields silently `nil` values (a `nil` `session-id` makes the channel's `:xf` drop **every** message).
- The registry is local to one backend instance. The message bus is shared (`mem:prod-infra/core`). Anything that must reach connections owned by another instance has to go through the bus — closing a connection from an RPC handler only ever works in a single-backend deployment.
## Revocation model
A subscription is authorized once, at `:subscribe-file`/`:subscribe-team`, and that decision used to be trusted for the whole connection life (GHSA-m53j-2766-6jqw, medium). Two independent mechanisms now bound it:
- **Announcement (primary).** A mutation that can change access publishes on `app.rpc.notifications/internal-revocation-topic` (`notify-permissions-changed` for one profile, `notify-team-permissions-changed` for one team id — the watcher resolves the members itself, so a large team is one event, not N). The `::revocation-watcher` Integrant component in `app.http.websocket` subscribes and calls `revalidate-profile-subscriptions`, which looks the profile up in `:by-profile` and re-checks every subscription with `file-readable?`/`team-readable?`: existence (the file gate is the same `sql:get-minimal-file` as the `:subscribe-file` gate, made non-throwing via `{:db/throw-if-not-exists false}`; the team gate is `db/get*` with `::sql/columns [:id :deleted-at]` — `deleted_at` must be in the projection or the soft-delete filter is a no-op) AND the `has-read-permissions?` predicates — and `sql:file-permissions` additionally filters `deleted_at` on file, project and team, as do `sql:get-team-id-for-file`/`-project` plus a final team-row check in `resolve-team-id`, so the organization-owner fallback grants nothing through soft-deleted file/project/team. The existence gate closes that fallback (expelled org owner, deleted resource). The event is only the trigger; the fresh check is the decision, which is what keeps a downgrade to `viewer` (still readable) and a non-member organization owner on a live resource from being mistaken for a revocation. Both publishers defer through `db/after-commit`, because the watcher checks on another connection and would otherwise see the pre-commit state. `:subscribe-team` deliberately has no existence gate of its own; only the re-checks add one, matching the file side (user decision, PR #12189 review).
- **Interval (safety net).** `start-relay` runs the relay as a `go-loop` that waits on the subscription channel and a one-shot tick channel together via `sp/alts!` with `:priority true` on the tick: when both hold something, the tick wins, so a steady stream of messages can never push the re-check away. The cadence is `subscription-revalidation-interval` (default `default-revalidation-interval`, 5 min, a code constant in `app.http.websocket` and **not** in the `config.clj` default map, same rule as session lifetimes). The deadline is fixed per cadence: a single tick channel covers a whole cadence and message turns reuse it (minting a fresh timeout per turn would both churn timers and starve under load exactly like an in-loop timer does); a new one is minted only after a tick fires. It covers what no announcement can: Nitrate transferring organization ownership (an HTTP call to another service) and rows deleted straight over SQL. Closing the channel ends the loop either way, because a take on a closed channel is always ready with nil — every way a subscription can end closes it, so a finished relay never keeps polling. Cost is three or four queries per subscription per interval (existence row + predicates), plus one Nitrate call per team per interval when the `:admin-console` flag is on (cached 30 s); one `timeout-chan` per subscription per cadence on promesa's default scheduler (one or two threads shared with every other scheduled task) — do not shorten the interval without sizing that. A tick abandoned at teardown (the subscription closed while it was pending) still fires its scheduled close harmlessly. Two places fail open, deliberately: `still-authorized?` (a transient DB error logs and keeps the subscription; `InterruptedException` is rethrown) and the `catch Throwable` in `watch-revocations`.
Which mutations announce: `delete-team-member`, `leave-team`, `update-team-member-role` (member only), `delete-team`, `delete-file`, `delete-project`, `move-files`, `move-project` (one team event each). Anything not listed here is covered only by the interval. `delete-share-link` deliberately does not: a share-link visitor never passes the websocket subscribe check, because `check-read-permissions!` is called without a `share-id` and that arity only considers team membership.
`close-file-subscription`/`close-team-subscription` are idempotent: no-op on a mismatched id, and they `dissoc` the subscription so a repeat call and `:pointer-update` do not publish to a dead topic. Closing the channel is what ends the relay (`take!` on a closed channel returns `nil`).
## Topic and layering
- `internal-revocation-topic` is a fixed uuid that is never a real resource id, so it cannot collide with a file, team or profile topic — and deliberately not the `uuid/zero` system topic, which is `sp/pipe`d straight into the client output and anything published there reaches every client. No client can subscribe to the internal topic: `:subscribe-file`/`:subscribe-team` only accept ids that already passed a permission check, and `:broadcast`/`:pointer-update` only republish to ids the server already holds.
- The topic and the publish helpers live in `app.rpc.notifications`, not in `app.http.websocket`: the command namespaces cannot require the watcher, because `app.http.websocket` already requires `app.rpc.commands.files`/`teams`. The watcher keeps the registry, which is the part it owns.
## Time and channel traps in this namespace
- The relay waits with `sp/alts!` over `[tick-ch channel]` with `:priority true`, following the `msgbus.clj` pattern (`start-io-loop` destructures the same `[val port]` shape and compares ports with `identical?`). A take on a closed channel is always ready with nil, which is what ends the loop — and what makes closing the channel from any teardown path prompt even while parked in `alts!`. `sp/timeout-chan` is one-shot: it closes once after the delay, and one abandoned at teardown still fires its scheduled close harmlessly.
- `ct/diff a b` returns **b − a** (`Duration/between`), so "time until a future instant" is `(inst-ms (ct/diff (ct/now) future))`.
- `ct/duration` unit keywords are singular: `{:millis 150}`, `{:minutes 5}`, never `:milliseconds`/`:days`.
- `still-authorized?` takes the check itself: `(boolean (check-access))`. Calling `(boolean pred)` on the closure is always true and silently disables the interval — this exact bug shipped once and no test caught it until the resubscribe-counting test existed. Naming rule here: no `?` for the action that queries and can fail (`check-access`), `?` only for the safe yes/no answer (`still-authorized?`).
## Tests
`backend/test/backend_tests/http_websocket_test.clj` drives the real Redis-backed msgbus, so the revocation tests are end-to-end: register the connection in the real registry, run the real command, wait for the subscription to drop, then assert no content arrives. A connection built without `register-connection` is invisible to the watcher, so the watcher-based tests must go through `open-registered-connection`. Interval tests shorten the config via `with-redefs` on `cf/get` rather than waiting minutes. `with-clean-registry` runs as an `:each` fixture, so no test can leak a live relay loop into the next one. `captured-events` forwards the keyword args through to the real `mbus/pub!` — rebuilding them positionally publishes to the topic `:message` with a `nil` payload and silently untests every other notice. Interval tests take access away with a direct row delete, never with a command: a command announces, so the test would pass even with the interval removed. The same rule covers the two counting/fan-out tests: `revocation-closes-subscriptions-that-are-no-longer-authorized` takes access away with `delete-team-membership!` (raw row delete, publishes nothing) because a command announcement lets the live watcher close both subscriptions before the test's own `revalidate-profile-subscriptions` call, and its `(= 2 …)` count then reads 0; `team-event-closes-every-member-subscription` soft-deletes the team row (`sql:team-members` joins `team_profile_rel` and `profile` only, so the watcher still resolves every member) and asserts nothing closed before one synthetic `notify-team-permissions-changed`, leaving the command path to `deleting-a-team-announces-once` and `deleting-a-team-closes-open-subscriptions`. The resubscribe-counting test is the one that proves finished relays stop polling. The soft-delete + org-owner re-check tests pair a direct row soft-delete with `with-redefs` on `nitrate/organization-owner-of-team?`: before the existence gate, the org-owner fallback kept the subscription open on a deleted resource. The soft-deleted-project/team-closes-org-owner-file-subscription tests pin the root fix in `app.rpc.permissions` (`resolve-team-id` refuses dead file/project/team for every caller of the fallback). `watcher-consumes-revocation-events-from-the-bus` takes access away with a direct row delete and drives the watcher with a synthetic bus event — never a command and never a second `notify-*` — so the "still subscribed" assertion cannot race an after-commit hook and a broken command announcement cannot mask a broken watcher. `moving-a-project-closes-source-team-subscriptions` pins the move-project asymmetry: the file subscription drops, the source-team subscription survives (the member is still a member).
`revocation-watcher-schema-resolves-every-ref` covers the `::revocation-watcher` params schema, which no other test compiles. It existed broken for a while: the three injected keys were written as bare keywords, which is a valid malli *property* position and not a key, so the schema raised `:malli.core/invalid-schema` — invisible unless `PENPOT_FLAGS=enable-backend-asserts` was set, and a crash of the whole system build rather than one test (see `mem:backend/testing` on the flag).
## See also
- `mem:backend/auth-permissions-product-domains` for the permission predicates and the not-found semantics that the re-checks reuse.
- `mem:prod-infra/core` for why the bus crosses backends and the registry does not.
- `mem:frontend/routing-app-shell-subtleties` for the client side: the frontend already reacts to `team-membership-change` by navigating away, which is why revocation needs no new client message.
- `.agents/plans/2026-10-07-websocket-subscription-revocation.md` for the original plan, and the sibling `.agents/plans/2026-10-05-leave-team-file-access-cleanup.md` whose hardened permission queries this re-verification depends on (without live team membership required in `sql:file-permissions`, the re-check would keep reporting access for a member who already left).

View File

@ -8,6 +8,7 @@ How to shape a function in this codebase. Each rule came from a review round; do
- **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is. - **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is.
- **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one. - **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one.
- **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them. - **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them.
- **All code-facing text MUST be in English** — identifiers, comments, docstrings, test descriptions. No exceptions; see the hard rule in `AGENTS.md`.
Reference implementation: the http metrics samplers in `app.http` (`sample-worker-metrics` / `sample-connector-metrics` are guard-free; `sample-http-metrics` guards at the boundary). Reference implementation: the http metrics samplers in `app.http` (`sample-worker-metrics` / `sample-connector-metrics` are guard-free; `sample-http-metrics` guards at the boundary).

View File

@ -8,5 +8,8 @@ Behaviors confirmed against the language/stdlib — do not re-derive from assump
- The analyzer reads `:async` only from the fn name meta and the `fn` operator meta; list-level meta is ignored (would make a MetaFn). `(fn ^:async [] …)` puts the meta on argv (pre/post only, NOT async). - The analyzer reads `:async` only from the fn name meta and the `fn` operator meta; list-level meta is ignored (would make a MetaFn). `(fn ^:async [] …)` puts the meta on argv (pre/post only, NOT async).
- Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself). - Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself).
- `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO). - `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO).
- Inside an `^:async` fn, a nested call that needs a temp binding (e.g. calling a local fn `(g (f x))`, whose arg gets a `G__` binding) compiles to `(await (async function(){…})())`: `await` unwraps a returned promise, so `(settled (upload (make-ctx)))` blocks until the upload promise settles instead of passing the promise on (a test that pushes the answer later hangs, idle). Bind each step in its own `let` binding.
- `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use. - `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
- Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant).
- The threading macros insert the threaded value at DIFFERENT positions: `->` and `some->` put it FIRST (`(-> x (f a))` → `(f x a)`), `->>` and `some->>` put it LAST (`(->> x (f a))` → `(f a x)`). `some->`/`some->>` also short-circuit on nil. With unary calls both nest the same way (`(-> x (f) (g))` and `(->> x (f) (g))` are both `(g (f x))`): what changes between them is the position inside each call, not the order of the steps. A pipeline that calls a fn whose first argument is a config/context (`app.features.fdata/resolve-file-data` is `(cfg, file)`) must use `->>`/`some->>`; with `->`/`some->` the arguments are swapped and the call silently returns the wrong value (it cost a broken `get-snapshot` in `app.features.file-snapshots`).
- `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`). - `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`).

View File

@ -0,0 +1,7 @@
# Clojure Logging Conventions
Structured logging through `app.common.logging` (`l/trc`/`l/dbg`/`l/info`/`l/wrn`/`l/error`/`l/fatal`): `:hint` carries the human-readable message, every other datum goes in its own keyword field (`:id`, `:container`, `:removed`, ...).
- `:hint` values start lowercase: `"fix dangling children"`, not `"Fix dangling children"`. Uppercase-first hints exist as legacy (~57 occurrences, the densest in `backend/src`; also `l/trc` in migration `legacy-28` and `log/error` in `app.common.files.repair`) — do not copy them, and do not mass-rewrite them unless the surrounding code needs the change.
- A name that requires uppercase (an env var like `PENPOT_MEDIA_PROCESSING_SERVICE_URI`, an acronym like `MCP` or `LDAP`) stays uppercase inside an otherwise lowercase sentence.
- Repair work done by a file migration logs at `l/trc`, only when it changes something, and with the ids it touched: the house style of `migrations.cljc` (`legacy-28` precedes; the file has no `warn`/`info` lines). The shipped `log4j2.xml` pins the `app` logger to `info` (static file, `monitorInterval="60"`), so those trace lines appear only after the operator raises the level; the durable record of a repaired file is the migration id in its `:migrations`.

View File

@ -0,0 +1,9 @@
# Promesa
Promise library of the monorepo (frontend CLJS and common, `funcool/promesa` 12.0.1 in `common/deps.edn`). Its API has two call styles and mixing them in one expression is the trap: `fmap`/`mcat`/`merr` are **fn first, promise last** (`->>` friendly) and `then`/`then'`/`catch`/`handle` are **promise first, fn last** (`->` friendly). Convention: `fmap`/`mcat`/`merr` inside `->>` pipelines, `then`/`catch` with `->`. A `then` written inside a `->>` reads backwards and hides whether the step carries a value or a promise.
- `fmap` (alias `map`, both `(f p)`) is functor map: `f` returns a plain value.
- `mcat` (alias `mapcat`, both `(f p)`) is monadic bind: `f` must return a promise, and the result mirrors it.
- `then` also unwraps a promise returned by `f`, which is why it covers both cases at the cost of not naming which one it is; `then'` maps without unwrapping.
- `merr` (fn first) and `catch` (promise first) are the error side.
- `all` takes a collection of promises (a lazy seq works); `let` chains sequential bindings.
- Seam with the rx pipelines of the frontend: `beicon.v2.core/from` builds an observable from "arrays, collections, and promise instance" (`external/beicon`), so `(->> p (p/mcat f) (rx/from) (rx/map g))` is the usual bridge (`app.main.data.imports`, `app.util.zip`).

View File

@ -26,6 +26,12 @@ Respect the intended abstraction direction in new/refactored code:
Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical. Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical.
## Build-mode dependent laziness
`app.common.data.macros/lazy-delay` returns a delay whose expression is evaluated eagerly by default. Only the builds that set the `app.common.data.macros/lazy-delay?` closure define to `true` defer it to the first deref; every production build does (frontend `main` and `worker`, exporter, library), so the Closure compiler replaces the constant and drops the eager branch. On the JVM it is always eager.
`app.common.schema/check-fn` resolves its schema through it: an invalid schema definition fails when the namespace loads in development and in tests, and only on the first check call in production. The other lazily built parts of `check-fn` (validator and explainer) are always deferred.
## Focused memory routing ## Focused memory routing
Model, schema, and persistence shape: Model, schema, and persistence shape:

View File

@ -6,7 +6,7 @@
- Do not treat nil as a distinct persisted state from absence. Import/export and cleanup paths may filter nil attrs away. - Do not treat nil as a distinct persisted state from absence. Import/export and cleanup paths may filter nil attrs away.
- Avoid Clojure-special naming in exported object attrs, especially boolean names ending in `?`; exported/imported data must survive JSON/SVG/Transit and external tooling. - Avoid Clojure-special naming in exported object attrs, especially boolean names ending in `?`; exported/imported data must survive JSON/SVG/Transit and external tooling.
- Any new shape attr that participates in component sync must be listed in `app.common.types.component/sync-attrs` with the correct touched group. Attrs absent from `sync-attrs` are ignored by component synchronization. - Any new shape attr that participates in component sync must be listed in `app.common.types.component/sync-attrs` with the correct touched group. Attrs absent from `sync-attrs` are ignored by component synchronization.
- Shape `:svg-attrs` is the one map whose keys are camelCase React prop names (`fillRule`, not `fill-rule`), because the SVG import path runs them through `app.common.svg/attrs->props`. The `.penpot` reader (`app.common.json/read-kebab-key`) rewrites every json key of every entry to kebab-case, nested maps included, so any code that decodes a binary file must run `:svg-attrs` back through `attrs->props` or the names are lost. `:svg-defs` and the `:content` tree of `svg-raw` shapes keep their source names and round-trip on their own. - Shape `:svg-attrs` keys are kebab-case (`fill-rule`, not `fillRule`): birth (`app.common.svg/attrs->kebab-props`, applied everywhere data lands on a shape) normalizes any outside spelling through the `svg-props` whitelist, and render boundaries (`get-svg-props`, `render-xml*`, `svg-node*`) translate back to camelCase for React. `:svg-defs` node attrs follow the same rule recursively; svg-raw `:content` attrs are kebab-ized spelling-only, never filtered — the sidebar reads `:stroke-style`, which is outside the whitelist. Non-spec attrs Penpot keeps live in `penpot-extra-attrs` (seeded with `:stroke-style`) and travel in source spelling. Excluded, each with its reason: `:tag` values (React and tag sets need them verbatim; the schema decoder restores keywords from stored strings), outer `:svg-defs` keys (id strings at birth, keywords or strings after JSON decode — accept both), all values (the kebab reader never touches values). Repair sites: birth plus migration `0035-fix-shape-svg-attrs-and-defs` (replaces the withdrawn `0021`); the binfile cleaner carries no svg repair.
## Cross-module update checklist ## Cross-module update checklist

View File

@ -52,3 +52,8 @@ its own drift. Comparing `:selrect.width` from shape A to
`:selrect.width` from shape B is comparing two values that `:selrect.width` from shape B is comparing two values that
"semantically" should be equal but were computed via different "semantically" should be equal but were computed via different
operation chains — exact equality will often be false. operation chains — exact equality will often be false.
## Library sync comparison
- `update-attrs` (`app.common.logic.libraries`) compares main vs copy attrs exactly first, then with tolerance for: `drifting-geometry-attrs` (`:x :y :width :height :selrect :points`, via `gsh/close-attrs?`); path and bool `:content` (`segment/content-close?`; bool shapes keep their path in `:content`); and text `:position-data` (`position-data-close?` over `mth/close-map?`: every numeric field of every entry within `mth/close?`, other fields such as fills, fonts and text exact). All other attrs (`:opacity`, radii, `:transform`, `:rotation`, text `:content`) stay exact: tolerance there hides real changes.
- `close-attrs?` pairs lists with `d/zip`, which truncates to the shorter list and treats nil vs vector as equal for `:points`/`:position-data`. Do not use it for variable-length data. That is why `:position-data` has its own helper, and why `attr-equal?` checks the `:points` count before calling it.

View File

@ -20,8 +20,10 @@
- Full referential/semantic validation currently runs only when file features contain `"components/v2"`. - Full referential/semantic validation currently runs only when file features contain `"components/v2"`.
- Validation starts at root plus orphan shapes, then validates component records. `validate-file!` raises `:validation :referential-integrity` with collected details. - Validation starts at root plus orphan shapes, then validates component records. `validate-file!` raises `:validation :referential-integrity` with collected details.
- `update-file` validates per save with the change-scoped variants: `validate-file-affected!` (referential, whole touched pages/components) and `validate-file-schema-affected!` (file schema with `:pages-index` reduced to the touched pages; non-page data checked whole). The schema variant maps change types to pages itself (`touched-page-ids`): a new change type that writes a page must be added there, or to `file-level-change-types` if it touches no page; an unknown type makes it check the whole file. The whole-file schema check loads and decodes every page, ~75–80% of a save on multi-page files.
- `repair-file` does not mutate data directly; it reduces validation errors into redo changes using `changes-builder`. Callers must apply or persist those changes. - `repair-file` does not mutate data directly; it reduces validation errors into redo changes using `changes-builder`. Callers must apply or persist those changes.
- `repair-file-until-valid` stops on: no errors; unchanged `:data` after apply (backend page/`PointerMap` `=` is pointer identity); stable error fingerprint `(frequencies (map #(dissoc % :shape) errors))` vs previous pass (check fingerprint before `repair-file`/`process-changes`); or `max-iterations`. Each `repair-error` builds from the original `file-data`, so N `:set`-based errors on one shape take N passes. - `repair-file-until-valid` stops on: no errors; unchanged `:data` after apply (backend page/`PointerMap` `=` is pointer identity); stable error fingerprint `(frequencies (map #(dissoc % :shape) errors))` vs previous pass (check fingerprint before `repair-file`/`process-changes`); or `max-iterations`. Each `repair-error` builds from the original `file-data`, so N `:set`-based errors on one shape take N passes.
- `:ref-shape-not-found` repair: when the copy root's local component has a `:main-instance-id` that resolves to no shape but a `:main-instance` shape of that component sits on its `:main-instance-page`, it emits `mod-component` pointing the id back to that main (keeps the copy linked; a repair scoped to the copy's page never sees the main's `:invalid-main-instance-id`). Otherwise near-main reassign, else detach the copy subtree.
- `:missing-slot` fires only for a REAL swap: a copy sub-head whose `shape-ref` is no longer a child of the near main parent. A pure positional mismatch (ref still a sibling elsewhere) is a reorder — valid, realigned by the async component sync; do not "repair" it by assigning swap slots (a slot freezes the child out of normal sync). `fix-missing-swap-slots` (migration 0019) follows the same membership rule. - `:missing-slot` fires only for a REAL swap: a copy sub-head whose `shape-ref` is no longer a child of the near main parent. A pure positional mismatch (ref still a sibling elsewhere) is a reorder — valid, realigned by the async component sync; do not "repair" it by assigning swap slots (a slot freezes the child out of normal sync). `fix-missing-swap-slots` (migration 0019) follows the same membership rule.
## Migrations ## Migrations
@ -30,4 +32,6 @@
- Migrations are an ordered set mixing legacy version-derived ids and newer named ids. Keep append order stable; `migrate` applies the set difference between available migrations and file migrations. - Migrations are an ordered set mixing legacy version-derived ids and newer named ids. Keep append order stable; `migrate` applies the set difference between available migrations and file migrations.
- `migrate-file` synthesizes legacy migration ids from old numeric versions when `:migrations` is absent, migrates legacy features, and records feature flags created through `cfeat/*new*`. - `migrate-file` synthesizes legacy migration ids from old numeric versions when `:migrations` is absent, migrates legacy features, and records feature flags created through `cfeat/*new*`.
- When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data. - When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data.
- Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it. - Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it.
- Rebase collision on a migration number: upstream keeps the id. Renumber the branch migration to the next free one, keep it last in `available-migrations`, and rename everything that spells the number — its test file and namespace (`files_migrations_00NN_test.cljc`), the test names, and comments in production code or memories that name it.
- Withdrawing a migration id: remove both the `defmethod` and the id from `available-migrations`; the replacement migration subsumes it and files that recorded the old id re-migrate through the replacement (persisted `:migrations` keep the old id via union — do not rewrite them, including playwright fixture JSONs). Idempotent/pure-cleanup migrations only; never withdraw one whose transform must not run again unless the replacement is safe for those files. Precedents: "0021-fix-shape-svg-attrs" -> "0035-fix-shape-svg-attrs-and-defs"; "legacy-36" -> "0036-clear-nil-keys-in-objects".

View File

@ -38,6 +38,10 @@ Useful builders:
`add-variant-with-copy` does not accept position params for children; use `gsh/absolute-move` after creation if positions matter. `add-variant-with-copy` does not accept position params for children; use `gsh/absolute-move` after creation if positions matter.
## Generative sampling caution
- `sg/sample` is unseeded per run. If a test filters samples by type (e.g. keep only `:svg-raw`) then `first` can yield nil on unlucky seeds; `assoc` onto nil builds a typeless map that fails `:multi` dispatch on `:type`. Pin `:seed` in `sg/sample` opts whenever a test depends on a filtered subtype being present (verified seed 42 deals svg-raw shapes on JVM and JS).
## Driving production paths ## Driving production paths
For shape mutations, prefer production-path helpers such as `cls/generate-update-shapes` plus `thf/apply-changes`. For component swaps with keep-touched behavior, use `tho/swap-component-in-shape` with `{:keep-touched? true}`. For shape mutations, prefer production-path helpers such as `cls/generate-update-shapes` plus `thf/apply-changes`. For component swaps with keep-touched behavior, use `tho/swap-component-in-shape` with `{:keep-touched? true}`.

View File

@ -6,8 +6,10 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories. - A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories.
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`. - Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`. - Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
- Structured logging conventions (`:hint` casing and fields, log levels): `mem:clojure/logging` — read before adding or changing any log call.
- Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics. - Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API. - Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
- Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables.
# Development workflow # Development workflow
@ -15,16 +17,19 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- Before `git commit` → `mem:workflow/creating-commits` (subject/body format, 80-char body wrapping enforced by `scripts/check-commit`, `AI-assisted-by: model-name` trailer) - Before `git commit` → `mem:workflow/creating-commits` (subject/body format, 80-char body wrapping enforced by `scripts/check-commit`, `AI-assisted-by: model-name` trailer)
- Before `gh issue create` → `mem:workflow/creating-issues` (title derivation, body template, labels, Issue Type) - Before `gh issue create` → `mem:workflow/creating-issues` (title derivation, body template, labels, Issue Type)
- Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, "Note:" line) - Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, "Note:" line)
- Before updating `CHANGES.md` → `mem:workflow/updating-changelog` (sources of truth, entry rules, the `:boom:`/`:rocket:` label contracts, pre-flight checks, anomaly report)
- Before a repo-wide pnpm version update → `mem:workflow/updating-pnpm` (workspace layout, `scripts/sync-pnpm-version` flow, the stamp-missing-field and ignored-builds gotchas, verification steps) - Before a repo-wide pnpm version update → `mem:workflow/updating-pnpm` (workspace layout, `scripts/sync-pnpm-version` flow, the stamp-missing-field and ignored-builds gotchas, verification steps)
- Before a Node version update → `mem:workflow/updating-node` (the six pin sites in `.nvmrc`, the devenv and image Dockerfiles and the plugins-deploy workflows, the DHI tag gotcha, what to verify) - Before a Node version update → `mem:workflow/updating-node` (the six pin sites in `.nvmrc`, the devenv and image Dockerfiles and the plugins-deploy workflows, the DHI tag and digest rules, what to verify)
- Before bumping a non-JS dependency pinned in `docker/images/` (JDK, ImageMagick) → `mem:workflow/docker-images`
- **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked. - **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked.
- **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via the `update-changelog` skill flow or on explicit user request. - **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or on explicit user request.
- You have access to the GitHub CLI `gh` or corresponding MCP tools. - You have access to the GitHub CLI `gh` or corresponding MCP tools.
- Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool. - Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool.
- Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`). - Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`).
- Align `let` binding values: when a `let` form has multiple bindings spanning several lines, align the value forms to the same column with spaces. - Align `let` binding values: when a `let` form has multiple bindings spanning several lines, align the value forms to the same column with spaces.
- If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files, fix them with `scripts/paren-repair` BEFORE running lint/format checks. See `mem:scripts/paren-repair` for usage. - If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files, fix them with `scripts/paren-repair` BEFORE running lint/format checks. See `mem:scripts/paren-repair` for usage.
- Never run anything that destroys data without explicit permission, including `drop-devenv`, `docker compose down -v`, `docker volume rm ...`. The user's real work lives in the volumes of the shared infra. - Never run anything that destroys data without explicit permission, including `drop-devenv`, `docker compose down -v`, `docker volume rm ...`. The user's real work lives in the volumes of the shared infra.
- **Never wait for background work with a sleep.** A long command moved to the background reports its own completion: do not poll it with `sleep 150` (or any other wait) inside a foreground command, and do not re-read its output file to check whether it finished. Keep working on something else, or end the response and wait for the notification.
# Project modules # Project modules
@ -57,11 +62,11 @@ The memory is structured in a way that you can get the critical information abou
- `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`. - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`.
- `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`. - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`.
- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`. - `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`.
- `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. - `scripts/changelog.py` — Changelog checks for `mem:workflow/updating-changelog`: `boom` (generate the `:boom:` section from `breaking change` issues), `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`.
- `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`. - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`.
- `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `<repo>/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`. - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `<repo>/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`.
- `scripts/setup-opencode` — Install the opencode CLI in-container as a global pnpm package in `$PNPM_HOME`. Not part of the devenv image; see `mem:devenv/core`. - `scripts/setup-opencode` — Install the opencode CLI in-container as a global pnpm package in `$PNPM_HOME`. Not part of the devenv image; see `mem:devenv/core`.
- `scripts/ci` — CI orchestration script: runs lint, tests, and format checks per module (`frontend backend common render-wasm exporter mcp plugins library`). Logs go to `.ci-logs/`; read the log file on failure. See `mem:scripts/ci`. - `scripts/ci` — CI orchestration script: runs lint, tests, and format checks per module, plus the production build on request (`--build`). Modules: `frontend backend common render-wasm exporter mcp plugins library media-processor`. Logs go to `.ci-logs/`; read the log file on failure. It is the preferred way to run the backend tests, and it exports `PENPOT_FLAGS` with `enable-backend-asserts`, which is what makes `clojure.core/assert` exist in the compiled test code. See `mem:scripts/ci`.
# Dependency graph # Dependency graph

View File

@ -5,7 +5,7 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
## Compose project layout ## Compose project layout
- `penpotdev-infra`: shared `postgres`, `rustfs`, `valkey`, `mailer`, `ldap`. File: `docker-compose.infra.yml`. - `penpotdev-infra`: shared `postgres`, `rustfs`, `valkey`, `mailer`, `ldap`. File: `docker-compose.infra.yml`.
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 (a.k.a. `main`) binds `$PWD`; ws1+ bind clones at `${PENPOT_WORKSPACES_DIR}/wsN/` (default `~/.penpot/penpot_workspaces/`), maintained by the developer. - `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 (a.k.a. `main`) binds `$PWD`; ws1+ bind clones at `${PENPOT_WORKSPACES_DIR}/wsN/` (default `${XDG_DATA_HOME:-~/.local/share}/penpot-devenv/workspaces/`, or the former `~/.penpot/penpot_workspaces/` when it already exists), maintained by the developer.
- Optional overlay `docker-compose.opencode.yml`: added by `instance-compose` as an extra `-f` only when `PENPOT_OPENCODE_CONFIG_DIR` is set (i.e. `run-devenv --opencode-config-dir DIR` ran in this process). Bind-mounts the host dir at `/home/penpot/.config/opencode` (`:z`). Flag-only, per-call; not read from ambient env. Parser `parse-opencode-config-dir` absolutizes (`~`, realpath) because compose resolves relative bind sources against the compose file's dir. Only instances brought up with the flag get the mount. - Optional overlay `docker-compose.opencode.yml`: added by `instance-compose` as an extra `-f` only when `PENPOT_OPENCODE_CONFIG_DIR` is set (i.e. `run-devenv --opencode-config-dir DIR` ran in this process). Bind-mounts the host dir at `/home/penpot/.config/opencode` (`:z`). Flag-only, per-call; not read from ambient env. Parser `parse-opencode-config-dir` absolutizes (`~`, realpath) because compose resolves relative bind sources against the compose file's dir. Only instances brought up with the flag get the mount.
- All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands. - All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands.
@ -15,6 +15,7 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is. - For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is.
- Per-instance test databases are created by `ensure-instance-test-database` (called from `start-instance`): `CREATE DATABASE` guarded by a `pg_database` existence check, so it is idempotent. Needed because `postgresql_init.sql` only runs on first volume creation. The backend applies migrations itself on first use, so an empty database is enough. - Per-instance test databases are created by `ensure-instance-test-database` (called from `start-instance`): `CREATE DATABASE` guarded by a `pg_database` existence check, so it is idempotent. Needed because `postgresql_init.sql` only runs on first volume creation. The backend applies migrations itself on first use, so an empty database is enough.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_s3_bucket()`. Never duplicates `defaults.env`. - `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_s3_bucket()`. Never duplicates `defaults.env`.
- Frontend admin-console is off by default in devenv: `docker-compose.main.yml` sets `PENPOT_DEVENV=true`, which makes `app.config` (`goog-define devenv-mode`) prepend `disable-admin-console` ahead of the user `config.js` flags; flags parse in order and user flags win, so `enable-admin-console` in `frontend/resources/public/js/config.js` re-enables it. Outside devenv (prod builds) the default is unset and the admin-console stays on, matching `flags/default`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails. - Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants ## Invariants
@ -29,7 +30,7 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
## Worker policy ## Worker policy
Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ inject it as false. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`. Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ inject it as false. Workers are pure fire-and-forget: `jobs/submit` inserts a row into the shared Postgres `job` table and returns (the legacy `task` table stays dormant); RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `jobs/submit` `dedupe` is racy across submitters); details in `mem:prod-infra/core`.
Each workspace is independent and can be started/stopped in any order. Shared infra (Postgres, RustFS, etc.) is shut down only when no instances remain running. Each workspace is independent and can be started/stopped in any order. Shared infra (Postgres, RustFS, etc.) is shut down only when no instances remain running.
@ -77,3 +78,42 @@ No `--delete` on the working-tree pass: gitignored caches in the workspace survi
- `start-devenv` / `log-devenv` / `drop-devenv`: legacy paths around ws0 + shared infra. `drop-devenv` never removes volumes. - `start-devenv` / `log-devenv` / `drop-devenv`: legacy paths around ws0 + shared infra. `drop-devenv` never removes volumes.
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present. `exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.
## Image and dependency pins
Two Dockerfile trees, both multi-arch (`linux/amd64,linux/arm64`):
`docker/devenv` (dev) and `docker/images` + `docker/imagemagick` (prod).
- Hand-pinned versions, one source each:
- `docker/devenv/Dockerfile`: `NODE_VERSION` (+ per-arch `NODE_SHA256`),
`PNPM_VERSION` (+ `PNPM_SHA256`), `CADDY_VERSION`, `CLOJURE_VERSION`,
Zulu JDK URL + `ESUM`, `RUSTUP_VERSION`/`RUST_VERSION`/
`EMSCRIPTEN_VERSION`, `CLJKONDO_VERSION`, `BABASHKA_VERSION`,
`CLJFMT_VERSION`, `PIXI_VERSION`, `GITHUB_CLI_VERSION`,
`AWS_CLI_VERSION` (+ `BINARY_SHA256`), `UV_VERSION`, `SERENA_VERSION`,
`postgresql-client-18`, and the `penpotapp/imagemagick:<tag>`
`COPY --from`.
- `docker/images/Dockerfile.backend`: Zulu JDK URL + `ESUM`. The `jlink`
`--add-modules` list must keep `jdk.management`
(`mem:backend/subtleties`).
- `docker/images/Dockerfile.{exporter,mcp,media-processor}`: `ARG
PNPM_VERSION` (+ arch SHAs, same values as devenv).
- Node and pnpm must agree in five places: `.nvmrc`, the devenv tarball pin,
the two `plugins-deploy-*` workflows that set `runtime: node@<version>`, and
the four Dockerfile pins. pnpm procedure: `mem:workflow/updating-pnpm`.
- `dhi.io` base images are pinned `tag@sha256:<index digest>`; Renovate moves
them (`penpot-renovate[bot]`, commits titled `:arrow_up: Update DHI base
images`). Tag and digest move together: a new tag with the old digest still
pulls the old image, and the digest must be the multi-arch index, not a
per-platform one.
- Every hand-pinned checksum is per architecture; one wrong value breaks half
of the multi-arch build.
- `SERENA_UPDATE_VERSION` in `docker-compose.main.yml` must never be lower than the Dockerfile's `SERENA_VERSION`. Any value makes `start-tmux.sh` install that Serena release on start, overriding the version baked into the image; a stale value silently downgrades Serena (it once replaced 1.7.0 with 1.6.1). If the image already has that version the install is an offline no-op; otherwise it downloads on every start (`run-devenv` recreates the container) and, offline, fails after ~25s and keeps the image version. Bump both together; empty means use the image version.
- Infra images live in `docker-compose.infra.yml`: `rustfs/rustfs` (owns the
S3 data volume), `axllent/mailpit` (SQLite in a volume), `valkey/valkey`,
`postgres:18.6`, `rroemhild/test-openldap`. Keep the volume names stable
when bumping them; replacement services get new volumes and leave the old
data untouched.
- Build locally: `docker build -f docker/devenv/Dockerfile docker/devenv/`
and `cd docker/images && ./build.sh <image> --local`. CI:
`build-docker-devenv.yml`, `build-docker.yml`.

View File

@ -0,0 +1,106 @@
# Auth Events and the Session Origin
`signin` is the frontend `identify` event emitted once per session from
`logged-in` (`app.main.data.auth`). It carries the profile plus a closed
vocabulary that says why the session started.
## Vocabulary
- `source`: `login` | `register`. Always present.
- `auth-backend`: `penpot` | `ldap` | `google` | `github` | `gitlab` | `oidc`.
Same key the profile already uses. Always present.
- `activation`: `immediate` | `email-verification`. Only on signups.
- `from-invitation`: boolean. Always present in `signin`
(`(some? (:invitation-token profile))`); `auth-origin` omits it when `nil`.
`auth-origin` (public, validated against `schema:auth-origin`) drops the nil
fields and checks with `(assert (sm/check ...))`: it throws in development and
tests, and `:elide-asserts true` removes it from the release build, so a bad
value can never break a real login. Emitters never call it directly.
## Where each value comes from
- `source` and `activation` travel as profile metadata (`::ev/source`,
`::ev/activation`), written by `origin-meta`. `login-from-token` derives both
from the token issuer with `iss->origin`: `:auth` → `login`; `:verify-email` →
`register` + `email-verification`. An unknown `:iss` produces no metadata, so
the assertion fires in development and the event goes without `:source` in
production.
- `auth-backend` and `from-invitation` are read from the profile.
- `profile-origin` composes the four and `logged-in` merges them into the event;
the emitters only declare the fact.
| Flow | `source` | `activation` | `from-invitation` |
|---|---|---|---|
| Password login | login | | false |
| Password login from an invitation link | login | | true |
| LDAP login | login | | false |
| Social login, existing active profile | login | | false |
| Signup, email verification disabled | register | immediate | false |
| Signup, email verification enabled | register | email-verification | false |
| Social signup, active profile | register | immediate | false |
| Social signup, inactive profile | register | email-verification | false |
| Signup from an invitation link, email verification disabled | register | immediate | true |
| Signup from an invitation link, email verification enabled | register | email-verification | true |
| Social login from an invitation link | login | | true |
| Social signup from an invitation link, email verification enabled | register | email-verification | true |
| LDAP login from an invitation link | login | | true |
LDAP is the explicit exception: the directory provisions the account on the
first login, but there is no user signup, so the flow is always `login` and
never carries `activation`. Do not "fix" it by inferring a signup.
## Invitations go through `logged-in`
- `login-from-register` accepts an optional `:invitation-token`;
`with-register-origin` decorates the profile and `logged-in` does the rest.
The four places that used to navigate by hand to `:auth-verify-token`
(`ui/auth/register.cljs` twice, `ui/static.cljs`, `ui/auth/login_modal.cljs`)
emit `login-from-register` with the token. The modal used to live in
`ui/viewer/login.cljs`; it was extracted to the shared
`ui/auth/login_modal.cljs` (`login-register-modal*`, also used by
`ui/mcp_authorize.cljs`).
- The invitation navigates through `get-invitation-nav`, which does not wait for
`::dtm/teams-fetched`: `fetch-teams` has no error handling, so depending on it
would leave the invitation unaccepted when `get-teams` fails. For invited
profiles `get-redirect-events` emits `rx/empty`, so the teams path cannot move
the user away from the token verification screen.
- The OIDC callback with an invitation mints an `:auth` token that carries the
invitation JWE in its claims (`backend/src/app/auth/oidc.clj`), so the
frontend follows the regular path (`login-from-token` → `logged-in`). That
flow visits the token verification screen twice.
- `ui/auth/login_modal.cljs` (the register modal: `:is-active` branch calls
`on-close`, or `on-register` when the caller passes one, as in the MCP
authorize flow) and `ui/static.cljs` (`rt/reload true`) `:is-active`
branches still bypass `logged-in`; they are a separate decision, not an
oversight.
## Backend side
- The `register-profile` success branches
(`backend/src/app/rpc/commands/auth.clj`) put `activation` in the event props
and emit **no** `action`: the name already says it is a signup, and a constant
action would only add noise. The retry branches
(`register-profile-retry`) keep `action` in `context`
(`ignore-because-blocked`, `ignore`, `resend-email-verification`,
`ignore-because-complaints`), where it distinguishes outcomes.
- The LDAP invitation branch of `login-with-ldap` returns the full profile plus
`:invitation-token`; a partial map would break the `auth-origin` assertion.
- `register-profile` does not accept the invitation: it mints the session and
returns the token. The acceptance is `accept-team-invitation`, emitted from
`verify-token`.
- `filter-telemetry-props` keeps `source` and `activation` in the anonymised
row; `:action` is not on the allowlist.
## Analytics
- `register-profile` is a track event, so the props adapter forwards
`activation` with no extra work.
- `signin` is an identify event built from a closed list of profile keys. It
only reaches analytics if the identify branch of Nexus forwards `source`,
`activation` and `from_invitation` (`external/nexus`, another repository).
## `::ev/source` is no longer dead metadata
It is read by `profile-origin`. Do not remove it as unused, and keep the
vocabulary in sync with `schema:auth-origin` and the table above.

View File

@ -49,16 +49,23 @@ Get `team-id` from `(:current-team-id @app.main.store/state)`. Get file ids from
## Runtime patching with `set!` ## Runtime patching with `set!`
Some frontend vars are deliberately mutable escape hatches for runtime instrumentation or circular-dependency patching. From `cljs_repl`, use `set!` for temporary debugging of CLJS vars such as `app.main.store/on-event`, `app.main.errors/reload-file`, `app.main.errors/is-plugin-error?`, `app.main.errors/last-report`, or `app.main.errors/last-exception`. These patches affect only the live browser runtime and disappear on reload or recompilation. Some frontend vars are deliberately mutable escape hatches for runtime instrumentation or circular-dependency patching. From `cljs_repl`, use `set!` for temporary debugging of CLJS vars such as `app.main.errors/reload-file`, `app.main.errors/is-plugin-error?`, `app.main.errors/last-report`, or `app.main.errors/last-exception`. These patches affect only the live browser runtime and disappear on reload or recompilation.
`set!` on `app.main.store/on-event` does nothing at runtime: the store captured the function when it was created. To watch events, subscribe to the store stream and dispose the subscription after:
```clojure ```clojure
;; Log non-noisy Potok events temporarily. ;; Log non-noisy Potok events temporarily.
(set! app.main.store/on-event (def sub
(fn [event] (beicon.v2.core/subs!
(when (potok.v2.core/event? event) (fn [event]
(.log js/console (potok.v2.core/repr-event event))))) (when (potok.v2.core/event? event)
(.log js/console (potok.v2.core/repr-event event))))
app.main.store/stream))
;; (beicon.v2.core/dispose! sub)
``` ```
`cljs_repl` starts every call in `cljs.user`; begin a call with `(in-ns 'my-ns)` to reuse helpers defined in another namespace.
Restore mutable hooks after debugging, or reload the frontend. Use JVM `alter-var-root` only for JVM Clojure; it is not the normal way to patch live CLJS browser vars. Restore mutable hooks after debugging, or reload the frontend. Use JVM `alter-var-root` only for JVM Clojure; it is not the normal way to patch live CLJS browser vars.
## Browser-console debug namespace ## Browser-console debug namespace

View File

@ -150,5 +150,5 @@ The frontend bundle is built once per SHA and restored by this job (`mem:fronten
Details: README, "Running in CI". Details: README, "Running in CI".
## Substrate ## Substrate
`mem:common/test-setup`, `mem:common/component-data-model`, `mem:common/component-swap-pipeline`, `mem:common/testing`, `mem:common/component-data-model`, `mem:common/component-swap-pipeline`,
`mem:frontend/testing`. `mem:frontend/testing`.

View File

@ -12,7 +12,7 @@ Frontend: CLJS SPA; React/Rumext; Potok; RxJS; okulary refs; SCSS modules; share
- `app.render_wasm.*`: frontend bridge to Rust/WASM renderer. - `app.render_wasm.*`: frontend bridge to Rust/WASM renderer.
- `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities. - `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities.
- `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app. - `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app.
- Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared permission rules are in `common/src/app/common/types/nitrate_permissions.cljc`. - Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared organization permission rules (`allowed?`, `can-send-invitations?`) are in `common/src/app/common/types/organization.cljc`.
- `app.config` external bridges (`external-feature-flag`, `external-session-id`, `external-context-info`, `external-notify-register-success`, `initialize-external-context-info`) delegate to `globalThis` hooks injected by the SaaS host. Never delete `external-feature-flag`, even with zero call sites: it is the seam for future A/B tests. - `app.config` external bridges (`external-feature-flag`, `external-session-id`, `external-context-info`, `external-notify-register-success`, `initialize-external-context-info`) delegate to `globalThis` hooks injected by the SaaS host. Never delete `external-feature-flag`, even with zero call sites: it is the seam for future A/B tests.
@ -46,6 +46,7 @@ Workspace behavior:
App shell and product flows: App shell and product flows:
- Routing, root app shell, websocket, and global errors: `mem:frontend/routing-app-shell-subtleties`. - Routing, root app shell, websocket, and global errors: `mem:frontend/routing-app-shell-subtleties`.
- Authentication events, the session origin vocabulary, and the invitation flow: `mem:frontend/auth-events`.
- Dashboard and viewer flows: `mem:frontend/dashboard-viewer-subtleties`. - Dashboard and viewer flows: `mem:frontend/dashboard-viewer-subtleties`.
- Plugin JS API runtime inside the frontend app: `mem:frontend/plugin-api-to-cljs-binding`. - Plugin JS API runtime inside the frontend app: `mem:frontend/plugin-api-to-cljs-binding`.

View File

@ -5,13 +5,17 @@
- Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id. - Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id.
- Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly. - Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly.
- Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications. - Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications.
- The v3 format carries no size per file: the size of one is the sum of its own entries plus the `objects/` its media and thumbnail metadata reference, the only link between a file and that shared storage.
- Those metadata entries are read one at a time on purpose: a container holds thousands and every read opens its own decompression stream, which makes Chromium fail the analysis — zip.js then masks the cause behind a `Cannot close a ERRORED writable stream` — while Firefox tolerates it. Do not turn this back into a parallel read. The read is best-effort: a reference that cannot be read leaves that file without size, never the whole package.
- The `:error` of an import entry is user-facing text, never an i18n key. The legacy SSE import stays in the backend for other clients.
- File creation/duplication strips file `:data` before putting file summaries into dashboard state. - File creation/duplication strips file `:data` before putting file summaries into dashboard state.
- The pending invitations are a team-admin only list (backend `:is-admin` already includes the owner). The dashboard hides the invitations entry from the header tabs and the team-options dropdown for non-admins, `invitations-section-content` keys the section on `:is-admin`, and `fetch-invitations` answers `insufficient-permissions` by emitting `invitations-fetched` with `[]`, so a demoted admin does not keep the cached emails and the header does not paint the invite button from a stale list. The organization `send-invitations` policy (`can-invite?`) does not gate the list, but it gates every action on it: inviting (header and empty-state buttons render disabled with the `dashboard.invite-profile-disabled` tooltip), the role selector (read-only label), row selection (so resend and delete never show) and copy link (`invitation-link-copyable?`). The backend enforces the same rule.
## Viewer ## Viewer
- `app.main.ui.ui` parses the viewer's `file-id` query param with `uuid/parse*`, so a missing or malformed id arrives as nil. `viewer-page*` calls `check-file-id!` (same namespace) during render, before mounting `viewer*`: it raises `ex/raise :type :not-found :code :missing-file-id`, which the error boundary turns into the not-found page. Without the guard the `dv/initialize` assertion fails on the nil file-id and the user gets an internal-error page. Test: `frontend-tests.ui.viewer-test/check-file-id-raises-not-found-for-invalid-file-id`. - `app.main.ui.ui` parses the viewer's `file-id` query param with `uuid/parse*`, so a missing or malformed id arrives as nil. `viewer-page*` calls `check-file-id!` (same namespace) during render, before mounting `viewer*`: it raises `ex/raise :type :not-found :code :missing-file-id`, which the error boundary turns into the not-found page. Without the guard the `dv/initialize` assertion fails on the nil file-id and the user gets an internal-error page. Test: `frontend-tests.ui.viewer-test/check-file-id-raises-not-found-for-invalid-file-id`.
- Viewer initialization sets `:current-file-id`, `:current-share-id`, and `:viewer-local`, then fetches the view-only bundle. Comment threads are fetched only for logged-in users. - Viewer initialization sets `:current-file-id`, `:current-share-id`, and `:viewer-local`, then fetches the view-only bundle. Comment threads are fetched only for logged-in users.
- Viewer bundle fetch sends the full supported feature set because anonymous shared viewers may not know team-enabled features. - Viewer bundle fetch sends the full supported feature set because anonymous shared viewers may not know team-enabled features.
- View-only bundles can contain pointer values in `:pages-index` and file data. Viewer resolves those fragments with `:get-file-fragment` before storing the bundle. - View-only bundles can contain pointer values in `:pages-index` and file data. Viewer resolves those fragments with `:get-file-fragment` before storing the bundle, through `app.main.features.pointer-map/resolve-pointer` (share-link scoped; the workspace and library loaders use `resolve-file`, which resolves the same two levels). Both fail loudly: a fragment that cannot be resolved rejects the stream and is logged with file id and fragment id instead of silently dropping the page.
- `bundle-fetched` indexes pages and precomputes viewer frames/all-frames, stores libraries/users/thumbnails/permissions under `:viewer`, then navigates to frame id, query index, or auto-selected frame. - `bundle-fetched` indexes pages and precomputes viewer frames/all-frames, stores libraries/users/thumbnails/permissions under `:viewer`, then navigates to frame id, query index, or auto-selected frame.
- Viewer zoom and interaction mode changes update both `:viewer-local` and the `:viewer` route query params. `update-zoom-querystring` guards with a query-param comparison (`not= current expected`) and uses `::rt/replace`; without that guard the load sequence (`bundle-fetched` → `zoom-to-fill` → `update-zoom-querystring` → `rt/nav`) re-runs on every navigation and crashes with React "maximum update depth exceeded" (2.18.0-RC5 regression, fixed in `31b73460c3`; regression test: `bundle-fetched-with-zoom-fill-url-does-not-navigate`). - Viewer zoom and interaction mode changes update both `:viewer-local` and the `:viewer` route query params. `update-zoom-querystring` guards with a query-param comparison (`not= current expected`) and uses `::rt/replace`; without that guard the load sequence (`bundle-fetched` → `zoom-to-fill` → `update-zoom-querystring` → `rt/nav`) re-runs on every navigation and crashes with React "maximum update depth exceeded" (2.18.0-RC5 regression, fixed in `31b73460c3`; regression test: `bundle-fetched-with-zoom-fill-url-does-not-navigate`).

View File

@ -15,6 +15,9 @@
## Runtime initialization and permissions ## Runtime initialization and permissions
- The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling. - The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling.
- `app.plugins/init-plugins-runtime` returns whether `ɵloadPlugin` was installed. It calls `preg/signal-runtime-ready` even on failure: `workspace.cljs` waits on that promise before `mcp/init`, so skipping it hangs MCP initialization forever.
- When SES bootstrap fails, `app.plugins` emits one `plugins-runtime-unavailable` telemetry event (with the SES error message), and `app.main.data.plugins/load-plugin!` shows the lazy `plugins.runtime-unavailable` toast when the user opens a plugin. `start-plugin!` (MCP path) stays silent by design.
- Runtime availability is detected by the existing `(fn? (unchecked-get global "ɵloadPlugin"))` check; do not add a second mechanism.
- Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id. - Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id.
- Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host. - Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host.
- The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code. - The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code.
@ -23,6 +26,9 @@
- Public Plugin API objects are lightweight handles, not durable snapshots. Most getters locate fresh state from `app.main.store/state` using hidden `$id`, `$file`, `$page`, etc. - Public Plugin API objects are lightweight handles, not durable snapshots. Most getters locate fresh state from `app.main.store/state` using hidden `$id`, `$file`, `$page`, etc.
- `not-valid` logs by default but throws when the plugin flag `throwValidationErrors` is enabled. The MCP execute-code handler deliberately enables that flag while running code. - `not-valid` logs by default but throws when the plugin flag `throwValidationErrors` is enabled. The MCP execute-code handler deliberately enables that flag while running code.
- A write to a removed shape always throws `Shape not found`, whatever that flag: setters through the shape proxy's `:on-set u/check-shape-writable` (runs outside the error handling), and methods that must not vanish silently (`resize`, `rotate`) call `u/check-shape-located` first. Methods run inside `:on-error (u/handle-error ...)`, which logs instead of throwing when the flag is off, so it rethrows errors whose `ex-data :type` is `::u/shape-not-found` or `::r/read-only`.
- A write while the workspace cannot take it (`r/read-only-reason`: save-failure lock `[:workspace-global :persistence-lock]`, `[:permissions :can-edit]` false, version preview `:preview-id`, lost render context `[:render-state :lost]`) always throws, whatever that flag. The `[:workspace-global :read-only?]` flag alone does not block: the Inspect tab sets it and plugins may still write. `r/check-permission` with `content:write`/`library:write` calls `r/check-writable` first, which covers most writers; writers without a permission check (most top-level context methods in `api.cljs`: `create*`, `group`, `align*`, …; z-order methods; text range `applyTypography`; ruler guide `remove`) call `r/check-writable` themselves, and shape setters get it through `u/check-shape-writable`. A new writer without a write-permission check must call it too. An async writer calls it again after each server await, before the next server write or commit (`rx/tap` on the response stream; `dwm/create-svg-shape-with-images` takes it as `:check-writable`), so a lock that lands mid-way rejects the promise with the read-only error. `r/has-permission?` is the check without it, used by the undo blocks so a block opened before a lock still closes. Comments (`comment:write`) stay writable: they do not go through file saves.
- `penpotMgmt` (`app.plugins.management`) checks permissions with `r/has-permission?` (`granted?`), never `r/check-permission`: its calls act on files through the backend, often from the dashboard with no file open, so the workspace read-only check (`check-writable`) must not block them.
- `naturalChildOrdering` and `throwValidationErrors` are stored per plugin under `[:plugins :flags plugin-id ...]`; changing default behavior affects automation and MCP diagnostics. - `naturalChildOrdering` and `throwValidationErrors` are stored per plugin under `[:plugins :flags plugin-id ...]`; changing default behavior affects automation and MCP diagnostics.
- Plugin data is stored under keyword namespaces: private data uses `(keyword "plugin" plugin-id)`, shared data uses `(keyword "shared" namespace)`. - Plugin data is stored under keyword namespaces: private data uses `(keyword "plugin" plugin-id)`, shared data uses `(keyword "shared" namespace)`.
@ -31,4 +37,7 @@
- Plugin listeners are watches on the global store and callbacks are debounced about 10ms. Callback exceptions are caught and logged so plugin code does not crash the app. - Plugin listeners are watches on the global store and callbacks are debounced about 10ms. Callback exceptions are caught and logged so plugin code does not crash the app.
- `selectionchange` callbacks receive arrays of shape id strings, while `filechange`, `pagechange`, and `shapechange` return proxies. - `selectionchange` callbacks receive arrays of shape id strings, while `filechange`, `pagechange`, and `shapechange` return proxies.
- `contentsave` fires only when persistence status transitions to `:saved`; it calls the callback with no value. - `contentsave` fires only when persistence status transitions to `:saved`; it calls the callback with no value.
- `penpot.waitForSave(timeout?)` (`app.plugins.persist/wait-for-save`; runtime forwarder checks `content:read`) waits on the **shared** persistence queue, not only the plugin's own commits. Flow: ~100ms intake race (buffer/queue non-empty vs deadline, capped to the timeout) → empty resolves; pending emits `::dps/force-persist` and races `dps/wait-persisted-or-error` against `::dps/rollback-rejected` / `::dps/rolled-back` (rollback rejects even if status later reads `:saved`). Default 60s. `:error` status at intake rejects at once (`earlier-failure`): `save-retrying` while `dps/save-retrying?` (slow retry cycle, recovers by itself), otherwise the stored `[:error :code]` with reopen advice (a terminal failure stays until the file is reopened, so it fails every call). Rejections are `Error`s with `code`: `save-timeout`, `rolled-back`, `save-retrying`, or the failure code (`save-failed` unless the backend gave one).
- Plugin timeouts validate through `u/valid-timeout?` (positive finite number ≤ `u/max-timeout`, signed 32-bit) and reject through `u/reject-not-valid`, which rejects an `Error` (never a bare string) so callers can read `.message`.
- `contentrollback` is not a state watch: it subscribes to `st/stream` for `::dps/rolled-back` (kept in `subscriptions`, disposed by `remove-listener`), not debounced, delivered async so the reverted state is in place. Payload `{fileIds pageIds shapeIds}` from `pcm/rollback-targets` (page shapes only, from the reverted commits' redo changes). When the rollback falls back to reloading the file, `::dps/rolled-back` waits for the reloaded data, so the callback never reads the rejected data. The runtime's `validEvents` must list every event name, or `penpot.on` rejects it; a stale runtime build rejects new events. The MCP plugin forwards these rollbacks with the next task response and the server puts a warning first in that tool response (`RollbackNotices`). Unsent ones survive a tab or plugin reload in `penpot.localStorage` (key `rollbacks/<file-id>`, `LocalStorageRollbackStore`; the MCP manifest in `app.main.data.workspace.mcp` and `mcp/packages/plugin/public/manifest.json` grant `allow:localstorage`, which the runtime checks on access); the entry goes once a response carried them.
- Plugin history `undoBlockBegin` creates a workspace undo transaction with a JS `Symbol`; `undoBlockFinish` commits that symbol. Missing finish eventually relies on the workspace transaction timeout. - Plugin history `undoBlockBegin` creates a workspace undo transaction with a JS `Symbol`; `undoBlockFinish` commits that symbol. Missing finish eventually relies on the workspace transaction timeout.

View File

@ -7,6 +7,8 @@
- `set-token!` and `replace-token!` are deliberately asymmetric. `set-token!` is a total no-op when the token already is the current one (no URL write, no report), so navigating to the screen you are on cannot grow the history stack. `replace-token!` always writes and always reports, because it is how the app corrects the URL in place (legacy `#/…` translation, cleaned share links) and that correction must reach the router even when the token did not change. This matches `Html5History`, which did the same; do not "fix" the asymmetry without deciding what navigates to the current screen. - `set-token!` and `replace-token!` are deliberately asymmetric. `set-token!` is a total no-op when the token already is the current one (no URL write, no report), so navigating to the screen you are on cannot grow the history stack. `replace-token!` always writes and always reports, because it is how the app corrects the URL in place (legacy `#/…` translation, cleaned share links) and that correction must reach the router even when the token did not change. This matches `Html5History`, which did the same; do not "fix" the asymmetry without deciding what navigates to the current screen.
- `app.util.dom/replace-history-state!` is the deliberate bypass used by `app.main.ui.dashboard`/`app.main.ui.static` to rewrite the URL **without** routing. Keep using it there instead of `replace-token!`, which would re-enter the router. - `app.util.dom/replace-history-state!` is the deliberate bypass used by `app.main.ui.dashboard`/`app.main.ui.static` to rewrite the URL **without** routing. Keep using it there instead of `replace-token!`, which would re-enter the router.
- Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required. - Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required.
- The organization SSO flow sends the **query string** (`rt/get-current-query-string`), not the URL and not the token: the backend builds the destination URL from `public-uri` + `?` + those params, so the client never chooses the origin of the redirect. The history token (`rt/get-current-token`, `location.search`) keeps its leading `?` because every other consumer concatenates it over the base path (`str cf/public-uri token`, `dom/append-query-param`, `dom/remove-query-param`); `rt/token->query-string` is the only bridge between the two shapes, so do not trim at a call site by hand; `rt/token->url` is the single client builder of an absolute destination from either shape (used by the SSO retry and error flow), so do not assemble `public-uri + "?" + params` at call sites. The gate is not the only caller: `dtm/resolve-login-team-id` runs from the root branch of `on-navigate` too, and there the URL carries its params instead of a screen (`?template=…`) — that call only reads `:authorized`, so it never follows the destination, which is exactly why a wrong destination there goes unnoticed. Do not add a rule that requires `screen`. See `mem:backend/auth-permissions-product-domains`.
- The router events expose their payload through `@event` (`IDeref`): `rt/navigate` returns `{:id :params :options}` and `rt/nav-raw` returns its `:href`/`:uri`. `rt/nav-raw` is the way out of the router (raw URL, outside the token scheme) and nothing in the app listens to `::rt/nav-raw`; the destination travels in the effect, so `@event` is the only way to tell where the user is being sent (tests included).
- Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete). - Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete).
- Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions. - Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions.
- The root app has two error layers, and they reset differently. `:exception` in the store renders `static/exception-page*` directly and is cleared by `rt/navigated`; a React render error is caught by `app.main.ui.error-boundary` (`react-error-boundary`), whose error lives in internal React state that only `reset-error-boundary` clears. Because every exit from the error page (the logo, the no-permission dialogs, the SSO dialog, the feedback link) only *navigates*, the boundary is given `:resetKeys [(eb/reset-key route)]` — the screen name — so a navigation counts as a fresh attempt at rendering. Three traps in that `resetKeys` value, all learned the hard way: it must be a real JS array (`#js [...]`), never a ClojureScript vector, because `react-error-boundary` reads `.length` and `.some` and a `PersistentVector` has neither, so `componentDidUpdate` throws `some is not a function`; it must hold the screen name as a *string*, because the library compares entries with `Object.is` and ClojureScript does not intern keywords, so two equal keywords are two objects and a keyword key would read as new on every route rebuilt, resetting the boundary on a param change inside the same screen; and it must be the screen rather than the params, so a param change does not replay the screen that just crashed. Navigating to the *same* screen therefore does not reset, and a reload is the only way out of that case. - The root app has two error layers, and they reset differently. `:exception` in the store renders `static/exception-page*` directly and is cleared by `rt/navigated`; a React render error is caught by `app.main.ui.error-boundary` (`react-error-boundary`), whose error lives in internal React state that only `reset-error-boundary` clears. Because every exit from the error page (the logo, the no-permission dialogs, the SSO dialog, the feedback link) only *navigates*, the boundary is given `:resetKeys [(eb/reset-key route)]` — the screen name — so a navigation counts as a fresh attempt at rendering. Three traps in that `resetKeys` value, all learned the hard way: it must be a real JS array (`#js [...]`), never a ClojureScript vector, because `react-error-boundary` reads `.length` and `.some` and a `PersistentVector` has neither, so `componentDidUpdate` throws `some is not a function`; it must hold the screen name as a *string*, because the library compares entries with `Object.is` and ClojureScript does not intern keywords, so two equal keywords are two objects and a keyword key would read as new on every route rebuilt, resetting the boundary on a param change inside the same screen; and it must be the screen rather than the params, so a param change does not replay the screen that just crashed. Navigating to the *same* screen therefore does not reset, and a reload is the only way out of that case.
@ -22,6 +24,9 @@
- `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler. - `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler.
- The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission. - The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission.
- Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason. - Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason.
- A team scoped navigation carries a team whenever any of the four tiers exists, and `app.main.data.common/resolve-team-id` decides it for every team scoped `go-to-*` helper of that namespace: the UUID the caller passes, `:current-team-id`, the team of the route already on screen (`?team-id`), then the profile `:default-team-id`. `go-to-feedback` and `go-to-viewer` are not team scoped and do not go through it, and `:team-id :default` keeps meaning the profile default, resolved before the rest. The route goes before the profile default because an emitter that runs from a team screen knows which team the user is looking at, and the personal team would open their file under another team. Two `rt/nav` emitters outside that namespace still decide their own id and can still emit nil, and they are the residual: `app.main.ui.nitrate.entry/nitrate-entry*` (the profile default) and the root branch of `app.main.ui.routes/on-query-navigate` (`dtm/resolve-login-team-id`, which answers nil when the profile has no default team). When none of the tiers exists the helper still fires the navigation with a nil id, so the not-found page appears instead of the click doing nothing.
- `app.main.ui/team-container*` raises `:not-found :invalid-team-id` for a dashboard or workspace route without a valid `team-id`; the raise is what keeps a broken URL from painting a blank shell, so it stays after the hook fix of #12728. `exception-section*` skips `errors/submit-report` for `:not-found`, so this failure never reaches PostHog and cannot be measured from the error dashboard. The exit from that page is the `error-container*` header (logo plus `not-found.no-permission.go-dashboard`), which emits `go-to-dashboard-recent` with the profile default and falls back to `rt/nav-root`; every not-found and authentication page already carries it, so there is no button to add there.
- A click cannot reach a team screen before its team is resolved: `team-container*` paints its children only when the route id matches the store team, and `refs/team` derives from `:current-team-id`, so with that key empty the dashboard and the workspace paint nothing clickable. The nil-id emissions come from events that run on their own (login, invitation acceptance, membership change, the sidebar logo, the workspace shortcuts) and from typed or bookmarked URLs.
## Store and websocket ## Store and websocket
@ -29,3 +34,4 @@ For general store mechanics such as `emit!`, `last-events`, persistence, and und
- Websocket initialization uses `cf/public-uri` joined with `ws/notifications`, converting `http/https` to `ws/wss`, and includes the current `session-id` as query param. - Websocket initialization uses `cf/public-uri` joined with `ws/notifications`, converting `http/https` to `ws/wss`, and includes the current `session-id` as query param.
- Reinitializing or finalizing websocket stops the previous receive stream. Incoming websocket payloads become Potok data events under `app.main.data.websocket/message`. - Reinitializing or finalizing websocket stops the previous receive stream. Incoming websocket payloads become Potok data events under `app.main.data.websocket/message`.
- A job the client created is followed with `app.main.data.jobs/watch-job`: it emits the `:job-event` messages of that job (filtered by `:job-id`) and one final outcome read from the row with `get-job`, which is read when the stream starts and again on every `opened` of the websocket, so a job that ended before the client started following it, or while the socket was down, still answers. The stream ends after that outcome, and a read that fails reaches the caller: the repo retries the transient transport errors of a GET, and anything else is a real failure the flow has to show. The `:stage` of a progress event is a stable key of the vocabulary of the worker, never a display string: `app.main.ui.jobs.progress` resolves it with a `case` of literal `tr` calls, falls back to a generic label for a stage it does not know, and the counter of the activity is named after its own stage (the counter of `:files` is the outer context).

View File

@ -17,7 +17,8 @@ Frontend testing is async-first: everything essentially asynchronous is modeled
- `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests. - `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests.
- `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`. - `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`.
- `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses). - `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses).
- Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for` (immediate check + bounded poll, fails instead of hanging), `observe` (stream → termination promise; asserts provided, timeout rejects). - Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for`, `wait-for-store`, `observe` (stream → termination promise; asserts provided, timeout rejects).
- `wait-for` waits on a **signal**, not on turns: `:signal` (an observable) re-checks the predicate on every emission and resolves on the event that changes the state; `:timeout-ms` (default 5000) only bounds failure then. Without a signal it polls against a real-time deadline — 50 turns were ~54 ms idle, and turns are not a unit of time. `wait-for-store` passes `ptk/input-stream` as the signal (the store notifies after applying the update, so the predicate sees the new state); the persistence fixture exposes `requests$` for `(= N (count @requests))`. A wait that resolves on an event consumes no turn. The waits with nothing to listen to stay on the poll: test atoms filled outside the store (`@cycles`, `@reports`, `@done`), `@failures`, and predicates over JS objects or closures.
- Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left). - Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left).
- Valid `^:async` placements: `mem:clojure/idioms`. - Valid `^:async` placements: `mem:clojure/idioms`.
@ -37,8 +38,15 @@ To assert over emitted event sequences, observe termination: subscribe through `
- `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`. - `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`.
- Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline. - Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline.
- Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis. - Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis.
- A wait that observes an effect the test asserts takes a signal. A wait that means "the side effect advanced before the next emit" has no event to listen to: keep the poll (or an explicit `settle`) and say so, because a signal would resolve inside the same `emit!` and the next step would overtake the pipeline (`persistence-rollback-test` drove its four emits this way).
- With `rp/cmd!` mocked, a store-state wait can resolve before the flow that changes it ends, and the mock scope then closes in flight: the next call hits the real HTTP layer (malformed URL from a nil `cf/public-uri`) and the error is reported on a later test. `unlink-file-from-library` dissocs the library before its async chain, so the only proof the chain ran inside the mock is its second RPC call; wait for that, through the mock.
- Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows). - Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows).
- A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop. - A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop.
- `app.main.errors` prints every handled cause with `ex/print-throwable` before notifying, and the app logger does not gate it. A test that runs a real handler end to end stubs that var (`main-errors-test/quiet-throwable`): without it, the 10 000-error governor test alone wrote 190 k of the 192 k lines of a run log (13,8 MB), and GitHub truncates a step log past ~4 MB, cutting the summary and any failing assertion. Same for a production `console.error` a test provokes on purpose: `code-gen-markup-svg-wasm-test` silences `(.-error js/console)` in a `try/finally` while two WASM panics go through `get-svg-markup` (2 blocks, 70 lines, and the exception dumps its own `data`/`arr`).
- The `:esm` test build cannot load the CommonJS `sax` module behind `tubax`, so SVG parsing fails in tests (`dwm/valid-svg-string?` answers false, `dwm/svg->clj` errors): mock both with the parsed data.
- A hook-order violation in a mounted component reaches neither the root's `onUncaughtError`/`onCaughtError`/`onRecoverableError` nor the markup: React logs an internal error to `console.error` and recovers, so the callbacks stay empty and the tree repaints. Assert on a captured `console.error` too. It only fires when a render *adds* hooks (nil → uuid); a render that calls none at all slips through React's bookkeeping, so the regression test needs the "hooks arrive" direction. `frontend-tests.ui.team-container-test` does both, with a jsdom document.
- `beicon.v2.core` has no `never`. A stream that must **not emit** is `(rx/empty)`: it completes at once, which is enough for `merge`, `filter` and `take` where only the emission matters. A stream that must **stay open** is a `(rx/subject)` nobody pushes to; `rx/empty` would complete the flow behind it. An undeclared `rx/never` compiles to an undefined var and the test dies as a rejected body, which `cljs.test` reports as a test with zero assertions instead of a failure.
- `set!` mocks are global and outlive the fixture, so a flow another module left running (a store's pending timer) can still reach the mocked transport mid-test; `with-persistence` records every command for that reason. Filter to the commands your layer issues — persistence only sends `update-file` — before counting or asserting emptiness, and never let a stray call release a `wait-for` on one.
From `frontend/`: From `frontend/`:
- Full unit test run (always builds, suppressed output): `pnpm run test:quiet`. - Full unit test run (always builds, suppressed output): `pnpm run test:quiet`.
@ -54,7 +62,7 @@ New frontend test namespaces must be required/listed in `frontend_tests/runner.c
## Playwright integration tests ## Playwright integration tests
Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, use `pnpm run test:e2e` or `pnpm run test:e2e --grep "pattern"` from `frontend/`; ensure dependencies are installed through `./scripts/setup` if the environment is not prepared. Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, run the whole suite with a worker count sized to the machine: `pnpm run test:e2e --workers=N` from `frontend/`, about one worker per two hardware threads, because each keeps two or three busy and holds about 2 GB of browser memory. The config's default of one worker leaves it slow; four take a 16-thread host from 48 minutes to 35 and fit a 64 GB machine beside a watching devenv, while a 32 GB machine takes two, or four with a slim devenv. `--grep "pattern"` narrows the run. Ensure dependencies are installed through `./scripts/setup` if the environment is not prepared.
Integration tests fake backend behavior by intercepting network/websocket traffic, so every RPC or websocket the page needs must be mocked. Use existing Page Object Models: Integration tests fake backend behavior by intercepting network/websocket traffic, so every RPC or websocket the page needs must be mocked. Use existing Page Object Models:
- `BasePage.mockRPC` intercepts RPC calls and already prefixes `/api/rpc/command/`; pass command names such as `get-profile`, not full URLs. - `BasePage.mockRPC` intercepts RPC calls and already prefixes `/api/rpc/command/`; pass command names such as `get-profile`, not full URLs.

View File

@ -7,10 +7,11 @@ high-coverage support reference, never the base.
## Workflow ## Workflow
- Canonicalize with `node ./scripts/translations.js sync -l <locale>` from `frontend/`: sorts entries, syncs `#:` comments from `en`, deletes keys missing in `en`. - Canonicalize with `node ./scripts/translations.js sync -l <locale>` from `frontend/`: sorts entries, syncs `#:` comments from `en`, deletes keys missing in `en`.
- `sync` never ADDS a key missing in the locale: it only syncs comments (for keys that already exist) and drops keys absent from `en`. A key added to `en` alone stays missing in `es` and its text silently falls back to English. Every new key is born in `en` AND `es` in the same change; compare the `msgid` sets to check (`comm -23 <(grep '^msgid ' en.po | sort -u) <(grep '^msgid ' es.po | sort -u)` must be empty).
- `sync` copies ALL comment flags from `en`, including `#, fuzzy`. Translated entries must NOT stay fuzzy: strip the flag after translating (mirror `es.po`, which keeps fuzzy only on genuinely untranslated entries). Fuzzy entries are excluded from `msgfmt --statistics` translated counts. - `sync` copies ALL comment flags from `en`, including `#, fuzzy`. Translated entries must NOT stay fuzzy: strip the flag after translating (mirror `es.po`, which keeps fuzzy only on genuinely untranslated entries). Fuzzy entries are excluded from `msgfmt --statistics` translated counts.
- Canonical files carry NO `#~` obsolete blocks (`en`/`es` have zero). Drop them; `sync` does not resurrect them. - Canonical files carry NO `#~` obsolete blocks (`en`/`es` have zero). Drop them; `sync` does not resurrect them.
- Definition of done: `msgfmt --check <locale>.po` exit 0 and `msgfmt --statistics` shows 0 fuzzy, 0 untranslated. - Definition of done: `msgfmt --check <locale>.po` exit 0, `msgfmt --statistics` shows 0 fuzzy, 0 untranslated, and the locale's `msgid` set is identical to `en.po`'s (no key missing, none extra).
- `rehash` scans `frontend/src` AND `common/src` for `(tr "key"` occurrences (any line counts, including `;;` comments) and refreshes `#:` refs in `en`, marking unreferenced keys `#, unused`. - `rehash` scans `frontend/src` AND `common/src` for `(tr "key"` occurrences (any line counts, including `;;` comments) and refreshes `#:` refs in `en`, marking unreferenced keys `#, unused`. The refs in `en` are broadly stale, so a repo-wide `rehash` rewrites hundreds of lines (measured: ~487 on `en` alone): it is a maintenance action on its own, never a step to run inside a change that only adds or edits a key.
- Dynamic keys are invisible to `rehash`: eliminate them instead of - Dynamic keys are invisible to `rehash`: eliminate them instead of
declaring them. Preference order: literal `(tr "key")` args only; declaring them. Preference order: literal `(tr "key")` args only;
never put a branch inside `tr`, hoist it out (`(if cond (tr "a") (tr "b"))`); never put a branch inside `tr`, hoist it out (`(if cond (tr "a") (tr "b"))`);
@ -28,6 +29,12 @@ high-coverage support reference, never the base.
## Entry rules ## Entry rules
- Placement: a key names the feature that renders it (`dashboard.import.*`,
`files-export-modal.*`, `jobs.*`); `labels.*` is the bucket for the short
generic words reused across screens (Accept, Cancel, Download…), never the
home of a feature's own vocabulary. Keep a feature's strings together: the
durable jobs write their progress stages as `jobs.progress.stage.*` and
their row statuses as `jobs.*`, not half in `labels.*`.
- Keys resolved dynamically from server-provided data carry a `#, backend` - Keys resolved dynamically from server-provided data carry a `#, backend`
flag (set it in `en`, `sync` copies it to the locales): do not remove or flag (set it in `en`, `sync` copies it to the locales): do not remove or
rename these entries, and keep their `%s` placeholders (the backend only rename these entries, and keep their `%s` placeholders (the backend only
@ -40,7 +47,6 @@ high-coverage support reference, never the base.
- Preserve verbatim: `%s`/`%d`, `{var}`/`{{...}}`, markdown `[text](%s)`, HTML tags, `\n` positions, brand names (Penpot), key names (Ctrl/Shift/Alt), technical terms (SVG, CSS, HSV, RGB). - Preserve verbatim: `%s`/`%d`, `{var}`/`{{...}}`, markdown `[text](%s)`, HTML tags, `\n` positions, brand names (Penpot), key names (Ctrl/Shift/Alt), technical terms (SVG, CSS, HSV, RGB).
## Catalan (ca) conventions ## Catalan (ca) conventions
- Normative IEC/Termcat Catalan. Address the user in VOSALTRES (2nd person plural): "Deseu", "Creeu", "Ja teniu un compte?". Buttons/menus use short imperatives ("Crea", "Mou", "Restaura"). - Normative IEC/Termcat Catalan. Address the user in VOSALTRES (2nd person plural): "Deseu", "Creeu", "Ja teniu un compte?". Buttons/menus use short imperatives ("Crea", "Mou", "Restaura").
- Established glossary (reuse exactly, do not re-coin): layer=capa, board=tauler, stroke=traç, fill=Emplenat, blur=Difuminat, shadow=Ombra, clipboard=porta-retalls, delete=Elimina, rename=Canvia el nom, shortcut=drecera, grid=graella (keep "grid" where the file already does, e.g. grid-layout editing), plugins/extensions UI=extensions, layout=Disposició, gradient=Degradat, wireframing kept as loanword. - Established glossary (reuse exactly, do not re-coin): layer=capa, board=tauler, stroke=traç, fill=Emplenat, blur=Difuminat, shadow=Ombra, clipboard=porta-retalls, delete=Elimina, rename=Canvia el nom, shortcut=drecera, grid=graella (keep "grid" where the file already does, e.g. grid-layout editing), plugins/extensions UI=extensions, layout=Disposició, gradient=Degradat, wireframing kept as loanword.
- Ela geminada uses the middle dot: Cancel·la, paral·lel, al·lega. - Ela geminada uses the middle dot: Cancel·la, paral·lel, al·lega.
@ -48,6 +54,20 @@ high-coverage support reference, never the base.
- Shortcut/action names (`shortcuts.*`) are noun/infinitive labels, not sentences. Error strings are direct, no hedging. - Shortcut/action names (`shortcuts.*`) are noun/infinitive labels, not sentences. Error strings are direct, no hedging.
- Same English source in different contexts may legitimately differ (verb "Copia" vs noun "Còpia"; "Desactivat" vs "Deshabilitada" agreeing with "drecera"). Normalize only true duplicates. - Same English source in different contexts may legitimately differ (verb "Copia" vs noun "Còpia"; "Desactivat" vs "Deshabilitada" agreeing with "drecera"). Normalize only true duplicates.
## Russian (ru) conventions
- Formal address with capital ВЫ ("Вы уверены…", "Вы собираетесь…"). Buttons/menus use infinitives ("Создать", "Удалить", "Дублировать"), never imperatives.
- Established glossary (reuse exactly, do not re-coin): layer=слой, board=доска (EN `board` is NEVER кадр — normalized in lote1: toolbar/viewer/shape-menu/header-menu/export-frames/undo/num-of-frames), path/draw tool=контур, stroke=обводка, fill=заливка, blur=размытие, shadow=тень, grid=сетка, clipboard=буфер обмена, shortcuts=сочетания клавиш (normalized in lote1: title/show-shortcuts/not-found), plugin=плагин, layout=макет (`labels.layout` fixed in lote1), gradient=градиент, token=токен, set=набор, theme=тема, file=файл, project=проект, team=команда, library=библиотека, component=компонент.
- No external RU style guide applies verbatim (Mozilla ru guide is marketing-only, GNOME glossary unmaintained, Adobe uses монтажная область for artboard but `ru.po` majority already says доска — keep доска for consistency).
- `words.ru.txt` `[function]` DELIBERATELY omits 1-letter words (в/с/к/о/у/а/и) and ending-colliding particles (на/по/не/ли/же/бы/его/их/те/или…): Russian inflects (проект→проекта, запросили=запрос+или), so listing them flags every inflected noun/verb. Never "complete" that list; put tripped valid words in `[ok]`.
- Never abbreviate т. д. / т. е. in `ru.po`: the lone т/д tokens poison the glue detector's frequency map and flag short words (тон→т+он). Write и так далее out.
- Tokens-domain glossary (lote2, reuse): alias=алиас, remap=переназначить, viewport=вьюпорт, set=набор, theme=тема, spread (shadow)=распространение, margins=внешние отступы, paddings=внутренние отступы, gaps=промежутки, type identifiers stay in English (Line Height, Font Size, innerShadow/dropShadow, none/Uppercase/Lowercase/Capitalize, thin/light/regular/bold/Italic).
- Editor-domain glossary (lote3, reuse): variant=вариант, property=свойство, guide=направляющая, board guide=направляющая доски, MCP server=MCP-сервер, size preset=пресет размера, border-radius section=скругление, path handler=маркер, canvas=холст, render=рендер/рендеринг, plugin (never extensión)=плагин.
- Avoid standalone те (those) in phrasing ("те же" → "такие же"): one lone те validates те+мы splits and flags every темы. Same poisoning class as т. д.
- `words.ru.txt` `[function]` also omits пока (dropped in lote4): with standalone за anywhere in the file, пока+за flags показа. Rare-typo detection loss accepted.
- Billing-domain glossary (lote4, reuse): plan names follow shipped `ru.po`, NOT es: Enterprise=Корпоративный, Professional=Профессиональный, Unlimited=Безлимитный, trial=пробный; subscription=подписка, storage=хранилище, seat editors=редакторы, viewer role=наблюдатель, amounts as `175 $/мес.` / `10 ГБ`; access token=токен доступа, MCP key=MCP-ключ (regenerate=перевыпустить), markdown `[text|target:self](%s)` kept verbatim.
- Closing-domain glossary (lote5, reuse): shortcut noun everywhere=сочетание (shortcuts section titles are noun/infinitive labels, never ярлыки); node=узел, segment=сегмент, measurements=размеры; WebGL unavailable flow uses гайд/руководство по неполадкам + рендеринг WebGL выключен/включён; `ru.po` reached 2275/2275 keys (100%) with 0 fuzzy, 0 untranslated per `msgfmt --statistics`.
## QA before commit ## QA before commit
- Run `node ./scripts/translations.js check -l <locale>` from `frontend/` (no default locale: pass `-l` explicitly): 0 errors required; review warnings by hand. Word lists live in `frontend/scripts/check-translations/words.<locale>.txt` (`[elision]` `[function]` `[common]` `[ok]` `[brands]`); new valid words that trip the gate go to `[ok]`; `check --self-test` covers the detector rules. Without a catalog only the universal checks run. `#, fuzzy` entries are skipped (known-pending, owned elsewhere). - Run `node ./scripts/translations.js check -l <locale>` from `frontend/` (no default locale: pass `-l` explicitly): 0 errors required; review warnings by hand. Word lists live in `frontend/scripts/check-translations/words.<locale>.txt` (`[elision]` `[function]` `[common]` `[ok]` `[brands]`); new valid words that trip the gate go to `[ok]`; `check --self-test` covers the detector rules. Without a catalog only the universal checks run. `#, fuzzy` entries are skipped (known-pending, owned elsewhere).

View File

@ -12,7 +12,7 @@
- Co-located SCSS modules are preferred. Use `app.main.style/stl` helpers from CLJS and design-system SCSS tokens/mixins instead of legacy global selectors or high-specificity nesting. - Co-located SCSS modules are preferred. Use `app.main.style/stl` helpers from CLJS and design-system SCSS tokens/mixins instead of legacy global selectors or high-specificity nesting.
- Keep CSS specificity low. Avoid nested selectors unless they target elements the component owns; CSS Modules already prevent class-name collisions. - Keep CSS specificity low. Avoid nested selectors unless they target elements the component owns; CSS Modules already prevent class-name collisions.
- Prefer CSS logical properties for directional spacing/layout (`padding-inline-start`, etc.). Physical `width`/`height` are still acceptable where they are clearer. - Prefer CSS logical properties for directional spacing/layout (`padding-inline-start`, etc.). Physical `width`/`height` are still acceptable where they are clearer. Never mix: a stylesheet logical property (`inline-size`) and an inline physical `width` on the same element cascade separately, and the logical mapping wins — a dynamically driven width must be `width` on both sides (import modal progress bar hit this). And an inline style passed from CLJS must be a JS object (`#js {:width …}`): for plain `[:div …]` elements the Rumext macro does not convert the `:style` map at runtime — a plain CLJS map reaches React raw and React never finds the keys (static style literals are converted at compile time, which is why `:style` maps seem to work elsewhere).
- Use named design-system variables/tokens for spacing, borders, fixed dimensions, colors, and typography. Avoid hardcoded px/rem values and deprecated `resources/styles/common/refactor/spacing.scss` variables. - Use named design-system variables/tokens for spacing, borders, fixed dimensions, colors, and typography. Avoid hardcoded px/rem values and deprecated `resources/styles/common/refactor/spacing.scss` variables.
- Use component-local CSS custom properties for variants and theming instead of one-off Sass variables when a component has multiple visual states. - Use component-local CSS custom properties for variants and theming instead of one-off Sass variables when a component has multiple visual states.
- Prefer DS typography components (`heading*`, `text*`) and typography mixins instead of plain text wrappers or deprecated typography mixins. - Prefer DS typography components (`heading*`, `text*`) and typography mixins instead of plain text wrappers or deprecated typography mixins.

File diff suppressed because one or more lines are too long

View File

@ -6,6 +6,7 @@
- `apply-modifiers*` is the lower-level commit path once object/text modifiers are ready. It updates frame guides, frame comment threads, and then emits `update-shapes` with `:reg-objects? true`. - `apply-modifiers*` is the lower-level commit path once object/text modifiers are ready. It updates frame guides, frame comment threads, and then emits `update-shapes` with `:reg-objects? true`.
- Transform commits restrict diff attrs to `transform-attrs` to avoid scanning unrelated shape attrs. - Transform commits restrict diff attrs to `transform-attrs` to avoid scanning unrelated shape attrs.
- Text transforms may carry derived `:position-data`; `assoc-position-data` attaches it while preserving the original text shape context. - Text transforms may carry derived `:position-data`; `assoc-position-data` attaches it while preserving the original text shape context.
- A drag commits when its `mse/drag-stopper` fires, `:interrupt` (Escape) included. The save-failure lock emits `::plk/cancel-gestures` first: drags built on `plk/gesture-stopper` (start-move, start-resize, start-rotate, start-move-line-point, box/curve/line drawing) see a `cancelled?` flag set before the stop and emit `dwt/cancel-transform` (`clear-local-transform` + `finish-transform`) or `cancel-drawing` instead. A new drag gesture must use it (or, in a component, drop the drag when read-only turns on): `commit-changes` refuses the commit anyway, but the preview (WASM modifiers/props) would stay.
## Component-copy touched suppression ## Component-copy touched suppression

View File

@ -13,19 +13,26 @@ enabling advanced AI-driven features in Penpot.
- Build Tool: TypeScript Compiler (tsc) + esbuild - Build Tool: TypeScript Compiler (tsc) + esbuild
- Package Manager: pnpm - Package Manager: pnpm
## General Principles ## Principles & Code Style
IMPORTANT: Use an idiomatic, object-oriented style. IMPORTANT: Use an idiomatic, object-oriented style.
In particular, this implies that, for any non-trivial interfaces, you use interfaces that expect explicitly typed abstractions * You keep each concern in exactly one home.
rather than mere functions (i.e. use the strategy pattern, for example). A mechanism whose parts are only correct in combination is implemented as a single component/class, and its parts are private to it;
expose a minimal public surface. Java-like encapsulation: If helper functions and constants are only used by one abstraction,
make them internal to the respective class.
* For any non-trivial interfaces, you use functions/methods that expect explicitly typed abstractions
rather than mere functions (i.e. use the strategy pattern, for example).
* Prefer declaring and instantiating concrete classes over defining only interfaces and using object literals.
Comments: Comments:
When describing parameters, methods/functions and classes, you use a precise style, where the initial (elliptical) phrase * When describing parameters, methods/functions and classes, you use a precise style, where the initial (elliptical) phrase
clearly defines *what* it is. Any details then follow in subsequent sentences. clearly defines *what* it is. Any details then follow in subsequent sentences.
* When describing what blocks of code do, you also use an elliptical style and start with a lower-case letter unless
the comment is a lengthy explanation with at least two sentences (in which case you start with a capital letter, as is
required for sentences).
When describing what blocks of code do, you also use an elliptical style and start with a lower-case letter unless Syntax:
the comment is a lengthy explanation with at least two sentences (in which case you start with a capital letter, as is * Never use if blocks (or other blocks) without braces, even for single-line cases.
required for sentences).
## Project Structure (Excerpt) ## Project Structure (Excerpt)
@ -50,7 +57,7 @@ mcp/
│ ├── src/ │ ├── src/
│ │ ├── main.ts # handles communication │ │ ├── main.ts # handles communication
│ │ └── plugin.ts # plugin implementation │ │ └── plugin.ts # plugin implementation
│ └── package.json # Includes @penpot-mcp/common dependency │ └── package.json # Plugin deps: published plugin-styles, linked plugin-types
└── prepare-api-docs # Python project for the generation of API docs └── prepare-api-docs # Python project for the generation of API docs
``` ```
@ -100,8 +107,36 @@ From the `mcp/` directory, run
(node reports `Promise resolution is still pending but the event loop has (node reports `Promise resolution is still pending but the event loop has
already resolved` for the file's root test). Inside tests the logger writes to already resolved` for the file's root test). Inside tests the logger writes to
stdout directly. `NODE_TEST_CONTEXT` is the env var `node --test` sets. stdout directly. `NODE_TEST_CONTEXT` is the env var `node --test` sets.
* `packages/plugin` takes `@penpot/plugin-styles` from the registry but links
`@penpot/plugin-types` to `plugins/libs/plugin-types` (`link:` in the manifest
and in `mcp/pnpm-lock.yaml`, with no registry entry, because a linked package
has no resolution): the published types do not declare the `penpotMgmt`
global the plugin needs. On a plugins libs version bump, only the styles
version moves. After editing the manifest, `pnpm install --lockfile-only` in
`mcp/` must leave the lock untouched.
* Cross-cutting testing principles and anti-patterns: `mem:testing`. * Cross-cutting testing principles and anti-patterns: `mem:testing`.
## OAuth for remote MCP clients
- Authorization/discovery URLs, self-contained client IDs, consent and PKCE/code binding, backend Redis state, deployment wiring, and compatibility limits: `mem:mcp/oauth`.
## Host and Origin allowlist (`RequestGuard`)
- `RequestGuard` checks `Host` and `Origin` (hostname only, port ignored) on `/mcp`, the REPL (Express middleware, 403) and the plugin WebSocket (close `1008` before any message is read). It stops web pages on the user's machine from driving a local server (DNS rebinding, cross-site WebSocket).
- Allowlist: `PENPOT_MCP_ALLOWED_HOSTS` (`*` turns the check off); unset → loopback only in single-user mode (local or remote: no other barrier, `execute_code` still runs in the user's session), any host in multi-user mode (relies on user tokens; behind nginx `/mcp/ws` the server sees `Host: penpot-mcp:4402`). A missing `Origin` passes (non-browser clients).
- Entries must be `*`, a hostname or a bracketed IPv6 address; one with a scheme, port or path makes `fromEnvironment` throw, so the server does not start. Startup logs a `warn` when the check is off in single-user mode.
- The plugin iframe has `allow-same-origin`, so its WebSocket carries the real plugin origin (`http://localhost:4400` locally, the Penpot origin when integrated).
- Test mocks of `PenpotMcpServer` passed to `PluginBridge` must provide `requestGuard`.
## Backend tools (`submit_feedback`)
- Tools that call the Penpot backend directly (not through the plugin) register only when `PENPOT_MCP_BACKEND_URI` is nonblank (`PenpotMcpServer.initTools`). Currently only `SubmitFeedbackTool`.
- `PenpotBackendClient` owns the HTTP call and the status-to-message mapping; it posts to `api/main/methods/send-mcp-feedback` with `Authorization: Token <MCP user token>`. The token comes from the session context (OAuth bearer or `?userToken=`); without one the tool fails before calling the backend.
- Tool zod schemas mirror the backend malli schema (lengths, blank, control characters); keep both in sync. The backend is the authority.
- Success means the mail job was queued, not delivered; the tool text says so. `formatArgs` hides the feedback from logs.
- Backend side: `send-mcp-feedback` in `app.rpc.commands.feedback`, configured by `PENPOT_MCP_FEEDBACK_EMAIL`; MCP-token auth rules in `mem:backend/auth-permissions-product-domains`.
- User docs: `mcp/README.md` (env tables), `docs/mcp/index.md` (tool list), `docs/technical-guide/configuration.md` (MCP feedback section).
## Devenv plugin/server wiring ## Devenv plugin/server wiring
In the normal Penpot devenv MCP path, the browser plugin does not discover or route through Postgres. The frontend provides the plugin extension API with `mcp.getServerUrl()`, currently derived from `frontend/src/app/config.cljs` as `penpotMcpServerURI` if set, otherwise `<public-uri>/mcp/ws`. The MCP plugin opens a direct WebSocket to that URL and appends the current MCP access token as a query parameter. In the normal Penpot devenv MCP path, the browser plugin does not discover or route through Postgres. The frontend provides the plugin extension API with `mcp.getServerUrl()`, currently derived from `frontend/src/app/config.cljs` as `penpotMcpServerURI` if set, otherwise `<public-uri>/mcp/ws`. The MCP plugin opens a direct WebSocket to that URL and appends the current MCP access token as a query parameter.
@ -113,5 +148,11 @@ For parallel devenvs, prefer same-origin MCP routing: each Penpot instance shoul
## Plugin reconnect policy ## Plugin reconnect policy
- The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS. - The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS.
- The MCP server emits `1008` for a duplicate connection on the same user token (`PluginBridge`) and for a missing `userToken` in multi-user mode. - The MCP server emits `1008` for a `Host`/`Origin` outside the `RequestGuard` allowlist, for a missing `userToken` in multi-user mode, for missing/invalid connection metadata, and to close a connection displaced by a newer one with the same user and session ID.
- A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via "Connect here". - Session IDs are deterministic per tab and file, so a reconnect reuses its ID while the old socket may be half-open: the newest connection wins (`PluginBridge.displaceConnection`). Across instances, each new connection publishes a session claim on a per-user Redis channel; an instance displaces its local connection only on a foreign claim that arrives after its own claim (channel order, no clocks). Discovery deduplicates by session ID.
- A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via the MCP menu.
## Rollback warnings
- `ExecuteCodeTaskHandler` awaits `penpot.waitForSave()` after the code and outside its swallow catch, so a save failure, save timeout or rollback during the wait fails that `execute_code` call itself. It hard-fails as "missing Plugin API support" when `penpot.waitForSave` is absent (old Penpot or stale runtime build, or broken `api.cljs` wiring). Both waits run only when `penpotMgmt.workspace.status` is `"ready"` (or there is no `penpotMgmt`), checked at each wait: dashboard calls skip them, because persistence `:error` outlives the workspace and would fail successful `penpotMgmt` calls. Budget: layout wait 30s before the code + save wait 60s after it, under `PENPOT_MCP_TOOL_TIMEOUT_S` (120s). `RollbackNotices` below only cover rollbacks that land after the call returned (concurrent edits, a response after a timeout).
- The plugin's `RollbackLog` (`packages/plugin/src/RollbackLog.ts`) holds the `contentrollback` events not yet sent with a task response; `Task.sendResponse` attaches them and acknowledges them once sent, and the server's `unclaimed-rollbacks` message (a response that came after its task timed out) puts them back. With a `RollbackStore` (`LocalStorageRollbackStore` over `penpot.localStorage`, one entry per file) every change is written through, so a reload of the tab or plugin keeps them; storage failures fall back to memory. Plugin tests run with `node --experimental-strip-types`, which rejects TypeScript parameter properties: declare fields explicitly.

View File

@ -0,0 +1,47 @@
# MCP OAuth
## Contract and scope
- MCP-specific authorization; existing Penpot login identifies the user on the approval page. No application vetting or persistent consent policy. Access requires user approval; registration alone grants nothing.
- Return the existing fixed MCP user token. On approval, enable `:mcp-enabled` and create the token only if no usable one exists; preserve an existing usable token. A token is unusable, and replaced, when expired (settings can give MCP tokens an expiry), when its JWT issuer is not `urn:penpot:mcp-token` (tokens issued before MCP tokens had their own issuer), or when this instance cannot decrypt it (e.g. secret key changed); the backend rejects all three. No refresh token, `expires_in`, or new token lifetime.
- No new client database table or schema migration. Client IDs must remain usable across separate authorization flows without server-side registration storage. Do not replace them with expiring Redis registrations.
- MCP transport continues matching user tokens to connected Penpot plugins; no new per-request token introspection or independent semantic validation. The logged-in Penpot instance remains the content-access boundary. Do not silently add resource-server validation or change token semantics.
- README scope: user/developer usage instructions; document `PENPOT_MCP_AUTH_URI` in its environment-variable table. Keep OAuth architecture and protocol details here and non-obvious invariants in source comments.
## Deployment and discovery
- MCP process: `PENPOT_MCP_AUTH_URI=https://penpot.example/api/mcp/oauth` with `--multi-user`. Unset/empty disables OAuth; configured OAuth requires multi-user. Absolute HTTPS, or HTTP on loopback for development; no userinfo, query, or fragment. Invalid configuration fails startup; startup logs OAuth status.
- Devenv: `mcp/scripts/start-mcp-devenv-oauth` starts multi-user mode with `PENPOT_MCP_AUTH_URI` derived from `PENPOT_PUBLIC_URI`. Manual opt-in only: agentic tooling needs the single-user `start-mcp-devenv`.
- Public MCP transport: `<Penpot public URI>/mcp/stream`, proxied to Node `/mcp`. Missing credentials with OAuth enabled: 401 plus `WWW-Authenticate` referencing `<auth base>/resource` and scope `mcp`.
- Bearer header takes precedence over legacy `?userToken=`. Malformed headers do not fall back to the query token. Existing query-token clients remain supported.
- Protected-resource metadata: `/api/mcp/oauth/resource`; issuer: `/api/mcp/oauth`; authorization-server metadata: `/.well-known/oauth-authorization-server/api/mcp/oauth`. All reach the backend. Bundled devenv/production nginx configs forward the well-known path; custom proxies must do likewise. Keep Penpot public URI and external URLs consistent.
- Node implementation: `mcp/packages/server/src/McpAuthentication.ts`, integrated by `PenpotMcpServer.ts`; OAuth-enabled tools advertise `_meta.securitySchemes`.
## Client identification
- `backend/src/app/mcp/oauth_clients.clj`: dynamic registration at `/api/mcp/oauth/register`, HTTPS client metadata documents (CIMD), and registrations obtained in advance via `(app.mcp.oauth-clients/register! {} metadata)`.
- DCR returns `penpot-mcp-v1.` plus unpadded base64url JSON: normalized callback URLs, optional name, authentication method, random nonce, optional secret hash. Fixed grant/response/scope values are reconstructed. Nonce distinguishes registrations with otherwise identical metadata.
- IDs are public, unsigned, unencrypted, non-expiring metadata; validate on every resolution. Client secrets are generated randomly and returned separately; only the hash enters the ID. No signing/encryption key or client record in PostgreSQL/Redis. Old experimental database-backed UUID IDs require registration again.
- An edited payload changes the ID. Authorization-code exchange compares the exact ID string against the approved request before resolving metadata. Do not replace this with comparison of decoded fields; code ownership and PKCE binding must survive metadata edits.
- Supported authentication: `none`, `client_secret_basic`, `client_secret_post`; no `private_key_jwt`. CIMD selects public-client `none` from the document's supported methods; other listed methods do not imply support here. Scope `mcp`; grant `authorization_code`; response `code`. CIMD may advertise additional grants (e.g. Codex lists `refresh_token`); require `authorization_code` support and normalize to the grant Penpot implements, rather than reject broader client capabilities.
- Entire ID capped at 2,048 ASCII characters. DCR returns `invalid_client_metadata` if encoded metadata exceeds this; shorter/fewer callback URLs reduce size. OAuth defines no fixed ID-length maximum (RFC 6749 §2.2); stateless IDs are explicitly described by RFC 7591 §A.5.2. Installed MCP client SDK 2.3.1 accepts a 2,048-character ID; this does not establish Codex/Claude compatibility.
- Redirect matching is exact except loopback HTTP port changes for native clients (`127.0.0.1`, `[::1]`, exact `localhost`). Host, path and query must still match; reject fragments/userinfo and lookalike localhost domains. Allow HTTPS, loopback HTTP, and reverse-domain native schemes.
- CIMD: fetch with existing SSRF-checked `app.http.client/req`, no redirect following, require matching document `client_id`, bounded 64-KiB body and request/body-read timeouts. Cache max 256 entries/one minute, further limited by HTTP max-age; no-store/no-cache/no max-age means no reuse.
## Approval and exchange
- `backend/src/app/http/mcp_oauth.clj`: separate router branch from ordinary RPC middleware; preserve OAuth snake_case JSON and form encoding. Session auth applies to approval request/decision only. Public metadata/register/token endpoints allow noncredentialed CORS; MCP transport CORS was not added.
- Errors: every failure answers with an OAuth `{"error": ...}` body, never the shared Penpot error shape. Failures without `:oauth-error` are also passed to `app.http.errors/handle` for logging only; its response is discarded. `wrap-errors` also wraps the session middleware of `/request` and `/decision`. Safe to log because these routes never set request `:params`.
- `/authorize`: validate client/callback, `code`, PKCE S256, scope/resource/state; store pending request for 600 seconds; redirect to `?screen=mcp-authorize&request=<opaque handle>`.
- `frontend/src/app/main/ui/mcp_authorize.cljs`: existing login with return URL; show application and account details; submit explicit Allow/Cancel. No fixed MCP token in frontend state or browser redirect. Approval refreshes profile and broadcasts MCP enablement; does not connect a plugin automatically.
- `/request`: first authenticated read binds request to profile plus hashed login-session ID and CSRF value, using compare-and-replace. Re-read after the attempt: concurrent GETs may win/lose binding; only the matching session may proceed.
- `/decision`: check exact Origin, session binding and CSRF, then atomically consume the request. Denial creates no token/enables nothing. Approval calls transactional `access-token/ensure-mcp-access!`, then stores code for 120 seconds. Code state holds token row ID, never the fixed token. Callback echoes OAuth `state` and RFC 9207 `iss`.
- `/token`: validate grant, exact client ID/callback/resource and PKCE; resolve current client metadata/authentication method and verify any secret; atomically consume validated code; fetch the unexpired token by original row ID/profile/type. Regeneration deletes the old token row, so a pending code cannot silently yield the replacement token. Return fixed `access_token`, `token_type: Bearer`, scope `mcp`.
- `backend/src/app/mcp/oauth_state.clj` and Lua: tenant/kind/hash-prefixed keys; random handles; SET NX EX must succeed before returning handle. Reads retain raw JSON for atomic comparison. Consume/delete and bind/replace compare the validated raw record in Lua; replacement preserves TTL. Redis failures fail closed. Backend Redis is required independently of Node's optional Redis plugin bridge.
- `backend/src/app/rpc/commands/access_token.clj`: `ensure-mcp-access!` decodes the stored token with `ex/ignoring` to apply the usable-token rule above. Lock profile row in both MCP generation and OAuth provisioning, including when no token exists. Serializes simultaneous approvals and settings regeneration. Provisioning must run within a transaction.
## Verification and limits
- Tests: `mcp_oauth_clients_test.clj`, `mcp_oauth_http_test.clj`, `mcp_oauth_state_test.clj`, OAuth provisioning in `rpc_access_tokens_test.clj`; MCP authentication/HTTP tests; frontend approval tests. Cover no-storage registration, size boundaries, edited-ID rejection, code replay/concurrency, session/CSRF binding, and fixed-token reuse.
- Live OpenAI/Codex/Claude compatibility and marketplace acceptance remain unverified. In particular, verify token persistence without `expires_in` and acceptance of the existing matching-only resource boundary; do not claim protocol conformance from SDK tests alone.
- Protocol references: https://www.rfc-editor.org/rfc/rfc6749.html#section-2.2 ; https://www.rfc-editor.org/rfc/rfc7591.html#appendix-A.5.2 ; https://www.rfc-editor.org/rfc/rfc7636.html ; https://www.rfc-editor.org/rfc/rfc9207.html .

View File

@ -26,6 +26,7 @@ No manual line wrapping: one line per bullet or paragraph, however long. Memorie
## Add/update threshold ## Add/update threshold
Add or update memories only with stable, non-obvious project conventions that avoid complex rediscovery in the future. Add or update memories only with stable, non-obvious project conventions that avoid complex rediscovery in the future.
Do not restate what the code already says (comments included): keep only what code cannot represent — decisions and their reasons, cross-file invariants, ordering constraints, environment and tool behavior, and traps that break builds.
Do not add: quick-read facts; generic language/framework knowledge; one-off task notes; volatile line-level details; behavior likely to change soon. Do not add: quick-read facts; generic language/framework knowledge; one-off task notes; volatile line-level details; behavior likely to change soon.
## Maintenance Actions ## Maintenance Actions

View File

@ -18,6 +18,7 @@
## Sandbox and global cleanup ## Sandbox and global cleanup
- `src/index.ts` calls `bootstrapSES()` (in `src/lib/bootstrap.ts`) at module load. If `repairIntrinsics` throws (e.g. SES 2.2+ rejects a `Temporal` wrapped by a browser extension), the error is caught, `getPluginsRuntimeError()` exposes it, and `initPluginsRuntime` installs no `ɵ*` globals: the frontend app still boots, plugins and MCP are unavailable in that environment. On failure, `Function.prototype.toString` has already been replaced but no intrinsics are hardened and no compartment/sandbox is created — never run plugins on that path.
- The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code. - The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code.
- Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json. - Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json.
- Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them. - Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them.

View File

@ -4,32 +4,36 @@ Backend (`app.config`, `PENPOT_*` env vars) is parameterized; deployments choose
## Services ## Services
- **PostgreSQL**: durable store. Profiles, teams, files, sessions, audit, `storage_object` metadata, the `task` queue, `scheduled_task` cron registry, migrations. File-data also lives here when the file-data backend is `legacy-db`/`db`. One shared DB across all backends. - **PostgreSQL**: durable store. Profiles, teams, files, sessions, audit, `storage_object` metadata, the `job` work queue (plus the dormant legacy `task` table), `scheduled_task` cron registry, migrations. File-data also lives here when the file-data backend is `legacy-db`/`db`. One shared DB across all backends.
- **Redis (Valkey-compatible)**: per-backend message bus and cache. Concrete uses: msgbus Pub/Sub for collaborative-editing broadcasts and team/profile-org notifications fired by RPC handlers (`app.rpc.notifications`, `files_update`, `teams`, `websocket`); file-summary cache gated by `enable-redis-cache`; rate-limit counters; and the dispatcher→runner work hand-off list `penpot.worker.queue:<tenant>:<queue>`. `PENPOT_REDIS_URI`. A burst of client-side `RedisCommandTimeoutException` (2s on `DEL`/`SET`, 10s on `BLPOP`) with the exporter healthy is the JVM paused or starved, not Redis: the timeout fires because nothing reads the reply. - **Redis (Valkey-compatible)**: per-backend message bus and cache. Concrete uses: msgbus Pub/Sub for collaborative-editing broadcasts and team/profile-org notifications fired by RPC handlers (`app.rpc.notifications`, `files_update`, `teams`, `websocket`); file-summary cache gated by `enable-redis-cache`; rate-limit counters; and the dispatcher→runner work hand-off list `penpot.worker.queue:<tenant>:<queue>`. `PENPOT_REDIS_URI`. A burst of client-side `RedisCommandTimeoutException` (2s on `DEL`/`SET`, 10s on `BLPOP`) with the exporter healthy is the JVM paused or starved, not Redis: the timeout fires because nothing reads the reply.
- **Object storage**: backends `:s3` and `:fs`. S3 in prod; devenv uses RustFS. Holds uploaded media, file-data when the file-data backend is `storage`, exports. Backend-side details (resolve, dedup, bucket set, object lifecycle, and file-data backends): `mem:backend/storage`. - **Object storage**: backends `:s3` and `:fs`. S3 in prod; devenv uses RustFS. Holds uploaded media, file-data when the file-data backend is `storage`, exports. Backend-side details (resolve, dedup, bucket set, object lifecycle, and file-data backends): `mem:backend/storage`.
- **SMTP mailer**: invitations, password resets, email verification (sent via the `:sendmail` worker task). - **SMTP mailer**: invitations, password resets, email verification (sent via the `:sendmail` worker job).
- **LDAP** (optional auth provider): helpers in `app.auth.*`, gated by `enable-login-with-ldap`. - **LDAP** (optional auth provider): helpers in `app.auth.*`, gated by `enable-login-with-ldap`.
## Task queue and worker model ## Task queue and worker model
Async tasks are enqueued via `wrk/submit!` (`app.worker`), which inserts a row into the shared Postgres `task` table tagged with `queue = "<tenant>:<queue-name>"`. Submission is **fire-and-forget** — RPC handlers never poll, never wait, and workers never publish to msgbus. The only completion signal is the `task` row's `status` / `completed_at` columns, which nothing in `rpc/` reads. Soft-delete RPCs return immediately after marking the top-level row, leaving the cascade and reaping to workers. Async jobs are enqueued via `jobs/submit` (`app.jobs`), which inserts a row into the shared Postgres `job` table tagged with `queue = "<tenant>:<queue-name>"`. Submission is **fire-and-forget** — RPC handlers never poll and never wait. The completion signal is the `job` row's `status` / `completed_at` columns, which the creation commands read back to answer with the state of the job they just created. A job submitted with `::jobs/profile-id` also publishes a `:job-event` message on that profile's msgbus topic after each transaction commits, and the binfile commands (`create-export-binfile-job`, `create-import-binfile-job`) are the callers that pass that owner, so the progress of their jobs does reach the msgbus. Soft-delete RPCs return immediately after marking the top-level row, leaving the cascade and reaping to workers.
Workers run on backends with `enable-backend-worker` in `PENPOT_FLAGS`. Each worker-enabled backend has a `dispatcher` (polls `task` with `FOR UPDATE SKIP LOCKED`, marks status='scheduled', RPUSHes claimed task IDs into **its own** Redis list) and one or more `runner`s per queue (BLPOP from that same local list, execute, update the Postgres row). The Redis hand-off list is purely intra-backend — cross-backend coordination happens at the Postgres row level. Workers run on backends with `enable-backend-worker` in `PENPOT_FLAGS`. Each worker-enabled backend has a `dispatcher` (polls `job` with `FOR UPDATE SKIP LOCKED`, marks status='scheduled', RPUSHes the JSON `[id scheduled-at]` payloads into **its own** Redis list) and one or more `runner`s per queue (BLPOP from that same local list, execute, update the Postgres row). The Redis hand-off list is purely intra-backend — cross-backend coordination happens at the Postgres row level. The queues are `default`, `webhooks`, `cron` and `binfile`: the last one carries the heavy work of a user job (`:worker-binfile-parallelism`, default 1), so a long export or import does not block the default runner.
## Cross-backend safety ## Cross-backend safety
Postgres row locking is the only correctness primitive: `task` claims via `FOR UPDATE SKIP LOCKED`, cron firing via `FOR UPDATE SKIP LOCKED` on the `scheduled_task` row, plus task-handler-internal locks (e.g. `file_gc_scheduler` locks candidate file rows). This makes the work-claim path safe across any number of worker-enabled backends. Postgres row locking is the only correctness primitive: `job` claims via `FOR UPDATE SKIP LOCKED`, cron firing via `FOR UPDATE SKIP LOCKED` on the `scheduled_task` row, plus job-handler-internal locks (e.g. `file_gc_scheduler` locks candidate file rows). This makes the work-claim path safe across any number of worker-enabled backends.
Two known race patterns survive multi-backend operation: Two known race patterns survive multi-backend operation:
- **Cron dedup is best-effort.** The lock on `scheduled_task` is released when the task body finishes. If two backends' cron timers fire for the same scheduled instant with a gap larger than the task body's runtime, both execute it. Penpot's cron entries are idempotent (`session-gc`, `objects-gc`, `storage-gc-*`, `tasks-gc`, `upload-session-gc`, `file-gc-scheduler`); the exceptions are `:telemetry` (would double-report) and `:audit-log-archive` (depends on archive target idempotency). - **Cron dedup is best-effort.** The lock on `scheduled_task` is released when the task body finishes. If two backends' cron timers fire for the same scheduled instant with a gap larger than the task body's runtime, both execute it. Penpot's cron entries are idempotent (`session-gc`, `objects-gc`, `storage-gc-*`, `storage-pending-gc`, `jobs-gc`, `file-gc-scheduler`); the exceptions are `:telemetry` (would double-report) and `:audit-log-consumer-nexus` / `:audit-log-consumer-admin-console` (depend on archive target idempotency).
- **`wrk/submit! ::dedupe true`** does a non-atomic `DELETE` then `INSERT`. Concurrent cross-backend submits can both bypass the `DELETE` (each sees the other's uncommitted insert as absent) and end up with duplicate `'new'` rows. Each row claims and runs once independently, so the underlying work is fine; the "at most one pending" guarantee weakens. - **`jobs/submit ::dedupe true`** does a non-atomic `DELETE` then `INSERT`. Concurrent cross-backend submits can both bypass the `DELETE` (each sees the other's uncommitted insert as absent) and end up with duplicate `'new'` rows. Each row claims and runs once independently, so the underlying work is fine; the "at most one pending" guarantee weakens.
Penpot in production lives with both: horizontal-scale deployments accept "exactly-once" as "essentially-once for idempotent operations." Devenv parallel instances handle it by running workers only on ws0 (see `mem:devenv/core`). Penpot in production lives with both: horizontal-scale deployments accept "exactly-once" as "essentially-once for idempotent operations." Devenv parallel instances handle it by running workers only on ws0 (see `mem:devenv/core`).
## Jobs observability
Worker-enabled, writable backends publish unified jobs metrics through the normal Prometheus registry. Counters describe submit, dispatch, terminal outcomes, retries, orphan recovery, rescheduling and total stored job events (unlabelled); histograms separate queue wait, handler execution and total job age. SQL-backed events are recorded after the outer PostgreSQL transaction commits. The `::jobs-metrics/sampler` component publishes backlog by status and the oldest pending-job age every 30 seconds. All labels are bounded operational values; job IDs, profile IDs, params and error text are excluded. Read-only backends do not start the sampler or the workers.
## Draining one instance ## Draining one instance
Each backend has a per-process availability flag (`app.health`, a global atom). While offline, `/readyz` answers `503 OFFLINE` without querying the database, so the load balancer removes that instance from rotation while requests in flight still finish. Operator surface: `app.srepl.main/set-offline` / `set-online`; a restart comes back online. In-flight worker tasks are safe: tasks claimed as `scheduled` return to `new` after 5 minutes via `sql:reschedule-lost` and another instance picks them up. Each backend has a per-process availability flag (`app.health`, a global atom). While offline, `/readyz` answers `503 OFFLINE` without querying the database, so the load balancer removes that instance from rotation while requests in flight still finish. Operator surface: `app.srepl.main/set-offline` / `set-online`; a restart comes back online. In-flight worker jobs are safe: jobs claimed as `scheduled` return to `new` after 5 minutes via `sql:reschedule-lost` and another instance picks them up.
## Touching production data ## Touching production data
@ -39,12 +43,17 @@ Production runs the deployed toolbox, not the devenv one:
- **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups. - **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups.
- **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them. - **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them.
- The read-only replica cuts long queries: see `mem:prod-infra/grafana`. - The read-only replica cuts long queries: see `mem:prod-infra/grafana`.
- A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table (file saves write to `file_change` on every save) build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block); the shipped migration uses `IF NOT EXISTS`, finds it and does nothing. A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_indexes` and `pg_index.indisvalid`). - A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block). A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_index` and `pg_index.indisvalid`).
- A partitioned parent cannot take `CREATE INDEX CONCURRENTLY` (`cannot create index on partitioned table ... concurrently`). Build one index per partition with `CONCURRENTLY`, then `CREATE INDEX ... ON ONLY` on the parent and `ALTER INDEX ... ATTACH PARTITION` for each child; the parent index only becomes valid when every partition is attached.
- Write the migration as plain, idempotent SQL: `ADD COLUMN IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DROP CONSTRAINT IF EXISTS` followed by `ADD CONSTRAINT`. Do not encode shape checks as `pg_catalog` introspection: they are easy to get wrong (a missing `NOT condeferred` accepts an `INITIALLY DEFERRED` foreign key) and they duplicate what the runbook verifies by hand. `IF NOT EXISTS` matches the name, not the definition, so the out-of-band procedure verifies the index shape and completeness.
- The runbook is the contract for a large instance: it builds and verifies the out-of-band work and then marks the migration step as applied (`INSERT INTO migrations (module, step) ...`), so the body does not run at startup. Verify the column, the pending rows, the index (valid, no predicate or expression, every partition attached) and the constraint shape before inserting the row; the marker is the only thing that skips the migration.
- A reference that a GC task resolves must be `DEFERRABLE INITIALLY IMMEDIATE`, never `INITIALLY DEFERRED`: the immediate check fires inside the transaction that catches the violation and retries the chunk; the deferred one fires at commit, outside the catch. A validated foreign key belongs out of band: `ADD FOREIGN KEY` runs as an `ALTER TABLE` and it scans the whole referencing table; `NOT VALID` on a partitioned table only exists since PostgreSQL 18.
- A backfill that removes a key an old reader uses (for example `metadata.storage-ref-id`, which the previous storage GC scans) must run with the old code stopped. Otherwise the old reader loses the reference and can delete a live object, and the rows it writes in the meantime miss the backfill.
- `ADD COLUMN` takes a brief `ACCESS EXCLUSIVE` even when `IF NOT EXISTS` skips it, and `ADD FOREIGN KEY` takes `ACCESS EXCLUSIVE` on the referencing table and `SHARE ROW EXCLUSIVE` on the referenced one. A migration accepts those locks; a deployment that cannot take them runs the runbook and marks the step as applied.
## See also ## See also
- Devenv composition and the ws0-only worker placement: `mem:devenv/core`. - Devenv composition and the ws0-only worker placement: `mem:devenv/core`.
- Storage backend resolution, dedup, bucket behavior, object lifecycle, and file-data lifecycle: `mem:backend/storage`. - Storage backend resolution, dedup, bucket behavior, object lifecycle, and file-data lifecycle: `mem:backend/storage`.
- Storage operation metrics (S3 API calls, logical ops, dedup, asset requests) and the logical-bucket vs physical-target distinction: `mem:backend/storage` (Metrics section). - Storage operation metrics (S3 API calls, logical ops, dedup, asset requests) and the logical-bucket vs physical-target distinction: `mem:backend/storage` (Metrics section).
- Storage operation metrics (S3 API calls, logical ops, dedup, asset requests) and the logical-bucket vs physical-target distinction: `mem:backend/storage` (Metrics section).
- Grafana (datasource ids and how to use them, main dashboard, panel conventions and gotchas): `mem:prod-infra/grafana`. - Grafana (datasource ids and how to use them, main dashboard, panel conventions and gotchas): `mem:prod-infra/grafana`.

View File

@ -36,6 +36,7 @@ From `render-wasm/`:
- Rust tests: `./test` or `cargo test <name>`. - Rust tests: `./test` or `cargo test <name>`.
- Cross-cutting testing principles and anti-patterns: `mem:testing`. - Cross-cutting testing principles and anti-patterns: `mem:testing`.
- Lint: `./lint`. - Lint: `./lint`.
- Format check: `cargo fmt --check`. - Format: `./fmt`; check only: `./fmt --check`.
- `./build`, `./test`, `./lint` and `./fmt` all run `setup`, which selects the Rust version pinned in `_build_env` (skip with `PENPOT_WASM_PREPARED=1`).
Do not change exported WASM function signatures without updating the corresponding frontend bridge and verifying the frontend renderer path. Do not change exported WASM function signatures without updating the corresponding frontend bridge and verifying the frontend renderer path.

View File

@ -1,12 +1,15 @@
# Changelog helper # Changelog helper
`scripts/changelog.py` holds the checks used by the `update-changelog` skill (extracted from the skill file so the skill stays workflow-only). Calls `scripts/gh.py` via subprocess; never call the GitHub API directly from this script. `scripts/changelog.py` holds the checks used by the changelog flow. The rules and the step-by-step live in `mem:workflow/updating-changelog`; this memory documents the subcommands. Calls `scripts/gh.py` via subprocess; never call the GitHub API directly from this script.
## Subcommands ## Subcommands
- `check-merged <PR...>` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries. - `check-merged <PR...>` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries.
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings; GHSA entries never appear here by design. - `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report (milestone mismatches type A/B plus `:rocket:` gaps C/D); rule violations never belong in the report, they are fixed in CHANGES.md first. - `security <MILESTONE> [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix <description> (<advisory URL>) (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default.
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design.
- `dev-only-bugs <MILESTONE> [--apply]` — list closed milestone bugs that never reached production, without `no changelog` (bugs carrying `release blocker` are analyzed too). Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or the milestone version itself (`only_unreleased_envs`; any production hint or other version keeps the bug in — versions compare as full semver with missing patch = 0, so for milestone 2.19.2 the released 2.19.0 keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
## Conventions ## Conventions
@ -15,4 +18,5 @@
- Version-section lookup accepts an optional `(suffix)` after the version header (e.g. `(Unreleased)`); a missing section is an error for `cross-ref` and an empty section for `report`. - Version-section lookup accepts an optional `(suffix)` after the version header (e.g. `(Unreleased)`); a missing section is an error for `cross-ref` and an empty section for `report`.
- PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms. - PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms.
- Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes. - Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes.
- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry); C/D are `:rocket:` gaps (warnings, never counted). - Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry), G `breaking change` issue missing from `:boom:` (counted, same contract as F); C/D are `:rocket:` gaps (warnings, never counted).
- `issue_excluded` (closed, non-Task, non-Rejected, no `no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift.

View File

@ -1,8 +1,15 @@
# CI (scripts/ci) # CI (scripts/ci)
`scripts/ci` runs CI-style checks — lint, tests, format — for one or more `scripts/ci` runs CI-style checks — lint, tests, format — and, on request,
monorepo modules and prints a per-task summary. It is the local equivalent the production build of each module, for one or more monorepo modules, and
of CI; use it to verify changes before declaring work done. prints a per-task summary. It is the local equivalent of CI; use it to
verify changes before declaring work done.
It also exports `PENPOT_FLAGS` with `enable-backend-asserts` appended (keeping
whatever the caller already set), which is what puts `clojure.core/assert`
into the compiled test code. Prefer it over calling `clojure -M:dev:test` by
hand for that reason alone: without the flag the suite passes while every
assertion is compiled out. GitHub Actions sets the same flag.
## When to use ## When to use
@ -19,6 +26,7 @@ Run from the repo root:
```bash ```bash
./scripts/ci MODULE... # lint + test + fmt per module ./scripts/ci MODULE... # lint + test + fmt per module
./scripts/ci --build MODULE # production build only
./scripts/ci --all --no-test # lint + fmt on all modules ./scripts/ci --all --no-test # lint + fmt on all modules
./scripts/ci --lint frontend # lint only ./scripts/ci --lint frontend # lint only
./scripts/ci --fix --no-test frontend # format files, skip tests ./scripts/ci --fix --no-test frontend # format files, skip tests
@ -26,14 +34,15 @@ Run from the repo root:
./scripts/ci --dry-run --all # preview what would run ./scripts/ci --dry-run --all # preview what would run
``` ```
Modules: `frontend backend common render-wasm exporter mcp plugins library`. Modules: `frontend backend common render-wasm exporter mcp plugins library
media-processor`.
Flags: Flags:
- Default tasks: `lint`, `test`, `fmt` (format check; `--fix` formats - Default tasks: `lint`, `test`, `fmt` (format check; `--fix` formats
instead). instead). Build is **not** in the default set.
- `--lint` / `--test` / `--fmt` run one task only; `--no-lint` / - `--lint` / `--test` / `--fmt` / `--build` run one task only; `--no-lint` /
`--no-test` / `--no-fmt` drop one task from the default set. `--no-test` / `--no-fmt` / `--no-build` drop one task from the default set.
- `--paren-repair` runs only the delimiter repair — it wraps - `--paren-repair` runs only the delimiter repair — it wraps
`scripts/paren-repair` over each module's Clojure/CLJS sources; see `scripts/paren-repair` over each module's Clojure/CLJS sources; see
`mem:scripts/paren-repair`. `mem:scripts/paren-repair`.
@ -42,6 +51,49 @@ Flags:
output; `--dry-run` prints commands without running; `--clean` removes output; `--dry-run` prints commands without running; `--clean` removes
the log directory. the log directory.
## Build task
`--build` compiles each module the way production does, reusing caches
(`.shadow-cljs`, `target/`, `node_modules`) instead of clearing them. It
does not package bundles: `manage.sh build-bundle` remains the way to
produce deployable artifacts.
| Module | Command | Output |
|---|---|---|
| `frontend` | `NODE_ENV=production pnpm run build:app:main && pnpm run build:app:libs && pnpm run build:app:assets` | `frontend/resources/public/`, `frontend/target/` |
| `backend` | `clojure -T:build jar` | `backend/target/penpot.jar` |
| `common` | *(none; skipped)* | — |
| `render-wasm` | `./build` (targets `frontend` and `export`) | `render-wasm/target/` |
| `exporter` | `NODE_ENV=production pnpm run build:app` | `exporter/target/` |
| `mcp` | `pnpm run build` | `mcp/*/dist` |
| `plugins` | `pnpm run build:runtime && pnpm run build:plugins && pnpm run build:styles-example && pnpm run build:doc` | `plugins/dist/` |
| `library` | `NODE_ENV=production clojure -M:dev:shadow-cljs release library` | `library/target/library/` |
| `media-processor` | `pnpm run types:check && pnpm run build` | `media-processor/dist/` |
- Every output path above is gitignored, so a build leaves `git status`
clean.
- The module `build` scripts in each `scripts/build` are **not** used: they
delete `node_modules`, wipe `resources/public` and run `pnpm install`,
which belongs to the bundle pipeline, not to a local check.
- `frontend` and `library` have a `build` script that clears the shadow-cljs
cache first; the task calls the inner commands to stay incremental.
`exporter` uses `build:app` for the same reason.
- `common` has no artifact of its own: its compilation is exercised by its
`test` task and by the builds of `frontend`, `backend` and `exporter`.
- `media-processor` is the ninth module. It has no lint, so that task shows
as skipped; its build runs `tsc --noEmit` before esbuild because esbuild
does not type-check.
- Cold-cache cost measured on the devenv: `library` ~120 s, `frontend` ~120 s,
`exporter` ~80 s, `backend` ~35 s. With warm caches the whole set (8
builds) takes ~4 min. The first run of `clojure -T:build jar` downloads
`tools.build` if it is not in `~/.m2`.
- `pnpm run` may trigger a dependency check/install when `node_modules`
drifts from the lockfile (`verify-deps-before-run`), so the first run
after a dependency change can need network.
- `render-wasm/build` takes a lock (`_build_lock`): never run two builds at
once in the same checkout.
## Logs and exit codes ## Logs and exit codes
- Full output of every task: `.ci-logs/<module>-<task>.log`. - Full output of every task: `.ci-logs/<module>-<task>.log`.
@ -53,12 +105,16 @@ Flags:
## Notes ## Notes
- `mcp` has no lint task (shows as skipped). `render-wasm` uses `./lint`, - `mcp` has no lint task and `media-processor` has no lint task; both show
`./test`, and `cargo fmt`. as skipped. `render-wasm` uses `./lint`, `./test`, and `./fmt`.
- `scripts/test_ci.py` (unittest, `python3 scripts/test_ci.py`) covers the
flag wiring, one build command per module and the help text through
`--dry-run`; it executes no real build.
- Exports `PENPOT_FLAGS` with `enable-backend-asserts` appended to any - Exports `PENPOT_FLAGS` with `enable-backend-asserts` appended to any
existing value, so backend test runs compile in and exercise existing value, so backend test runs compile in and exercise
precondition asserts; see `mem:backend/testing`. precondition asserts; see `mem:backend/testing`.
- Test tasks are long-running (backend: `clojure -M:dev:test`); use a - Test and build tasks are long-running (backend: `clojure -M:dev:test`;
generous timeout when calling it from an agent shell. frontend build: ~2 min warm); use a generous timeout when calling it from
an agent shell.
- Skill entry point: `.agents/skills/local-ci/SKILL.md`. - Skill entry point: `.agents/skills/local-ci/SKILL.md`.
- Testing principles and output discipline: `mem:testing`. - Testing principles and output discipline: `mem:testing`.

View File

@ -11,6 +11,7 @@ repository via GraphQL and REST APIs through the authenticated `gh` CLI.
- Comparing milestone issues against CHANGES.md to find missing entries. - Comparing milestone issues against CHANGES.md to find missing entries.
- Explicitly linking a GitHub issue to a pull request. - Explicitly linking a GitHub issue to a pull request.
- Listing or inspecting GitHub Security Advisories (GHSA). - Listing or inspecting GitHub Security Advisories (GHSA).
- Fetching issue comments (the `## Breaking change note` markdown for the changelog).
## Prerequisites ## Prerequisites
@ -39,7 +40,7 @@ python3 scripts/gh.py issues "2.16.0" --label "bug"
python3 scripts/gh.py issues "2.16.0" --label "bug,regression" python3 scripts/gh.py issues "2.16.0" --label "bug,regression"
# Exclude by label # Exclude by label
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" python3 scripts/gh.py issues "2.16.0" --exclude "no changelog"
# Show only issues NOT yet in CHANGES.md # Show only issues NOT yet in CHANGES.md
python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md
@ -80,7 +81,17 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts. Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts.
**Output**: JSON array to stdout; progress to stderr. **Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both.
### `comments`
Fetch every comment of one or more issues via REST (paginated, 100 per page), oldest first. Same input styles as `issue`: numbers, `--file`, `--stdin`.
```bash
python3 scripts/gh.py comments 11235 11236
```
**Output fields per comment**: `issue`, `id`, `author`, `created_at`, `body`. JSON array to stdout; progress to stderr. `changelog.py boom` reads the `## Breaking change note` comment from this output.
### `link-issue` ### `link-issue`
@ -91,7 +102,7 @@ python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
# Short alias: python3 scripts/gh.py link <ISSUE_NUMBER> <PR_NUMBER> # Short alias: python3 scripts/gh.py link <ISSUE_NUMBER> <PR_NUMBER>
``` ```
The command resolves both node IDs and calls `addCloseIssueReferences`. It trusts the successful mutation instead of re-querying, because GitHub does not reliably report mutation-created links through `closedByPullRequestsReferences(userLinkedOnly: true)`. It is safe to rerun, works for merged PRs, and does not close an issue retroactively. JSON goes to stdout; progress and errors go to stderr; a missing issue/PR or a failed mutation exits non-zero. The command resolves both node IDs and calls `addCloseIssueReferences`. It trusts the successful mutation instead of re-querying, because GitHub does not reliably report mutation-created links through `closedByPullRequestsReferences(userLinkedOnly: true)`. It is safe to rerun and works for merged PRs. It does not guarantee the issue stays open: adding `Closes #N` to the body of an already-merged PR closed the issue immediately when tried (#12896 / PR #12067, 2026-11-09). JSON goes to stdout; progress and errors go to stderr; a missing issue/PR or a failed mutation exits non-zero.
### `advisories` ### `advisories`
@ -107,6 +118,9 @@ python3 scripts/gh.py advisories --severity critical
# Filter by state # Filter by state
python3 scripts/gh.py advisories --state triage python3 scripts/gh.py advisories --state triage
# Filter by patched version declared in the advisory
python3 scripts/gh.py advisories --patched-version 2.19.1
# Get full detail for a single advisory # Get full detail for a single advisory
python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7 python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
``` ```
@ -115,6 +129,8 @@ python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
**Detail output** (single advisory) adds: description, vulnerabilities (package, version ranges), credits, timestamps. **Detail output** (single advisory) adds: description, vulnerabilities (package, version ranges), credits, timestamps.
`--patched-version` filters the raw advisories before formatting: the match is an exact token of `vulnerabilities[*].patched_versions` (split on `,`/`;`), so `2.19.1` never matches `2.19.10`. Every state is included (`triage`, `draft`, `published`, `closed`, `withdrawn`). The raw `vulnerabilities` stay internal to the filter: the list output keeps the summary schema.
**Output**: JSON to stdout; progress to stderr. **Output**: JSON to stdout; progress to stderr.
## Key principles ## Key principles

View File

@ -73,6 +73,8 @@ a test that reproduces it:
- **One Assertion Per Concept** — each test verifies one behavior; split - **One Assertion Per Concept** — each test verifies one behavior; split
compound assertions compound assertions
- **Descriptive Test Names** — names read like specifications - **Descriptive Test Names** — names read like specifications
- **English Only** — test names, `testing` strings, comments, and every
code-facing text MUST be in English; see the hard rule in `AGENTS.md`.
## Prefer Real Implementations Over Mocks ## Prefer Real Implementations Over Mocks

View File

@ -43,10 +43,9 @@ that breaks them is wrong, even if the code is right.
- For another commit: `./scripts/check-commit -c <ref>` - For another commit: `./scripts/check-commit -c <ref>`
- **NEVER** hand-wave the body as "one long line". If a line exceeds 80, - **NEVER** hand-wave the body as "one long line". If a line exceeds 80,
break it at a space. break it at a space.
- **NEVER add a `Co-authored-by:` trailer.** It is forbidden; `scripts/check-commit` fails the commit when any line matches it (case-insensitive). - **`Co-authored-by:` is for human co-authors only** (`Co-authored-by: Name <email>`). AI providers, agents and bots must not be credited that way; use `AI-assisted-by:` instead. `scripts/check-commit` rejects a trailer value that names an AI provider or agent (`Claude`, `Copilot`, `Cursor`, `Codex`, `Gemini`, `Aider`, `Cline`, `Windsurf`, `DeepSeek`, `OpenCode`, `devin-ai`, `amazon q`, and similar) or carries a bot identity (`[bot]`, `bot@`). `NON_HUMAN_COAUTHOR_PATTERN` is the list; extend it when a new tool starts adding trailers. A person whose own name or email matches one of those tokens (a surname like `Cline`, an address at a provider) is rejected too; amend the trailer or credit them with `Reviewed-by:`.
- CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build. - CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build.
- Exceptions inside the body (do not wrap these): `Signed-off-by:`, - Exceptions inside the body (do not wrap these): `Signed-off-by:`, `Co-authored-by:`, `AI-assisted-by:` trailers, and lines carrying a URL.
`AI-assisted-by:` trailers, and lines carrying a URL.
**AI-assisted-by trailer rules:** **AI-assisted-by trailer rules:**
- Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash` - Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash`
@ -56,6 +55,14 @@ that breaks them is wrong, even if the code is right.
`:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight `:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight
## Atomic Commits
A PR squash-merges into one commit by default, so the split inside the branch
does not reach `main` and nothing depends on it. When the PR asks for a
regular merge (`mem:workflow/creating-prs` → Merge Strategy), every commit
must stand alone: it builds, passes the linter, and needs no later commit on
the same branch to make sense.
## Referencing Issues ## Referencing Issues
Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link. Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link.

View File

@ -35,11 +35,36 @@ Command what should be built. Format: `[Imperative verb] [what] in/on [where]`.
| Field | Rule | | Field | Rule |
|-------|------| |-------|------|
| **Labels** | `community contribution` (PRs from non-core) · skip workflow labels (`backport candidate`, `team-qa`) · do **not** add `bug` or `enhancement` labels (use Issue Type instead) | | **Labels** | `community contribution` (PRs from non-core) · `breaking change` when the change breaks compatibility (see below) · skip workflow labels (`backport candidate`, `team-qa`) · do **not** add `bug` or `enhancement` labels (use Issue Type instead) |
| **Milestone** | Use the current or next planned milestone. Fetch available milestones: `gh api repos/penpot/penpot/milestones --jq '.[].title'`. If unsure, omit. | | **Milestone** | Use the current or next planned milestone. Fetch available milestones: `gh api repos/penpot/penpot/milestones --jq '.[].title'`. If unsure, omit. |
| **Project** | Always `Main` (project number 8). Use `--project "Main"` flag. | | **Project** | Always `Main` (project number 8). Use `--project "Main"` flag. |
| **Issue Type** | See Issue Type section below. Cannot be set via `gh issue create` — use GraphQL after creation. | | **Issue Type** | See Issue Type section below. Cannot be set via `gh issue create` — use GraphQL after creation. |
## Breaking changes
When the issue introduces a breaking change or a deprecation, it is the unit
`mem:workflow/updating-changelog` turns into a block under
`### :boom: Breaking changes & Deprecations`. The section is generated from
the issue (`scripts/changelog.py boom`), so the issue must carry both:
1. The `breaking change` label.
2. A `## Breaking change note` **comment** (not a body section). Everything
after that marker line is copied to the changelog: paragraphs, bullet
lists, code blocks and headings are kept (headings are demoted to level 5
in `CHANGES.md`). Write it for self-hosters: what changes, what they must
do, and what breaks if they do nothing. Keep it short: explain the
implications and link the runbook or docs for the detail, never paste the
runbook itself. Links must point to a long-lived branch (`staging` or
`main`), never to a feature branch.
Post the note as a new comment. Do not add a breaking-change section to the
issue body and do not hand-write the changelog block. When the issue comes
from a PR, reuse the text of the PR's breaking-change section.
Do not hard-wrap the note: one line per paragraph or bullet.
The full changelog rules (sections, exclusions, `:rocket:`, pre-flight and
anomaly report) live in `mem:workflow/updating-changelog`.
## Issue Body Template ## Issue Body Template
Write the body to a temp file to avoid shell quoting issues: Write the body to a temp file to avoid shell quoting issues:
@ -237,6 +262,9 @@ Identify:
description). Omit internal details. Use the templates above. description). Omit internal details. Use the templates above.
- **Issue Type:** use the mapping table above (also handles `:bug:` / - **Issue Type:** use the mapping table above (also handles `:bug:` /
`:sparkles:` / `:tada:` title prefixes). `:sparkles:` / `:tada:` title prefixes).
- **Breaking changes:** when the PR body has a breaking-change section, post
its user-facing text as a `## Breaking change note` comment on the issue and
add the `breaking change` label (see **Breaking changes** above).
### Create the issue ### Create the issue
@ -287,7 +315,7 @@ Add `Closes #<ISSUE_NUMBER>` to the PR body for readable context, then run the e
python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER> python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
``` ```
The command creates the GitHub Development link by calling `addCloseIssueReferences` and trusts the successful mutation (GitHub does not reliably report mutation-created links back through the API). It is safe to rerun and does not close an issue retroactively when the PR is already merged. Do not rely on the body keyword as the assignment operation. The command creates the GitHub Development link by calling `addCloseIssueReferences` and trusts the successful mutation (GitHub does not reliably report mutation-created links back through the API). It is safe to rerun. Do not rely on it to keep the issue open: adding `Closes #N` to the body of an already-merged PR closed the issue immediately when tried (#12896 / PR #12067, 2026-11-09; the timeline shows a cross-reference plus a ClosedEvent with no closer). For changelog units that outcome is fine — a merged fix means a closed issue — but be aware when linking old merged PRs. Do not rely on the body keyword as the assignment operation.
### Clean up ### Clean up
@ -303,6 +331,8 @@ rm -f /tmp/issue-body.md /tmp/pr-body.md
label or the author is not a core team member, add the label to the issue. label or the author is not a core team member, add the label to the issue.
- **Don't put implementation details in the issue body** — the issue is for - **Don't put implementation details in the issue body** — the issue is for
users, QA, and changelog readers. users, QA, and changelog readers.
- **Drop the PR's `no issue required` label** once the issue exists: it only
marks PRs that need no changelog unit, and it is stale from then on.
## Creating Issues from Draft Body ## Creating Issues from Draft Body

View File

@ -32,6 +32,19 @@ PR titles follow commit title conventions:
See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters. See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters.
## Merge Strategy
Policy: `CONTRIBUTING.md` → "Merging", under "Pull Requests".
- Squash merge is the default, so the PR's own commits never reach `main`
and the PR title becomes the landed subject.
- A regular merge is possible on request only. The request is the **first
line** of the PR body, above the `**Note:**` line:
`> **Merge:** do not squash; each commit is self-contained.`
- Ask for it only when every commit on the branch is self-contained (builds
and passes lint on its own). Default to squash unless the user asks for
the opposite.
## Description Body ## Description Body
Include concise sections covering: Include concise sections covering:
@ -60,13 +73,15 @@ PR descriptions follow this structure:
``` ```
The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR. The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
When the PR requests a regular merge instead of a squash merge, its
`**Merge:**` line goes above the `**Note:**` line (see Merge Strategy).
## Writing Principles ## Writing Principles
- **Write for humans.** The diff shows what changed. The description explains why. - **Write for humans.** The diff shows what changed. The description explains why.
- **Be concise.** Focus on reasoning: What was the problem? Why did it happen? How did you solve it? - **Be concise.** Focus on reasoning: What was the problem? Why did it happen? How did you solve it?
- **Prefer bullets over paragraphs.** Short bullet items, grouped by area with bold lead-ins where helpful, are far easier to digest than prose; keep any remaining paragraph to a few sentences. - **Prefer bullets over paragraphs.** Short bullet items, grouped by area with bold lead-ins where helpful, are far easier to digest than prose; keep any remaining paragraph to a few sentences.
- **No manual line wraps.** Markdown renders adapting to the viewport; hard-wrapped lines degrade rendering. One line per paragraph or bullet, however long. - **No manual line wraps.** Markdown renders adapting to the viewport; hard-wrapped lines degrade rendering. One line per paragraph or bullet, however long. This applies to PR descriptions and to PR/issue comments and review replies alike.
- **Skip the obvious.** Don't explain what `git diff` already shows. - **Skip the obvious.** Don't explain what `git diff` already shows.
### What NOT to Include ### What NOT to Include
@ -80,7 +95,7 @@ The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
## Explicit Issue Assignment ## Explicit Issue Assignment
- For each GitHub issue that a PR resolves, run `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` after creating or editing the PR. Do not rely on `Closes #NNNN` in the body; it is only human-readable context. - For each GitHub issue that a PR resolves, run `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` after creating or editing the PR. Do not rely on `Closes #NNNN` in the body; it is only human-readable context.
- The command calls `addCloseIssueReferences` and trusts the successful mutation: GitHub does not reliably report mutation-created links back through `closedByPullRequestsReferences(userLinkedOnly: true)`, so the command exits non-zero only when a link target is missing or the mutation fails. It is safe to rerun and also works for an already merged PR; it does not close an issue retroactively. - The command calls `addCloseIssueReferences` and trusts the successful mutation: GitHub does not reliably report mutation-created links back through `closedByPullRequestsReferences(userLinkedOnly: true)`, so the command exits non-zero only when a link target is missing or the mutation fails. It is safe to rerun and also works for an already merged PR. Do not rely on it to keep the issue open: adding `Closes #N` to the body of an already-merged PR closed the issue immediately when tried (#12896 / PR #12067, 2026-11-09).
- Skip this process for `Relates to #NNNN` and Taiga references, which do not represent a closing relationship. - Skip this process for `Relates to #NNNN` and Taiga references, which do not represent a closing relationship.
## Before Opening ## Before Opening

View File

@ -0,0 +1,7 @@
# Non-JS dependencies in the production images
Node and pnpm pins: `mem:workflow/updating-node` and `mem:workflow/updating-pnpm`. Only the parts the Dockerfiles cannot show:
- JDK: two pins that move together — `docker/devenv/Dockerfile` (`setup-jvm` stage, full JDK) and `docker/images/Dockerfile.backend` (JDK + `jlink` runtime). Same build, same sums in both. They compile and run the same backend: `manage.sh` builds the bundle inside the devenv image (`build-backend-bundle` → `build "backend"`) and `backend/scripts/run.template.sh` starts the runtime with `--enable-preview`, which requires the runtime major to match the compiler major. A runtime line behind the compiler line is a silent trap.
- JDK line: tracks a non-LTS line, so there is no minor to pick — the last build of the line is the ceiling; moving line is a major and needs a real build to validate the `jlink` module list (`--add-modules` in `Dockerfile.backend`). JDK 27 keeps all of them; check with `bin/jlink --add-modules <list>` against the downloaded JDK before bumping. Source of the URL and both sums: the Azul metadata API (`api.azul.com/metadata/v1/zulu/packages/?java_version=<n>&os=linux&arch=<x64|aarch64>&java_package_type=jdk&archive_type=tar.gz&latest=true&release_status=ga&availability_types=CA`), then confirm the sums by downloading the tarball and running `sha256sum`.
- ImageMagick: publish `penpotapp/imagemagick:<ver>` **before** bumping the `COPY --from` refs in the backend, exporter and devenv Dockerfiles; otherwise both builds break on the missing tag.

View File

@ -0,0 +1,221 @@
# Updating the changelog
> Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human release managers who need full context. That is why it uses plain language and examples. It is the canonical home of the changelog flow; the `update-changelog` skill is only an entry point that routes here.
`CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory whose `patched_versions` include the version. It is never written from scratch by hand; the only manual parts are the wording of a description when the title is not enough and the advisory summaries (polished on GitHub, the source of the generated security entries).
The commands live in `scripts/gh.py` (GitHub queries) and `scripts/changelog.py` (changelog checks). Their flags and internals are documented in `mem:scripts/gh` and `mem:scripts/changelog`.
## 1. When to run it
- Before a release, to fill the section of the version being released.
- When new issues are added to an existing milestone.
- When asked to audit or refresh `CHANGES.md`.
Editing `CHANGES.md` happens only on explicit request or as part of the release process; see `mem:critical-info`.
## 2. Sources of truth
| Question | Source |
|---|---|
| Which issues belong to the version? | GitHub milestone |
| Which security fixes? | Advisories whose `patched_versions` include the version |
| Which section? | Issue Type |
| Is it a breaking change? | `breaking change` label |
| Is it a highlight? | `release highlight` label |
- **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
- **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug).
- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry.
- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge.
- **Excluded issues**: `no changelog`, `Task` issue type, `Rejected` project status. `release blocker` is a workflow label (the issue blocks the release), not a changelog exclusion: bugs carrying it are analyzed by `dev-only-bugs` (section 8, step 3), and closed non-bug issues carrying it are reviewed by hand (section 8, step 3).
- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3).
- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own.
- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly.
## 3. Layout of `CHANGES.md`
The newest version goes at the top, right after the `# CHANGELOG` header. Inside a version, the section order is fixed:
```markdown
# CHANGELOG
## 2.20.0 (Unreleased)
### :boom: Breaking changes & Deprecations
#### <Breaking change title>
<content>
[#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :rocket: Epics and highlights
- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237))
### :lock: Security fixes
- Fix <user-facing description of the vulnerability> (<advisory URL>) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :tada: Features
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :sparkles: Enhancements
- Improve <user-facing description of the existing behaviour> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :bug: Bugs
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
```
- `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update.
- **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it.
- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories.
- One blank line between the last entry of a section and the next heading.
- No manual line wrapping: one line per bullet or paragraph, however long.
## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`)
The entry shape is:
```markdown
- <description> (by @github_username) [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR), [#PR2](https://github.com/penpot/penpot/pull/PR2))
```
- **Description**: derived from the issue title, not the PR title. Strip emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`) and write the user-facing behaviour in imperative mood. Examples: `Plugin API token methods fail with schema validation error` → `Fix Plugin API token methods failing with schema validation error`; `Custom uploaded font family names are not sanitized` → `Sanitize font family names on custom uploaded fonts`.
- **Issue is the primary link**: the issue is the changelog unit, the PR is the implementation. Multiple fix PRs go comma-separated inline.
- **Community attribution**: when the issue or its PR carries `community contribution`, add `(by @username)` before the issue link. Use the PR author, not the issue author.
- **Closed issue with no PR** in the milestone: link the issue only, no PR reference.
- **PR with no linked issue**: link a matching closed issue in the milestone when one exists; otherwise skip it, because the issue is the changelog unit.
- **False-positive PR links**: a PR may claim to close an unrelated issue (ancient PR, cross-project reference). If the titles are clearly unrelated or the PR predates the issue by years, treat it as a data glitch and ignore it.
## 5. `:boom:`: breaking changes and deprecations
Each breaking change is a `####` block, not a bullet. The title is the issue title without its emoji prefix; the content comes from a comment on the issue; the references go on their own line at the end.
The **issue** must carry:
1. The `breaking change` label.
2. A comment whose first line is `## Breaking change note`. Everything after that marker line is the changelog content: paragraphs, bullet lists, code blocks, tables, headings (they are demoted to `#####` on write), and so on. The marker itself is case-insensitive and accepts `#`, `##` or `###`; use `##`.
Example comment:
```markdown
## Breaking change note
The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed.
Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)).
```
Generated block:
```markdown
#### Enable the Admin Console by default
The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed.
Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)).
[#12807](https://github.com/penpot/penpot/issues/12807) (PR: [#12475](https://github.com/penpot/penpot/pull/12475))
```
Rules for the note:
- **Write it for admins**: what changes, what they must do, and what breaks if they do nothing. Do not paste the full runbook; explain the implications and link the runbook or docs.
- **Link a long-lived branch**, `staging` or `main`, never a feature branch (`issue-1234`). The link may 404 until the PR merges; that is expected.
- **Do not add issue or PR links**: the generator appends them.
- **Do not hard-wrap the text**: one line per paragraph or bullet. The generator copies lines verbatim, so wrapped lines end up wrapped in `CHANGES.md`.
- **One note per issue**. If several comments carry the marker, the generator uses the last one and warns.
- **No note**: the generator warns on stderr and emits only the heading and references; fix the issue, not `CHANGES.md`.
- **`####`, not `###`**: `###` is the level of the changelog sections. An entry at `###` would render as a sibling section and the scripts that read and rewrite `:boom:` would cut the section there. The note author never writes the `####`: the generator adds it.
- **Only merged closing PRs** are referenced; unmerged or superseded PRs are skipped.
- **Never hand-edit the section**. Generate it with `boom` (section 8). The note lives on GitHub; edit it there.
## 6. `:rocket:`: epics and highlights
- Lists exactly the milestone issues with the `release highlight` label, and nothing else. Product decides highlights by labelling issues; a regeneration cannot undo that choice.
- Every entry copies the text and references of the issue's regular entry and MUST carry both an issue link and a PR reference.
- If no issue is labelled, leave the section out; do not pick highlights yourself.
- Never edit `:rocket:` by hand: add or remove the label and regenerate.
## 7. `:lock:`: security fixes
Generated from the advisories that declare the version in `patched_versions`; never written by hand.
- **Source and filter**: `python3 scripts/gh.py advisories --patched-version <VERSION>` keeps the advisories whose `vulnerabilities[*].patched_versions` contain the version as an exact token (so `2.19.1` never matches `2.19.10`). Any state is valid — `triage`, `draft`, `published` — because the advisory may still be unpublished when the changelog is written.
- **Fill the advisories first**: `patched_versions` must be filled before generating the changelog. The release process publishes the advisories with (or after) the release; the changelog does not wait for publication.
- **PR reference**: a merged PR in the milestone labelled `pr pt` whose body contains the GHSA id contributes its number inline; several PRs go comma-separated. An advisory with no PR is legal: the entry links only the advisory.
- **Missing `patched_versions`**: when a `pr pt` PR references an advisory that declares no matching `patched_versions`, `security` prints a warning on stderr and leaves it out. Fix the advisory, not `CHANGES.md`.
- **Entry format**: `- Fix <description> (<advisory URL>) (PR: [#PR](...))`. The description is `Fix` plus the advisory summary, with the first word decapitalized unless it starts with an acronym (`SVG`, `CVE-...`).
- **Wording**: the flow may polish the description. Re-running `security --write` rewrites it from the summary, so the durable place for wording is the advisory summary on GitHub; keep it a problem statement, because the generator adds `Fix`.
- **Empty version**: no matching advisories → the subsection is not inserted (and is removed by `--write`).
- **Never hand-edit the section**: generate it with `security` (section 8).
## 8. Flow
1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious.
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs "<VERSION>"` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration. Bugs carrying `release blocker` are analyzed like any other bug.
- **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these.
- **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or the milestone version itself. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10).
- **Closed non-bug issues carrying `release blocker`** have no automatic check: `python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog"` lists them (closed, non-Task). Show them to the user: the internal ones get the `no changelog` label, the rest keep their entry.
4. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
7. **Reconcile the existing text** (section 9).
8. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
9. **Generate `:rocket:`** from the `release highlight` label (section 6).
10. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
12. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
## 9. Pre-flight: reconcile before writing
Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` before adding the new section:
1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports. The same holds after merging a patch branch into `staging`: the fix keeps its released patch entry and leaves the newer, unreleased section (the merge conflict in `CHANGES.md` is resolved by keeping both sections, newest first, minus the duplicate).
2. **Stale milestone assignment** (the issue moved out of this milestone) → remove the entry.
3. **Newly applied exclusion labels** (`no changelog`) → remove the entry. (`release blocker` is not an exclusion label; it never removes an entry by itself.)
4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry.
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently.
6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section.
7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry.
8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it.
9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section.
## 10. Anomaly report
`report` classifies problems it cannot decide alone. The total counts A, B, E, F and G; C and D are warnings.
- **A — issue in this milestone, referenced PR in another milestone (or none).** The claim "this issue is fixed by this PR in this milestone" is inconsistent. Fix the milestone on the issue or the PR, or remove the entry.
- **B — PR in this milestone whose closing issue is in another milestone.** Same fix. An issue with no milestone is not an anomaly (another, probably private, project).
- **E — `:boom:` entry whose issue lacks the `breaking change` label.** The entry stays in the changelog: either label the issue or move the entry to its regular section.
- **F — `:rocket:` out of sync with the `release highlight` label.** A rule violation: `:rocket:` must list exactly the labelled issues. Fix the label or re-run the flow.
- **G — `breaking change` issue missing from `:boom:`.** A rule violation: `:boom:` must list exactly the labelled issues. Regenerate the section or relabel.
- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights).
- **D — `:rocket:` entry without issue AND PR references** (gap).
**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total.
Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run.
## 11. Pitfalls
- **Open issues with provisional entries**: a PR author may add the entry before the merge. `boom` will not generate it until the issue closes; keep the hand-written block identical to what the generator would produce.
- **Labels, not emoji**: title prefixes are cosmetic and often missing; the section comes from Issue Type and the two labels.
- **Taiga references**: old entries link to Taiga. Resolve them to the GitHub issue via the description text or the PRs that mention the Taiga URL, then link issue and PR.
- **Long-lived links only**: runbooks and docs referenced from a note must point to `staging` or `main`, never to a feature branch that will be deleted.
- **Generated sections**: `:boom:`, `:rocket:` and `:lock:` are generated. Never patch them by hand; edit the label, the note comment or the advisory and re-run.
- **Deleted or transferred issues**: an issue referenced in the changelog that returns 410/404 (deleted or moved out of the repo) must lose its entry in the pre-flight. Its security fixes, if any, stay in `:lock:` through the advisory.
## 12. Where the code is
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`.
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`.
- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`.
- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**.

View File

@ -1,15 +1,16 @@
# Updating Node Across the Toolchain # Updating Node Across the Toolchain
No single source of truth: nothing reads `.nvmrc` at build time, and the version is repeated in a different shape per file. Six sites move together. No single source of truth: nothing reads `.nvmrc` at build time, and the version is repeated in a different shape per file. Six sites move together, and the image ones carry a digest beside the tag.
## Pin sites ## Pin sites
- `.nvmrc` → `v<X.Y.Z>`, with the leading `v`. - `.nvmrc` → `v<X.Y.Z>`, with the leading `v`.
- `docker/devenv/Dockerfile`, `setup-node` stage → `ENV NODE_VERSION=v<X.Y.Z>`; downloads `https://nodejs.org/dist/${NODE_VERSION}/node-${NODE_VERSION}-linux-{arm64,x64}.tar.gz`. Node is the only download in that file with **no** checksum verification (pnpm, JDK and AWS CLI do verify), so bumping Node has no hash to recompute. - `docker/devenv/Dockerfile`, `setup-node` stage → `ENV NODE_VERSION=v<X.Y.Z>`; downloads `https://nodejs.org/dist/${NODE_VERSION}/node-${NODE_VERSION}-linux-{arm64,x64}.tar.gz`, one tarball per arch, and verifies each against a `NODE_SHA256` right below its `BINARY_URL`. Both hashes come from `SHASUMS256.txt` in that same dist directory, so a bump must recompute the two; only the arch SHAs change, the structure stays.
- `docker/images/Dockerfile.mcp` → two stages: `FROM dhi.io/node:<X.Y.Z>-debian13-dev AS build` and `FROM dhi.io/node:<X.Y.Z>-debian13 AS image`. The release stage carries no `-dev`. - `docker/images/Dockerfile.mcp` → two stages: `FROM dhi.io/node:<X.Y.Z>-debian13-dev AS build` and `FROM dhi.io/node:<X.Y.Z>-debian13 AS image`. The release stage carries no `-dev`.
- `docker/images/Dockerfile.media-processor` and `docker/images/Dockerfile.exporter` → single `FROM dhi.io/node:<X.Y.Z>-debian13-dev`. - `docker/images/Dockerfile.media-processor` and `docker/images/Dockerfile.exporter` → single `FROM dhi.io/node:<X.Y.Z>-debian13-dev`.
- `.github/workflows/plugins-deploy-api-doc.yml` and `.github/workflows/plugins-deploy-styles-doc.yml` → `runtime: node@<X.Y.Z>` in the `pnpm/setup` step. Its `working-directory: plugins` makes pnpm's own version come from the manifest, so this pin is the Node one; keep it equal to `.nvmrc` minus the `v`. - `.github/workflows/plugins-deploy-api-doc.yml` and `.github/workflows/plugins-deploy-styles-doc.yml` → `runtime: node@<X.Y.Z>` in the `pnpm/setup` step. Its `working-directory: plugins` makes pnpm's own version come from the manifest, so this pin is the Node one; keep it equal to `.nvmrc` minus the `v`.
- DHI tags take the bare version; only the devenv `NODE_VERSION` carries the `v`. - DHI tags take the bare version; only the devenv `NODE_VERSION` carries the `v`.
- Every `dhi.io` base image in the repo (the node ones above, plus `debian-base` and `nginx`) is pinned by digest: `FROM dhi.io/<image>:<tag>@sha256:<digest>`. Tag and digest are a pair; Renovate's `Update DHI base images` commits rewrite both in one go, so a hand-made tag bump that leaves the digest behind keeps the old image.
## Gotchas ## Gotchas
@ -17,9 +18,11 @@ No single source of truth: nothing reads `.nvmrc` at build time, and the version
- Node 24.x is the Krypton LTS line: take the newest patch of that major unless a major bump is the point. - Node 24.x is the Krypton LTS line: take the newest patch of that major unless a major bump is the point.
- Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites. - Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites.
- A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift. - A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift.
- The newer version wins in both directions. A maintenance→main merge (`main-2.18.x` into `main`) can carry the newer Node while `.nvmrc` and the `packageManager` fields stay on the older pair; then adopt the newer set and finish the bump on all six sites plus the 35 `packageManager` fields.
- Node stays on the 24.x line even when the rest of `.github/` moves on: `99db795966` (`update GitHub Actions`) took the two `plugins-deploy-*` `runtime:` pins to `node@26.13.1` on develop while `.nvmrc` stayed on `v24.22.3`. The project keeps every site on the `.nvmrc` value; resolve that conflict back to the 24.x pin instead of taking develop's.
- Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import. - Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import.
## Verification ## Verification
- `rg '<old version>'` finds nothing outside `CHANGES.md` and untracked scratch files (a repo-wide grep without path filters hits multi-megabyte ClojureScript analyzer caches: always scope it). - `rg '<old version>'` finds nothing outside `CHANGES.md` and untracked scratch files (a repo-wide grep without path filters hits multi-megabyte ClojureScript analyzer caches: always scope it). Grep `dhi\.io/node` as well: a line whose tag moved but whose digest did not still pulls the old image.
- After a devenv rebuild, `node --version` matches `.nvmrc` and `pnpm --version` matches `PNPM_VERSION` in `docker/devenv/Dockerfile`. - After a devenv rebuild, `node --version` matches `.nvmrc` and `pnpm --version` matches `PNPM_VERSION` in `docker/devenv/Dockerfile`.

View File

@ -23,12 +23,12 @@ file (never pipe tool output through filters).
- The store survives `node_modules` cleans. It is content-addressed and - The store survives `node_modules` cleans. It is content-addressed and
integrity-verified, so it cannot go stale; staleness lives in integrity-verified, so it cannot go stale; staleness lives in
node_modules. Only `scripts/clean-node-modules --store` removes it. node_modules. Only `scripts/clean-node-modules --store` removes it.
- Every first-party `package.json` (35 files: workspace roots plus - Every first-party `package.json` (34 files: workspace roots plus
members) must carry a `packageManager` field with the identical members) must carry a `packageManager` field with the identical
`pnpm@<version>+sha512.<hash>` value. Do not let them drift. `pnpm@<version>+sha512.<hash>` value. Do not let them drift.
`external/` (vendored trees with their own lifecycles), `.opencode/`, `external/` (vendored trees with their own lifecycles), `.opencode/`,
`.pnpm-store/`, `bundles/` and `docker/images/bundle-*` copies are `.pnpm-store/`, `bundles/`, `target/` build output and
never stamped. `docker/images/bundle-*` copies are never stamped.
- pnpm is a system binary everywhere (devenv image, CI runners, Docker - pnpm is a system binary everywhere (devenv image, CI runners, Docker
images). Nothing may call corepack: it is gone from Node 25+. pnpm images). Nothing may call corepack: it is gone from Node 25+. pnpm
auto-downloads the `packageManager` version on mismatch auto-downloads the `packageManager` version on mismatch
@ -60,7 +60,7 @@ file (never pipe tool output through filters).
3. From the repo root, on a host whose system pnpm is the target version 3. From the repo root, on a host whose system pnpm is the target version
(rebuilt devenv), run `scripts/sync-pnpm-version`. It resolves the (rebuilt devenv), run `scripts/sync-pnpm-version`. It resolves the
integrity hash via `pnpm view` + node (no npm, no corepack, no integrity hash via `pnpm view` + node (no npm, no corepack, no
python3) and stamps the identical field into all 35 files, replacing python3) and stamps the identical field into all 34 files, replacing
only the value line (each file keeps its own indent) and inserting only the value line (each file keeps its own indent) and inserting
the key after `"name"`/`"version"` where missing. Explicit version the key after `"name"`/`"version"` where missing. Explicit version
instead: `scripts/sync-pnpm-version <version>`. The explicit form works instead: `scripts/sync-pnpm-version <version>`. The explicit form works
@ -71,6 +71,7 @@ file (never pipe tool output through filters).
4. Run `scripts/sync-pnpm-version --install` (plain `pnpm install` in 4. Run `scripts/sync-pnpm-version --install` (plain `pnpm install` in
each of the 11 workspace roots) to refresh lockfile metadata, then each of the 11 workspace roots) to refresh lockfile metadata, then
fix any failing workspace (see gotchas) and re-run that directory. fix any failing workspace (see gotchas) and re-run that directory.
For a version-only bump, `pnpm install --lockfile-only` in each root is enough: it rewrites only the pnpm self-reference entries and needs no node_modules. The repo-root `pnpm-lock.yaml` is rewritten by itself on the first `pnpm` call that self-installs the target version.
## Gotchas ## Gotchas
@ -88,14 +89,17 @@ file (never pipe tool output through filters).
so a nested one silently forks install and lockfile behavior. Do not so a nested one silently forks install and lockfile behavior. Do not
reintroduce it. reintroduce it.
- `media-processor/target/package.json` is a generated, gitignored copy that - `media-processor/target/package.json` is a generated, gitignored copy that
the script also stamps, so stamping and `--check` report 36 files while the script also stamps, so stamping and `--check` report 35 files while
only 35 are tracked. A stale copy there makes `--check` fail with nothing only 34 are tracked. A stale copy there makes `--check` fail with nothing
wrong in the repo: delete `media-processor/target` and re-run. wrong in the repo: delete `media-processor/target` and re-run.
- Expect metadata-only lockfile diffs when only the pnpm version moves: - Expect metadata-only lockfile diffs when only the pnpm version moves:
the pnpm self-reference entries, plus a new `packageManagerDependencies` the pnpm self-reference entries, plus a new `packageManagerDependencies`
section in lockfiles last written by older pnpm. Large diffs mean section in lockfiles last written by older pnpm. Large diffs mean
re-resolution; inspect them before accepting. re-resolution; inspect them before accepting.
- A backport merge of main into staging can bring a newer `PNPM_VERSION` in `docker/` while every `packageManager` field stays behind, since main receives partial `.github/`+`docker/` backports from develop. Resolve those conflicts with main's version and run `scripts/sync-pnpm-version <version>` in the same merge. - A backport merge of main into staging can bring a newer `PNPM_VERSION` in `docker/` while every `packageManager` field stays behind, since main receives partial `.github/`+`docker/` backports from develop. Resolve those conflicts with main's version and run `scripts/sync-pnpm-version <version>` in the same merge.
- Direction does not decide the winner, the version does. A maintenance→main merge (`main-2.18.x` into `main`) can carry the newer Node and pnpm pins in `docker/` and the `plugins-deploy-*` workflows while main's `.nvmrc` and 35 `packageManager` fields stay on the older pair. Take the newer pair on every pin site (`docker/`, workflows, `.nvmrc`, `packageManager`) and run `scripts/sync-pnpm-version <version>` in the same merge; check with `--check` plus `pnpm install --lockfile-only --frozen-lockfile` per workspace.
- Those merges conflict on the pnpm self-reference block in all 11 lockfiles (importers `packageManagerDependencies`, `packages/@pnpm/exe.*@<ver>`, `pnpm@<ver>` and the matching `snapshots` entries). Resolve each workspace's lockfile keeping its own importer/package resolution and swapping only that block to the newer version. Do not start from the older side and regenerate with `pnpm install --lockfile-only`: pnpm re-resolves every changed range upward (this merge took root `@playwright/test` from 1.63.0 to 1.64.1 and `@modelcontextprotocol/server` from 2.3.1 to 2.4.0), which pollutes the merge with unrelated bumps. If a workspace later needs a real re-resolution after a manifest bump, `pnpm install --lockfile-only --fix-lockfile` accepts the hand-merged file and only fills what the manifests demand.
- `pnpm install` cannot run in `frontend` or `exporter` when `github.com` does not resolve: the `@penpot/svgo` git dependency triggers a `git fetch`. Frontend's lockfile entry carries no integrity, so pnpm always refetches; exporter's has one and still works. For frontend, resolve the self-reference hunks only.
## Verification ## Verification

View File

@ -1,42 +1,54 @@
# the name by which the project can be referenced within Serena/when chatting with the LLM. # the name by which the project can be referenced within Serena/when chatting with the LLM.
project_name: "penpot" project_name: "penpot"
# list of languages for which language servers are started (LSP backend only); choose from: # list of language servers to start when using the LSP backend; choose from:
# ada al angular ansible bash # ada al angular ansible astro
# bsl clojure cpp cpp_ccls crystal # bash bsl clojure cpp cpp_ccls
# csharp csharp_omnisharp cue dart elixir # crystal csharp csharp_omnisharp cue dart
# elm erlang fortran fsharp gdscript # deno elixir elm erlang fortran
# go groovy haskell haxe hlsl # fsharp gdscript gleam go groovy
# html java json julia kotlin # haskell haxe hlsl html java
# latex lean4 lua luau markdown # json julia julia_fatou kotlin latex
# matlab msl nix ocaml pascal # lean4 lua lua_emmylua luau markdown
# perl php php_phpactor php_phpantom powershell # matlab msl nextflow nix ocaml
# python python_jedi python_pyrefly python_ty r # pascal perl php php_devsense php_phpactor
# rego ruby ruby_solargraph rust scala # php_phpantom powershell python python_basedpyright python_jedi
# scss solidity svelte swift systemverilog # python_pyrefly python_ty qml r rego
# terraform toml typescript typescript_vts vue # ruby ruby_solargraph rust scala scss
# yaml zig # solidity svelte swift systemverilog terraform
# toml typescript typescript_vts vue wolfram
# yaml zig
# (This list may be outdated; generated with scripts/print_language_list.py; # (This list may be outdated; generated with scripts/print_language_list.py;
# For the current list, see values of Language enum here: # For the current list, see values of the LanguageServerId enum here:
# https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py) # https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
# For some languages, there are alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.) # For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
# Note: # Note:
# - For C, use cpp # - For C, use cpp
# - For JavaScript, use typescript # - For JavaScript, use typescript
# - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root) # - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
# - For Astro projects, use astro (subsumes typescript/javascript for .astro projects; requires npm)
# - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm) # - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
# - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
# - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three) # - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
# - For Free Pascal/Lazarus, use pascal # - For Free Pascal/Lazarus, use pascal
# - External Python adapters may add further registered IDs; install the adapter package first
# and then use its ID here, for example: example
# Special requirements: # Special requirements:
# Some languages require additional setup/installations. # Some language servers require additional setup/installations.
# See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers # See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
# When using multiple languages, the first language server that supports a given file will be used for that file. # When using multiple language servers, the first language server that supports a given file will be used for that file.
# The first language is the default language and the respective language server will be used as a fallback. # The first language server is the default language and the respective language server will be used as a fallback.
# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored. # Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
language_servers:
- clojure
- typescript
- rust
# same list for Serena 1.x, which only reads this key; keep both lists equal
languages: languages:
- clojure - clojure
- typescript - typescript
- rust - rust
# the encoding used by text files in the project # the encoding used by text files in the project
# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings # For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
@ -177,3 +189,18 @@ activation_command:
# maximum time in seconds to wait for activation_command to complete before killing it (default 180s). # maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
# must be a positive number. # must be a positive number.
activation_command_timeout: 180.0 activation_command_timeout: 180.0
# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.get_diagnostics_for_symbol") to include in the REPL
# that would otherwise be disabled (particularly optional methods, which are disabled by default).
# This extends the existing inclusions (e.g. from the global configuration).
included_apis: []
# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.find_symbol") to exclude from the REPL.
# This extends the existing exclusions (e.g. from the global configuration).
excluded_apis: []
# The interface through which the agent (LLM) accesses Serena's functionality (overrides the global setting).
# Valid values: tools, REPL (see the global configuration for details); leave empty to use the global setting.
# Note: the interface is fixed at startup. If a project is activated post-init, its setting is not applied.
agent_interface: REPL

View File

@ -10,7 +10,8 @@
If the user pushes, treat that commit as final from the agent's side. If the user pushes, treat that commit as final from the agent's side.
- **Never edit `CHANGES.md` by hand** in commits or PRs. The changelog is - **Never edit `CHANGES.md` by hand** in commits or PRs. The changelog is
generated from GitHub milestones during the release process; update it only generated from GitHub milestones during the release process; update it only
via the `update-changelog` skill flow or on explicit user request. via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or
on explicit user request.
- **Never pipe test output directly to filters** (`| head`, `| tail`, `| grep`, etc.). - **Never pipe test output directly to filters** (`| head`, `| tail`, `| grep`, etc.).
Always redirect to a file first: `command > /tmp/output.txt 2>&1`, then read/grep the file. Always redirect to a file first: `command > /tmp/output.txt 2>&1`, then read/grep the file.
This prevents hiding test failures. See `mem:testing` for details. This prevents hiding test failures. See `mem:testing` for details.
@ -27,12 +28,18 @@
- Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, AI note) - Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, AI note)
Don't infer format from the title of a previous commit/issue/PR — the memory Don't infer format from the title of a previous commit/issue/PR — the memory
is the source of truth. is the source of truth.
- **Everything in code MUST be in English.** Identifiers (functions,
variables, types, namespaces), code comments, docstrings, test
descriptions, log hints — always English, regardless of the language the
author works in. Tests are not an exception. The only place another
language may appear in the codebase is product copy rendered through the
i18n system.
## CRITICAL: Read module memories BEFORE writing any code ## CRITICAL: Read module memories BEFORE writing any code
Do this **before planning, before coding, before touching any file**: Do this **before planning, before coding, before touching any file**:
1. Read `critical-info` (use `serena_read_memory critical-info` or read `.serena/memories/critical-info.md`). 1. Read `critical-info` through Serena (see "Accessing memories" below) or read `.serena/memories/critical-info.md`.
It describes the project structure and tells you which modules exist. It describes the project structure and tells you which modules exist.
2. From `critical-info`, identify which modules your task affects. 2. From `critical-info`, identify which modules your task affects.
3. Read each affected module's **core memory** — the name is `<module>/core` 3. Read each affected module's **core memory** — the name is `<module>/core`
@ -103,9 +110,12 @@ When working on a task:
## Accessing memories ## Accessing memories
- **If `serena_read_memory` / `serena_list_memories` tools are available**: use them. - **If Serena's `serena_repl` tool is available** (Serena 2.x): call `s.mem.read_memory(name)`
`serena_read_memory` takes a memory name (e.g. `critical-info`, `frontend/core`). and `s.mem.list_memories()` through it.
- **If tools are NOT available**: read the filesystem directly. - **If Serena's `read_memory` / `list_memories` tools are available** (Serena 1.x; some clients
show them as `serena_read_memory` / `serena_list_memories`): use them.
- Either way, `read_memory` takes a memory name (e.g. `critical-info`, `frontend/core`).
- **If Serena is NOT available**: read the filesystem directly.
Memory name `mem:foo/bar` maps to file `.serena/memories/foo/bar.md`. Memory name `mem:foo/bar` maps to file `.serena/memories/foo/bar.md`.
## Cross-reference convention ## Cross-reference convention
@ -152,6 +162,6 @@ precision while maintaining a strong focus on maintainability and performance.
- `scripts/nrepl-eval.mjs` — Evaluate Clojure code via nREPL (backend + frontend). - `scripts/nrepl-eval.mjs` — Evaluate Clojure code via nREPL (backend + frontend).
- `scripts/check-commit` — Validate commit messages against Penpot's commit guidelines. - `scripts/check-commit` — Validate commit messages against Penpot's commit guidelines.
- `scripts/check-fmt-clj` — Check Clojure formatting without modifying files. - `scripts/check-fmt-clj` — Check Clojure formatting without modifying files.
- `scripts/ci` — CI orchestration script for running lint, tests, and format checks across modules. See `mem:scripts/ci`. - `scripts/ci` — CI orchestration script for running lint, tests, format checks and, on request (`--build`), production builds across modules. See `mem:scripts/ci`.
- `scripts/gh.py` — Multi-purpose GitHub CLI helper. Subcommands: `issues` (list issues in a milestone), `prs` (fetch PR details), `advisories` (list/inspect security advisories). See `python3 scripts/gh.py --help`. - `scripts/gh.py` — Multi-purpose GitHub CLI helper. Subcommands: `issues` (list issues in a milestone), `prs` (fetch PR details), `advisories` (list/inspect security advisories). See `python3 scripts/gh.py --help`.

View File

@ -1,6 +1,184 @@
# CHANGELOG # CHANGELOG
## 2.19.0 (Unreleased) ## 2.21.0
### :boom: Breaking changes & Deprecations
### :lock: Security fixes
- Fix real-time collaboration WebSocket never re-validates file access after `:subscribe-file`, letting revoked users keep receiving live edit content (https://github.com/penpot/penpot/security/advisories/GHSA-m53j-2766-6jqw) (PR: [#12189](https://github.com/penpot/penpot/pull/12189))
### :tada: Features
- Auto-link libraries during import based on slugified name [#9263](https://github.com/penpot/penpot/issues/9263) (PR: [#9958](https://github.com/penpot/penpot/pull/9958))
- Implement basic bullet lists [#12630](https://github.com/penpot/penpot/issues/12630) (PR: [#12720](https://github.com/penpot/penpot/pull/12720))
### :sparkles: Enhancements
- Improve performance of payload decoding in the workspace (by @lukel97) [#12916](https://github.com/penpot/penpot/issues/12916) (PR: [#12704](https://github.com/penpot/penpot/pull/12704))
- Keep the search when changing library in the Swap component menu (by @desty) [#12335](https://github.com/penpot/penpot/issues/12335) (PR: [#12873](https://github.com/penpot/penpot/pull/12873))
### :bug: Bugs
- Fix workspace stuttering when switching pages in a large file (by @lukel97) [#12917](https://github.com/penpot/penpot/issues/12917) (PR: [#12811](https://github.com/penpot/penpot/pull/12811))
- Fix workspace freezes when undoing or redoing a move of a board with many components (by @lukel97) [#12913](https://github.com/penpot/penpot/issues/12913) (PR: [#12870](https://github.com/penpot/penpot/pull/12870))
- Fix dashboard freezes for about a second after duplicating a large file (by @lukel97) [#12914](https://github.com/penpot/penpot/issues/12914) (PR: [#12911](https://github.com/penpot/penpot/pull/12911))
- Fix not being able to collapse a token set group after reordering it [#12625](https://github.com/penpot/penpot/issues/12625) (PR: [#12671](https://github.com/penpot/penpot/pull/12671))
- Fix slow variant switch with the WebGL renderer [#12968](https://github.com/penpot/penpot/issues/12968) (PR: [#12985](https://github.com/penpot/penpot/pull/12985))
- Fix login page GitHub logo showing with low contrast in the light theme [#12938](https://github.com/penpot/penpot/issues/12938) (PR: [#12943](https://github.com/penpot/penpot/pull/12943))
- Fix leak and black screen on wasm-critical errors [#12977](https://github.com/penpot/penpot/issues/12977) (PR: [#12978](https://github.com/penpot/penpot/pull/12978))
## 2.20.0
### :boom: Breaking changes & Deprecations
#### Add table partitioning for file-change
The file history table (`file_change`) is now partitioned by file, with the same 16 partitions as the file payload table (`file_data`). Queries that filter by file, and the join between the two tables, resolve against a single partition, so they stay fast as the table grows. The rewrite also removes the dead columns and stores the table in its final layout; the space the old table held comes back when it is dropped. A save no longer writes the file features on every revision, and the team snapshot quota is a single lookup.
The four snapshot RPC methods (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot` and `unlock-file-snapshot`) now require `file-id`. The frontend already sends it; API clients that call them must add it.
On small and medium instances the table is rewritten automatically on boot; there is nothing to do. On large instances (hundreds of GB in `file_change`) it does not fit in the startup transaction: run it by hand **before upgrading to 2.20.0** following the [large-instance runbook](https://github.com/penpot/penpot/blob/staging/operations/2.20.0-file-change-partition.md). The copy runs while the instance is online, followed by a cutover of a few minutes with the backend and the workers stopped. The old table is kept for rollback; check the disk headroom before starting, and drop the old table days later to get the space back.
[#12416](https://github.com/penpot/penpot/issues/12416) (PR: [#12417](https://github.com/penpot/penpot/pull/12417))
#### Protect the file data storage objects with a foreign key in the backend
This release moves the file data storage pointer out of `file_data.metadata->>'storage-ref-id'` and into a new `file_data.storage_object_id` column with a foreign key to `storage_object(id)`. The metadata key is gone: nothing writes or reads it.
What it means for self-hosted instances:
- The release is a coordinated restart. The new code reads the column; the migration drops `storage-ref-id` from the rows it backfills. Rolling back to the previous binary is not supported after this release runs.
- A fresh database or a small instance needs nothing special: the startup migration does the whole job.
- A large instance should not let the startup migration run. Build the column, the index and the foreign key online beforehand, stop the instances, run the backfill only if a row still carries `metadata.storage-ref-id`, verify the end state, and mark the migration as applied. The full procedure is documented in [`operations/2.20.0-file-data-storage-object-id.md`](https://github.com/penpot/penpot/blob/main/operations/2.20.0-file-data-storage-object-id.md).
- If the storage backend was never enabled, there is nothing to backfill and the stopped window is just the restart.
If you upgrade blindly: on fresh databases and small instances the startup migration completes the change. On a large instance, the foreign key validation scan can block writes for a long time; follow the runbook instead.
[#12768](https://github.com/penpot/penpot/issues/12768) (PR: [#12769](https://github.com/penpot/penpot/pull/12769))
### :rocket: Epics and highlights
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
### :tada: Features
- Add a feedback channel to the MCP server [#12490](https://github.com/penpot/penpot/issues/12490) (PR: [#12551](https://github.com/penpot/penpot/pull/12551))
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Implement full text format paste within Penpot [#12047](https://github.com/penpot/penpot/issues/12047) (PR: [#12137](https://github.com/penpot/penpot/pull/12137))
- Add team, project and file management to the Plugin API for global plugins [#12170](https://github.com/penpot/penpot/issues/12170) (PR: [#12169](https://github.com/penpot/penpot/pull/12169))
- Serve Penpot under a subpath [#3194](https://github.com/penpot/penpot/issues/3194) (PR: [#12358](https://github.com/penpot/penpot/pull/12358))
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
- Add OAuth support for MCP client connections [#12034](https://github.com/penpot/penpot/issues/12034) (PR: [#12480](https://github.com/penpot/penpot/pull/12480))
### :sparkles: Enhancements
- Auto-scroll sortable lists when dragging an item near their top or bottom edge (by @ShreyashAgare26) [#10478](https://github.com/penpot/penpot/issues/10478) (PR: [#11595](https://github.com/penpot/penpot/pull/11595))
- Make WebGL the default renderer for new accounts [#11855](https://github.com/penpot/penpot/issues/11855) (PR: [#12880](https://github.com/penpot/penpot/pull/12880))
- Prevent the hosted MCP server from running execute_code twice when the plugin is open in two tabs [#12008](https://github.com/penpot/penpot/issues/12008) (PR: [#11927](https://github.com/penpot/penpot/pull/11927))
- Allow detaching typography tokens [#11887](https://github.com/penpot/penpot/issues/11887) (PR: [#11969](https://github.com/penpot/penpot/pull/11969))
- Complete the Russian translations [#11955](https://github.com/penpot/penpot/issues/11955) (PR: [#11956](https://github.com/penpot/penpot/pull/11956))
- Run .penpot file import and export as durable background jobs [#12194](https://github.com/penpot/penpot/issues/12194) (PR: [#12056](https://github.com/penpot/penpot/pull/12056), [#12192](https://github.com/penpot/penpot/pull/12192))
- Open files and projects in a new tab with Ctrl/Cmd+click in the dashboard [#12228](https://github.com/penpot/penpot/issues/12228) (PR: [#12230](https://github.com/penpot/penpot/pull/12230))
- Add an RPC method to read file-level plugin data [#12258](https://github.com/penpot/penpot/issues/12258) (PR: [#10171](https://github.com/penpot/penpot/pull/10171))
- Add a get-file-tokens RPC method to read a file's design tokens [#12260](https://github.com/penpot/penpot/issues/12260) (PR: [#12172](https://github.com/penpot/penpot/pull/12172))
- Show the progress and the size of each file in the import and export dialogs [#12340](https://github.com/penpot/penpot/issues/12340) (PR: [#12192](https://github.com/penpot/penpot/pull/12192))
- Focus the search field when opening the color tokens tab of the color picker [#12344](https://github.com/penpot/penpot/issues/12344) (PR: [#12345](https://github.com/penpot/penpot/pull/12345))
- Harden the file data offload job with a row lock, per-file transactions, throttle and metrics [#12398](https://github.com/penpot/penpot/issues/12398) (PR: [#12403](https://github.com/penpot/penpot/pull/12403))
- Render email templates inside the mail job in the backend [#12445](https://github.com/penpot/penpot/issues/12445) (PR: [#12448](https://github.com/penpot/penpot/pull/12448))
- Run local WASM exports for shapes and boards without freezing the UI [#12512](https://github.com/penpot/penpot/issues/12512) (PR: [#12515](https://github.com/penpot/penpot/pull/12515))
- Use WASM to generate SVG markup in the code inspector when render-wasm is active [#12567](https://github.com/penpot/penpot/issues/12567) (PR: [#12608](https://github.com/penpot/penpot/pull/12608))
- Close the workspace search with the Escape key (by @filipsajdak) [#9540](https://github.com/penpot/penpot/issues/9540) (PR: [#12374](https://github.com/penpot/penpot/pull/12374))
- Restrict management RPC methods to their declared shared keys [#12730](https://github.com/penpot/penpot/issues/12730) (PR: [#12736](https://github.com/penpot/penpot/pull/12736))
- Reduce memory used by duplicate shape ids in the workspace [#12791](https://github.com/penpot/penpot/issues/12791) (PR: [#12792](https://github.com/penpot/penpot/pull/12792))
- Reduce memory consumption on shape decoding [#12789](https://github.com/penpot/penpot/issues/12789) (PR: [#12790](https://github.com/penpot/penpot/pull/12790))
- Notify other users when a snapshot is restored [#12425](https://github.com/penpot/penpot/issues/12425) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Normalize the configured tenant to lowercase [#12842](https://github.com/penpot/penpot/issues/12842) (PR: [#12843](https://github.com/penpot/penpot/pull/12843))
- Speed up saves on large files by checking only the pages a save changes [#12602](https://github.com/penpot/penpot/issues/12602) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Retry saves after short server failures and keep the workspace open when a save fails [#12601](https://github.com/penpot/penpot/issues/12601) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Switch the workspace to read-only mode when the file can't be saved [#12507](https://github.com/penpot/penpot/issues/12507) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Validate upload content against real file bytes in media uploads [#12796](https://github.com/penpot/penpot/issues/12796) (PR: [#12801](https://github.com/penpot/penpot/pull/12801))
- Add the file and MCP session to audit events sent from plugins and the MCP [#12902](https://github.com/penpot/penpot/issues/12902) (PR: [#12972](https://github.com/penpot/penpot/pull/12972))
- Add skip-if-active option to job submission in the backend jobs substrate [#12265](https://github.com/penpot/penpot/issues/12265) (PR: [#12266](https://github.com/penpot/penpot/pull/12266))
- Add HTML paste from clipboard in text editor v3 [#10465](https://github.com/penpot/penpot/issues/10465)
- Skip loading the thumbnail rasterizer in WebGL renderer sessions to cut workspace memory [#13029](https://github.com/penpot/penpot/issues/13029) (PR: [#12835](https://github.com/penpot/penpot/pull/12835))
- Speed up uploading page shapes to WASM [#13030](https://github.com/penpot/penpot/issues/13030) (PR: [#12854](https://github.com/penpot/penpot/pull/12854))
### :bug: Bugs
- Fix font upload failing with a regex error [#10436](https://github.com/penpot/penpot/issues/10436) (PR: [#11951](https://github.com/penpot/penpot/pull/11951))
- Fix dashboard file thumbnails loading slow for large multi-page files [#12053](https://github.com/penpot/penpot/issues/12053) (PR: [#12054](https://github.com/penpot/penpot/pull/12054))
- Fix Plugin API resolving a font family token to its separate words when the family name has several words [#11748](https://github.com/penpot/penpot/issues/11748) (PR: [#11750](https://github.com/penpot/penpot/pull/11750))
- Fix file import recreating circular library references when the exported libraries use each other [#12052](https://github.com/penpot/penpot/issues/12052) (PR: [#12062](https://github.com/penpot/penpot/pull/12062))
- Fix asset redirect responses advertising a cache lifetime of about 1000 days instead of 24 hours (by @ners) [#12761](https://github.com/penpot/penpot/issues/12761) (PR: [#12758](https://github.com/penpot/penpot/pull/12758))
- Fix an internal error on typography tokens due to circular dependencies on other token types [#9580](https://github.com/penpot/penpot/issues/9580)
- Fix the MCP server rejecting request bodies over 100kb (express.json default) with PayloadTooLargeError (by @GitNimay) [#12080](https://github.com/penpot/penpot/issues/12080) (PR: [#12450](https://github.com/penpot/penpot/pull/12450))
- Fix library component, color and typography counters showing zero when opening files from the dashboard (by @lukel97) [#12826](https://github.com/penpot/penpot/issues/12826) (PR: [#12787](https://github.com/penpot/penpot/pull/12787))
- Fix workspace crash with Maximum call stack size exceeded when applying a library update [#12576](https://github.com/penpot/penpot/issues/12576) (PR: [#12661](https://github.com/penpot/penpot/pull/12661))
- Fix canvas rendering an imported shape without its hole when the SVG fill rule is evenodd [#11947](https://github.com/penpot/penpot/issues/11947) (PR: [#12006](https://github.com/penpot/penpot/pull/12006))
- Fix Plugin API component.instance() and createShapeFromSvg() nesting the new shape into whatever board is at (0, 0) of the current page (by @makesomethingshit) [#12086](https://github.com/penpot/penpot/issues/12086) (PR: [#12499](https://github.com/penpot/penpot/pull/12499))
- Fix view mode breaking zoom, board navigation, and comments when the URL has no page id [#12684](https://github.com/penpot/penpot/issues/12684) (PR: [#12687](https://github.com/penpot/penpot/pull/12687))
- Fix autosave stopping and valid edits being lost when part of a save fails validation [#12553](https://github.com/penpot/penpot/issues/12553) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix dashboard duplicate copying a file without the edits that are still saving [#12603](https://github.com/penpot/penpot/issues/12603) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Alt measurements still showing a shape that was deleted [#12605](https://github.com/penpot/penpot/issues/12605) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Plugin API ignoring changes to a deleted shape without an error [#12606](https://github.com/penpot/penpot/issues/12606) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix typography sample overflowing its row in locales with a wider sample (ko, ar) (by @desty) [#11486](https://github.com/penpot/penpot/issues/11486) (PR: [#11487](https://github.com/penpot/penpot/pull/11487))
- Fix token not showing when selecting two text layers (by @ShreyashAgare26) [#11924](https://github.com/penpot/penpot/issues/11924) (PR: [#11944](https://github.com/penpot/penpot/pull/11944))
- Fix inner shadow fill not working in the SVG render while it works in WebGL [#11967](https://github.com/penpot/penpot/issues/11967) (PR: [#12141](https://github.com/penpot/penpot/pull/12141))
- Fix @penpot/plugin-styles loading Work Sans from a third-party CDN (by @ump45nose) [#12023](https://github.com/penpot/penpot/issues/12023) (PR: [#12087](https://github.com/penpot/penpot/pull/12087))
- Fix schema error on swatch component for image fills [#12135](https://github.com/penpot/penpot/issues/12135) (PR: [#12136](https://github.com/penpot/penpot/pull/12136))
- Fix worker storing more shape data than needed [#12740](https://github.com/penpot/penpot/issues/12740) (PR: [#12840](https://github.com/penpot/penpot/pull/12840))
- Fix worker not freeing pages [#12738](https://github.com/penpot/penpot/issues/12738) (PR: [#12751](https://github.com/penpot/penpot/pull/12751))
- Fix editing the L value in HSL mode resetting H and S to 0 (by @makesomethingshit) [#12615](https://github.com/penpot/penpot/issues/12615) (PR: [#12629](https://github.com/penpot/penpot/pull/12629))
- Restrict dashboard plugins to the MCP plugin and localhost plugins [#12948](https://github.com/penpot/penpot/issues/12948) (PR: [#12949](https://github.com/penpot/penpot/pull/12949))
- Fix workspace memory growing on every page switch with the WebGL renderer [#13028](https://github.com/penpot/penpot/issues/13028) (PR: [#12877](https://github.com/penpot/penpot/pull/12877))
- Fix workspace showing React error #310 when the team container renders without a team id (by @filipsajdak) [#12728](https://github.com/penpot/penpot/issues/12728) (PR: [#12920](https://github.com/penpot/penpot/pull/12920))
## 2.19.2
### :bug: Bugs fixed
- Fix referential integrity error when copying component variants with nested components across libraries and teams [#10704](https://github.com/penpot/penpot/issues/10704) (PR: [#12928](https://github.com/penpot/penpot/pull/12928))
- Fix migration 0033 inserting nil shape entries for dangling child references that blocked file opening [#12936](https://github.com/penpot/penpot/issues/12936) (PR: [#12979](https://github.com/penpot/penpot/pull/12979))
- Fix switching a variant and adding a component instance taking over 10 seconds in files with many token-styled shapes [#13019](https://github.com/penpot/penpot/issues/13019) (PR: [#13021](https://github.com/penpot/penpot/pull/13021))
- Fix autosave failing after uploading an SVG as a component in the assets panel, and on other mains named unlike their component [#12773](https://github.com/penpot/penpot/issues/12773) (PR: [#12794](https://github.com/penpot/penpot/pull/12794))
- Fix `enable-oidc-registration` not working when password login is disabled [#12964](https://github.com/penpot/penpot/issues/12964) (PR: [#12974](https://github.com/penpot/penpot/pull/12974))
- Fix self-hosted backend opening 60 database connections at startup and keeping them open while idle (by @UsamaSadiq) [#12622](https://github.com/penpot/penpot/issues/12622) (PR: [#13052](https://github.com/penpot/penpot/pull/13052))
## 2.19.1
### :lock: Security fixes
- Fix editor-role team members retrieving the signed invitation token of any pending invitation (https://github.com/penpot/penpot/security/advisories/GHSA-4mmc-j8rh-74pf) (PR: [#12892](https://github.com/penpot/penpot/pull/12892))
- Fix share-link page-scope escape in `get-profiles-for-file-comments` leaking the identity of comment authors outside the link's scope (https://github.com/penpot/penpot/security/advisories/GHSA-89cq-3wv7-987r) (PR: [#12888](https://github.com/penpot/penpot/pull/12888))
- Fix missing authorization and unbounded array in mark-all-threads-as-read that let any authenticated user write the status of inaccessible threads (https://github.com/penpot/penpot/security/advisories/GHSA-j4vc-cmf6-jhr7) (PR: [#12894](https://github.com/penpot/penpot/pull/12894))
- Fix incomplete SVG sanitizer escaping that let a "sanitized" SVG carry live script (CVE-2026-105694) (https://github.com/penpot/penpot/security/advisories/GHSA-qwhj-rchj-58mg) (PR: [#12887](https://github.com/penpot/penpot/pull/12887))
### :bug: Bugs fixed
- Fix component instance (variant) inside a layout resetting to its original size after a shared library update of an unrelated component [#12670](https://github.com/penpot/penpot/issues/12670) (PR: [#12851](https://github.com/penpot/penpot/pull/12851))
## 2.19.0
### :boom: Breaking changes & Deprecations
#### Migrate legacy file payloads to file_data and delete orphan revisions
File payloads now live in one place: every file state and snapshot is kept in `file_data`, and the orphan `file_change` rows that kept old payloads alive forever are removed. No read path reached those rows and nothing ever collected them, so the upgrade also reclaims the database and TOAST space they held. There is no API or UI change.
On small and medium instances this happens automatically on boot and there is nothing to do. On large instances (millions of snapshots, hundreds of GB in TOAST) the work does not fit in the startup transaction: run it by hand **before upgrading to 2.19.0** following the [large-instance runbook](https://github.com/penpot/penpot/blob/staging/operations/2.19.0-migrate-legacy-file-data.md). The runbook tells you whether your instance needs it, keeps the instance online while the data moves, and verifies the result before you deploy.
[#12784](https://github.com/penpot/penpot/issues/12784) (PR: [#12785](https://github.com/penpot/penpot/pull/12785))
#### Enable the Admin Console by default
The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed. Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)).
[#12807](https://github.com/penpot/penpot/issues/12807) (PR: [#12475](https://github.com/penpot/penpot/pull/12475))
### :rocket: Epics and highlights ### :rocket: Epics and highlights
@ -74,6 +252,38 @@
- Fix thumbnails failing on older Chromium browsers and each thumbnail rendering several times [#11878](https://github.com/penpot/penpot/issues/11878) (PR: [#11879](https://github.com/penpot/penpot/pull/11879)) - Fix thumbnails failing on older Chromium browsers and each thumbnail rendering several times [#11878](https://github.com/penpot/penpot/issues/11878) (PR: [#11879](https://github.com/penpot/penpot/pull/11879))
- Fix internal error when opening the libraries modal [#11943](https://github.com/penpot/penpot/issues/11943) (PR: [#11946](https://github.com/penpot/penpot/pull/11946)) - Fix internal error when opening the libraries modal [#11943](https://github.com/penpot/penpot/issues/11943) (PR: [#11946](https://github.com/penpot/penpot/pull/11946))
- Fix Plugin API `connectLibrary` linking unpublished files as libraries [#12001](https://github.com/penpot/penpot/issues/12001) (PR: [#12002](https://github.com/penpot/penpot/pull/12002)) - Fix Plugin API `connectLibrary` linking unpublished files as libraries [#12001](https://github.com/penpot/penpot/issues/12001) (PR: [#12002](https://github.com/penpot/penpot/pull/12002))
- Fix incorrect padding at the top of the tokens and themes section [#9585](https://github.com/penpot/penpot/issues/9585) (PR: [#12422](https://github.com/penpot/penpot/pull/12422))
- Fix token tree path handling for repeated and nested segments (by @makesomethingshit) [#11588](https://github.com/penpot/penpot/issues/11588) (PR: [#11591](https://github.com/penpot/penpot/pull/11591))
- Fix comment input duplicating text when confirming a Japanese IME composition with Enter [#11757](https://github.com/penpot/penpot/issues/11757) (PR: [#11768](https://github.com/penpot/penpot/pull/11768))
- Fix file not saving after deleting a variant [#11758](https://github.com/penpot/penpot/issues/11758) (PR: [#12216](https://github.com/penpot/penpot/pull/12216))
- Fix component copies not syncing when a shape is moved out of a main component and another change follows at once [#11978](https://github.com/penpot/penpot/issues/11978) (PR: [#11981](https://github.com/penpot/penpot/pull/11981))
- Fix MCP plugin showing a version mismatch warning on every patch release (by @robertkodra) [#12007](https://github.com/penpot/penpot/issues/12007) (PR: [#12009](https://github.com/penpot/penpot/pull/12009))
- Fix workspace crash when switching to the path tool while drawing a shape [#12100](https://github.com/penpot/penpot/issues/12100) (PR: [#12101](https://github.com/penpot/penpot/pull/12101))
- Fix referential integrity error when swapping a component [#12144](https://github.com/penpot/penpot/issues/12144) (PR: [#12333](https://github.com/penpot/penpot/pull/12333))
- Fix workspace saving text shapes on every page open when a custom font is missing [#12151](https://github.com/penpot/penpot/issues/12151) (PR: [#12164](https://github.com/penpot/penpot/pull/12164))
- Fix swapped nested components losing their absolute position after a library update [#12176](https://github.com/penpot/penpot/issues/12176) (PR: [#12556](https://github.com/penpot/penpot/pull/12556))
- Fix theme selector showing the editor message when the tokens source is a library [#12177](https://github.com/penpot/penpot/issues/12177) (PR: [#12203](https://github.com/penpot/penpot/pull/12203))
- Fix untranslated texts in the This file tab of the libraries modal [#12178](https://github.com/penpot/penpot/issues/12178) (PR: [#12204](https://github.com/penpot/penpot/pull/12204))
- Fix workspace staying in the saving state and losing unsaved edits when a save request gets no answer [#12222](https://github.com/penpot/penpot/issues/12222) (PR: [#12223](https://github.com/penpot/penpot/pull/12223))
- Fix scrub cursor showing on numeric inputs with mixed values in the sidebar [#12227](https://github.com/penpot/penpot/issues/12227) (PR: [#12229](https://github.com/penpot/penpot/pull/12229))
- Fix internal error when copying a swapped nested component [#12289](https://github.com/penpot/penpot/issues/12289) (PR: [#12434](https://github.com/penpot/penpot/pull/12434))
- Show an error when uploading a custom font fails [#12304](https://github.com/penpot/penpot/issues/12304) (PR: [#12315](https://github.com/penpot/penpot/pull/12315))
- Fix React warning about the unknown `tooltipClass` prop in layout-item size inputs [#12311](https://github.com/penpot/penpot/issues/12311) (PR: [#12329](https://github.com/penpot/penpot/pull/12329))
- Fix guides inside a board still being interactive [#12322](https://github.com/penpot/penpot/issues/12322) (PR: [#12460](https://github.com/penpot/penpot/pull/12460))
- Fix pasted text not being visible in an empty text box until it is resized [#12438](https://github.com/penpot/penpot/issues/12438) (PR: [#12859](https://github.com/penpot/penpot/pull/12859))
- Fix files failing referential integrity validation staying unsaveable in every session [#12464](https://github.com/penpot/penpot/issues/12464) (PR: [#12524](https://github.com/penpot/penpot/pull/12524), [#12662](https://github.com/penpot/penpot/pull/12662))
- Fix unicode characters not rendering in text layers with the new render engine [#12469](https://github.com/penpot/penpot/issues/12469) (PR: [#12471](https://github.com/penpot/penpot/pull/12471))
- Fix text image fills staying blank when media loads after text layout [#12563](https://github.com/penpot/penpot/issues/12563) (PR: [#12564](https://github.com/penpot/penpot/pull/12564))
- Fix swapped nested icons moving outside the layout after a library update [#12657](https://github.com/penpot/penpot/issues/12657) (PR: [#12716](https://github.com/penpot/penpot/pull/12716))
- Fix garbled bullets before the element counts in the libraries modal [#12677](https://github.com/penpot/penpot/issues/12677) (PR: [#12678](https://github.com/penpot/penpot/pull/12678))
- Fix token dropdown of numeric inputs showing the scrub cursor [#12679](https://github.com/penpot/penpot/issues/12679) (PR: [#12680](https://github.com/penpot/penpot/pull/12680))
- Fix variant property names and values accepting `,` and `=` [#12683](https://github.com/penpot/penpot/issues/12683) (PR: [#12688](https://github.com/penpot/penpot/pull/12688))
- Fix changing a variant property from another page leaving the main component label out of date [#12692](https://github.com/penpot/penpot/issues/12692) (PR: [#12693](https://github.com/penpot/penpot/pull/12693))
- Fix `:referential-integrity` errors when moving variant mains in or out of a set [#12696](https://github.com/penpot/penpot/issues/12696) (PR: [#12706](https://github.com/penpot/penpot/pull/12706))
- Fix `:referential-integrity` errors when creating, combining or renaming variant sets [#12734](https://github.com/penpot/penpot/issues/12734) (PR: [#12779](https://github.com/penpot/penpot/pull/12779))
- Fix save errors after masking a variant with Ctrl+M [#12748](https://github.com/penpot/penpot/issues/12748) (PR: [#12749](https://github.com/penpot/penpot/pull/12749))
- Fix icons in component instances not being shown on the canvas [#12752](https://github.com/penpot/penpot/issues/12752)
- Fix upload endpoints accepting non-multipart upload paths and validation error bodies being truncated [#12896](https://github.com/penpot/penpot/issues/12896) (PR: [#12067](https://github.com/penpot/penpot/pull/12067))
### :sparkles: New features & Enhancements ### :sparkles: New features & Enhancements
@ -89,7 +299,7 @@
- Apply tokens to stroke per side [#9842](https://github.com/penpot/penpot/issues/9842) (PR: [#10913](https://github.com/penpot/penpot/pull/10913)) - Apply tokens to stroke per side [#9842](https://github.com/penpot/penpot/issues/9842) (PR: [#10913](https://github.com/penpot/penpot/pull/10913))
- Add individual stroke widths per side (top, right, bottom, left) [#9854](https://github.com/penpot/penpot/issues/9854) (PR: [#10648](https://github.com/penpot/penpot/pull/10648)) - Add individual stroke widths per side (top, right, bottom, left) [#9854](https://github.com/penpot/penpot/issues/9854) (PR: [#10648](https://github.com/penpot/penpot/pull/10648))
- Expand the token tree right after importing it (by @AKnassa) [#9819](https://github.com/penpot/penpot/issues/9819) (PR: [#10664](https://github.com/penpot/penpot/pull/10664)) - Expand the token tree right after importing it (by @AKnassa) [#9819](https://github.com/penpot/penpot/issues/9819) (PR: [#10664](https://github.com/penpot/penpot/pull/10664))
- Update the selection size badge live during resize, replacing the resize tooltip [#10259](https://github.com/penpot/penpot/issues/10259) - Update the selection size badge live during resize, replacing the resize tooltip [#10259](https://github.com/penpot/penpot/issues/10259) (PR: [#11884](https://github.com/penpot/penpot/pull/11884))
- Add color variants, rotation correction, viewport flip and hide-when-too-small to the selection size badge [#10264](https://github.com/penpot/penpot/issues/10264) (PR: [#11884](https://github.com/penpot/penpot/pull/11884)) - Add color variants, rotation correction, viewport flip and hide-when-too-small to the selection size badge [#10264](https://github.com/penpot/penpot/issues/10264) (PR: [#11884](https://github.com/penpot/penpot/pull/11884))
- Support single-file SVG export with the WebGL render [#10546](https://github.com/penpot/penpot/issues/10546) - Support single-file SVG export with the WebGL render [#10546](https://github.com/penpot/penpot/issues/10546)
- Persist the hide resolved comments preference and separate the filter options in the comments menu [#10686](https://github.com/penpot/penpot/issues/10686) (PR: [#10694](https://github.com/penpot/penpot/pull/10694)) - Persist the hide resolved comments preference and separate the filter options in the comments menu [#10686](https://github.com/penpot/penpot/issues/10686) (PR: [#10694](https://github.com/penpot/penpot/pull/10694))
@ -113,13 +323,36 @@
- Make "Fix when scrolling" work in the WebGL render [#11684](https://github.com/penpot/penpot/issues/11684) (PR: [#11752](https://github.com/penpot/penpot/pull/11752)) - Make "Fix when scrolling" work in the WebGL render [#11684](https://github.com/penpot/penpot/issues/11684) (PR: [#11752](https://github.com/penpot/penpot/pull/11752))
- Complete Catalan translations [#11739](https://github.com/penpot/penpot/issues/11739) (PR: [#11741](https://github.com/penpot/penpot/pull/11741)) - Complete Catalan translations [#11739](https://github.com/penpot/penpot/issues/11739) (PR: [#11741](https://github.com/penpot/penpot/pull/11741))
- Remove the optional client-provided id from creation RPC endpoints [#11783](https://github.com/penpot/penpot/issues/11783) (PR: [#11784](https://github.com/penpot/penpot/pull/11784)) - Remove the optional client-provided id from creation RPC endpoints [#11783](https://github.com/penpot/penpot/issues/11783) (PR: [#11784](https://github.com/penpot/penpot/pull/11784))
- Allow color tokens for the canvas background [#11791](https://github.com/penpot/penpot/issues/11791) (PR: [#11923](https://github.com/penpot/penpot/pull/11923)) - Allow color tokens for the canvas background [#11791](https://github.com/penpot/penpot/issues/11791) (PR: [#11923](https://github.com/penpot/penpot/pull/11923), [#11950](https://github.com/penpot/penpot/pull/11950))
- Show a "no assets yet" message for empty files in the libraries modal [#11811](https://github.com/penpot/penpot/issues/11811) (PR: [#11818](https://github.com/penpot/penpot/pull/11818)) - Show a "no assets yet" message for empty files in the libraries modal [#11811](https://github.com/penpot/penpot/issues/11811) (PR: [#11818](https://github.com/penpot/penpot/pull/11818))
- Upgrade the MCP SDK to v2 and adopt the new protocol version [#11827](https://github.com/penpot/penpot/issues/11827) (PR: [#11841](https://github.com/penpot/penpot/pull/11841)) - Upgrade the MCP SDK to v2 and adopt the new protocol version [#11827](https://github.com/penpot/penpot/issues/11827) (PR: [#11841](https://github.com/penpot/penpot/pull/11841))
- Remove legacy SSE support from the MCP server [#11846](https://github.com/penpot/penpot/issues/11846) (PR: [#11841](https://github.com/penpot/penpot/pull/11841)) - Remove legacy SSE support from the MCP server [#11846](https://github.com/penpot/penpot/issues/11846) (PR: [#11841](https://github.com/penpot/penpot/pull/11841))
- Delete selected path handles with Backspace or Delete [#11858](https://github.com/penpot/penpot/issues/11858) (PR: [#11896](https://github.com/penpot/penpot/pull/11896)) - Delete selected path handles with Backspace or Delete [#11858](https://github.com/penpot/penpot/issues/11858) (PR: [#11896](https://github.com/penpot/penpot/pull/11896))
- Use the tooltip component in the token pill info [#11886](https://github.com/penpot/penpot/issues/11886) (PR: [#11900](https://github.com/penpot/penpot/pull/11900)) - Use the tooltip component in the token pill info [#11886](https://github.com/penpot/penpot/issues/11886) (PR: [#11900](https://github.com/penpot/penpot/pull/11900))
- Hide the duplicate cursor when alt-dragging a shape that cannot be duplicated (by @0xTHAC0) [#11905](https://github.com/penpot/penpot/issues/11905) (PR: [#11195](https://github.com/penpot/penpot/pull/11195)) - Hide the duplicate cursor when alt-dragging a shape that cannot be duplicated (by @0xTHAC0) [#11905](https://github.com/penpot/penpot/issues/11905) (PR: [#11195](https://github.com/penpot/penpot/pull/11195))
- Document the team auto-add behavior with `disable-email-verification` [#11376](https://github.com/penpot/penpot/issues/11376) (PR: [#11425](https://github.com/penpot/penpot/pull/11425))
- Make the workspace recover from interrupted saves and show a proper error instead of the internal error page [#11799](https://github.com/penpot/penpot/issues/11799)
- Add component management operations to the Plugin API [#12301](https://github.com/penpot/penpot/issues/12301) (PR: [#12303](https://github.com/penpot/penpot/pull/12303))
- Record the imported file ids in the import-binfile audit event [#12485](https://github.com/penpot/penpot/issues/12485) (PR: [#12491](https://github.com/penpot/penpot/pull/12491))
## 2.18.7
### :bug: Bugs fixed
- Fix binfile import accepting library relations outside the package and self-relations [#12895](https://github.com/penpot/penpot/issues/12895) (PR: [#12862](https://github.com/penpot/penpot/pull/12862))
## 2.18.6
### :bug: Bugs fixed
- Fix upload endpoints storing server files when the path points outside the temp dir [#12771](https://github.com/penpot/penpot/issues/12771) (PR: [#12772](https://github.com/penpot/penpot/pull/12772))
## 2.18.5
### :bug: Bugs fixed
- Fix the Admin Console failing on every URL on Docker deployments by reverting the subpath support [#12358](https://github.com/penpot/penpot/pull/12358)
## 2.18.4 ## 2.18.4
### :bug: Bugs fixed ### :bug: Bugs fixed

View File

@ -20,6 +20,7 @@ Center](https://help.penpot.app/).
- [Title format](#title) - [Title format](#title)
- [Description](#description) - [Description](#description)
- [Review process](#review-process) - [Review process](#review-process)
- [Merging](#merging)
- [What we won't accept](#what-we-wont-accept) - [What we won't accept](#what-we-wont-accept)
- [Good first issues](#good-first-issues) - [Good first issues](#good-first-issues)
- [Commit Guidelines](#commit-guidelines) - [Commit Guidelines](#commit-guidelines)
@ -111,8 +112,8 @@ Branch names are not enforced, but we recommend the following:
- Otherwise, use a short, descriptive name with words separated by hyphens - Otherwise, use a short, descriptive name with words separated by hyphens
and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`). and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`).
Since PRs are squash-merged, the branch name does not survive into the Since PRs are squash-merged by default, the branch name does not survive
commit history — what matters is the [PR title](#title). into the commit history — what matters is the [PR title](#title).
### Format ### Format
@ -156,9 +157,27 @@ for more concrete information.
- Address review feedback by **pushing new commits** — do not - Address review feedback by **pushing new commits** — do not
force-push during review, as it breaks comment threads. force-push during review, as it breaks comment threads.
- PRs require at least **one approval** before merge. - PRs require at least **one approval** before merge.
- We use **squash-merge** by default. The PR title becomes the final - We **squash-merge** by default; see [Merging](#merging) for the reason
and for how to ask for a regular merge. The PR title becomes the final
commit message, so follow the [title format](#title) above. commit message, so follow the [title format](#title) above.
### Merging
PRs are **squash-merged** by default. One PR lands as one commit, which keeps
the change atomic: the history stays readable, and `git bisect` never stops
on an intermediate commit that fails the tests or the linter.
This is the default, not a hard rule. If you keep your commits atomic and
self-contained — each one builds, passes the linter and makes sense on its
own — you may ask for a regular merge instead. Put the request in the first
line of the PR description, above everything else:
> **Merge:** do not squash; each commit is self-contained.
Maintainers honor the request when the commits hold up, and squash-merge
otherwise. After a squash merge the PR title becomes the commit subject on
the main branch; see [Title format](#title).
### What we won't accept ### What we won't accept
To save time on both sides, please avoid submitting PRs that: To save time on both sides, please avoid submitting PRs that:
@ -225,8 +244,11 @@ Commit messages must follow this format:
those prefixes to an existing subject) those prefixes to an existing subject)
- **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted) - **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted)
- Separate the subject from the body with a **blank line** - Separate the subject from the body with a **blank line**
- Never add a `Co-authored-by` trailer — it is forbidden and fails - Use `Co-authored-by:` only for a human co-author
`./scripts/check-commit` (`Co-authored-by: Name <email>`). AI providers, agents and bots are not
allowed there; for AI participation add an `AI-assisted-by: <model-name>`
trailer instead. `./scripts/check-commit` rejects the trailer when its value
names one of them.
You can check a commit against these rules with `./scripts/check-commit`. You can check a commit against these rules with `./scripts/check-commit`.
CI runs the same script on every new commit in CI runs the same script on every new commit in

View File

@ -13,9 +13,6 @@
<a href="https://tree.taiga.io/project/penpot/" rel="nofollow"> <a href="https://tree.taiga.io/project/penpot/" rel="nofollow">
<img alt="Managed with Taiga.io" src="https://img.shields.io/badge/managed%20with-TAIGA.io-709f14.svg"> <img alt="Managed with Taiga.io" src="https://img.shields.io/badge/managed%20with-TAIGA.io-709f14.svg">
</a> </a>
<a href="https://gitpod.io/#https://github.com/penpot/penpot" rel="nofollow">
<img alt="Gitpod ready-to-code" src="https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod">
</a>
</p> </p>
<p align="center"> <p align="center">

View File

@ -40,7 +40,6 @@
nrepl/nrepl {:mvn/version "1.8.0"} nrepl/nrepl {:mvn/version "1.8.0"}
org.postgresql/postgresql {:mvn/version "42.7.13"} org.postgresql/postgresql {:mvn/version "42.7.13"}
org.xerial/sqlite-jdbc {:mvn/version "3.53.4.0"}
com.zaxxer/HikariCP {:mvn/version "7.1.0"} com.zaxxer/HikariCP {:mvn/version "7.1.0"}
@ -54,6 +53,8 @@
org.jsoup/jsoup {:mvn/version "1.23.2"} org.jsoup/jsoup {:mvn/version "1.23.2"}
com.j256.simplemagic/simplemagic {:mvn/version "1.17"}
at.yawk.lz4/lz4-java at.yawk.lz4/lz4-java
{:mvn/version "1.12.0"} {:mvn/version "1.12.0"}

View File

@ -4,7 +4,7 @@
"license": "MPL-2.0", "license": "MPL-2.0",
"author": "Kaleidos INC Sucursal en España SL", "author": "Kaleidos INC Sucursal en España SL",
"private": true, "private": true,
"packageManager": "pnpm@12.15.0+sha512.b3951d6d6ff4a9f13dc7cf56cd28d2b8eb37dbcc9c68e52e1e1191873ea9f82a017287ab50baadc5ccc494f036c67c0e47eb13608c2fa36f47711b61ea128e61", "packageManager": "pnpm@12.21.0+sha512.66fd8278ee28935ce951792d34367ff3af5c432a2756445823029533835b8da529bbe0be5130afd56ed0713fb8789e47bad82d93bd5ae2fe54b15562d6276ef0",
"repository": { "repository": {
"type": "git", "type": "git",
"url": "https://github.com/penpot/penpot" "url": "https://github.com/penpot/penpot"

131
backend/pnpm-lock.yaml generated
View File

@ -7,153 +7,162 @@ importers:
configDependencies: {} configDependencies: {}
packageManagerDependencies: packageManagerDependencies:
pnpm: pnpm:
specifier: 12.15.0 specifier: 12.21.0
version: 12.15.0 version: 12.21.0
packages: packages:
'@pnpm/exe.android-arm64@12.15.0': '@pnpm/exe.android-arm64@12.21.0':
resolution: {integrity: sha512-eVCqmscY70LGvgRzu3mi8IBWBFGix+a6mNZoWmXxESsBdeiKEF9xQSuaeel10yn+zI7TXy2jlI63vFLzETk+aQ==} resolution: {integrity: sha512-iaWidxR5B9NUBpwMznCQu1yUpKB6BjTY5BYX2lORj76VsfINdj8Z8XWSzSNrGI8qlxa28Jzdp5fiaspzOaJZlA==}
cpu: [arm64] cpu: [arm64]
os: [android] os: [android]
'@pnpm/exe.android-x64@12.15.0': '@pnpm/exe.android-x64@12.21.0':
resolution: {integrity: sha512-Gx/WbkFcdUFUg8g8AShezcS9EiQxfYvdAm2hfKLO8sfJmJvKOCt5fzlgHWDzmGhoclibgW+tUGZ/KF4F4wOTCA==} resolution: {integrity: sha512-35e5v3wlwjIYxDywq+R6/z8CE4gLOEgRxt18yqz1tMRLmx2mnW+CFJI2Q8+srTNjn7gykXUC0x1k195JNikC5A==}
cpu: [x64] cpu: [x64]
os: [android] os: [android]
'@pnpm/exe.darwin-arm64@12.15.0': '@pnpm/exe.darwin-arm64@12.21.0':
resolution: {integrity: sha512-N0ZykiDWCvqSBJvt71j7cSoBR993FqvYZFtmpCtqvW3pguiI+NAgraLPA5OTwu5mV+/d8hYpVcPktJA3xFDLtA==} resolution: {integrity: sha512-F5OyMX/6zLSowpTZEQRojbCmPS1Pq+fyRaIinslsIe6Glh3qbyQ52u+Fe0eEzSaRuIbWZkXmAAXSWh6Q5Wrlgg==}
cpu: [arm64] cpu: [arm64]
os: [darwin] os: [darwin]
'@pnpm/exe.darwin-x64@12.15.0': '@pnpm/exe.darwin-x64@12.21.0':
resolution: {integrity: sha512-7Ntr2HlBUxzwWOg7NwDxCbRW/3vXuqKO+hcXm+w3SBi9rrHY53li1wLqWVT7Gjxd+lVEHM1QJZo6WJEGFU8C6A==} resolution: {integrity: sha512-EzpvcsaE6hrrgYeEiAFYd2rSgFybYyxXhkAY7K3y6Hg3loVfcyJ81hHczpFzi8lW5RXtUlFECRPZ8ZAMRFQYYg==}
cpu: [x64] cpu: [x64]
os: [darwin] os: [darwin]
'@pnpm/exe.freebsd-x64@12.15.0': '@pnpm/exe.freebsd-x64@12.21.0':
resolution: {integrity: sha512-btOiyL7fL7W4uQGH0aqFai2TNC3IoKcZ7gNHBVbEFZt2dWrnrD0SufL3iGs3fLlmb4OtuU35uf4Zpr9V9Jh9mA==} resolution: {integrity: sha512-wNjuEnmdLks4aayPu/QZWQ5tQM9Jn+odUp/vwQM8BXRYoulkTZhtZTHoD60N6mUdLWef1hgUeUh8llNGEUsimw==}
cpu: [x64] cpu: [x64]
os: [freebsd] os: [freebsd]
'@pnpm/exe.linux-arm64-musl@12.15.0': '@pnpm/exe.linux-arm64-musl@12.21.0':
resolution: {integrity: sha512-pNC38k+O+90FOjbPNSnw0E7apq1PKTmz3fFe1amn+lys/U5kL++uzir8bRiho8ONcyA3d9EU7wdqcY0DKkj+JA==} resolution: {integrity: sha512-Ceit9zn4g5ZqCJtx1fV8eJE7p8aeXMrGZ+Dc01vRqZLIUNgNeI+Ur0ruyqxtsnJx6rGADyXzohumxPLHjaJ5AQ==}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
libc: [musl] libc: [musl]
'@pnpm/exe.linux-arm64@12.15.0': '@pnpm/exe.linux-arm64@12.21.0':
resolution: {integrity: sha512-qML/8aCXslh8td/w4U+HJNqUcZ2+m6EfTbWScIzZRmyPsRFN+xB4LP1U/hymxc1iSJiKtG56HJx2fvUErWFIGw==} resolution: {integrity: sha512-4Oj/iF2TDCe2lcH0n4TupPR5sSdrNLclJB6xBElLMPCUOAvBhdf2rRlSaO5M4erJ6g4Fyy7YTXTpcmfFsKyxEA==}
cpu: [arm64] cpu: [arm64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-ppc64@12.15.0': '@pnpm/exe.linux-ppc64@12.21.0':
resolution: {integrity: sha512-tjRDUUl7RCJ6T/6eoNPxOqkwU4Mzi5RtTXmSAxujyz37OTwwsMU7Ps087Zk4gHcUvRSeyF40ZFsew3vtE+nmUA==} resolution: {integrity: sha512-s2o7rdSDeUTBFsNqxptpZXlTgT6cojmMVUOvTQvFNSv6Easian02ha4tR32fwwD97Au4YzIIjMCwywCF4D82Sw==}
cpu: [ppc64] cpu: [ppc64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-riscv64@12.15.0': '@pnpm/exe.linux-riscv64@12.21.0':
resolution: {integrity: sha512-bF1uBWEfbwaeryz6lIiaJ8AWD082otZfvzGX5l+6LZRpRwUEpJHUsJrORaYP5/cNMv5ZR2p6TvvcJcQzwtDzmw==} resolution: {integrity: sha512-EYv3GzxeBTITCuM3yGHHbpRFpUvyulkXpQ6LsjKAQNL3iaba1E6h9iDmlouLb3Io4AMMsfdk0/uG6poQ41uewA==}
cpu: [riscv64] cpu: [riscv64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-s390x@12.15.0': '@pnpm/exe.linux-s390x@12.21.0':
resolution: {integrity: sha512-zKMhIzsexYMToeLmqV5jPsvV/f9vwsJb2kUH2NFYzalbxImeBrlA4kL7GwF8gD4pagV6jYYcJb8RjbZIdjHptg==} resolution: {integrity: sha512-8VXGNGVNEfs7pL9GTZOzFxjFEnMt4WL3rLOADR5/kPQ7ROTV9GfpVI2DPO/zArXFSkvfc9Zad/WypHLbvzoaTQ==}
cpu: [s390x] cpu: [s390x]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.linux-x64-musl@12.15.0': '@pnpm/exe.linux-x64-musl@12.21.0':
resolution: {integrity: sha512-A9J0Jg0Rdx+qQU4qXtwyST5tto9IOu1zi4fDk35nbc6DWahggbcKtFvZPKFwL2qgUXGXOX//hXv6z/sp6emO8A==} resolution: {integrity: sha512-MnC5vpIXYubhDZCgCwlfJMg7Cg6HupN2IuouT9gD/tUmOfVdQYqeyYccWbthyo/Jeto8de1oQlxeRN1+1Hl2DQ==}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
libc: [musl] libc: [musl]
'@pnpm/exe.linux-x64@12.15.0': '@pnpm/exe.linux-x64@12.21.0':
resolution: {integrity: sha512-yc+QlZR+9+nr29J4fRzhMoapgxTBDndq/+peyoq1XUufnQFnfbXUMFudDIKzHi545linGqmOmXfn0NoufNgcOg==} resolution: {integrity: sha512-8eufCjHXspKbNS/kh8q4Q0gqJoOuvC4Q5pgos6XS8By/nAMtP+jf8F1Cw9er840kfAX+PJ/ts+A9qnLE91rvXQ==}
cpu: [x64] cpu: [x64]
os: [linux] os: [linux]
libc: [glibc] libc: [glibc]
'@pnpm/exe.win32-arm64@12.15.0': '@pnpm/exe.netbsd-x64@12.21.0':
resolution: {integrity: sha512-fPXZxuY1VclkOga9bc53qNCXcegfeoBR8+Ue+AospDwKzuGGSHll0oDxkLfkmpjyRqpe7bH5DyQPYp/xSMaIVA==} resolution: {integrity: sha512-eyh1M8hQZCkLWxuva7S/fc1SbFbgqg0/5di0dgWmyiuc2Io3dbWVihDun8X+XM8zsR5uhIDvvZpbS9aZQbCovg==}
cpu: [x64]
os: [netbsd]
'@pnpm/exe.win32-arm64@12.21.0':
resolution: {integrity: sha512-7xTxxUXo/grRLivSeIl6B21gnsz7ooe5EMPcJmQqwxc8WLJVReN8MzrlHru8bqXfVr4X3+6ZGZscntKenlCBqQ==}
cpu: [arm64] cpu: [arm64]
os: [win32] os: [win32]
'@pnpm/exe.win32-x64@12.15.0': '@pnpm/exe.win32-x64@12.21.0':
resolution: {integrity: sha512-nrd+Tp1We1DGj4DGG5Bea9lZNh9y1PjlLBdEwdInS4D0h0UIDtM7JaGGGDJhNzIwcHASyihWI7HLV+dtG+fhYg==} resolution: {integrity: sha512-9TpkJemVApxJ1Z1w0ZFGN+AuSx4nJOqXSJUqMqDvaXTYwGOzvBiiWdFP9GN0ketiWCQS1Q0/pza2E7I3pP0OBQ==}
cpu: [x64] cpu: [x64]
os: [win32] os: [win32]
pnpm@12.15.0: pnpm@12.21.0:
resolution: {integrity: sha512-s5UdbW/0qfE9x89WzSjSuOs328ycaOUuHhGRhz6p+CoBcoerULqtxczElPA2xnwOR+sTYIwvo29HcRth6hKOYQ==} resolution: {integrity: sha512-Zv2CeO4ok1zpUXktNDZ/869cQyonVkRYIwKVM4NbjaUpu+C+UTCv1W7QcT+4eJ5Hutgtk71a4v5UsVVi1idu8A==}
engines: {node: '>=18'} engines: {node: '>=18'}
hasBin: true hasBin: true
snapshots: snapshots:
'@pnpm/exe.android-arm64@12.15.0': '@pnpm/exe.android-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.android-x64@12.15.0': '@pnpm/exe.android-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.darwin-arm64@12.15.0': '@pnpm/exe.darwin-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.darwin-x64@12.15.0': '@pnpm/exe.darwin-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.freebsd-x64@12.15.0': '@pnpm/exe.freebsd-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-arm64-musl@12.15.0': '@pnpm/exe.linux-arm64-musl@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-arm64@12.15.0': '@pnpm/exe.linux-arm64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-ppc64@12.15.0': '@pnpm/exe.linux-ppc64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-riscv64@12.15.0': '@pnpm/exe.linux-riscv64@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-s390x@12.15.0': '@pnpm/exe.linux-s390x@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-x64-musl@12.15.0': '@pnpm/exe.linux-x64-musl@12.21.0':
optional: true optional: true
'@pnpm/exe.linux-x64@12.15.0': '@pnpm/exe.linux-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.win32-arm64@12.15.0': '@pnpm/exe.netbsd-x64@12.21.0':
optional: true optional: true
'@pnpm/exe.win32-x64@12.15.0': '@pnpm/exe.win32-arm64@12.21.0':
optional: true optional: true
pnpm@12.15.0: '@pnpm/exe.win32-x64@12.21.0':
optional: true
pnpm@12.21.0:
optionalDependencies: optionalDependencies:
'@pnpm/exe.android-arm64': 12.15.0 '@pnpm/exe.android-arm64': 12.21.0
'@pnpm/exe.android-x64': 12.15.0 '@pnpm/exe.android-x64': 12.21.0
'@pnpm/exe.darwin-arm64': 12.15.0 '@pnpm/exe.darwin-arm64': 12.21.0
'@pnpm/exe.darwin-x64': 12.15.0 '@pnpm/exe.darwin-x64': 12.21.0
'@pnpm/exe.freebsd-x64': 12.15.0 '@pnpm/exe.freebsd-x64': 12.21.0
'@pnpm/exe.linux-arm64': 12.15.0 '@pnpm/exe.linux-arm64': 12.21.0
'@pnpm/exe.linux-arm64-musl': 12.15.0 '@pnpm/exe.linux-arm64-musl': 12.21.0
'@pnpm/exe.linux-ppc64': 12.15.0 '@pnpm/exe.linux-ppc64': 12.21.0
'@pnpm/exe.linux-riscv64': 12.15.0 '@pnpm/exe.linux-riscv64': 12.21.0
'@pnpm/exe.linux-s390x': 12.15.0 '@pnpm/exe.linux-s390x': 12.21.0
'@pnpm/exe.linux-x64': 12.15.0 '@pnpm/exe.linux-x64': 12.21.0
'@pnpm/exe.linux-x64-musl': 12.15.0 '@pnpm/exe.linux-x64-musl': 12.21.0
'@pnpm/exe.win32-arm64': 12.15.0 '@pnpm/exe.netbsd-x64': 12.21.0
'@pnpm/exe.win32-x64': 12.15.0 '@pnpm/exe.win32-arm64': 12.21.0
'@pnpm/exe.win32-x64': 12.21.0
--- ---
lockfileVersion: '9.0' lockfileVersion: '9.0'

View File

@ -6,66 +6,74 @@
;; 60s for operations that can be slow (file updates, images, fonts, thumbnails, ;; 60s for operations that can be slow (file updates, images, fonts, thumbnails,
;; snapshots, imports) and 30s for the rest. ;; snapshots, imports) and 30s for the rest.
;; Note: a queue bounds how many waiters pile up, a timeout bounds how long one ;; Note: a queue bounds how many waiters pile up, a timeout bounds how long one
;; waiter waits. Every entry sets a timeout; the by-profile ones also set a ;; waiter waits. Every entry sets a timeout and a queue: a command takes the
;; queue, which is where a pile-up hurts. ;; limits it declares before the root ones, so the queue of a limit is where
;; its own pile-up waits. The root limits bound what runs at the same time,
;; not what waits for it.
;; The production file lives outside this repository: mirror the shape there,
;; not these numbers. It has to list every id a command declares: an id without
;; an entry runs with no limiter at all.
{:update-file/global {:update-file/global
{:permits 20 :timeout 60000} {:permits 20 :queue 50 :timeout 60000}
:update-file/by-profile :update-file/by-profile
{:permits 1 :queue 5 :timeout 60000} {:permits 1 :queue 5 :timeout 60000}
:process-font/global :process-font/global
{:permits 4 :timeout 60000} {:permits 4 :queue 10 :timeout 60000}
:process-font/by-profile :process-font/by-profile
{:permits 1 :timeout 60000} {:permits 1 :queue 5 :timeout 60000}
:process-image/global :process-image/global
{:permits 8 :timeout 60000} {:permits 8 :queue 20 :timeout 60000}
:process-image/by-profile :process-image/by-profile
{:permits 1 :timeout 60000} {:permits 1 :queue 5 :timeout 60000}
:auth/global :auth/global
{:permits 8 :timeout 30000} {:permits 8 :queue 20 :timeout 30000}
:root/global :root/global
{:permits 40 :timeout 30000} {:permits 40 :queue 100 :timeout 30000}
:root/by-profile :root/by-profile
{:permits 10 :queue 30 :timeout 30000} {:permits 10 :queue 30 :timeout 30000}
:file-thumbnail-ops/global :file-thumbnail-ops/global
{:permits 20 :timeout 60000} {:permits 20 :queue 50 :timeout 60000}
:file-thumbnail-ops/by-profile :file-thumbnail-ops/by-profile
{:permits 2 :timeout 60000} {:permits 2 :queue 10 :timeout 60000}
:submit-audit-events/by-profile :submit-audit-events/by-profile
{:permits 1 :queue 3 :timeout 30000} {:permits 1 :queue 3 :timeout 30000}
:restore-file-snapshot/global :restore-file-snapshot/global
{:permits 3 :timeout 60000} {:permits 3 :queue 5 :timeout 60000}
:restore-file-snapshot/by-profile :restore-file-snapshot/by-profile
{:permits 1 :queue 2 :timeout 60000} {:permits 1 :queue 2 :timeout 60000}
:create-file-snapshot/global :create-file-snapshot/global
{:permits 3 :timeout 60000} {:permits 3 :queue 5 :timeout 60000}
:create-file-snapshot/by-profile :create-file-snapshot/by-profile
{:permits 1 :queue 2 :timeout 60000} {:permits 1 :queue 2 :timeout 60000}
:send-user-feedback/global :send-user-feedback/global
{:permits 4 :timeout 30000} {:permits 4 :queue 10 :timeout 30000}
:send-user-feedback/by-profile :send-user-feedback/by-profile
{:permits 1 :queue 3 :timeout 30000} {:permits 1 :queue 3 :timeout 30000}
:import-binfile/global :import-binfile/global
{:permits 4 :timeout 60000} {:permits 4 :queue 10 :timeout 60000}
:import-binfile/by-profile :import-binfile/by-profile
{:permits 1 :queue 2 :timeout 60000} {:permits 1 :queue 2 :timeout 60000}
:profile-plugin-ops/global :create-import-binfile-job/global
{:permits 4 :timeout 30000} {:permits 4 :queue 10 :timeout 60000}
:create-import-binfile-job/by-profile
{:permits 1 :queue 2 :timeout 60000}
:profile-plugin-ops/by-profile :profile-plugin-ops/by-profile
{:permits 1 :queue 2 :timeout 30000}} {:permits 1 :queue 2 :timeout 30000}}

View File

@ -62,13 +62,14 @@ export PENPOT_FLAGS="\
disable-tiered-file-data-storage \ disable-tiered-file-data-storage \
enable-file-validation \ enable-file-validation \
enable-file-schema-validation \ enable-file-schema-validation \
enable-debug-save-faults \
enable-redis-cache \ enable-redis-cache \
enable-link-preview \ enable-link-preview \
enable-subscriptions"; enable-subscriptions";
# Uncomment for nexus integration testing # Uncomment for nexus integration testing
# export PENPOT_FLAGS="$PENPOT_FLAGS enable-audit-log-archive"; # export PENPOT_FLAGS="$PENPOT_FLAGS enable-nexus";
# export PENPOT_AUDIT_LOG_ARCHIVE_URI="http://localhost:6070/api/audit"; # export PENPOT_NEXUS_URI="http://localhost:6070/";
# Default deletion delay for devenv # Default deletion delay for devenv
export PENPOT_DELETION_DELAY="24h" export PENPOT_DELETION_DELAY="24h"
@ -92,6 +93,7 @@ export JAVA_OPTS="\
-XX:+UseShenandoahGC \ -XX:+UseShenandoahGC \
-XX:+UseCompactObjectHeaders \ -XX:+UseCompactObjectHeaders \
-XX:ShenandoahGCMode=generational \ -XX:ShenandoahGCMode=generational \
-XX:NativeMemoryTracking=summary \
-XX:-OmitStackTraceInFastThrow \ -XX:-OmitStackTraceInFastThrow \
--sun-misc-unsafe-memory-access=allow \ --sun-misc-unsafe-memory-access=allow \
--enable-preview \ --enable-preview \

View File

@ -629,7 +629,7 @@
(l/warn :hint "received incomplete user info object (please set correct scopes)" :info info) (l/warn :hint "received incomplete user info object (please set correct scopes)" :info info)
(ex/raise :type :internal (ex/raise :type :internal
:code :incomplete-user-info :code :incomplete-user-info
:hint "inconmplete user info" :hint "incomplete user info"
:info info))) :info info)))
;; If the provider is OIDC, we can proceed to check ;; If the provider is OIDC, we can proceed to check
@ -690,15 +690,113 @@
(assoc :query (u/map->query-string params)))] (assoc :query (u/map->query-string params)))]
(redirect-response uri)))) (redirect-response uri))))
(defn- public-uri
"The configured public URI, parsed, with the app base path normalized with
a trailing slash."
[]
(u/ensure-path-slash (u/uri (cf/get :public-uri))))
(defn- public-uri-base
"The public URI as a string, with the app base path normalized with a
trailing slash. Every URL the server builds starts here, so the app base
path is never lost."
[]
(str (public-uri)))
(defn- redirect-with-organization-sso-error (defn- redirect-with-organization-sso-error
[{:keys [dest-url organization-id organization-name]}] [{:keys [dest-url organization-id organization-name]}]
(-> (str (or dest-url (cf/get :public-uri))) (-> (str (or dest-url (public-uri-base)))
(u/append-query-param :sso-error true) (u/append-query-param :sso-error true)
(u/append-query-param :organization-id organization-id) (u/append-query-param :organization-id organization-id)
(cond-> organization-name (cond-> organization-name
(u/append-query-param :organization-name organization-name)) (u/append-query-param :organization-name organization-name))
(redirect-response))) (redirect-response)))
(defn- routable-params?
"True when `params` are params of a route: at least one, and no key carries
a scheme, a path or a fragment. `screen=workspace&team-id=…` and the root
params of the application (`template=…`) qualify; `https://evil.example/`
does not."
[params]
(and (seq params)
(every? #(nil? (re-find #"[/:?#]" (name %))) (keys params))))
(defn build-dest-url
"Return the absolute URL for `query-string` under the current public URI.
The query string is the route of the client (`screen=…&params`, and the root
of the application may carry only its params, like `template=…`): it does not
carry the `?` that separates it from the path, which is put back here, so the
destination is always built by the server and the client only picks the route.
A value that still comes with the separator is trimmed, so a client bundled
before that change keeps its destination.
The params are parsed and rendered back, so a value that is not a route never
becomes a destination: it is ignored with a warning and the user lands on the
public URI, which is also where a blank one lands."
[query-string]
(try
(let [base (public-uri-base)
query (str query-string)
route (if (str/starts-with? query "?")
(subs query 1)
query)
params (u/query-string->map route)]
(cond
(str/blank? query)
base
(not (routable-params? params))
(do (l/warn :hint "ignoring a redirect target that is not a route"
:query-string (subs route 0 (min 200 (count route))))
base)
:else
(str base "?" (u/map->query-string params))))
(catch Exception e
;; The destination must never break the SSO flow: a query string the
;; server cannot parse lands on the public URI instead.
(l/warn :hint "could not build the redirect target" :cause e)
(public-uri-base))))
(defn- same-origin?
"True when `uri` has the same origin as `base`: scheme, host (case
insensitive) and port, with the default port made explicit."
[uri base]
(let [default-port (fn [scheme] (if (= "https" (str/lower (str scheme))) 443 80))
port (fn [u] (or (:port u) (default-port (:scheme u))))]
(and (= (str/lower (str (:scheme uri)))
(str/lower (str (:scheme base))))
(= (str/lower (str (:host uri)))
(str/lower (str (:host base))))
(= (port uri) (port base)))))
(defn- under-base-path?
"True when the path of `uri` is under the base path of `base`."
[uri base]
(str/starts-with? (str (:path uri)) (:path base)))
(defn- trusted-dest-url
"Return `dest-url` when it points inside the current public URI, nil
otherwise.
The destination is built by the server (see `build-dest-url`), but state
tokens minted before that was the case may carry any URL, so the callback
checks the destination before redirecting to it. The check compares the
parsed origin, so neither a userinfo prefix
(`https://penpot.example.com@evil.example/`) nor a look-alike host
(`https://penpot.example.com.evil.example/`) can pass, and it requires
the path to be under the base path of the public URI, so the destination
stays inside the application."
[dest-url]
(when (some? dest-url)
(let [dest (try (u/uri (str dest-url)) (catch Exception _ nil))
base (public-uri)]
(when (and (some? dest)
(same-origin? dest base)
(under-base-path? dest base))
(str dest-url)))))
(defn- redirect-to-register (defn- redirect-to-register
[cfg info provider] [cfg info provider]
(let [info (assoc info (let [info (assoc info
@ -915,11 +1013,18 @@
(defn build-organization-sso-auth-redirect-uri (defn build-organization-sso-auth-redirect-uri
"Build the OIDC authorization redirect URI for an organization SSO config. "Build the OIDC authorization redirect URI for an organization SSO config.
Raises if the config is incomplete or OIDC discovery fails." Raises if the config is incomplete or OIDC discovery fails.
[cfg sso & {:keys [dest-url organization-id provider]}]
`query-string` is the route of the client (`screen=…&params`, the query
string proper, without its separator): the destination is built here, so it
can never point outside the public URI.
`profile-id` is the profile that started the flow; it travels in the
state token so the callback only grants the organization SSO entry to that
very session."
[cfg sso & {:keys [query-string organization-id provider profile-id]}]
(let [organization-id (or organization-id (:organization-id sso)) (let [organization-id (or organization-id (:organization-id sso))
issuer (organization-sso-discovery-uri sso) issuer (organization-sso-discovery-uri sso)
dest-url (or dest-url (str (cf/get :public-uri)))] dest-url (build-dest-url query-string)]
(when-not issuer (when-not issuer
(raise-invalid-sso-config (raise-invalid-sso-config
:hint "missing issuer" :hint "missing issuer"
@ -929,6 +1034,7 @@
state-token (tokens/generate cfg {:iss "oidc" state-token (tokens/generate cfg {:iss "oidc"
:dest-url dest-url :dest-url dest-url
:organization-id organization-id :organization-id organization-id
:profile-id profile-id
:issuer issuer :issuer issuer
:exp (ct/in-future "4h")})] :exp (ct/in-future "4h")})]
(build-auth-redirect-uri oidc-provider state-token)) (build-auth-redirect-uri oidc-provider state-token))
@ -1024,23 +1130,39 @@
(defn- organization-sso-callback-handler (defn- organization-sso-callback-handler
"Handle the organization-SSO branch of the OIDC callback: state carries "Handle the organization-SSO branch of the OIDC callback: state carries
:dest-url — exchange the authorization code with the OIDC provider to :dest-url — exchange the authorization code with the OIDC provider to
verify authentication actually occurred, then redirect back to dest-url." verify authentication actually occurred, then redirect back to dest-url.
The state also carries the :profile-id that started the flow: only that
session is granted the organization SSO entry, so a state prepared by
someone else cannot mark the session of the one who completes it."
[cfg request state code] [cfg request state code]
(let [dest-url (:dest-url state)] (let [dest-url (trusted-dest-url (:dest-url state))]
(try (try
(let [organization-id (:organization-id state) (let [organization-id (:organization-id state)
sso (nitrate/call cfg :get-organization-sso {:organization-id organization-id}) sso (nitrate/call cfg :get-organization-sso {:organization-id organization-id})
provider (prepare-organization-sso-provider cfg sso) provider (prepare-organization-sso-provider cfg sso)
_info (get-info cfg provider state code) _info (get-info cfg provider state code)
session (session/get-session request) session (session/get-session request)
exp (ct/in-future {:hours 4})] exp (ct/in-future {:hours 4})
(when (and session organization-id) ;; A state minted before the profile claim existed never
(let [props (-> (or (:props session) {}) ;; matches, and neither does the session of another profile.
(update :sso assoc organization-id exp))] same-profile? (and (some? session)
(session/update-session (::session/manager cfg) (assoc session :props props)))) (some? (:profile-id state))
(submit-organization-sso-auth-event (= (:profile-id state) (:profile-id session)))]
cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded") (if (and (some? session) (not same-profile?))
(redirect-response dest-url)) (submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-failed"
:failure-reason "session-mismatch")
(do
(when (and session organization-id)
(let [props (-> (or (:props session) {})
(update :sso assoc organization-id exp))]
(session/update-session (::session/manager cfg) (assoc session :props props))
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-token-issued")))
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded")))
(redirect-response (or dest-url (public-uri-base))))
(catch Throwable cause (catch Throwable cause
(let [{:keys [code]} (ex-data cause)] (let [{:keys [code]} (ex-data cause)]
(binding [l/*context* (errors/request->context request)] (binding [l/*context* (errors/request->context request)]
@ -1058,10 +1180,36 @@
(let [organization-id (:organization-id state) (let [organization-id (:organization-id state)
organization-name (:name (nitrate/call cfg :get-organization-summary {:organization-id organization-id}))] organization-name (:name (nitrate/call cfg :get-organization-summary {:organization-id organization-id}))]
(redirect-with-organization-sso-error (redirect-with-organization-sso-error
{:dest-url dest-url {:dest-url (or dest-url (public-uri-base))
:organization-id organization-id :organization-id organization-id
:organization-name organization-name})))))) :organization-name organization-name}))))))
(def oidc-provider-types
"Backends authenticating through an OIDC-compatible provider.
Only these backends take the SSO registration gate."
#{"oidc" "google" "github" "gitlab"})
(defn sso-registration-allowed?
"SSO registration is allowed when public registration or explicit
OIDC registration is enabled. Shared by the callback gate and the
`register-profile` token gate — keep one definition."
[]
(or (contains? cf/flags :registration)
(contains? cf/flags :oidc-registration)))
(defn- valid-invitation-for?
"Returns true when `invitation-token` is a verifiable
`:team-invitation` JWT issued for `email`. Never throws: a garbage
token means there is no valid invitation."
[cfg invitation-token email]
(try
(boolean
(when (and (some? invitation-token) (some? email))
(let [invitation (tokens/verify cfg {:token invitation-token :iss :team-invitation})]
(= (profile/clean-email (:member-email invitation))
(profile/clean-email email)))))
(catch Throwable _ false)))
(defn- callback-handler (defn- callback-handler
[cfg {:keys [params] :as request}] [cfg {:keys [params] :as request}]
(if-let [error (get params :error)] (if-let [error (get params :error)]
@ -1094,8 +1242,8 @@
(redirect-with-error "email-domain-not-allowed") (redirect-with-error "email-domain-not-allowed")
:else :else
(if (or (contains? cf/flags :registration) (if (or (sso-registration-allowed?)
(contains? cf/flags :oidc-registration)) (valid-invitation-for? cfg (:invitation-token info) (:email info)))
(redirect-to-register cfg info provider) (redirect-to-register cfg info provider)
(redirect-with-error "registration-disabled"))) (redirect-with-error "registration-disabled")))
@ -1113,11 +1261,15 @@
:else :else
(let [sxf (session/create-fn cfg profile info) (let [sxf (session/create-fn cfg profile info)
token (or (:invitation-token info) ;; The session token is always an :auth one; the
(tokens/generate cfg ;; invitation token, when present, travels inside its
{:iss :auth ;; claims so the frontend goes through the regular
:exp (ct/in-future "15m") ;; token verification path and then `logged-in`.
:profile-id (:id profile)})) token (tokens/generate cfg
{:iss :auth
:exp (ct/in-future "15m")
:profile-id (:id profile)
:invitation-token (:invitation-token info)})
;; If proceed, update profile on the database ;; If proceed, update profile on the database
profile (update-profile-with-info cfg profile info) profile (update-profile-with-info cfg profile info)

View File

@ -9,7 +9,7 @@
for recently imported shapes." for recently imported shapes."
(:require (:require
[app.common.data :as d] [app.common.data :as d]
[app.common.svg :as csvg] [app.common.geom.matrix :as gmt]
[app.common.types.shape :as cts] [app.common.types.shape :as cts]
[app.common.uuid :as uuid])) [app.common.uuid :as uuid]))
@ -106,14 +106,11 @@
:reverse-column :column-reverse :reverse-column :column-reverse
dir)))) dir))))
(defn- fix-svg-attrs (defn- fix-svg-transform
"The json reader of the binfile rewrites every key of every entry to "The json reader decodes `:svg-transform` as a plain map; the svg defs
kebab-case, but `:svg-attrs` keys are react prop names and are stored renderer multiplies it as a matrix, so turn it back into one."
in camelCase. `attrs->props` is the transform the svg import path
already applies to them, so running it again restores the names; it
is idempotent, so shapes that come in correct are left untouched."
[shape] [shape]
(d/update-when shape :svg-attrs csvg/attrs->props)) (d/update-when shape :svg-transform gmt/coerce))
(defn clean-shape-post-decode (defn clean-shape-post-decode
"A shape procesor that expected to be executed after schema decoding "A shape procesor that expected to be executed after schema decoding
@ -123,7 +120,7 @@
(fix-shape-shadow-color) (fix-shape-shadow-color)
(fix-root-shape) (fix-root-shape)
(fix-legacy-flex-dir) (fix-legacy-flex-dir)
(fix-svg-attrs))) (fix-svg-transform)))
(defn- fix-container (defn- fix-container
[container] [container]

View File

@ -24,7 +24,6 @@
[app.common.weak :as weak] [app.common.weak :as weak]
[app.config :as cf] [app.config :as cf]
[app.db :as db] [app.db :as db]
[app.db.sql :as sql]
[app.features.fdata :as fdata] [app.features.fdata :as fdata]
[app.features.file-migrations :as fmigr] [app.features.file-migrations :as fmigr]
[app.loggers.audit :as-alias audit] [app.loggers.audit :as-alias audit]
@ -43,7 +42,6 @@
(def ^:dynamic *state* nil) (def ^:dynamic *state* nil)
(def ^:dynamic *options* nil) (def ^:dynamic *options* nil)
(def ^:dynamic *reference-file* nil)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; DEFAULTS ;; DEFAULTS
@ -74,11 +72,11 @@
;; Maximum total decompressed size allowed for all JSON/text zip entries ;; Maximum total decompressed size allowed for all JSON/text zip entries
;; combined within a single import job: 500MiB. Bounds the case where many ;; combined within a single import job: 500MiB. Bounds the case where many
;; entries, each individually under default-max-text-entry-size, still sum ;; entries, each individually under default-max-text-entry-size, still sum
;; to an unreasonable total. Set well above the 200MiB it replaces because ;; to an unreasonable total. Set this high because legitimate giant
;; legitimate giant exports exist: hundreds of thousands of KB-sized ;; exports exist: hundreds of thousands of KB-sized entries (pages,
;; entries (pages, shapes) that never trip the per-entry cap but sum past ;; shapes) that never trip the per-entry cap but sum past 400MiB. The
;; 400MiB. The per-entry cap still bounds single-entry DEFLATE ;; per-entry cap still bounds single-entry DEFLATE amplification, so this
;; amplification, so this only moves the cumulative ceiling. ;; only moves the cumulative ceiling.
(def ^:const default-max-text-total-size (def ^:const default-max-text-total-size
(* 1024 1024 500)) (* 1024 1024 500))
@ -130,18 +128,12 @@
(defn initial-state (defn initial-state
[] []
{:storage-objects #{} {:storage-objects #{}
:files #{} :files #{}})
:teams #{}
:projects #{}})
(defn collect-storage-objects (defn collect-storage-objects
[state items] [state items]
(update state :storage-objects into xf-map-media-id items)) (update state :storage-objects into xf-map-media-id items))
(defn collect-summary
[state key items]
(update state key into xf-map-media-id items))
(defn lookup-index (defn lookup-index
[id] [id]
(when id (when id
@ -159,7 +151,7 @@
[index obj & attrs] [index obj & attrs]
(reduce (fn [index attr-fn] (reduce (fn [index attr-fn]
(let [old-id (attr-fn obj) (let [old-id (attr-fn obj)
new-id (if (::overwrite *options*) old-id (uuid/next))] new-id (uuid/next)]
(assoc index old-id new-id))) (assoc index old-id new-id)))
index index
attrs)) attrs))
@ -209,6 +201,7 @@
p.team_id, p.team_id,
coalesce(fd.backend, 'legacy-db') AS backend, coalesce(fd.backend, 'legacy-db') AS backend,
fd.metadata AS metadata, fd.metadata AS metadata,
fd.storage_object_id AS storage_object_id,
fd.data AS data fd.data AS data
FROM file AS f FROM file AS f
LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id) LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id)
@ -218,139 +211,230 @@
(str sql:files-with-data " WHERE f.id = ?")) (str sql:files-with-data " WHERE f.id = ?"))
(def sql:get-file-without-data (def sql:get-file-without-data
(str "WITH files AS (" sql:files-with-data ")" "SELECT f.id,
"SELECT f.id, f.project_id,
f.project_id, f.created_at,
f.created_at, f.modified_at,
f.modified_at, f.deleted_at,
f.deleted_at, f.name,
f.name, f.is_shared,
f.is_shared, f.has_media_trimmed,
f.has_media_trimmed, f.revn,
f.revn, f.ignore_sync_until,
f.ignore_sync_until, f.comment_thread_seqn,
f.comment_thread_seqn, f.features,
f.features, f.version,
f.version, f.vern,
f.vern, p.team_id
f.team_id FROM file AS f
FROM files AS f INNER JOIN project AS p ON (p.id = f.project_id)
WHERE f.id = ?")) WHERE f.id = ?")
(defn- migrate-file (defn- migrate-file
[{:keys [::db/conn] :as cfg} {:keys [read-only?]} {:keys [id] :as file}] "Apply the pending migrations to the file.
(binding [pmap/*load-fn* (partial fdata/load-pointer cfg id)
pmap/*tracked* (pmap/create-tracked)]
(let [libs (delay (get-resolved-file-libraries cfg file))
;; For avoid unnecesary overhead of creating multiple
;; pointers and handly internally with objects map in their
;; worst case (when probably all shapes and all pointers
;; will be readed in any case), we just realize/resolve them
;; before applying the migration to the file.
file (-> (fdata/realize cfg file)
(fmg/migrate-file libs))]
(if (or read-only? (db/read-only? conn)) With `persist-after-migration?` the migrated map is also persisted, which is
file only safe from a map read under the `file` row lock (see `get-file*`). The
(do ;; When file is migrated, we break the rule of no returned map is then the reattached one: decoded data where the pages the
;; perform mutations on get operations and update the migration did not change keep their pointers and their fragment ids, plus
;; file with all migrations applied the `:storage-object-id` of the row the write left, so a caller that
(update-file! cfg file) persists again touches the current object. Without `persist-after-migration?`
(fmigr/resolve-applied-migrations cfg file)))))) the realized map is returned, which is what a read-only context can do.
`get-file*` realizes the result when the caller asks for it."
[cfg {:keys [id] :as file} {:keys [persist-after-migration?]}]
(fdata/with-tracked-pointers cfg id
(let [libs (delay (get-resolved-file-libraries cfg file))
;; Capture the pointer layout before realizing: the migration
;; runs on plain data, and unchanged positions must keep their
;; original fragment ids instead of being re-wrapped with new
;; ones on persist.
original (fdata/original-pointers (:data file))
objects-map-before? (contains? (:features file) "fdata/objects-map")
realized (-> (fdata/realize cfg file)
(fmg/migrate-file libs))
objects-map-added? (and (not objects-map-before?)
(contains? (:features realized) "fdata/objects-map"))]
(if persist-after-migration?
(let [file (if objects-map-added?
realized
(fdata/reattach-unchanged original realized))
written (update-file! cfg file)]
;; When the file is migrated, we break the rule of not
;; performing mutations on get operations and update the file
;; with all migrations applied. The reattached map is also what
;; the read returns: the caller realizes it only when it asks.
(assoc (fmigr/resolve-applied-migrations cfg file)
:storage-object-id (:storage-object-id written)))
realized))))
(defn- fetch-row
"Read the file row. With `load-data?` the file data columns come along, but
they are not resolved nor decoded (see `attach-file-data`). With
`load-migrations?` the applied migrations are resolved, which is what
`need-migration?` expects. A skip-locked read answers nil instead of raising
when the row is gone."
[{:keys [::db/conn] :as cfg} id
{:keys [include-deleted?
skip-locked?
lock-for-update?
lock-for-share?
throw-if-not-exists?
load-migrations?
load-data?]}]
(let [sql (if load-data?
sql:get-file
sql:get-file-without-data)
sql (cond
lock-for-update?
(str sql " FOR UPDATE of f")
lock-for-share?
(str sql " FOR SHARE of f")
:else
sql)
sql (if (and skip-locked? (or lock-for-share? lock-for-update?))
(str sql " SKIP LOCKED")
sql)
file (db/get-with-sql conn [sql id] {::db/throw-if-not-exists false
::db/remove-deleted (not include-deleted?)})
file (some-> file
(d/update-when :features db/decode-pgarray #{})
(d/update-when :metadata fdata/decode-metadata))
file (if (and file load-migrations?)
(fmigr/resolve-applied-migrations cfg file)
file)]
(or file
(when-not (or skip-locked? (not throw-if-not-exists?))
(ex/raise :type :not-found
:code :object-not-found
:hint "database object not found"
:table :file
:file-id id)))))
(defn- attach-file-data
"Resolve the data of an already read row and decode it when `decode?`. It
completes the map it receives instead of reading the row again. It does not
realize: `get-file*` owns that step, once per path."
[cfg file {:keys [decode?]}]
(let [file (fdata/resolve-file-data cfg file)]
(if decode?
(fdata/decode-file-data cfg file)
file)))
(defn- fetch-file
"Read the row with its data resolved and decoded, under the requested lock."
[cfg id options]
(when-let [file (fetch-row cfg id options)]
(attach-file-data cfg file options)))
(def ^:private
default-get-file-options
{:load-migrations? true
:throw-if-not-exists? true
:persist-after-migration? true
:load-data? true
:migrate? true
:decode? true})
(defn- get-file* (defn- get-file*
[{:keys [::db/conn] :as cfg} id [{:keys [::db/conn] :as cfg} id options]
{:keys [migrate?
realize?
decode?
skip-locked?
include-deleted?
load-data?
throw-if-not-exists?
lock-for-update?
lock-for-share?]
:or {lock-for-update? false
lock-for-share? false
load-data? true
migrate? true
decode? true
include-deleted? false
throw-if-not-exists? true
realize? false}
:as options}]
(assert (db/connection? conn) "expected cfg with valid connection") (assert (db/connection? conn) "expected cfg with valid connection")
(when (and (not load-data?)
(or lock-for-share? lock-for-share? skip-locked?))
(throw (IllegalArgumentException. "locking is incompatible when `load-data?` is false")))
(let [sql (let [{:keys [migrate?
(if load-data? realize?
sql:get-file decode?
sql:get-file-without-data) load-data?
lock-for-share?
read-only?]
:as options}
(merge default-get-file-options options)
sql read-only?
(cond (or read-only? (db/read-only? conn))]
lock-for-update?
(str sql " FOR UPDATE of f")
lock-for-share? (when (and realize? (or (not load-data?) (not decode?)))
(str sql " FOR SHARE of f") (throw (IllegalArgumentException. "invalid options, you only can realize when loading data and decoding")))
:else (when (and lock-for-share? load-data?)
sql) (throw (IllegalArgumentException. "invalid options, `:lock-for-share?` is only allowed on row-only reads")))
sql (cond
(if skip-locked? ;; The caller only wants the row: that read is the answer.
(str sql " SKIP LOCKED") (not load-data?)
sql) (fetch-row cfg id options)
file (and migrate? decode? (not read-only?))
(db/get-with-sql conn [sql id] (when-let [file (fetch-row cfg id options)]
{::db/throw-if-not-exists false (if (fmg/need-migration? file)
::db/remove-deleted (not include-deleted?)}) (when-let [file (fetch-file cfg id (assoc options :lock-for-update? true))]
(cond-> file
(fmg/need-migration? file)
(as-> $ (migrate-file cfg $ options))
file realize?
(-> file (->> (fdata/realize cfg))))
(d/update-when :features db/decode-pgarray #{})
(d/update-when :metadata fdata/decode-metadata))]
(if file (let [file (attach-file-data cfg file options)]
(if load-data? (cond-> file
(let [file realize?
(->> file (->> (fdata/realize cfg))))))
(fmigr/resolve-applied-migrations cfg)
(fdata/resolve-file-data cfg))
will-migrate? :else
(and migrate? (fmg/need-migration? file))] (when-let [file (fetch-file cfg id options)]
(cond-> file
(and migrate? decode? (fmg/need-migration? file))
(as-> $ (migrate-file cfg $ (assoc options :persist-after-migration? false)))
(if decode? realize?
(cond->> (fdata/decode-file-data cfg file) (->> (fdata/realize cfg)))))))
(and realize? (not will-migrate?))
(fdata/realize cfg)
will-migrate?
(migrate-file cfg options))
file))
file)
(when-not (or skip-locked? (not throw-if-not-exists?))
(ex/raise :type :not-found
:code :object-not-found
:hint "database object not found"
:table :file
:file-id id)))))
(defn get-file (defn get-file
"Get file, resolve all features and apply migrations. "Get file, resolve all features and apply migrations.
Runs inside a transaction: it joins the caller's when the cfg already
carries a connection and opens one otherwise, so the migration write
commits with the read. A read-only read (`:read-only?`) takes no lock and
writes nothing, so it runs on a plain connection instead of opening a
transaction of its own; with a cfg that already carries a connection it
still joins the caller's. Accepts a cfg map only: a bare pool or a
connection raises `IllegalArgumentException`.
Useful when you have plan to apply massive or not surgical Useful when you have plan to apply massive or not surgical
operations on file, because it removes the overhead of lazy fetching operations on file, because it removes the overhead of lazy fetching
and decoding." and decoding.
Options:
- `:realize?` returns plain data instead of fragment pointers; it
requires `:load-data?` and `:decode?`.
- `:decode?` decodes the stored blob (default true).
- `:load-data?` reads the data columns (default true). A row-only read
(`false`) is the only mode that accepts `:lock-for-share?`.
- `:migrate?` applies the pending migrations on read (default true).
- `:load-migrations?` resolves the applied migrations on a row read
(default true).
- `:read-only?` applies the pending migrations in memory only and skips
the transaction. It does not take a lock.
- `:lock-for-update?`, `:skip-locked?`, `:include-deleted?` and
`:throw-if-not-exists?` are the usual read knobs. A lock needs a
read-write transaction from the caller: PostgreSQL rejects row locks in
a read-only transaction.
- `:persist-after-migration?` is internal: `get-file*` turns it off
for the in-memory migration of a read-only context. Do not pass it."
[cfg file-id & {:as opts}] [cfg file-id & {:as opts}]
(db/run! cfg get-file* file-id opts)) (when-not (map? cfg)
(throw (IllegalArgumentException.
"expected cfg map, not a pool or a connection")))
(if (:read-only? opts)
(db/run! cfg get-file* file-id opts)
(db/tx-run! cfg get-file* file-id opts)))
(defn clean-file-features (defn clean-file-features
[file] [file]
@ -378,19 +462,27 @@
inner join project as p on (p.id = f.project_id) inner join project as p on (p.id = f.project_id)
where fpr.file_id = ? where fpr.file_id = ?
and fpr.profile_id = ? and fpr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1 and exists (select 1
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id where tpr.team_id = p.team_id
and tpr.profile_id = fpr.profile_id) and tpr.profile_id = fpr.profile_id
and t.deleted_at is null)
union all union all
select tpr.is_owner, select tpr.is_owner,
tpr.is_admin, tpr.is_admin,
tpr.can_edit tpr.can_edit
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
inner join project as p on (p.team_id = tpr.team_id) inner join project as p on (p.team_id = tpr.team_id)
inner join file as f on (p.id = f.project_id) inner join file as f on (p.id = f.project_id)
where f.id = ? where f.id = ?
and tpr.profile_id = ? and tpr.profile_id = ?
and t.deleted_at is null
and p.deleted_at is null
and f.deleted_at is null
union all union all
select ppr.is_owner, select ppr.is_owner,
ppr.is_admin, ppr.is_admin,
@ -400,10 +492,14 @@
inner join project as p on (p.id = ppr.project_id) inner join project as p on (p.id = ppr.project_id)
where f.id = ? where f.id = ?
and ppr.profile_id = ? and ppr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1 and exists (select 1
from team_profile_rel as tpr from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id where tpr.team_id = p.team_id
and tpr.profile_id = ppr.profile_id)") and tpr.profile_id = ppr.profile_id
and t.deleted_at is null)")
(defn- get-file-permissions* (defn- get-file-permissions*
[conn profile-id file-id] [conn profile-id file-id]
@ -451,47 +547,31 @@
[cfg project-id] [cfg project-id]
(db/get cfg :project {:id project-id})) (db/get cfg :project {:id project-id}))
(def ^:private sql:get-teams (def ^:private sql:get-files-rels
"SELECT t.* FROM team WHERE id = ANY(?)") "SELECT flr.*,
fls.synced_at
(defn get-teams FROM file_library_rel AS flr
[cfg ids] JOIN file AS l
(let [conn (db/get-connection cfg) ON flr.library_file_id = l.id
ids (db/create-array conn "uuid" ids)] LEFT JOIN file_library_sync AS fls
(->> (db/exec! conn [sql:get-teams ids]) ON fls.file_id = flr.file_id
(map decode-row-features)))) AND fls.library_file_id = flr.library_file_id
WHERE flr.file_id = ANY(?)
(defn get-team AND l.deleted_at IS NULL
[cfg team-id] ORDER BY flr.file_id, flr.library_file_id;")
(-> (db/get cfg :team {:id team-id})
(decode-row-features)))
(defn get-fonts
[cfg team-id]
(db/query cfg :team-font-variant
{:team-id team-id
:deleted-at nil}))
(defn get-files-rels (defn get-files-rels
"Given a set of file-id's, return all matching relations with the libraries" "Given a set of file-id's, return all matching relations with the libraries.
Rows come back in stable `(file-id, library-file-id)` order so export
and import resolve cycles deterministically."
[cfg ids] [cfg ids]
(assert (set? ids) "expected a set of uuids") (assert (set? ids) "expected a set of uuids")
(assert (every? uuid? ids) "expected a set of uuids") (assert (every? uuid? ids) "expected a set of uuids")
(db/run! cfg (fn [{:keys [::db/conn]}] (db/run! cfg (fn [{:keys [::db/conn]}]
(let [ids (db/create-array conn "uuid" ids) (let [ids (db/create-array conn "uuid" ids)]
sql (str "SELECT flr.*," (db/exec! conn [sql:get-files-rels ids])))))
" fls.synced_at"
" FROM file_library_rel AS flr"
" JOIN file AS l"
" ON flr.library_file_id = l.id"
" LEFT JOIN file_library_sync AS fls"
" ON fls.file_id = flr.file_id"
" AND fls.library_file_id = flr.library_file_id"
" WHERE flr.file_id = ANY(?)"
" AND l.deleted_at IS NULL;")]
(db/exec! conn [sql ids])))))
(def ^:private sql:upsert-file-library-sync (def ^:private sql:upsert-file-library-sync
"INSERT INTO file_library_sync (file_id, library_file_id, synced_at) "INSERT INTO file_library_sync (file_id, library_file_id, synced_at)
@ -526,6 +606,20 @@
(->> (db/exec! conn [sql:get-libraries ids']) (->> (db/exec! conn [sql:get-libraries ids'])
(into #{} xf-map-id)))))) (into #{} xf-map-id))))))
(defn library-link-status
"Check whether linking `file-id` to `library-id` is safe. Returns
`:ok` when the link can be created, `:self-link` when both ids are
the same, and `:circular` when the link would close a dependency
cycle (i.e. `file-id` is already among the transitive libraries of
`library-id`). Shared by the UI command (`link-file-to-library`,
which raises) and the v3 importer (`import-file-relations`, which
skips and logs), so both stay consistent."
[cfg file-id library-id]
(cond
(= file-id library-id) :self-link
(contains? (get-libraries cfg [library-id]) file-id) :circular
:else :ok))
(defn get-file-object-thumbnails (defn get-file-object-thumbnails
"Return all file object thumbnails for a given file." "Return all file object thumbnails for a given file."
[cfg file-id] [cfg file-id]
@ -534,15 +628,6 @@
:deleted-at nil}) :deleted-at nil})
(not-empty))) (not-empty)))
(defn get-file-thumbnail
"Return the thumbnail for the specified file-id"
[cfg {:keys [id revn]}]
(db/get* cfg :file-thumbnail
{:file-id id
:revn revn
:data nil}
{::sql/columns [:media-id :file-id :revn]}))
(def ^:private sql:get-missing-media-references (def ^:private sql:get-missing-media-references
"SELECT fmo.* "SELECT fmo.*
FROM file_media_object AS fmo FROM file_media_object AS fmo
@ -701,7 +786,12 @@
(let [library-ids (get-libraries cfg [file-id]) (let [library-ids (get-libraries cfg [file-id])
data (reduce (fn [data library-id] data (reduce (fn [data library-id]
(if-let [library (get-file cfg library-id :include-deleted? true)] (if-let [library (get-file cfg library-id :include-deleted? true)]
(cond-> (ctf/absorb-assets data (:data library)) ;; The library is read without `:realize?`, so its
;; `:components` may be a pointer map. Bind the
;; library's own loader: the absorb walks the
;; components and would raise without one.
(cond-> (fdata/with-pointers cfg library-id
(ctf/absorb-assets data (:data library)))
(= library-id (cfo/get-tokens-source data)) (= library-id (cfo/get-tokens-source data))
(cfo/absorb-tokens-lib (:data library))) (cfo/absorb-tokens-lib (:data library)))
data)) data))
@ -784,18 +874,33 @@
(dissoc :metadata) (dissoc :metadata)
(dissoc :migrations))) (dissoc :migrations)))
(defn file->response
"Strip the internal keys of a file map before it leaves the backend in a
response. The file data storage pointer is bookkeeping for the write path:
a `file-data` object is authentication-only, so a leaked id would let any
logged-in profile fetch the blob from `/assets/by-id`. The metadata key of
the old pointer is stripped too: a row the migration does not backfill keeps
it, and the metadata schema is open, so it would travel in the response."
[file]
(-> file
(dissoc :storage-object-id)
(d/update-when :metadata #(dissoc % :storage-ref-id))))
(defn- file->file-data-params (defn- file->file-data-params
[{:keys [id] :as file} & {:as opts}] [{:keys [id] :as file}]
(let [created-at (or (:created-at file) (ct/now)) (let [created-at (or (:created-at file) (ct/now))
modified-at (or (:modified-at file) created-at)] modified-at (or (:modified-at file) created-at)]
(d/without-nils (-> (d/without-nils
{:id id {:id id
:type "main" :type "main"
:file-id id :file-id id
:data (:data file) :data (:data file)
:metadata (:metadata file) :metadata (:metadata file)
:created-at created-at :created-at created-at
:modified-at modified-at}))) :modified-at modified-at})
;; The storage object the persisted row pointed at (nil for a new or
;; imported file). `upsert!` touches it before replacing the row.
(assoc :storage-object-id (:storage-object-id file)))))
(defn insert-file! (defn insert-file!
"Insert a new file into the database table. Expectes a not-encoded file. "Insert a new file into the database table. Expectes a not-encoded file.
@ -819,34 +924,63 @@
:cause cause) :cause cause)
(throw cause)))) (throw cause))))
(->> (file->file-data-params file) ;; An INSERT never replaces a persisted pointer: `duplicate-file` hands
;; the source file map here, and touching the source object would be a
;; write on a live object.
(->> (file->file-data-params (dissoc file :storage-object-id))
(fdata/upsert! cfg)) (fdata/upsert! cfg))
nil)) nil))
(defn update-file! (defn update-file!
"Update an existing file on the database. Expects not encoded file." "Update an existing file on the database. Expects a not-encoded file map,
[{:keys [::db/conn] :as cfg} {:keys [id] :as file} & {:as opts}] as returned by `get-file`, and optionally the fields to change as keyword
args; they are merged into the file map before persisting.
(if (::reset-migrations? opts false) The storage object the persisted row pointed at travels in the map
(fmigr/reset-migrations! conn file) (`:storage-object-id`), and `upsert!` touches it before replacing the row.
(fmigr/upsert-migrations! conn file)) That is why the caller must pass the map it read instead of building a
partial one: nothing is queried here. The params only change mutable fields
of the file: the pointer and the identity (`:id`, `:file-id`) always come
from the read map.
(let [file Returns the map `fdata/upsert!` answers with: `:written` and the
(encode-file cfg file) `:storage-object-id` of the object this write created, so a caller that
persists again touches the current object instead of the replaced one."
[{:keys [::db/conn] :as cfg} file & {:as params}]
file-params ;; A map read with `:load-data? false` has no `:data`; persisting it would
(file->params (dissoc file :id)) ;; wipe the file content. `ex/raise` instead of `assert`: the guard must
;; exist in the deployed binary, where `clojure.core/assert` is compiled
;; out.
(when-not (contains? file :data)
(ex/raise :type :internal
:code :missing-file-data
:hint "expected a file map read with data"
:file-id (:id file)))
file-data-params (let [reset? (::reset-migrations? params false)
(file->file-data-params file)] ;; The pointer and the identity are bookkeeping of the persisted row,
;; not editable fields: they can only come from the read map. A pointer
;; in the params would skip the touch and leak the replaced blob; an
;; `:id` would retarget the UPDATE.
file (merge file (dissoc params ::reset-migrations?
:storage-object-id
:id
:file-id))]
(db/update! conn :file file-params (if reset?
{:id id} (fmigr/reset-migrations! conn file)
{::db/return-keys false}) (fmigr/upsert-migrations! conn file))
(fdata/upsert! cfg file-data-params) (let [file (encode-file cfg file)
nil)) file-params (file->params (dissoc file :id))]
(db/update! conn :file file-params
{:id (:id file)}
{::db/return-keys false})
(fdata/upsert! cfg (file->file-data-params file)))))
(defn save-file! (defn save-file!
"Applies all the final validations and perist the file, binfile "Applies all the final validations and perist the file, binfile
@ -859,8 +993,7 @@
(let [file (-> file (let [file (-> file
(assoc :created-at timestamp) (assoc :created-at timestamp)
(assoc :modified-at timestamp) (assoc :modified-at timestamp)
(cond-> (not (::overwrite cfg)) (assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5})))
(assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5}))))
(update :features (update :features
(fn [features] (fn [features]
(-> (::features cfg #{}) (-> (::features cfg #{})
@ -877,9 +1010,7 @@
(when (ex/exception? result) (when (ex/exception? result)
(l/error :hint "file schema validation error" :cause result)))) (l/error :hint "file schema validation error" :cause result))))
(if (::overwrite cfg) (insert-file! cfg file opts)))
(update-file! cfg file (assoc opts ::reset-migrations? true))
(insert-file! cfg file opts))))
(def ^:private sql:get-file-libraries (def ^:private sql:get-file-libraries
"WITH RECURSIVE libs AS ( "WITH RECURSIVE libs AS (
@ -944,7 +1075,12 @@
(let [library-ids (->> (get-file-libraries conn (:id file)) (let [library-ids (->> (get-file-libraries conn (:id file))
(map :id) (map :id)
(cons (:id file))) (cons (:id file)))
load-fn #(get-file cfg % :migrate? false)] ;; Fragment rows are file-local and callers bind the loader of
;; the file they work on, so each library realizes its pointers
;; here, with its own id, before anything walks its data.
load-fn (fn [library-id]
(->> (get-file cfg library-id :migrate? false)
(fdata/realize-pointers cfg)))]
(weak/loadable-weak-value-map library-ids load-fn {id file}))) (weak/loadable-weak-value-map library-ids load-fn {id file})))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@ -1062,3 +1198,46 @@
{:bytes bytes {:bytes bytes
:size (alength ^bytes bytes) :size (alength ^bytes bytes)
:hash (sto/calculate-hash (java.io.ByteArrayInputStream. bytes))}))) :hash (sto/calculate-hash (java.io.ByteArrayInputStream. bytes))})))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; REQUEST CFG
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn resolve-export-type
"The canonical export type of a request on the legacy RPC: an explicit
`type` wins, and the deprecated boolean flags are translated. The
creation command of a job does not need it, because it asks the caller
for the type instead of guessing it."
[{:keys [type include-libraries embed-assets]}]
(cond
(some? type) type
(true? include-libraries) :include-libraries
(true? embed-assets) :merge-libraries
:else :detach-libraries))
(defn export-cfg
"The domain cfg of an export: the frozen set of files, how their
libraries are handled, and where the package is written."
[cfg {:keys [file-ids export-type output]}]
(-> cfg
(assoc ::file-ids file-ids)
(assoc ::export-type export-type)
(assoc ::output output)))
(defn import-cfg
"The domain cfg of an import: the destination team/profile/project, the
name, the input and the anti-zip-bomb limits. The caller resolves the
team, because what allows it to import there differs between the RPC and
a job; what both must share is fixed here."
[cfg {:keys [profile-id project-id team name input]}]
(-> cfg
(assoc ::features (cfeat/get-team-enabled-features cf/flags team))
(assoc ::project-id project-id)
(assoc ::profile-id profile-id)
(assoc ::team-id (:id team))
(assoc ::name name)
(assoc ::input input)
(assoc ::import-max-binary-entry-size (cf/get :binfile-import-max-binary-entry-size))
(assoc ::import-max-text-entry-size (cf/get :binfile-import-max-text-entry-size))
(assoc ::import-max-text-total-size (cf/get :binfile-import-max-text-total-size))
(assoc ::import-max-zip-entries (cf/get :binfile-import-max-zip-entries))))

View File

@ -0,0 +1,120 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns app.binfile.jobs
"The progress bridge of the binfile jobs.
A job has no SSE channel, so the `:progress` points that `bf.v1` and
`bf.v3` emit become `progress` events of the job through
`jobs/heartbeat`, which throttles them and drops the ones that do not
fit. The adapter turns a tap into a self-contained milestone: a `:stage`
of a single vocabulary (the two formats name the same work differently)
and the counters the tap carries, with the outer scopes in course as
context. The reader puts the outer counters in every tap inside them,
so the adapter holds no state between taps. The ids the taps carry are
not part of the job event contract.
The handlers in `app.tasks.export-binfile` and
`app.tasks.import-binfile` run the core through `run` and
guard it with `check-active`: a job that was cancelled, or that
already reached a terminal state, raises an `:interrupt` right there on
the run thread, so the work stops at its next event point and the
transaction of its handler rolls back with it. A run whose last tap
happened before the interruption is discarded once it returns, which is
what makes that transaction roll back too."
(:require
[app.jobs :as jobs]
[app.util.events :as events]))
(defn check-active
"Raise an `:interrupt` when the job is no longer active.
The substrate owns both the check and the error; this is the adapter's
entry point to it, used by the two handlers before their run and by the
tests. Answers with the cfg, so a handler can thread it."
[cfg context]
(jobs/check-active cfg (:id context)))
(def ^:private stage-of
"The stage a tap section becomes: one vocabulary for the two formats, so
a client resolves each stage once. The counter of the activity is named
after its own stage, which is what lets a client render
`counters[stage]` without a table of equivalences."
{:file :files
:page :pages
:media :media
:thumbnail :thumbnails
:thumbnails :thumbnails
:component :components
:color :colors
:typography :typographies
:tokens-lib :tokens
:tokens-status :tokens
:storage-object :storage-objects
:storage-objects :storage-objects
:relations :relations
:manifest :manifest
:v1/metadata :metadata
:v1/files :files
:v1/rels :relations
:v1/sobjects :storage-objects})
(defn- stage
"The stage of a tap section; a section without a name of its own keeps
it, so a new tap is never dropped from the stream."
[section]
(get stage-of section section))
(defn- counter
"The counter a tap reports, when it reports one: a tap that only marks a
point of the run has no units to count."
[{:keys [current total]}]
(when (some? current)
(cond-> {:current current}
(some? total)
(assoc :total total))))
(defn- milestone
"The milestone a tap becomes: what the job is doing now and the counters
the tap carries, with the outer scopes in course as context.
The tap of a scope carries its outer counters, so the milestone is
self-contained and the adapter remembers nothing between taps: a client
that misses one (the substrate throttles the events) still renders the
truth from the next."
[{:keys [section outer-counters] :as tap}]
(let [stage (stage section)
own (counter tap)]
(cond-> {:stage stage}
(or (some? outer-counters) (some? own))
(assoc :counters (cond-> (or outer-counters {})
(some? own) (assoc stage own))))))
(defn run
"Run `f` with the progress taps of the core turned into milestones of the
job.
`f` is called with the cfg, so the adapter forwards what it receives
and a caller can pass the core directly.
Every tap is reported: the substrate throttles the durable events, and
a milestone that does not fit loses no information because the next one
carries the same state.
The taps run on this same thread: the sink calls the heartbeat inline,
so an `:interrupt` it raises aborts the core right there, inside the
transaction of its handler, which rolls back."
[cfg context f]
(let [job-id (:id context)
sink (fn [[type data]]
(when (= :progress type)
(jobs/heartbeat cfg
:job-id job-id
:progress (milestone data))))]
(binding [events/*sink* sink]
(let [result (f cfg)]
(check-active cfg context)
result))))

View File

@ -329,13 +329,13 @@
[:v1/metadata :v1/files :v1/rels :v1/sobjects])))) [:v1/metadata :v1/files :v1/rels :v1/sobjects]))))
(defmethod write-section :v1/metadata (defmethod write-section :v1/metadata
[{:keys [::output ::bfc/ids ::bfc/include-libraries] :as cfg}] [{:keys [::output ::bfc/file-ids ::bfc/include-libraries] :as cfg}]
(if-let [fids (get-files cfg ids)] (if-let [fids (get-files cfg file-ids)]
(let [lids (when include-libraries (let [lids (when include-libraries
(bfc/get-libraries cfg ids)) (bfc/get-libraries cfg file-ids))
ids (into fids lids)] file-ids (into fids lids)]
(write-obj! output {:version cf/version :files ids}) (write-obj! output {:version cf/version :files file-ids})
(vswap! bfc/*state* assoc :files ids)) (vswap! bfc/*state* assoc :files file-ids))
(ex/raise :type :not-found (ex/raise :type :not-found
:code :files-not-found :code :files-not-found
:hint "unable to retrieve files for export"))) :hint "unable to retrieve files for export")))
@ -482,9 +482,11 @@
(bfm/apply-pending-migrations! cfg) (bfm/apply-pending-migrations! cfg)
;; Knowing that the ids of the created files are in index, ;; Knowing that the ids of the created files are in index,
;; just lookup them and return them in the same shape as v3. ;; just lookup them and return them in the same shape as v3. The
;; metadata of an old package may repeat a file id, so the answer
;; carries one id per imported file.
(let [files (-> bfc/*state* deref :files)] (let [files (-> bfc/*state* deref :files)]
{:file-ids (into [] (keep #(get-in @bfc/*state* [:index %])) files)}))))) {:file-ids (into [] (comp (keep #(get-in @bfc/*state* [:index %])) (distinct)) files)})))))
(defmethod read-import :v1 (defmethod read-import :v1
[options] [options]
@ -510,89 +512,114 @@
(defmethod read-section :v1/files (defmethod read-section :v1/files
[{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}] [{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}]
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))] (let [imported (volatile! #{})]
(let [file (read-obj! input) (doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
media (read-obj! input) (let [file (read-obj! input)
media (read-obj! input)
file-id (:id file)]
file-id (:id file) (when (not= file-id expected-file-id)
file-id' (bfc/lookup-index file-id) (ex/raise :type :validation
:code :inconsistent-penpot-file
:found-id file-id
:expected-id expected-file-id
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
file (bfc/clean-file-features file) (if (contains? @imported file-id)
thumbnails (:thumbnails file)] ;; Old exports could repeat a file id in the metadata (the file was
;; also one of its own libraries). The entry is read above to keep
;; the stream in sync, but the file is imported only once.
(l/warn :hint "ignoring repeated file entry"
:file-id (str file-id)
::l/sync? true)
(when (not= file-id expected-file-id) (let [file-id' (bfc/lookup-index file-id)
(ex/raise :type :validation file (bfc/clean-file-features file)
:code :inconsistent-penpot-file thumbnails (:thumbnails file)]
:found-id file-id
:expected-id expected-file-id
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
(l/dbg :hint "processing file" (vswap! imported conj file-id)
:id (str file-id)
:features (:features file)
:version (-> file :data :version)
:media (count media)
:thumbnails (count thumbnails)
::l/sync? true)
(when (seq thumbnails) (l/dbg :hint "processing file"
(let [thumbnails (remap-thumbnails thumbnails file-id')] :id (str file-id)
(l/dbg :hint "updated index with thumbnails" :features (:features file)
:total (count thumbnails) :version (-> file :data :version)
::l/sync? true) :media (count media)
(vswap! bfc/*state* update :thumbnails bfc/into-vec thumbnails))) :thumbnails (count thumbnails)
::l/sync? true)
(when (seq media) (when (seq thumbnails)
;; Update index with media (let [thumbnails (remap-thumbnails thumbnails file-id')]
(l/dbg :hint "update index with media" :total (count media) ::l/sync? true) (l/dbg :hint "updated index with thumbnails"
(vswap! bfc/*state* update :index bfc/update-index (map :id media)) :total (count thumbnails)
::l/sync? true)
(vswap! bfc/*state* update :thumbnails bfc/into-vec thumbnails)))
;; Store file media for later insertion (when (seq media)
(l/dbg :hint "update media references" ::l/sync? true) ;; Update index with media
(vswap! bfc/*state* update :media into (map #(update % :id bfc/lookup-index)) media)) (l/dbg :hint "update index with media" :total (count media) ::l/sync? true)
(vswap! bfc/*state* update :index bfc/update-index (map :id media))
(let [file (-> file ;; Store file media for later insertion
(assoc :id file-id') (l/dbg :hint "update media references" ::l/sync? true)
(cond-> (and (= idx 0) (some? name)) (vswap! bfc/*state* update :media into (map #(update % :id bfc/lookup-index)) media))
(assoc :name name))
(assoc :project-id project-id)
(dissoc :thumbnails))
file (bfc/process-file system file)]
;; All features that are enabled and requires explicit migration are (let [file (-> file
;; added to the state for a posterior migration step. (assoc :id file-id')
(doseq [feature (-> (::bfc/features system) (cond-> (and (= idx 0) (some? name))
(set/difference cfeat/no-migration-features) (assoc :name name))
(set/difference (:features file)))] (assoc :project-id project-id)
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature file-id'])) (dissoc :thumbnails))
file (bfc/process-file system file)]
(l/dbg :hint "create file" :id (str file-id') ::l/sync? true) ;; All features that are enabled and requires explicit migration
(bfc/save-file! system file ::db/return-keys false) ;; are added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features system)
(set/difference cfeat/no-migration-features)
(set/difference (:features file)))]
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature file-id']))
file-id')))) (l/dbg :hint "create file" :id (str file-id') ::l/sync? true)
(bfc/save-file! system file ::db/return-keys false)
file-id')))))))
(defmethod read-section :v1/rels (defmethod read-section :v1/rels
[{:keys [::db/conn ::bfc/input ::bfc/timestamp]}] [{:keys [::db/conn ::bfc/input ::bfc/timestamp] :as cfg}]
(let [rels (read-obj! input) (let [rels (->> (read-obj! input)
(sort-by (fn [{:keys [file-id library-file-id]}]
[(str file-id) (str library-file-id)])))
ids (into #{} (-> bfc/*state* deref :files))] ids (into #{} (-> bfc/*state* deref :files))]
;; Insert all file relations
(doseq [{:keys [library-file-id] :as rel} rels] (doseq [{:keys [library-file-id] :as rel} rels]
(let [rel (-> rel (let [rel (-> rel
(update :file-id bfc/lookup-index) (update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))] (update :library-file-id bfc/lookup-index))]
(if (contains? ids library-file-id) (if (contains? ids library-file-id)
(do (case (bfc/library-link-status cfg (:file-id rel) (:library-file-id rel))
(l/dbg :hint "create file library link" :self-link
:file-id (:file-id rel) (l/warn :hint "ignoring self-referencing library link"
:lib-id (:library-file-id rel) :file-id (:file-id rel)
::l/sync? true) :lib-id (:library-file-id rel)
(let [rel-params (dissoc rel :synced-at)] ::l/sync? true)
(db/insert! conn :file-library-rel rel-params)
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params) :circular
:library-file-id (:library-file-id rel-params) (l/warn :hint "ignoring circular library link"
:synced-at (or (:synced-at rel) :file-id (:file-id rel)
timestamp)}))) :lib-id (:library-file-id rel)
::l/sync? true)
:ok
(do
(l/dbg :hint "create file library link"
:file-id (:file-id rel)
:lib-id (:library-file-id rel)
::l/sync? true)
(let [rel-params (dissoc rel :synced-at)]
(db/insert! conn :file-library-rel rel-params)
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params)
:library-file-id (:library-file-id rel-params)
:synced-at (or (:synced-at rel)
timestamp)}))))
(l/warn :hint "ignoring file library link" (l/warn :hint "ignoring file library link"
:file-id (:file-id rel) :file-id (:file-id rel)
@ -689,12 +716,12 @@
`::bfc/embed-assets`: instead of including the libraries, embed in the `::bfc/embed-assets`: instead of including the libraries, embed in the
same file library all assets used from external libraries." same file library all assets used from external libraries."
[{:keys [::bfc/ids] :as cfg} output] [{:keys [::bfc/file-ids] :as cfg} output]
(dm/assert! (dm/assert!
"expected a set of uuid's for `::bfc/ids` parameter" "expected a set of uuid's for `::bfc/file-ids` parameter"
(and (set? ids) (and (set? file-ids)
(every? uuid? ids))) (every? uuid? file-ids)))
(dm/assert! (dm/assert!
"expected instance of jio/IOFactory for `input`" "expected instance of jio/IOFactory for `input`"

View File

@ -1,445 +0,0 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns app.binfile.v2
"A sqlite3 based binary file exportation with support for exportation
of entire team (or multiple teams) at once."
(:refer-clojure :exclude [read])
(:require
[app.binfile.common :as bfc]
[app.common.data :as d]
[app.common.features :as cfeat]
[app.common.logging :as l]
[app.common.time :as ct]
[app.common.transit :as t]
[app.common.uuid :as uuid]
[app.config :as cf]
[app.db :as db]
[app.db.sql :as sql]
[app.loggers.audit :as-alias audit]
[app.loggers.webhooks :as-alias webhooks]
[app.storage :as sto]
[app.storage.tmp :as tmp]
[app.util.events :as events]
[app.worker :as-alias wrk]
[clojure.set :as set]
[cuerdas.core :as str]
[datoteka.io :as io]
[promesa.util :as pu])
(:import
java.sql.DriverManager))
(set! *warn-on-reflection* true)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; LOW LEVEL API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- create-database
([cfg]
(let [path (tmp/tempfile :prefix "penpot.binfile." :suffix ".sqlite")]
(create-database cfg path)))
([cfg path]
(let [db (DriverManager/getConnection (str "jdbc:sqlite:" path))]
(assoc cfg ::db db ::path path))))
(def ^:private
sql:create-kvdata-table
"CREATE TABLE kvdata (
tag text NOT NULL,
key text NOT NULL,
val text NOT NULL,
dat blob NULL
)")
(def ^:private
sql:create-kvdata-index
"CREATE INDEX kvdata__tag_key__idx
ON kvdata (tag, key)")
(defn- setup-schema!
[{:keys [::db]}]
(db/exec-one! db [sql:create-kvdata-table])
(db/exec-one! db [sql:create-kvdata-index]))
(defn- write!
[{:keys [::db]} tag k v & [data]]
(db/insert! db :kvdata
{:tag (d/name tag)
:key (str k)
:val (t/encode-str v {:type :json-verbose})
:dat data}
{::db/return-keys false}))
(defn- read-blob
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:dat]})]
(:dat obj)))
(defn- read-seq
([{:keys [::db]} tag]
(->> (db/query db :kvdata
{:tag (d/name tag)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str)))
([{:keys [::db]} tag k]
(->> (db/query db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str))))
(defn- read-obj
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:val]})]
(-> obj :val t/decode-str)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; IMPORT/EXPORT IMPL
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(declare ^:private write-project!)
(declare ^:private write-file!)
(defn- write-team!
[cfg team-id]
(let [team (bfc/get-team cfg team-id)
fonts (bfc/get-fonts cfg team-id)]
(events/tap :progress
{:op :export
:section :write-team
:id team-id
:name (:name team)})
(l/trc :hint "write" :obj "team"
:id (str team-id)
:fonts (count fonts))
(when-let [photo-id (:photo-id team)]
(vswap! bfc/*state* update :storage-objects conj photo-id))
(vswap! bfc/*state* update :teams conj team-id)
(vswap! bfc/*state* bfc/collect-storage-objects fonts)
(write! cfg :team team-id team)
(doseq [{:keys [id] :as font} fonts]
(vswap! bfc/*state* update :team-font-variants conj id)
(write! cfg :team-font-variant id font))))
(defn- write-project!
[cfg project]
(events/tap :progress
{:op :export
:section :write-project
:id (:id project)
:name (:name project)})
(l/trc :hint "write" :obj "project" :id (str (:id project)))
(write! cfg :project (str (:id project)) project)
(vswap! bfc/*state* update :projects conj (:id project)))
(defn- write-file!
[cfg file-id]
(let [file (bfc/get-file cfg file-id :realize? true)
thumbs (bfc/get-file-object-thumbnails cfg file-id)
media (bfc/get-file-media cfg file)
rels (bfc/get-files-rels cfg #{file-id})]
(events/tap :progress
{:op :export
:section :write-file
:id file-id
:name (:name file)})
(vswap! bfc/*state* (fn [state]
(-> state
(update :files conj file-id)
(update :file-media-objects into bfc/xf-map-id media)
(bfc/collect-storage-objects thumbs)
(bfc/collect-storage-objects media))))
(write! cfg :file file-id file)
(write! cfg :file-rels file-id rels)
(run! (partial write! cfg :file-media-object file-id) media)
(run! (partial write! cfg :file-object-thumbnail file-id) thumbs)
(when-let [thumb (bfc/get-file-thumbnail cfg file)]
(vswap! bfc/*state* bfc/collect-storage-objects [thumb])
(write! cfg :file-thumbnail file-id thumb))
(l/trc :hint "write" :obj "file"
:thumbnails (count thumbs)
:rels (count rels)
:media (count media))))
(defn- write-storage-object!
[{:keys [::sto/storage] :as cfg} id]
(let [sobj (sto/get-object storage id)
data (with-open [input (sto/get-object-data storage sobj)]
(io/read input))]
(l/trc :hint "write" :obj "storage-object" :id (str id) :size (:size sobj))
(write! cfg :storage-object id (meta sobj) data)))
(defn- read-storage-object!
[{:keys [::sto/storage ::bfc/timestamp] :as cfg} id]
(let [mdata (d/update-when (read-obj cfg :storage-object id)
:content-type bfc/normalize-content-type)
data (read-blob cfg :storage-object id)
;; NOTE: v2 bundles are legacy and no longer produced; the
;; sanitizer call stays as defense in depth and is covered by
;; the shared helper unit tests, with no dedicated v2
;; import test.
clean (when (bfc/svg-object? mdata)
(bfc/sanitize-imported-svg mdata data))
data (or (:bytes clean) data)
hash (or (:hash clean) (sto/calculate-hash data))
content (-> (sto/content data)
(sto/wrap-with-hash hash))
params (-> mdata
(assoc ::sto/content content)
(assoc ::sto/deduplicate? true)
(assoc ::sto/touched-at timestamp))
sobject (sto/put-object! storage params)]
(vswap! bfc/*state* update :index assoc id (:id sobject))
(l/trc :hint "read" :obj "storage-object"
:id (str id)
:new-id (str (:id sobject))
:size (:size sobject))))
(defn read-team!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} team-id]
(l/trc :hint "read" :obj "team" :id (str team-id))
(let [team (read-obj cfg :team team-id)
team (-> team
(update :id bfc/lookup-index)
(update :photo-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-team
:id team-id
:name (:name team)})
(db/insert! conn :team
(update team :features db/encode-pgarray conn "text")
::db/return-keys false)
(doseq [font (->> (read-seq cfg :team-font-variant)
(filter #(= team-id (:team-id %))))]
(let [font (-> font
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(update :woff1-file-id bfc/lookup-index)
(update :woff2-file-id bfc/lookup-index)
(update :ttf-file-id bfc/lookup-index)
(update :otf-file-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(db/insert! conn :team-font-variant font
::db/return-keys false)))
team))
(defn read-project!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} project-id]
(l/trc :hint "read" :obj "project" :id (str project-id))
(let [project (read-obj cfg :project project-id)
project (-> project
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-project
:id project-id
:name (:name project)})
(db/insert! conn :project project
::db/return-keys false)))
(defn read-file!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} file-id]
(l/trc :hint "read" :obj "file" :id (str file-id))
(let [file (-> (read-obj cfg :file file-id)
(update :id bfc/lookup-index)
(update :project-id bfc/lookup-index))
file (bfc/process-file cfg file)]
(events/tap :progress
{:op :import
:section :read-file
:id file-id
:name (:name file)})
;; All features that are enabled and requires explicit migration are
;; added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features cfg)
(set/difference cfeat/no-migration-features)
(set/difference (:features file)))]
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature (:id file)]))
(bfc/save-file! cfg file ::db/return-keys false))
(doseq [thumbnail (read-seq cfg :file-object-thumbnail file-id)]
(let [thumbnail (-> thumbnail
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index))
file-id (:file-id thumbnail)
thumbnail (update thumbnail :object-id
#(str/replace-first % #"^(.*?)/" (str file-id "/")))]
(db/insert! conn :file-tagged-object-thumbnail thumbnail
{::db/return-keys false})))
(doseq [rel (read-obj cfg :file-rels file-id)]
(let [rel (-> rel
(update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))]
(db/insert! conn :file-library-rel rel
::db/return-keys false)
(bfc/upsert-file-library-sync! conn (assoc rel :synced-at timestamp))))
(doseq [media (read-seq cfg :file-media-object file-id)]
(let [media (-> media
(update :id bfc/lookup-index)
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index)
(update :thumbnail-id bfc/lookup-index))]
(db/insert! conn :file-media-object media
::db/return-keys false
::sql/on-conflict-do-nothing true))))
(def ^:private empty-summary
{:teams #{}
:files #{}
:projects #{}
:file-media-objects #{}
:team-font-variants #{}
:storage-objects #{}})
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PUBLIC API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn export-team!
[cfg team-id]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (create-database cfg)]
(l/inf :hint "start"
:operation "export"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [cfg]
(setup-schema! cfg)
(binding [bfc/*state* (volatile! empty-summary)]
(write-team! cfg team-id)
(run! (partial write-project! cfg)
(bfc/get-team-projects cfg team-id))
(run! (partial write-file! cfg)
(bfc/get-team-files-ids cfg team-id))
(run! (partial write-storage-object! cfg)
(-> bfc/*state* deref :storage-objects))
(write! cfg :manifest "team-id" team-id)
(write! cfg :manifest "objects" (deref bfc/*state*))
(::path cfg))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "export"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))
(defn import-team!
[cfg path]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (-> (create-database cfg path)
(assoc ::bfc/timestamp (ct/now)))]
(l/inf :hint "start"
:operation "import"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
(db/exec-one! conn ["SET idle_in_transaction_session_timeout = 0"])
(db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])
(binding [bfc/*state* (volatile! {:index {}})]
(let [objects (read-obj cfg :manifest "objects")]
;; We first process all storage objects, they have
;; deduplication so we can't rely on simple reindex. This
;; operation populates the index for all storage objects.
(run! (partial read-storage-object! cfg) (:storage-objects objects))
;; Populate index with all the incoming objects
(vswap! bfc/*state* update :index
(fn [index]
(-> index
(bfc/update-index (:teams objects))
(bfc/update-index (:projects objects))
(bfc/update-index (:files objects))
(bfc/update-index (:file-media-objects objects))
(bfc/update-index (:team-font-variants objects)))))
(let [team-id (read-obj cfg :manifest "team-id")
team (read-team! cfg team-id)
features (cfeat/get-team-enabled-features cf/flags team)
cfg (assoc cfg ::bfc/features features)]
(run! (partial read-project! cfg) (:projects objects))
(run! (partial read-file! cfg) (:files objects))
;; (run-pending-migrations! cfg)
team)))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "import"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))

View File

@ -231,6 +231,32 @@
(def validate-file-thumbnail (def validate-file-thumbnail
(sm/check-fn schema:file-thumbnail)) (sm/check-fn schema:file-thumbnail))
;; --- PROGRESS
(defn- tap-progress
"Emit a progress tap of the core, with the counters of the outer scopes
in course when the cfg has any.
The outer counters live in `::outer-counters` of the cfg while a scope
is being processed (today only `:files`, tomorrow `:teams` and
`:projects` too), so every tap inside carries them as `:outer-counters`:
the stages inside do not repeat them, and whoever reads the taps (the
jobs adapter) builds its report from one tap alone instead of
remembering the last position it saw. A tap that happens outside any
scope carries no outer counters, which is the truth about it."
[cfg data]
(events/tap :progress
(cond-> data
(some? (::outer-counters cfg))
(assoc :outer-counters (::outer-counters cfg)))))
(defn- with-outer-counter
"Return cfg with the counter of `scope` set to `position`, keeping the
counters of the other outer scopes already in course, so entering a
nested scope never wipes the ones above it."
[cfg scope position]
(assoc-in cfg [::outer-counters scope] position))
;; --- EXPORT IMPL ;; --- EXPORT IMPL
(defn- write-entry! (defn- write-entry!
@ -269,8 +295,10 @@
(defn- export-storage-objects (defn- export-storage-objects
[{:keys [::output] :as cfg}] [{:keys [::output] :as cfg}]
(let [storage (sto/resolve cfg)] (let [storage (sto/resolve cfg)
(doseq [id (-> bfc/*state* deref :storage-objects not-empty)] ids (-> bfc/*state* deref :storage-objects not-empty)
total (count ids)]
(doseq [[index id] (d/enumerate ids)]
(let [sobject (sto/get-object storage id) (let [sobject (sto/get-object storage id)
smeta (meta sobject) smeta (meta sobject)
ext (cmedia/mtype->extension (:content-type smeta)) ext (cmedia/mtype->extension (:content-type smeta))
@ -282,7 +310,8 @@
(write-entry! output path params) (write-entry! output path params)
(events/tap :progress {:section :storage-object :id id}) (tap-progress cfg {:section :storage-object :id id
:current (inc index) :total total})
(with-open [input (sto/get-object-data storage sobject)] (with-open [input (sto/get-object-data storage sobject)]
(.putNextEntry ^ZipOutputStream output (ZipEntry. (str "objects/" id ext))) (.putNextEntry ^ZipOutputStream output (ZipEntry. (str "objects/" id ext)))
@ -310,7 +339,7 @@
thumbnails (bfc/get-file-object-thumbnails cfg file-id)] thumbnails (bfc/get-file-object-thumbnails cfg file-id)]
(events/tap :progress {:section :file :id file-id :name (:name file)}) (tap-progress cfg {:section :file :id file-id :name (:name file)})
(vswap! bfc/*state* update :files assoc file-id (vswap! bfc/*state* update :files assoc file-id
{:id file-id {:id file-id
@ -344,7 +373,8 @@
(write-entry! output path page) (write-entry! output path page)
(events/tap :progress {:section :page :id page-id :name (:name page) :file-id file-id}) (tap-progress cfg {:section :page :id page-id :name (:name page) :file-id file-id
:current (inc index) :total (count pages)})
(doseq [[shape-id shape] objects] (doseq [[shape-id shape] objects]
(let [path (str "files/" file-id "/pages/" page-id "/" shape-id ".json") (let [path (str "files/" file-id "/pages/" page-id "/" shape-id ".json")
@ -355,30 +385,33 @@
(vswap! bfc/*state* bfc/collect-storage-objects media) (vswap! bfc/*state* bfc/collect-storage-objects media)
(vswap! bfc/*state* bfc/collect-storage-objects thumbnails) (vswap! bfc/*state* bfc/collect-storage-objects thumbnails)
(doseq [{:keys [id] :as media} media] (doseq [[index {:keys [id] :as item}] (d/enumerate media)]
(let [path (str "files/" file-id "/media/" id ".json") (let [path (str "files/" file-id "/media/" id ".json")
media (encode-media media)] encoded (encode-media item)]
(events/tap :progress {:section :media :id id :file-id file-id}) (tap-progress cfg {:section :media :id id :file-id file-id
(write-entry! output path media))) :current (inc index) :total (count media)})
(write-entry! output path encoded)))
(doseq [thumbnail thumbnails] (doseq [[index thumbnail] (d/enumerate thumbnails)]
(let [data (cth/parse-object-id (:object-id thumbnail)) (let [data (cth/parse-object-id (:object-id thumbnail))
path (str "files/" file-id "/thumbnails/" (:tag data) "/" (:page-id data) path (str "files/" file-id "/thumbnails/" (:tag data) "/" (:page-id data)
"/" (:frame-id data) ".json") "/" (:frame-id data) ".json")
data (-> data data (-> data
(assoc :media-id (:media-id thumbnail)) (assoc :media-id (:media-id thumbnail))
(encode-file-thumbnail))] (encode-file-thumbnail))]
(events/tap :progress {:section :thumbnails :id (:object-id thumbnail) :file-id file-id}) (tap-progress cfg {:section :thumbnails :id (:object-id thumbnail) :file-id file-id
:current (inc index) :total (count thumbnails)})
(write-entry! output path data))) (write-entry! output path data)))
(doseq [[id component] components] (doseq [[index [id component]] (d/enumerate components)]
(let [path (str "files/" file-id "/components/" id ".json") (let [path (str "files/" file-id "/components/" id ".json")
component (encode-component component)] component (encode-component component)]
(events/tap :progress {:section :component :id id :file-id file-id}) (tap-progress cfg {:section :component :id id :file-id file-id
:current (inc index) :total (count components)})
(write-entry! output path component))) (write-entry! output path component)))
(doseq [[id color] colors] (doseq [[index [id color]] (d/enumerate colors)]
(let [path (str "files/" file-id "/colors/" id ".json") (let [path (str "files/" file-id "/colors/" id ".json")
color (-> (encode-color color) color (-> (encode-color color)
(dissoc :file-id)) (dissoc :file-id))
@ -386,20 +419,22 @@
(and (contains? color :path) (and (contains? color :path)
(str/empty? (:path color))) (str/empty? (:path color)))
(dissoc :path))] (dissoc :path))]
(events/tap :progress {:section :color :id id :file-id file-id}) (tap-progress cfg {:section :color :id id :file-id file-id
:current (inc index) :total (count colors)})
(write-entry! output path color))) (write-entry! output path color)))
(doseq [[id object] typographies] (doseq [[index [id object]] (d/enumerate typographies)]
(let [path (str "files/" file-id "/typographies/" id ".json") (let [path (str "files/" file-id "/typographies/" id ".json")
typography (encode-typography object)] typography (encode-typography object)]
(events/tap :progress {:section :typography :id id :file-id file-id}) (tap-progress cfg {:section :typography :id id :file-id file-id
:current (inc index) :total (count typographies)})
(write-entry! output path typography))) (write-entry! output path typography)))
(when (and tokens-lib (when (and tokens-lib
(not (ctob/empty-lib? tokens-lib))) (not (ctob/empty-lib? tokens-lib)))
(let [path (str "files/" file-id "/tokens.json") (let [path (str "files/" file-id "/tokens.json")
encoded-tokens (encode-tokens-lib tokens-lib)] encoded-tokens (encode-tokens-lib tokens-lib)]
(events/tap :progress {:section :tokens-lib :file-id file-id}) (tap-progress cfg {:section :tokens-lib :file-id file-id})
(write-entry! output path encoded-tokens))) (write-entry! output path encoded-tokens)))
(when tokens-status (when tokens-status
@ -415,18 +450,19 @@
encoded-status (-> (encode-tokens-status tokens-status) encoded-status (-> (encode-tokens-status tokens-status)
(cfo/ids->names effective-tokens-lib))] (cfo/ids->names effective-tokens-lib))]
(events/tap :progress {:section :tokens-status :file-id file-id}) (tap-progress cfg {:section :tokens-status :file-id file-id})
(write-entry! output path encoded-status))))) (write-entry! output path encoded-status)))))
(defn- export-files (defn- export-files
[{:keys [::bfc/ids ::bfc/export-type ::output] :as cfg}] [{:keys [::bfc/file-ids ::bfc/export-type ::output] :as cfg}]
(let [original-ids ids (let [original-ids file-ids
ids (into ids (when (= export-type :include-libraries) (bfc/get-libraries cfg ids))) file-ids (into file-ids (when (= export-type :include-libraries)
rels (if (= export-type :include-libraries) (bfc/get-libraries cfg file-ids)))
(->> (bfc/get-files-rels cfg ids) rels (if (= export-type :include-libraries)
(mapv (juxt :file-id :library-file-id))) (->> (bfc/get-files-rels cfg file-ids)
[]) (mapv (juxt :file-id :library-file-id)))
[])
;; Compute external libraries: referenced by original files but ;; Compute external libraries: referenced by original files but
;; not included in the export set. Only relevant for :link-later. ;; not included in the export set. Only relevant for :link-later.
@ -451,10 +487,11 @@
(vswap! bfc/*state* assoc :files (d/ordered-map)) (vswap! bfc/*state* assoc :files (d/ordered-map))
;; Write all the exporting files ;; Write all the exporting files
(doseq [[index file-id] (d/enumerate ids)] (doseq [[index file-id] (d/enumerate file-ids)]
(-> cfg (-> cfg
(assoc ::file-id file-id) (assoc ::file-id file-id)
(assoc ::file-seqn index) (with-outer-counter :files {:current (inc index)
:total (count file-ids)})
(export-file))) (export-file)))
;; Write manifest file ;; Write manifest file
@ -747,17 +784,19 @@
(defn- read-file-colors (defn- read-file-colors
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
(->> (get-in entries-index [:colors (str file-id)]) (let [entries (get-in entries-index [:colors (str file-id)])]
(reduce (fn [result {:keys [id entry]}] (->> (d/enumerate entries)
(let [object (->> (read-entry cfg input entry) (reduce (fn [result [index {:keys [id entry]}]]
(decode-color) (let [object (->> (read-entry cfg input entry)
(validate-color))] (decode-color)
(events/tap :progress {:section :color :id id :file-id file-id}) (validate-color))]
(if (= id (:id object)) (tap-progress cfg {:section :color :id id :file-id file-id
(assoc result id object) :current (inc index) :total (count entries)})
result))) (if (= id (:id object))
{}) (assoc result id object)
(not-empty))) result)))
{})
(not-empty))))
(defn- read-file-components (defn- read-file-components
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
@ -776,15 +815,18 @@
(reduce-kv (fn [objects id shape] (reduce-kv (fn [objects id shape]
(assoc objects id (bfl/clean-shape-pre-decode shape))) (assoc objects id (bfl/clean-shape-pre-decode shape)))
objects objects
objects))))] objects))))
entries
(get-in entries-index [:components (str file-id)])]
(->> (get-in entries-index [:components (str file-id)]) (->> (d/enumerate entries)
(reduce (fn [result {:keys [id entry]}] (reduce (fn [result [index {:keys [id entry]}]]
(let [object (->> (read-entry cfg input entry) (let [object (->> (read-entry cfg input entry)
(clean-component-pre-decode) (clean-component-pre-decode)
(decode-component) (decode-component)
(clean-component-post-decode))] (clean-component-post-decode))]
(events/tap :progress {:section :component :id id :file-id file-id}) (tap-progress cfg {:section :component :id id :file-id file-id
:current (inc index) :total (count entries)})
(if (= id (:id object)) (if (= id (:id object))
(assoc result id object) (assoc result id object)
result))) result)))
@ -793,22 +835,24 @@
(defn- read-file-typographies (defn- read-file-typographies
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
(->> (get-in entries-index [:typographies (str file-id)]) (let [entries (get-in entries-index [:typographies (str file-id)])]
(reduce (fn [result {:keys [id entry]}] (->> (d/enumerate entries)
(let [object (->> (read-entry cfg input entry) (reduce (fn [result [index {:keys [id entry]}]]
(decode-typography) (let [object (->> (read-entry cfg input entry)
(validate-typography))] (decode-typography)
(events/tap :progress {:section :typography :id id :file-id file-id}) (validate-typography))]
(if (= id (:id object)) (tap-progress cfg {:section :typography :id id :file-id file-id
(assoc result id object) :current (inc index) :total (count entries)})
result))) (if (= id (:id object))
{}) (assoc result id object)
(not-empty))) result)))
{})
(not-empty))))
(defn- read-file-tokens-lib* (defn- read-file-tokens-lib*
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
(when-let [{:keys [entry]} (first (get-in entries-index [:tokens (str file-id)]))] (when-let [{:keys [entry]} (first (get-in entries-index [:tokens (str file-id)]))]
(events/tap :progress {:section :tokens-lib :file-id file-id}) (tap-progress cfg {:section :tokens-lib :file-id file-id})
(->> (read-entry cfg input entry nil) (->> (read-entry cfg input entry nil)
(decode-tokens-lib) (decode-tokens-lib)
(validate-tokens-lib)))) (validate-tokens-lib))))
@ -829,7 +873,7 @@
(defn- read-file-tokens-status (defn- read-file-tokens-status
[{:keys [::bfc/input ::entries-index] :as cfg} file-id tokens-lib] [{:keys [::bfc/input ::entries-index] :as cfg} file-id tokens-lib]
(when-let [{:keys [entry]} (first (get-in entries-index [:tokens-status (str file-id)]))] (when-let [{:keys [entry]} (first (get-in entries-index [:tokens-status (str file-id)]))]
(events/tap :progress {:section :tokens-status :file-id file-id}) (tap-progress cfg {:section :tokens-status :file-id file-id})
(->> (read-entry cfg input entry nil) (->> (read-entry cfg input entry nil)
(cfo/names->ids tokens-lib) (cfo/names->ids tokens-lib)
(decode-tokens-status) (decode-tokens-status)
@ -851,19 +895,21 @@
(defn- read-file-pages (defn- read-file-pages
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
(->> (get-in entries-index [:pages (str file-id)]) (let [entries (get-in entries-index [:pages (str file-id)])]
(keep (fn [{:keys [id entry]}] (->> (d/enumerate entries)
(let [page (->> (read-entry cfg input entry) (keep (fn [[index {:keys [id entry]}]]
(decode-page)) (let [page (->> (read-entry cfg input entry)
page (dissoc page :options)] (decode-page))
(events/tap :progress {:section :page :id id :file-id file-id}) page (dissoc page :options)]
(when (= id (:id page)) (tap-progress cfg {:section :page :id id :file-id file-id
(let [objects (read-file-shapes cfg file-id id)] :current (inc index) :total (count entries)})
(assoc page :objects objects)))))) (when (= id (:id page))
(sort-by :index) (let [objects (read-file-shapes cfg file-id id)]
(reduce (fn [result {:keys [id] :as page}] (assoc page :objects objects))))))
(assoc result id (dissoc page :index))) (sort-by :index)
(d/ordered-map)))) (reduce (fn [result {:keys [id] :as page}]
(assoc result id (dissoc page :index)))
(d/ordered-map)))))
(defn- read-file-thumbnails (defn- read-file-thumbnails
[{:keys [::bfc/input ::entries-index] :as cfg} file-id] [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
@ -904,8 +950,11 @@
:plugin-data plugin-data})) :plugin-data plugin-data}))
(defn- import-file (defn- import-file
[{:keys [::db/conn ::bfc/project-id ::manifest] :as cfg} {file-id :id file-name :name}] [{:keys [::db/conn ::bfc/project-id ::manifest] :as cfg}
(let [file-id' (bfc/lookup-index file-id) {file-id :id file-name :name}
position]
(let [cfg (with-outer-counter cfg :files position)
file-id' (bfc/lookup-index file-id)
file (read-file cfg file-id) file (read-file cfg file-id)
media (read-file-media cfg file-id) media (read-file-media cfg file-id)
thumbnails (read-file-thumbnails cfg file-id)] thumbnails (read-file-thumbnails cfg file-id)]
@ -917,16 +966,19 @@
:version (:version file) :version (:version file)
::l/sync? true) ::l/sync? true)
(tap-progress cfg {:section :file :file-id file-id})
(vswap! bfc/*state* update :index bfc/update-index media :id) (vswap! bfc/*state* update :index bfc/update-index media :id)
(doseq [item media] (doseq [[index item] (d/enumerate media)]
(let [params (-> item (let [params (-> item
(update :id bfc/lookup-index) (update :id bfc/lookup-index)
(assoc :file-id file-id') (assoc :file-id file-id')
(d/update-when :media-id bfc/lookup-index) (d/update-when :media-id bfc/lookup-index)
(d/update-when :thumbnail-id bfc/lookup-index))] (d/update-when :thumbnail-id bfc/lookup-index))]
(events/tap :progress {:section :media :id (:id params) :file-id file-id}) (tap-progress cfg {:section :media :id (:id params) :file-id file-id
:current (inc index) :total (count media)})
(l/dbg :hint "inserting media object" (l/dbg :hint "inserting media object"
:file-id (str file-id') :file-id (str file-id')
@ -936,10 +988,9 @@
:old-id (str (:id item)) :old-id (str (:id item))
::l/sync? true) ::l/sync? true)
(db/insert! conn :file-media-object params (db/insert! conn :file-media-object params)))
::db/on-conflict-do-nothing? (::bfc/overwrite cfg))))
(doseq [item thumbnails] (doseq [[index item] (d/enumerate thumbnails)]
(let [media-id (bfc/lookup-index (:media-id item)) (let [media-id (bfc/lookup-index (:media-id item))
object-id (-> (assoc item :file-id file-id') object-id (-> (assoc item :file-id file-id')
(cth/fmt-object-id)) (cth/fmt-object-id))
@ -953,13 +1004,12 @@
:media-id (str media-id) :media-id (str media-id)
::l/sync? true) ::l/sync? true)
(events/tap :progress {:section :thumbnail :file-id file-id :object-id object-id}) (tap-progress cfg {:section :thumbnail :file-id file-id :object-id object-id
:current (inc index) :total (count thumbnails)})
(db/insert! conn :file-tagged-object-thumbnail params (db/insert! conn :file-tagged-object-thumbnail params
::db/on-conflict-do-nothing? true))) ::db/on-conflict-do-nothing? true)))
(events/tap :progress {:section :file :file-id file-id})
(let [data (-> (read-file-data cfg file-id (:tokens-source file)) (let [data (-> (read-file-data cfg file-id (:tokens-source file))
(d/without-nils) (d/without-nils)
(assoc :id file-id') (assoc :id file-id')
@ -987,27 +1037,55 @@
file-id'))) file-id')))
(defn- import-file-relations (defn- import-file-relations
"Import library relations from the manifest, ignoring relations whose
endpoints are not both part of the package. Of the rest, skip edges that
would close a dependency cycle or link a file to itself (same check as
`link-file-to-library`, see `bfc/library-link-status`). Relations are
processed in sorted manifest order and the first edge wins: an edge
that would close a cycle with previously imported relations is skipped
and logged."
[{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}] [{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}]
(events/tap :progress {:section :relations}) (let [relations (sort-by (fn [[file-id libr-id]] [(str file-id) (str libr-id)])
(doseq [[file-id libr-id] (:relations manifest)] (:relations manifest))
file-ids (into #{} (map :id) (:files manifest))]
(doseq [[index [file-id libr-id]] (d/enumerate relations)]
(tap-progress cfg {:section :relations
:current (inc index)
:total (count relations)})
(let [file-id (bfc/lookup-index file-id) (if (and (contains? file-ids file-id)
libr-id (bfc/lookup-index libr-id)] (contains? file-ids libr-id))
(let [file-id (bfc/lookup-index file-id)
libr-id (bfc/lookup-index libr-id)]
(case (bfc/library-link-status cfg file-id libr-id)
:self-link
(l/warn :hint "ignoring self-referencing library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
(when (and file-id libr-id) :circular
(l/dbg :hint "create file library link" (l/warn :hint "ignoring circular library link"
:file-id (str file-id) :file-id (str file-id)
:lib-id (str libr-id) :lib-id (str libr-id)
::l/sync? true) ::l/sync? true)
(let [rel-params {:file-id file-id
:library-file-id libr-id}] :ok
(db/insert! conn :file-library-rel rel-params) (do
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))) (l/dbg :hint "create file library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
(let [rel-params {:file-id file-id
:library-file-id libr-id}]
(db/insert! conn :file-library-rel rel-params)
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))
(l/warn :hint "ignoring invalid file library relation"
:file-id (str file-id)
:lib-id (str libr-id))))))
(defn- import-storage-objects (defn- import-storage-objects
[{:keys [::bfc/input ::entries-index ::bfc/timestamp] :as cfg}] [{:keys [::bfc/input ::entries-index ::bfc/timestamp] :as cfg}]
(events/tap :progress {:section :storage-objects})
;; IMPORTANT: we strongly do not reuse the main connection that can ;; IMPORTANT: we strongly do not reuse the main connection that can
;; run inside a transaction because the storage upload process can ;; run inside a transaction because the storage upload process can
;; fail in the middle of uploading and leave garbage on the underlying ;; fail in the middle of uploading and leave garbage on the underlying
@ -1016,9 +1094,12 @@
;; what the storage subsystem registers in other parallel ;; what the storage subsystem registers in other parallel
;; transaction ;; transaction
(let [storage (sto/resolve cfg) (let [storage (sto/resolve cfg)
entries (:objects entries-index)] entries (:objects entries-index)
total (count entries)]
(doseq [{:keys [id entry]} entries] (doseq [[index {:keys [id entry]}] (d/enumerate entries)]
(tap-progress cfg {:section :storage-objects
:current (inc index) :total total})
(let [object (-> (read-entry cfg input entry) (let [object (-> (read-entry cfg input entry)
(decode-storage-object) (decode-storage-object)
(d/update-when :content-type bfc/normalize-content-type) (d/update-when :content-type bfc/normalize-content-type)
@ -1200,12 +1281,15 @@
(vswap! bfc/*state* update :index assoc old-lib-id library-id))) (vswap! bfc/*state* update :index assoc old-lib-id library-id)))
(let [files (get manifest :files) (let [files (get manifest :files)
file-ids (reduce (fn [result file] total (count files)
file-ids (reduce (fn [result [index file]]
(let [name' (get file :name) (let [name' (get file :name)
file (assoc file :name name')] file (assoc file :name name')]
(conj result (import-file cfg file)))) (conj result (import-file cfg file
{:current (inc index)
:total total}))))
[] []
files) (d/enumerate files))
;; Build map of file-id to file-name for resolution ;; Build map of file-id to file-name for resolution
files-info (into {} (map (fn [file-id manifest-file] files-info (into {} (map (fn [file-id manifest-file]
[file-id (:name manifest-file)]) [file-id (:name manifest-file)])
@ -1220,56 +1304,6 @@
{:file-ids file-ids {:file-ids file-ids
:resolution resolution}))) :resolution resolution})))
(defn- invalidate-thumbnails
[cfg file-id]
(let [storage (sto/resolve cfg ::db/reuse-conn true)
sql-1
(str "update file_tagged_object_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")
sql-2
(str "update file_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")]
(run! #(sto/touch-object! storage %)
(sequence
(keep :media-id)
(concat
(db/exec! cfg [sql-1 file-id])
(db/exec! cfg [sql-2 file-id]))))))
(defn- import-file-and-overwrite*
[{:keys [::manifest ::bfc/file-id] :as cfg}]
(when (not= 1 (count (:files manifest)))
(ex/raise :type :validation
:code :invalid-condition
:hint "unable to perform in-place update with binfile containing more than 1 file"
:manifest manifest))
(bfc/configure-import-transaction cfg)
(let [ref-file (bfc/get-minimal-file cfg file-id ::db/for-update true)
file (first (get manifest :files))
cfg (assoc cfg ::bfc/overwrite true)]
(vswap! bfc/*state* update :index assoc (:id file) file-id)
(binding [bfc/*options* cfg
bfc/*reference-file* ref-file]
(import-storage-objects cfg)
(import-file cfg file)
(invalidate-thumbnails cfg file-id)
(bfm/apply-pending-migrations! cfg)
{:file-ids [file-id]
:resolution {}})))
(defn- import-files (defn- import-files
[{:keys [::bfc/timestamp ::bfc/input] :or {timestamp (ct/now)} :as cfg}] [{:keys [::bfc/timestamp ::bfc/input] :or {timestamp (ct/now)} :as cfg}]
@ -1317,35 +1351,34 @@
(cfeat/check-supported-features! features))) (cfeat/check-supported-features! features)))
(events/tap :progress {:section :manifest}) (tap-progress cfg {:section :manifest})
(binding [bfc/*state* (volatile! {:media [] :index {}})] (binding [bfc/*state* (volatile! {:media [] :index {}})]
(if (::bfc/file-id cfg) (db/tx-run! cfg import-files*))))
(db/tx-run! cfg import-file-and-overwrite*)
(db/tx-run! cfg import-files*)))))
;; --- PUBLIC API ;; --- PUBLIC API
(defn export-files! (defn export-files!
"Do the exportation of a specified file in custom penpot binary "Do the exportation of a specified file in custom penpot binary
format. There are some options available for customize the output: format. The cfg carries what `bfc/export-cfg` froze: `::bfc/file-ids`
(the files to export), `::bfc/export-type` (how linked libraries are
handled) and `::bfc/output` (where the package is written).
`::bfc/export-type`: determines how linked libraries are handled. `::bfc/export-type` determines how linked libraries are handled.
Valid values: Valid values: `:include-libraries` (include linked libraries),
`:include-libraries` (include linked libraries),
`:merge-libraries` (embed library assets in the file), `:merge-libraries` (embed library assets in the file),
`:detach-libraries` (treat assets as basic objects), `:detach-libraries` (treat assets as basic objects),
`:link-later` (preserve component metadata for relinking on import)." `:link-later` (preserve component metadata for relinking on import)."
[{:keys [::bfc/ids] :as cfg} output] [{:keys [::bfc/file-ids ::bfc/output] :as cfg}]
(assert (assert
(and (set? ids) (every? uuid? ids)) (and (set? file-ids) (every? uuid? file-ids))
"expected a set of uuid's for `::bfc/ids` parameter") "expected a set of uuid's for `::bfc/file-ids` parameter")
(assert (assert
(satisfies? jio/IOFactory output) (satisfies? jio/IOFactory output)
"expected instance of jio/IOFactory for `input`") "expected instance of jio/IOFactory for `::bfc/output`")
(let [id (uuid/next) (let [id (uuid/next)
tp (ct/tpoint) tp (ct/tpoint)
@ -1371,6 +1404,11 @@
(vreset! ab true) (vreset! ab true)
nil) nil)
(catch clojure.lang.ExceptionInfo cause
(if (= :interrupt (:type (ex-data cause)))
(vreset! ab true)
(throw cause)))
(catch Throwable cause (catch Throwable cause
(vreset! cs cause) (vreset! cs cause)
(vreset! ab true) (vreset! ab true)
@ -1385,6 +1423,12 @@
(defn import-files! (defn import-files!
[{:keys [::bfc/input] :as cfg}] [{:keys [::bfc/input] :as cfg}]
(when (::bfc/file-id cfg)
(ex/raise :type :validation
:code :not-supported
:hint "in-place import is not supported"
:file-id (::bfc/file-id cfg)))
(assert (assert
(and (uuid? (::bfc/profile-id cfg)) (and (uuid? (::bfc/profile-id cfg))
(uuid? (::bfc/project-id cfg))) (uuid? (::bfc/project-id cfg)))

View File

@ -10,6 +10,7 @@
[app.common.data :as d] [app.common.data :as d]
[app.common.exceptions :as ex] [app.common.exceptions :as ex]
[app.common.flags :as flags] [app.common.flags :as flags]
[app.common.logging :as l]
[app.common.schema :as sm] [app.common.schema :as sm]
[app.common.time :as ct] [app.common.time :as ct]
[app.common.uri :as u] [app.common.uri :as u]
@ -77,6 +78,11 @@
:telemetry-uri "https://telemetry.penpot.app/" :telemetry-uri "https://telemetry.penpot.app/"
:jobs-lease (ct/duration {:minutes 30})
:jobs-retention (ct/duration {:days 7})
:jobs-user-ttl (ct/duration {:days 7})
:jobs-request-timeout (ct/duration {:minutes 2})
:media-max-file-size (* 1024 1024 30) ; 30MiB :media-max-file-size (* 1024 1024 30) ; 30MiB
:font-max-file-size (* 1024 1024 30) ; 30MiB :font-max-file-size (* 1024 1024 30) ; 30MiB
@ -97,12 +103,44 @@
:quotes-upload-sessions-per-profile 5 :quotes-upload-sessions-per-profile 5
:quotes-upload-chunks-per-session 20 :quotes-upload-chunks-per-session 20
:quotes-export-jobs-per-profile 10
:quotes-import-jobs-per-profile 10
:upload-max-chunk-size (* 1024 1024 30) ; 30MiB :upload-max-chunk-size (* 1024 1024 30) ; 30MiB
;; SSRF protection ;; SSRF protection
:ssrf-allowed-hosts #{} :ssrf-allowed-hosts #{}
:ssrf-extra-blocked-cidrs #{}}) :ssrf-extra-blocked-cidrs #{}})
(def schema:tenant
"Tenant identifier: hostname-label-style, lowercase (letters, digits
and hyphens). The value is normalized to lowercase when the
configuration is read, so an operator writing `PENPOT_TENANT=ACME` and
one writing `acme` name the same tenant: the value is compared as an
exact string in the keys and rows it lands in, and one uppercase
letter would otherwise split one tenant into two.
It is interpolated into Redis keys separated by dots (msgbus
topics, rate-limit buckets, exporter job keys, which are also matched
with a glob pattern), so `.` and whitespace are rejected at startup
because one tenant would then be able to read or overwrite another's
keys.
`%`, `_` and `:` are rejected too, and the reason they were added is
gone: the job queue used to be filtered with a LIKE over a
`<tenant>:<queue>` prefix and now has a `tenant` column. They are kept
because narrowing the rule is a startup contract change with no real
tenant asking for it, and a tenant that has one would have to be
renamed. Tenants using those characters must be renamed before
upgrading."
[:re #"^[a-z0-9-]+$"])
(defn- normalize-tenant
"Lowercase the configured tenant before it is validated, so
`PENPOT_TENANT=ACME` and `acme` mean the same tenant everywhere."
[config]
(cond-> config
(string? (:tenant config)) (update :tenant str/lower)))
(def schema:config (def schema:config
(do #_sm/optional-keys (do #_sm/optional-keys
[:map {:title "config"} [:map {:title "config"}
@ -110,7 +148,7 @@
[:admins {:optional true} [::sm/set ::sm/email]] [:admins {:optional true} [::sm/set ::sm/email]]
[:secret-key {:optional true} :string] [:secret-key {:optional true} :string]
[:tenant {:optional false} :string] [:tenant {:optional false} schema:tenant]
[:is-saas ::sm/boolean] [:is-saas ::sm/boolean]
[:public-uri {:optional false} ::sm/uri] [:public-uri {:optional false} ::sm/uri]
[:host {:optional false} :string] [:host {:optional false} :string]
@ -121,6 +159,10 @@
[:http-server-io-threads {:optional true} ::sm/int] [:http-server-io-threads {:optional true} ::sm/int]
[:http-server-max-worker-threads {:optional true} ::sm/int] [:http-server-max-worker-threads {:optional true} ::sm/int]
;; How often a websocket subscription re-checks that its access still
;; holds. The default lives in `app.http.websocket`, not here.
[:subscription-revalidation-interval {:optional true} ::ct/duration]
;; Explicit CORS allowlist used when the :cors flag is enabled. ;; Explicit CORS allowlist used when the :cors flag is enabled.
;; Configured via PENPOT_ALLOWED_ORIGINS as a comma/whitespace ;; Configured via PENPOT_ALLOWED_ORIGINS as a comma/whitespace
;; separated list of origins (e.g. "https://plugins.example.com"). ;; separated list of origins (e.g. "https://plugins.example.com").
@ -173,22 +215,29 @@
[:deletion-delay {:optional true} ::ct/duration] [:deletion-delay {:optional true} ::ct/duration]
[:file-clean-delay {:optional true} ::ct/duration] [:file-clean-delay {:optional true} ::ct/duration]
[:jobs-lease {:optional true} ::ct/duration]
[:jobs-retention {:optional true} ::ct/duration]
[:jobs-user-ttl {:optional true} ::ct/duration]
[:jobs-request-timeout {:optional true} ::ct/duration]
[:telemetry-enabled {:optional true} ::sm/boolean] [:telemetry-enabled {:optional true} ::sm/boolean]
[:default-blob-version {:optional true} ::sm/int] [:default-blob-version {:optional true} ::sm/int]
[:allow-demo-users {:optional true} ::sm/boolean] [:allow-demo-users {:optional true} ::sm/boolean]
[:error-report-webhook {:optional true} :string] [:error-report-webhook {:optional true} :string]
[:user-feedback-destination {:optional true} :string] [:user-feedback-destination {:optional true} :string]
[:mcp-feedback-email {:optional true} ::sm/email]
[:default-rpc-rlimit {:optional true} [::sm/vec :string]] [:default-rpc-rlimit {:optional true} [::sm/vec :string]]
[:rpc-rlimit-config {:optional true} ::fs/path] [:rpc-rlimit-config {:optional true} ::fs/path]
[:rpc-climit-config {:optional true} ::fs/path] [:rpc-climit-config {:optional true} ::fs/path]
[:audit-log-archive-uri {:optional true} :string] [:nexus-uri {:optional true} ::sm/uri]
[:audit-log-http-handler-concurrency {:optional true} ::sm/int] [:audit-log-http-handler-concurrency {:optional true} ::sm/int]
[:default-executor-parallelism {:optional true} ::sm/int] ;; REVIEW [:default-executor-parallelism {:optional true} ::sm/int] ;; REVIEW
[:worker-default-parallelism {:optional true} ::sm/int] [:worker-default-parallelism {:optional true} ::sm/int]
[:worker-webhook-parallelism {:optional true} ::sm/int] [:worker-webhook-parallelism {:optional true} ::sm/int]
[:worker-cron-parallelism {:optional true} ::sm/int]
[:worker-binfile-parallelism {:optional true} ::sm/int]
[:database-password {:optional true} [:maybe :string]] [:database-password {:optional true} [:maybe :string]]
[:database-uri {:optional true} ::sm/uri] [:database-uri {:optional true} ::sm/uri]
@ -196,6 +245,12 @@
[:database-readonly {:optional true} ::sm/boolean] [:database-readonly {:optional true} ::sm/boolean]
[:database-min-pool-size {:optional true} ::sm/int] [:database-min-pool-size {:optional true} ::sm/int]
[:database-max-pool-size {:optional true} ::sm/int] [:database-max-pool-size {:optional true} ::sm/int]
;; pgjdbc prepared statement cache (see app.db). Size is in bytes and is
;; converted to the whole MiB the driver accepts. Absent keys mean the
;; driver defaults; see docs/technical-guide/configuration.md.
[:database-prepared-statement-cache-threshold {:optional true} [::sm/int {:min -1}]]
[:database-prepared-statement-cache-queries {:optional true} [::sm/int {:min 0}]]
[:database-prepared-statement-cache-size {:optional true} [::sm/int {:min 0}]]
[:quotes-teams-per-profile {:optional true} ::sm/int] [:quotes-teams-per-profile {:optional true} ::sm/int]
[:quotes-access-tokens-per-profile {:optional true} ::sm/int] [:quotes-access-tokens-per-profile {:optional true} ::sm/int]
@ -207,6 +262,8 @@
[:quotes-font-variants-per-team {:optional true} ::sm/int] [:quotes-font-variants-per-team {:optional true} ::sm/int]
[:quotes-comment-threads-per-file {:optional true} ::sm/int] [:quotes-comment-threads-per-file {:optional true} ::sm/int]
[:quotes-comments-per-file {:optional true} ::sm/int] [:quotes-comments-per-file {:optional true} ::sm/int]
[:quotes-export-jobs-per-profile {:optional true} ::sm/int]
[:quotes-import-jobs-per-profile {:optional true} ::sm/int]
[:quotes-snapshots-per-file {:optional true} ::sm/int] [:quotes-snapshots-per-file {:optional true} ::sm/int]
[:quotes-snapshots-per-team {:optional true} ::sm/int] [:quotes-snapshots-per-team {:optional true} ::sm/int]
[:quotes-team-access-requests-per-team {:optional true} ::sm/int] [:quotes-team-access-requests-per-team {:optional true} ::sm/int]
@ -362,13 +419,31 @@
(def explain-config (def explain-config
(sm/explainer schema:config)) (sm/explainer schema:config))
(defn- normalize-file-data-backend
"The file data moved to `file_data` (migration `0157-migrate-legacy-file-data`), so
`file.data` and `file_change.data` are read-only leftovers and the `legacy-db`
back end is read-only: a write against it fails on every save. Accepting it would
start a healthy instance that cannot save a single file, so it reads as `db`,
which serves the legacy rows too. Warn instead of aborting: the operator may
still be halfway through the migration."
[config]
(if (= "legacy-db" (c/get config :file-data-backend))
(do
(l/warn :hint "file-data-backend=legacy-db is read-only, using db"
:file-data-backend "legacy-db"
:replacement "db")
(assoc config :file-data-backend "db"))
config))
(defn read-config (defn read-config
"Reads the configuration from environment variables and decodes all "Reads the configuration from environment variables and decodes all
known values." known values."
[& {:keys [prefix default] :or {prefix "penpot"}}] [& {:keys [prefix default] :or {prefix "penpot"}}]
(->> (read-env prefix) (->> (read-env prefix)
(merge default) (merge default)
(decode-config))) (normalize-tenant)
(decode-config)
(normalize-file-data-backend)))
(def version (def version
(v/parse (or (some-> (io/resource "version.txt") (v/parse (or (some-> (io/resource "version.txt")
@ -422,6 +497,41 @@
[& segments] [& segments]
(apply join-uri (c/get config :public-uri) segments)) (apply join-uri (c/get config :public-uri) segments))
(defn get-jobs-lease
"Max time a job can run without touching modified_at (heartbeat or
progress) before the dispatcher marks it as `aborted` (system-side
terminal, never retried, reported with an error log)."
[]
(or (c/get config :jobs-lease)
(ct/duration {:minutes 30})))
(defn get-jobs-request-timeout
"Default timeout for the ephemeral request! calls (waiting for the
reply-key blpop); can be overridden per call. Any override is applied
by raising the pooled connection command timeout for the duration of
the call, which the pool restores on return."
[]
(or (c/get config :jobs-request-timeout)
(ct/duration {:minutes 2})))
(defn get-jobs-retention
"How long terminal (completed/failed/cancelled/aborted) internal job rows are
kept before the jobs GC deletes them. The legacy `task` table is not
touched by the jobs GC: while both versions run in parallel it is
cleaned by the legacy `tasks-gc` of that version."
[]
(or (c/get config :jobs-retention)
(ct/duration {:days 7})))
(defn get-jobs-user-ttl
"How long a user-facing job, and the artifact it owns, is kept before
the jobs GC deletes the row and marks the object for the storage GC.
Not the same as `:jobs-retention`, which only sweeps terminal internal
rows."
[]
(or (c/get config :jobs-user-ttl)
(ct/duration {:days 7})))
(defn get (defn get
"A configuration getter. Helps code be more testable." "A configuration getter. Helps code be more testable."
([key] ([key]

Some files were not shown because too many files have changed in this diff Show More