The params schema of the ::revocation-watcher key named
:app.nitrate/client as a bare keyword. Inside [:map ...] a bare
keyword is not a key but a schema property, so malli resolved it
as a registry lookup. Nothing had registered that keyword, and
compiling the schema raised :malli.core/invalid-schema, which took
down the whole system build instead of one assertion.
Nothing caught it because the assertion only runs when
PENPOT_FLAGS carries enable-backend-asserts, so the suite passed
with the schema broken. With the flag the backend died before a
single test ran.
The client is declared as an explicit pair, the way
auth.oidc/schema:routes-params already does. It is optional
because init-key yields nil when the :admin-console flag is off,
and a required key would fail the assert on a normal boot.
The new test compiles the schema and pins both halves of that
contract, following the pattern already used for the three worker
schemas in backend-tests.worker-test.
AI-assisted-by: space-bunny-free
Restore the `or` of `=` comparisons for presence message types
instead of the `contains?` set literal. The literal builds a new
set on every forwarded message while `or` over interned keywords
allocates nothing, on a per-message path.
AI-assisted-by: muse-spark-1.3-contributor
Fold the subscription re-check into the relay loop itself: wait on
the subscription channel and a one-shot tick channel with `alts!`
and priority on the tick. A single tick channel covers a whole
cadence and message turns reuse it, so the cadence holds under load
without churning timers. Drops `schedule-revalidation` and the
`promesa.exec` require. No behavior change; the focused suite
passes untouched.
AI-assisted-by: muse-spark-1.3-contributor
Rename the `start-relay` callback param from `close!` to `close-fn`,
following the no-new-`!` direction and matching the sibling
`on-forward` callback that already carries no `!`.
Replace the two fixed-path `assoc-in` calls in `register-connection`
and `unregister-connection` with `update` plus `assoc`, as asked on
the PR. Both parent maps always exist in the atom, so the behavior
is identical.
AI-assisted-by: muse-spark-1.3-contributor
Rename the zero-arg permission check from `authorized?` to
`check-access` in `still-authorized?` and `start-relay`, so the
call `(still-authorized? check-access)` reads as running a check
instead of passing a predicate to a predicate. No behavior change:
`?` stays only on the safe yes/no answer.
AI-assisted-by: muse-spark-1.3-contributor
Follow-up to the GHSA-m53j-2766-6jqw fix on issue-12155, addressing
the reviewCode findings F1-F23.
The revalidation task is now cancelled when the relay ends (F1): the
closed channel is what ends it, because every way a subscription can
end closes it. Before, each ended subscription left a task polling the
database every 5 minutes for the life of the process while retaining
the closed connection.
The permission query now filters soft-deleted files, projects and
teams (F2), matching its sibling queries. The websocket re-check
therefore sees what the subscribe handler sees, and `delete-file`,
`delete-project` and the newly announced `delete-team` actually close
file subscriptions.
Team-wide changes publish one event per team instead of one per member
(F4): the watcher resolves the members itself, so a large team cannot
overflow the bounded publish buffers. Both publishers defer through the
existing `db/after-commit` (F3), so the watcher never checks
pre-commit state.
The interval is now genuinely tested (F6): revoking access through a
direct row delete, the way an external change would, plus a negative
control proving the tests fail with the scheduler removed. The
`captured-events` helper forwards keyword args correctly instead of
republishing to a garbage topic (F5), and the registry fixture runs per
test so no relay leaks between tests (F8).
The fail-open policy is pinned in both directions with the interrupt
rethrown (F7), the internal topic is a fixed uuid like every other
topic (F21), the watcher stops on shutdown (F22), the config key sits
with the other HTTP knobs (F19), and the memory no longer claims a
Promesa defect that the repo itself contradicts (F10).
Closes#12155
AI-assisted-by: space-bunny-free
The announcement path is the primary mechanism, but it cannot see an
access change that fires no command here: Nitrate transferring the
ownership of an organization is an HTTP call to another service, and
rows deleted straight over SQL publish nothing. Both keep a
subscription alive until the connection happens to close.
Each relay now re-checks access every `subscription-revalidation-interval`
(5 minutes by default, overridable) and drops the subscription when it
no longer holds. The default is a code constant in `app.http.websocket`
rather than the `config.clj` default map, matching how session
lifetimes are configured.
The re-check is a scheduled task rather than a timer inside the relay
loop, on purpose: driven by the loop, a steady stream of edits would
push the check away forever and a busy file would never be re-checked
at all.
`start-relay` also replaces the `sp/pipe` of the team subscription and
the ad-hoc `go-loop` of the file one, so both now share the same
forwarding and re-check path.
`still-authorized?` is the one place that fails open: a transient
database error logs and keeps the subscription rather than kicking a
user out of an editor, and the next tick tries again.
Closes#12155
AI-assisted-by: space-bunny-free
The revocation watcher only helps if the mutations that can take
access away actually announce it. Seven of them now do, after the write
that changed access:
- delete-team-member and leave-team announce the affected member.
- update-team-member-role announces too. A role change never removes
read on its own, but announcing lets the watcher decide from fresh
data instead of assuming, which is what keeps a downgrade to viewer
from being mistaken for a revocation.
- delete-file, delete-project, move-files and move-project announce to
the whole team. Which profiles had the content open is only known to
their own connections, and over-announcing is cheap: the watcher
re-checks and closes nothing for members who keep access.
delete-share-link is deliberately excluded. A share-link visitor never
passes the websocket subscribe check, because `check-read-permissions!`
is called without a share-id and that arity only considers team
membership, so there is no subscription to revoke.
leave-team also gains the notification it never had: it published
nothing at all on the leaver's own topic, so a session that left kept
its editor open with no idea, while a removed member was told.
The topic and the publish helper move to `app.rpc.notifications`, next
to the other bus notifications. The command namespaces cannot require
the watcher itself: `app.http.websocket` already requires these command
namespaces, so that would be a cycle. The watcher keeps the subscription
registry, which is the part it owns.
Closes#12155
AI-assisted-by: space-bunny-free
GHSA-m53j-2766-6jqw. A subscription is authorized once, when it is
opened, and that decision is trusted for the rest of the connection's
life. A member whose access is revoked keeps receiving the live file
diff for as long as their tab stays open.
Closing the connection from the mutation does not work: the registry
is local to one backend instance, while the RPC that revokes access
runs on whichever instance received the request. So the revocation
travels over the message bus instead, which every instance shares, and
each instance re-verifies the subscriptions it owns.
`revalidate-profile-subscriptions` looks the profile up in the new
index, then re-checks every file and team subscription with the same
`has-read-permissions?` predicates that `:subscribe-file` and
`:subscribe-team` authorize with. The event is only the trigger; the
fresh check is the decision, so the watcher keeps no second copy of the
permission rules. That is what makes a downgrade to viewer keep its
subscription, and keeps the read-only access of a non-member
organization owner.
`close-team-subscription` mirrors `close-file-subscription` for team
traffic. Both now forget the subscription instead of leaving a stale
one behind, which makes them idempotent and stops `:pointer-update`
from publishing to a topic the connection no longer relays.
The internal topic is separate from the `uuid/zero` system topic, which
is piped straight to the client. No client can reach it:
`:subscribe-file` and `:subscribe-team` only accept ids that already
passed a permission check.
Closes#12155
AI-assisted-by: space-bunny-free
The connection registry was a flat map from connection id to
connection data, so finding the connections of one profile meant
walking every connection. `on-connect` and `on-disconnect` also
`swap!`ed a whole-map `assoc`/`dissoc`, which is O(n) per connection
churn.
The registry now holds `:connections` and `:by-profile` in the same
atom, so one `swap!` keeps both consistent and a profile lookup is a
single map access. Both maps are updated together by
`register-connection` / `unregister-connection`; a profile whose last
connection leaves is dropped from the index instead of lingering with
an empty set.
Also extracts `close-file-subscription` out of the
`:unsubscribe-file` handler so a revocation can tear a subscription
down through exactly the same path the client uses: announce
`:leave-file`, close the relay channel, purge the bus. Closing the
channel is what stops the `:subscribe-file` go-loop, since `take!` on
a closed channel returns nil.
Adds `connections-for-profile` and `repl-get-connections-for-profile`
so an operator can inspect a profile's connections from the REPL the
same way as for a file or a team.
The new helpers take no `!` suffix: per `mem:clojure/design-rules` the
default is no `!`, and these are neither `app.db` API nor transient
helpers. Existing `!` names such as `mbus/pub!` are legacy and stay
until their own refactor.
The registry stays local to one backend instance. That is documented
on the var: reaching connections owned by another instance has to go
through the message bus, which is what the revocation watcher will do.
Closes#12155
AI-assisted-by: space-bunny-free
GHSA-m53j-2766-6jqw. A websocket subscription checks read permission
once, at subscribe time, then relays every message published to the
topic for the rest of the connection's life. A member whose access is
revoked keeps receiving live file content as long as the tab stays
open, because nothing re-checks the permission.
These tests reproduce the leak against the real message bus, so the
failure is observed end to end rather than assumed:
- revoked-member-stops-receiving-file-changes: after delete-team-member
the socket still receives the :file-change diff.
- revoked-member-stops-receiving-library-changes: leave-team keeps the
team subscription alive, leaking :library-change diffs.
- role-downgrade-keeps-revocation-irrelevant: a downgrade to :viewer
keeps read access, so the subscription must survive it. Guards
against a fix that closes subscriptions too eagerly.
The two revocation tests fail on the current code; the role test
passes and pins the behavior a fix must not break.
Closes#12155
AI-assisted-by: space-bunny-free
A plugin with "scope": "global" keeps running across the dashboard
and files and can find and open the user's projects and files. Any
manifest could ask for this without the user being told.
Add the allow:global permission. The plugin registry adds it to
every manifest with global scope, so the install dialog lists it,
and a plugin that becomes global in a later version shows the
permissions update dialog again. The permission is valid in the
shared schema, which the backend checks when saving, and in the
runtime manifest schema. The built-in MCP plugin declares it too.
Document the scope property and the new permission for plugin
developers.
* ✨ Run binfile import and export as durable jobs
Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.
Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.
How:
- The substrate gains the pieces a user-facing job needs: optional
job-def metadata (family, resource-role), an expiry on submit, the
owner in the runner context, and job resources in their own bucket,
written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
business params, the registry checks the family, the job-def decodes
and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
vocabulary with self-contained counters, throttled heartbeats, and an
interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
the package descriptor; the import re-checks the edition permission,
runs the core in a transaction so a cancellation rolls it back, and
releases the consumed package and its temporary copy.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Handle externally-cancelled binfile jobs end to end
The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.
- The export emits `{:cancelled true}` per file and the modal paints a
neutral cancelled row instead of success; a message without an
artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
they replace.
Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Pre-translate the cancelled import message
The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Move cancelled strings under generic labels keys
The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add minor improvements
* ♻️ Rename export job command to create-export-binfile-job
Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.
submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.
The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Polish the import and export status rows
The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.
In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Show the cancelled export and fix the import error
The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).
The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.
The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Reflect the real job params in the creation commands
The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.
Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.
A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Stop the test system from running job runners
The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.
That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.
Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Upload the package one chunk at a time and as a percentage
The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.
The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.
AI-assisted-by: deepseek-v4.1-flash
* 🌐 Complete the Spanish catalogue
es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).
sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Move the job status keys into the jobs namespace
The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.
Rename them so the whole job vocabulary is one namespace:
labels.queued -> jobs.queued
labels.export-cancelled -> jobs.export-cancelled
labels.import-cancelled -> jobs.import-cancelled
No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.
AI-assisted-by: deepseek-v4.1-flash
* 🔥 Remove the translation keys nothing references
Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:
labels.uploading-file a duplicate of jobs.progress.stage.upload
since the upload stage says "Uploading file…"
labels.downloading-file dead since the durable jobs replaced the old
import and export progress rows
inspect.attributes.stroke.alignment.center
inspect.attributes.stroke.alignment.inner
inspect.attributes.stroke.alignment.outer
workspace.toolbar.mcp-connect-here
The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
Giant legitimate exports hold hundreds of MB of JSON across
hundreds of thousands of KB-sized entries without tripping any
per-entry cap, so the 200MiB cumulative default rejected them out
of the box. Raise it to 500MiB; the 20MiB per-entry cap still
bounds single-entry inflation. Pin the floor with a regression
test.
AI-assisted-by: muse-spark-1.3-contributor
The S3 file download opens the temp file with CREATE_NEW, so it
fails when the path already exists. tempfile creates the empty
file on reserve, which made every large download depend on the
SDK deleting the placeholder between its own retries.
Add tmp/temp-path, which reserves a unique path and registers
cleanup without creating the file, and use it for S3 downloads.
The old retry on FileAlreadyExists stays as a safety net for a
name clash.
AI-assisted-by: muse-spark-1.3-contributor
Delete the team-scoped file, project and pin relations on leave-team
and delete-team-member, require a live team membership in the file,
project, search and listing permission queries, and clean
pre-existing orphan rows with a migration. Former members now get
not-found on file actions instead of keeping read and rename access
(GHSA-v9r9-h77c-55m2).
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Sanitize SVG namespace-prefixed elements and attributes
The SVG sanitizer compared qualified tag and attribute names against
its blocklists. The XML parser keeps the namespace prefix in the
keyword name (❌script), so a script element written with a prefix
bound to the SVG namespace passed sanitization untouched and was
later served as image/svg+xml, executing in the Penpot origin.
Match tags and attributes on the local name (the part after the last
colon) instead, so dangerous elements and attributes are removed no
matter which prefix they use. Legitimately prefixed attributes such
as xlink:href keep working.
Closes#12071
AI-assisted-by: glm-5.3-flash
* ♻️ Cover prefixed SVG vectors and hoist sanitizer blocklist
Apply the code review findings on top of the sanitizer fix:
- Add a test for the javascript: href vector using a prefix bound to
the xlink namespace, the other payload variant the old exact-key
check missed.
- Add a test asserting that legitimately prefixed attributes such as
xml:space are preserved by the local-name matching.
- Hoist the dangerous-tags blocklist next to the other sanitizer
policy defs and bind the keyword name once in local-name.
Relates to #12071
AI-assisted-by: glm-5.3-flash
* ✨ Gate error reporters behind :error-reporting flag
Error reporters used to start on every boot with no way to
switch them off. They now only start when the new
:error-reporting flag is on (off by default, enable with
PENPOT_FLAGS=enable-error-reporting). The database reporter
needs the flag alone; mattermost needs the flag plus its
webhook url. Shutdown with the flag off is safe, and the
database loop now honors its runtime switch.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Add audit-log reference memory and update dev tooling
Adds the top-level audit-log reference memory with all event
flows and turns backend/audit-log into a redirect stub.
Also includes the staged dev-tooling updates (playwright
deps, mdts args, gitignore).
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Assign server-side initiator to frontend audit events
Frontend audit events now carry a server-assigned initiator
derived from the x-client header (penpot-frontend goes to
app, penpot-admin-console goes to admin-console, legacy
penpot-nitrate stays on admin-console, all else goes to
app). Client-sent values are always overwritten, and the
key survives on telemetry shadow rows.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep initiator on frontend telemetry context
The strips-pii-keys test still asserted the old contract
(initiator stripped). The initiator only names the sending
app, so it is not personal data and must survive the
telemetry filter like on the backend channel.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Document initiator on PostHog and browser origins
Complete the audit-log memory for the latest changes: the
initiator property forwarded to PostHog on both channels
and the eventOrigin rule for admin-console browser events.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix unscaped argument in script
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Isolate backend-test database per devenv instance
Each wsN now gets its own backend-test database
(penpot_test_wsN) and Valkey DB (6+N) on the shared
infra, so parallel test runs no longer wipe each
other. The main database stays shared on purpose.
manage.sh creates the test database on bring-up and
passes PENPOT_TEST_* into the main container. Test
helpers already read those vars, so no runtime code
changes. scripts/psql and db-schema gain --ws.
AI-assisted-by: muse-spark-1.3-contributor-free
* 🐛 Fix per-instance test database creation on bring-up
psql -c does not reliably mix SQL with psql-only
commands, so the CREATE DATABASE ... WHERE NOT
EXISTS ... \gexec one-liner never ran. Use two
plain SQL round-trips instead (check pg_database,
then CREATE DATABASE) and print what happens, so a
future failure shows up in the bring-up output.
AI-assisted-by: muse-spark-1.3-contributor-free
Run the audit archive cron when :nexus or :admin-console is on. Ship
allowlisted events to Admin Console first (with row ids for
idempotency), then the full chunk to Nexus when :nexus is set. Mark
archived_at for the whole chunk on success, including nitrate-only
mode. Rename the gate flag from :audit-log-archive to :nexus.
AI-assisted-by: Composer
Co-authored-by: Cursor <cursoragent@cursor.com>
* ✨ Add size limits to profile props and plugin registry
Bound the total serialized size of profile settings to 2 MiB
(:profile-props-max-size), checked on the merged result before
persisting, with a controlled :props-too-large error. Profiles
that already exceed the limit can still shrink but cannot grow.
Cap plugin registry entries in the shared schema (code 1 MiB, 50
plugins max, bounded name/host/description/icon) and restore rate
limiting on the plugin RPCs (profile-mutations bucket, one write
at a time per profile). The plugin manager now asks for
confirmation before removal and ignores repeated clicks while a
persist request is in flight.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Enforce plugin count cap, byte sizes and removal guard
Enforce the declared 50-plugin cap in add-profile-plugin with a
specific :too-many-plugins error (updates of existing entries
still pass); the cap lives in a shared max-plugins constant.
Measure profile props size in UTF-8 bytes instead of chars so
multibyte content cannot slip past the limit.
Cover install/remove persist logic with mocked-RPC frontend tests
(release semantics, in-flight dedupe, validation vs rollback
split) and add the missing boundary tests in common.
Expose the in-flight persist set from the plugin registry and
disable the remove button of entries being saved.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix rollback loops and restore paths in plugin registry
Restore the previous plugin version instead of dropping the entry
when a validation error rejects an update of an installed plugin.
Make compensating writes one-shot with terminal callbacks so a
persistent failure cannot ping-pong between install and remove.
Restores keep the original list position; the unused public
plugin-persisting? predicate is removed.
Pin count-before-size precedence with a dedicated test and fix
translation source refs to their canonical lines.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Guard notifications write and fix restore ordering
Route update-profile-notifications through check-props-size! so
oversized profiles cannot grow through that path; document the
exempt system writers. Remove the duplicated stale entries in
en.po, keeping the canonical translation refs.
Restore rejected plugin updates at their original list position
instead of leaving the optimistic move in place.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Skip no-op plugin removal and clarify size comments
Return early from remove-profile-plugin when the id is absent:
no wasted write, no size check, and no manufactured :plugins key
that could spuriously fail on oversized profiles.
Clarify that per-field string caps count chars while the byte
budget is enforced by profile-props-max-size.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 📎 Fix formatting in rlimit.edn for profile operations
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Fix formatting of import-binfile/global entry
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Simplify props size check and tighten plugin entry caps
Measure props with transit bytes directly instead of the
PGobject string roundtrip.
Rename check-props-size! to check-props-size: single hard limit
on the merged props, no growth comparison, and return props so
writers thread the check into the update.
Move the 2 MiB default into default-props-max-size on the
profile namespace, still overridable with the optional
:profile-props-max-size config entry.
Tighten registry-entry :code and :icon to 500 chars: they hold
manifest paths, not content.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix compatibility problems
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ♻️ Scope organization notifications to specific WebSocket topics
Publish team/org notifications to team-id and organization-id topics
instead of broadcasting to all connections via uuid/zero. Dashboard and
workspace now subscribe to their current team and organization on
initialization, receiving only relevant events.
Backend: added subscribe-organization/unsubscribe-organization WebSocket
handlers and updated :close to clean up organization subscriptions.
Modified notify-team-change, notify-organization-deletion, and
notify-organization-change-sso to publish to specific topics.
Frontend: dashboard and workspace now subscribe to team-id and
organization-id (when applicable) on initialization, with nil-guard
in topic filters.
Closes#11455
AI-assisted-by: longcat-2.0
* 🔧 Remove unused session-id binding in unsubscribe-organization
Clj-kondo lint fix.
AI-assisted-by: longcat-2.0
* 🐛 Fix permission check on team ws connection
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Enforce idle and absolute session expiration
Sessions now expire on two server-side conditions: an idle window
(PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE, default 7d) and an absolute cap
from creation (PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE_ABSOLUTE, default
30d, enforced by the token :exp claim). The daily session-gc task
deletes rows that exceed either window, so idle sessions can no
longer be replayed and active sessions are not deleted at the idle
window.
Also remove the legacy v1 HTTP sessions: the http_session table and
the string-id / :ver 0 token code paths are gone. Any v1 cookie now
requires a fresh login.
Document the session expiration configuration in the technical guide
and add a backend memory describing the token, renewal and GC model.
Closes#11646
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Address session-expiration review findings F1-F4
Fix the unreadable test (a stray paren broke whole-suite
discovery), enforce idle expiration on every request in
wrap-authz, fail boot fast when the absolute cap sits below
the idle window, and align config defaults with the memory
rule while fixing its migration number and stale reference.
Closes#11646
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add Malli schema for storage metadata with dual decode
Phase 1 of the storage_object.metadata migration: reads accept both
Transit and plain JSON (sniffed by the marker) and always return the
normalized shape; writes validate against a closed per-bucket Malli
schema and still serialize as Transit unless the new
:storage-metadata-as-json config flag is set.
The 0155 migration normalizes existing rows inside Transit (reference
to bucket, default bucket, drop of chunk leftovers) and is
idempotent; large instances should fake it and run the batched
script instead.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address review findings on storage metadata Phase 1
Collapse the dead :reference leg of the gc-touched bucket fallback
(the decode always sets :bucket on non-nil metadata, so it is only
reachable with a NULL column) and fix its comment.
Pin the write flag off in the transit-assuming metadata tests so the
suite proves the same with the flag set, and add coverage for the
flag rollback contract, JSON hash survival, NULL metadata in
gc-touched, and the 0155 normalization statements.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Backfill NULLs, canonical buckets, comment fix
Backfill NULL metadata columns in 0155 via coalesce (the key-missing
rule already matches them), derive valid-buckets from the Malli schema
dispatch entries so the list lives in one place, and correct the
lookup-bucket fallback comment to NULL columns.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Defer corrupt metadata rows in storage gc-touched
Decode touched rows individually so one non-map metadata value no
longer aborts the whole chunk: corrupt rows are logged and deferred
exactly one day in the same transaction, keeping their metadata
intact for a later repair, while healthy rows process normally.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address storage metadata phase 1 review findings
Address the review findings on the storage metadata phase 1 branch:
- Fix put-and-delete-object: it stored the object with
::sto/expired-at, so the row was already deleted and del-object!
returned false. Add delete-expired-object-returns-false to keep
the expired-delete case covered.
- Cache the Malli decoder and encoder per process. Building them
compiles the closed multi-dispatch schema, and decode-metadata
runs on every read path (get-object, dedup probes, GC batches).
- Catch Exception instead of Throwable in try-decode-row so JVM
Errors are not deferred as corrupt metadata.
- Add penpot_storage_gc_poison_total, emitted from
storage-gc-touched; wire ::mtx/metrics into its handler.
- Anchor the encoding sniff to the start of the document so a
plain JSON value that begins with a Transit-looking prefix is
not read as Transit.
- Cover every bucket on both encodings, a JSON roundtrip through
the jsonb column, nil metadata, the canonical bucket set and a
poison-only GC chunk.
- Rename private check-metadata! to check-metadata.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Simplify the storage metadata schema to a single map
Replace the per-bucket :multi dispatch with a single closed map: the
bucket is validated with ::sm/one-of over metadata-buckets (now a plain
set) and the remaining keys are typed optional fields. Per-bucket
enforcement shrinks to a one-line :fn guard requiring :file-id and :id
for file-data, whose ids the GC reads to resolve references.
- Drop the dead (sm/register! ::metadata ...): nothing references the
schema by keyword.
- Define tempfile-bucket and upload-session-bucket in the schema and
alias them from app.storage, removing duplicated literals.
- Keep content-type required and the map closed, so an unknown bucket
or key still fails fast on write.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop input coercion from encode-metadata
encode-metadata no longer runs the json-transformer decoder before
validation. On the write path its only effect was coercing string
UUIDs to UUID, and every producer already passes native UUIDs (the
RPC profile-id, uuid/random, or binfile ids decoded as ::sm/uuid).
Reads keep decoding, so stored Transit or JSON values still come
back as native types.
- Replace encode-accepts-string-uuids with
encode-rejects-string-uuids, pinning the stricter contract.
- Pass native UUIDs in encode-writes-plain-json-with-flag.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Document each statement in the storage metadata migration
Move the per-statement rules out of the header and add a comment to each
UPDATE explaining what it does: drop chunk leftovers, promote the legacy
"~:reference" to "~:bucket", drop residual "~:reference", and backfill the
default bucket. The header keeps the scope, the encoding note, the `->`
vs `?` note and the large-instance warning.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap wrapped lines in the backend storage memory
One line per bullet or paragraph, as mem:memory-maintenance requires.
Only formatting; no content change.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Defer storage GC poison rows in their own transaction
process-chunk! no longer takes poison-ids; it only processes the healthy
chunk. The deferral moves to defer-poison! and process-touched! runs it in
its own transaction, separate from the freeze/delete work. The loop still
drains while there is chunk or poison, so a batch made only of poison rows
does not leave healthy rows behind the LIMIT 10 waiting for the next run.
Add a regression test: ten poison rows plus one healthy row with a later
touched_at are all handled in the same run.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Declare per-bucket metadata requirements in one map
Replace the file-data-specific predicate with bucket-requirements, a map
from bucket to the extra keys it must carry. metadata-buckets is derived
from its keys and a single generic :fn enforces presence, so a new bucket
and its contract are one entry. organization now requires
:organization-id; file-data keeps requiring :file-id and :id.
Update the http-assets test helper to set organization-id for its
organization objects.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC helpers
Rename the internal helpers in app.storage.gc-touched (process-chunk,
defer-poison, mark-freeze-in-bulk, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC deleted helpers
Rename the internal helpers in app.storage.gc-deleted (clean-deleted,
delete-sobjects, delete-give-up, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix dedup lookup for JSON-encoded storage metadata
get-database-object-by-hash only matched the Transit keys, so once the
:storage-metadata-as-json flag wrote plain JSON rows the dedup stopped
finding them and duplicated blobs. Match both encodings with a UNION ALL
of two indexable branches.
- Add migration 0156 with the plain-key dedup index; the legacy 0068
index stays until Transit support is removed.
- Cover it with a JSON dedup test and a Transit -> JSON cross test.
AI-assisted-by: deepseek-v4.1-flash
The import-svg-attrs-asset helper called (first result) on the
map returned by import-files!, yielding a clojure.lang.MapEntry
instead of a file UUID. PostgreSQL rejected the MapEntry as a
query parameter with "Can't infer the SQL type".
Extract the id via (first (:file-ids result)) as the neighboring
test already does.
AI-assisted-by: longcat-2.5-preview-free
The event was written twice per accepted organization invitation. The
backend submitted it, and the browser then re-submitted a copy of the
props that the backend had already put in the response under
`:organization-invitation-audit` (`handle-token :team-invitation` in
`verify-token.cljs`). Both rows carried the same name with different prop
vocabularies, and the browser copy only existed when the browser finished
the flow.
Emit the event from the backend only. It now also carries the three props
that lived in the browser copy: the organization member count before the
add, the add source, and whether the invitee also joined a team. The
origin moves to the event context as `:event-origin`. The response no
longer includes `:organization-invitation-audit`, so the browser stops
emitting the event and `verify-token.cljs` drops its
`app.main.data.event` require.
The `accept-*` events of this command now share one prop vocabulary:
`:profile-id` for the accepting profile, `:invited-by` for the inviter
and `:profile-email` for the email, replacing the mix of
`:user-id`/`:user-who-send-invitation` and `:email`.
Audit consumers of `accept-organization-invitation` now see one row per
acceptance instead of two, and must read the new prop names.
AI-assisted-by: space-bunny-free
The json reader of the binfile v3 import rewrites every key of every
entry to kebab-case, nested maps included. Shape `:svg-attrs` is the one
map whose keys are camelCase react prop names, because the svg import
path runs them through `attrs->props`, so an attribute exported as
`fillRule` came back as `:fill-rule` and was stored that way.
The renderer looks the attribute up by its camelCase name, does not find
it and falls back to the default fill rule, so a shape exported with
`fillRule: evenodd` was painted without its hole, and the attributes
panel showed `fill-rule`.
`clean-shape-post-decode` already runs on every shape right after the
schema decode, for page shapes and component objects alike, so the
repair goes there: run `:svg-attrs` back through `attrs->props`, the
same transform that built the keys. It is idempotent, so shapes that
arrive correct are left untouched.
The new tests import a real export that carries the attribute, for
page shapes and component shapes.
Closes#11954
AI-assisted-by: space-bunny-free
* 🐛 Attribute audit events to the caller, not the response
prepare-rpc-event took the event profile-id from the result map
whenever it carried one, before falling back to the caller. Any
command returning a response with a :profile-id key silently
credited the action to somebody else.
get-error-report returns the report with its decoded content
merged in, and that content holds the profile that owned the
report, so privileged reads were logged against the users whose
crashes were being inspected. verify-token on a team invitation
returns the inviter's profile-id, so accepting an invitation was
logged against the inviter.
Resolution is now ::audit/profile-id metadata, then
::rpc/profile-id, then the zero uuid; the response is never
consulted. The two verify-token branches that relied on it now
declare the profile in the result metadata. Every other command
either already declared it or returns no :profile-id; all 30
registered command namespaces were checked.
The tests that pinned the old behavior are replaced by ones
covering the new contract.
AI-assisted-by: space-bunny-free
* 🐛 Coerce the audit profile-id override to a uuid
The only sanctioned way for a command to override the profile of an
audit event is the ::audit/profile-id metadata, and the value is set
by hand in a dozen commands, some of them reading it from token
claims or other sources we do not type.
schema:event requires a uuid and submit* swallows the validation
error, so a string did not fail loudly: the row was dropped silently.
Values that cannot become a uuid are now discarded with a warning
and the event falls back to the caller, which is always a valid uuid.
A uuid, the common case, exits on the first check.
AI-assisted-by: space-bunny-free
* 🐛 Migrate openUIApi schema to Zod v4 function syntax
Zod 4 removed z.function().args(), which broke the
plugins-runtime build with implicit-any errors on every
openUIApi parameter and knock-on possibly-null errors on
the modal in plugin-manager.
Declare the inputs with z.function({ input: [...] }) so the
parameter and return types infer again; behavior is unchanged.
Add a regression spec covering delegation, optional args and
rejection of invalid theme and title values.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Fix deprecated markdown-it-anchor permalink option in docs
Migrate docs Eleventy config to the markdown-it-anchor v10 API.
Replace the deprecated boolean permalink option with
linkInsideHeader, keeping the same symbol and class.
Bump markdown-it-anchor to v10 and related docs deps.
AI-assisted-by: muse-spark-1.3-contributor
* ⬆️ Update deps
* ⬆️ Update base docker images
* 📎 Fix mcp tests
Bump the devenv and image Dockerfiles to pnpm 12.6.0
(latest stable; 12.8.0 is still on the next tag) with
fresh SHA256 pins, and opencode to 1.18.33 plus
opencode2 to 2.0.18 with fresh checksums.
Stamp all 35 package.json files via
scripts/sync-pnpm-version and refresh the 11 lockfiles;
diffs are metadata-only, with no dependency re-resolution.
Drop the last corepack calls from the media-processor
build script and its generated image setup: pnpm now
ships as a system binary and self-heals version drift.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add renderer option on create-demo-profile command
Allow create-demo-profile to take an optional renderer (svg or
wasm) and store it on profile props. When omitted, no renderer is
written so the Penpot default still applies.
Closes#11893
AI-assisted-by: Muse Spark 1.3 Free
* 📚 Fix renderer doc version on create-demo-profile
Correct the ::doc/changes version for the new renderer param from
2.20 to 2.18.1.
AI-assisted-by: Muse Spark 1.3 Free
