Depth gate: the compare reads the op log and applies the depth gate
before it derives the branch, the same order merge and update from
main use, so a log too deep to replay refuses before the derive runs.
The refusal names the materialise and the delete, the two remedies the
gates do not refuse, in the hint and in the notification copy.
Resolutions: the conflict answers of merge-file-branch and
update-branch-from-main carry the main revision the conflicts were
computed against, and the client sends it as expected-main-revn when
resolutions are applied before the dialog's diff fetch returns.
Libraries: absorbing skips branch files, so unpublishing or deleting a
shared library writes no payload on a branch and the branch keeps its
own link; a soft-deleted library resolves nothing there until the
branch's next update from main brings main's absorbed copies. The
absorbed write passes the file's features to update-file!, which the
hand-built params dropped, so the pointer-map fragments the absorption
regenerates are persisted and no payload references a fragment no
write produced.
Queries: the project file count, the team file count, the list of
files that use a library, and the socket-REPL team duplication skip
branch files.
Snapshots: the branch refusal lives in check-restorable!, which the
restore command calls before its backup snapshot and the two
socket-REPL restore helpers reach through restore!.
Migration 0155: the base_branch_revn backfill repositions only the
rows whose branch file carries a data payload, so a re-run changes
nothing on the branches the op-log model creates.
Tests: the branch merge tests run in JavaScript, which the common test
runner lists; new backend tests pin the gates, the absorption skip,
the query filters, the restore refusals, the migration re-run, and the
conflict answers' revision; a new frontend test pins the token the
senders carry with resolutions.
AI-assisted-by: mixed models
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The branch comments called the set of ids a document's references are
written in (main's file id and media rows, or the branch's) a frame.
Penpot's code uses frame for a board, through `:frame-id`, so a reader
could not tell which sense a comment meant. The comments, the
docstrings of `get-base-data`, `branch-base-data`, and `remap-changes`,
and one test name now say id space. Every board-sense use of frame is
unchanged.
AI-assisted-by: mixed models
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Four comments cited docket entry ids that resolve only on the machine
the set lives on, and the reviewer this branch is written for cannot
follow them. Each citation stood beside the number or the rule it
records, so the sentences stand without it.
AI-assisted-by: mixed models
Merge engine: a same-parent reorder, an addition under a parent main
deleted, and a branch deleting a container main edited each lost work
silently. Each now follows a policy read from
backend/resources/app/branch-merge-policies.edn, re-read when the file
changes, with the defaults: merge the reorder, and raise a conflict on
the deleted parent or container. Added siblings keep the branch's order.
A :mov-page index counts the target's full page list, a cleared page
background reaches main, a per-attribute resolution can remove an
attribute, and a token set reorder survives a rename on the other side.
Conflict payloads name :main and :branch by document in both
directions. Reorder detection costs nothing measurable on a page of
20,201 objects.
Data versions: each op-log row records the data version it was written
at, the branch replays its log in segments and migrates the document
at each boundary, and the comparison base is migrated like main and the
branch, so an upgrade no longer shows as false conflicts.
Writers: a binfile import, an admin repair, file GC and a snapshot
restore refuse a branch file, whose payload nothing reads, and the
branch base pin cannot be renamed into a user snapshot. A branch save
runs the ordinary media fix-up and logs the rewritten references. An
update from main deletes the branch's copy of media main deleted and
squashes no phantom media changes.
Commands and UI: the pull-request commands publish their messages after
the commit, the listing takes one connection per comparison and shows a
warning instead of "in sync" when a comparison fails, materialize takes
:branch-file-id, and a dashboard fetch that never answers ends in an
error state instead of a spinner, with a refetch on reconnect.
backend/dev/branch_merge_cases.clj builds one example file per merge
policy case for demonstrating the alternatives.
Two models wrote this change, claude-opus-5-5 and mimo-v2.6-pro, and
the work switched between them, so the trailer reads mixed models.
AI-assisted-by: mixed models
The source notes on files_branch.clj listed 32 review findings. This
commit fixes the 21 that are defects of the code; each fix carries a
regression test that failed before it and passes after.
Concurrency: merge, update from main, rename, archive and delete now
re-read the branch row under the advisory lock and work off that row,
so a concurrent merge or update can no longer leave a ten-year base
pin behind or release a superseded base twice. Delete and archive take
the branch file lock, create and reopen of a pull request refuse a
branch that a concurrent merge closed, and create checks the branch
quota under the lock. Merge, update and delete publish their msgbus
messages only after the transaction commits.
Merge, update and materialize re-validate after the repair: merge and
update refuse a result that stays invalid, materialize logs it. The
change log row carries the repair changes, and the three paths drop
the cached file summary under :redis-cache.
Contracts: a :conflict refusal answers 409 instead of 500, resolution
values must be :main or :branch, and update from main accepts the
expected-main-revn token the frontend now sends from a loaded diff.
Smaller fixes: create refuses a blank name, writes no file_data row,
and a rename reaches the branch file's own name. Deleting a kept
merged branch keeps its merged status. get-branch-diff reports the
creation time of the current base snapshot. Rename, archive and delete
report their duration and outcome. An image used only by a library
colour is copied into the target file. Dead fallbacks and dead
deleted-at checks are gone.
AI-assisted-by: claude-opus-5-5
Comments written for the duplicate-file prototype still described a
branch as a copy of its file, a merge as clean-only, token diffing as
missing, a review as a read-only snapshot, and file_change rows as
living for the team deletion delay. Each now states what the code at
this revision does: a branch is a file row derived from a pinned base
plus its op log, every change kind except :page-attrs is translated,
merges apply resolutions, a review opens the live editable branch, an
update squashes the log and repositions the base, and file_change rows
expire after one hour. The audited docstring names the props it sets,
the step 6 comment says each revn counter starts at its own side, and
handle-file-merged names both topics it arrives on.
Comment, docstring, and SQL-comment lines only. clj-kondo reports no
errors on the touched Clojure files. Translations are untouched.
AI-assisted-by: claude-opus-5-5
Data loss, reproduced twice rather than inferred. A branch stores no
data of its own, so its whole work lives in `file_branch_change`, and
`update-branch-from-main` had a fast path that deleted every one of
those rows whenever the update found nothing of main's to apply. The
comment there assumed the branch was in sync with main, which holds only
when the branch has no changes of its own, and `branch-diverged?` was
already computed two lines above and consulted only inside
`reposition-base!`.
A branch with one fill edit of `#bb0000` and an untouched main, updated
with no resolutions, returned `:updated` and came back `#000000` with an
empty log. The user asked to catch up with a main that had not moved and
lost their edit. Through the modal the same defect made the resolution
that says "keep mine" the one that discards it.
The fast path is now guarded by `(not branch-diverged?)`, so a diverged
branch falls through to the squash that re-derives its net against the
new base. The predicate is the right one: it compares the branch file's
own revision counter against the revision the base was last positioned
at, and a log row and a revision bump are one event, because
`persist-branch-file!` is the log's only appender. The comment at the
guard states the invariant, which is that the log may be emptied only
when what it encodes already sits in the base.
One correction to how the fast path was described while this was being
chased: `(empty? changes)` does not mean the branch has nothing of its
own. In the update direction the change set is MAIN's contribution, so a
branch that is behind main in content already took the squash path and
still does. The fast path is reached when main has nothing to contribute,
which is what the third new test pins.
The three sibling paths that also empty or replace a log were each
judged and are sound, because each one writes the state elsewhere inside
the same transaction before emptying.
Three regression tests, one per reproduction plus the case the fast path
exists for, asserting through the derived document rather than through
log row counts, because a legitimate squash rewrites the log while
preserving the state.
Verified: the whole backend suite, 923 tests and 5,550 assertions with 0
failures. The two live demo branches were checked read-only and still
hold their one op-log row each, so the hand-test fixture was never
damaged.
Two defects with one cause, which is that a comparison and a write want
different identifier frames and the code used one frame for both.
Bringing main into a branch compared in the branch's frame. It remapped
main and the base into that frame, which puts the branch's inherited
content where it cannot agree with the base: everything the branch
inherited still names main, so every reference the remap rewrote read as
a difference. On a demo branch that was 364 of 368 reported conflicts,
and none of them was an edit anybody made. The comparison now runs in
main's frame, exactly as the compare view and the listing summary
already do, and the changes it produces are translated into the branch's
frame before they are applied and logged. That translation is the new
`branch_merge.cljc::remap-changes`, which is a separate function rather
than a call to `remap-refs` for a reason its docstring gives: `relink-refs`
finds a reference where the attribute and the value travel together, and
a change vector carries the identifier alone. The write map is renamed to
`apply-map`, so no later reader can reach for it as the comparison map.
The compare also decoded its merge base twice. The derive resolves and
decodes the pinned snapshot in order to fold the op log over it, and the
caller then decoded the same bytes again for the comparison's third
version, which is about 406 ms on the design-system file and is paid by
every uncached branch summary in a listing as well. The derive now
offers the pre-fold value it already holds, behind an explicit opt-in, so
an ordinary read gains neither a key nor a cost, and a branch read that
forgets the flag raises rather than comparing against nothing. The
invariant a comment now states is that the comparison's base is the
pre-fold value, which is sound because the fold is pure: `process-changes`
threads a value, every method builds persistently, and `PointerMap`'s
`assoc` and `without` return new objects, so the pre-fold base shares
structure with the folded result and costs no extra memory for the pages
the log did not touch.
The new test asserts both halves, because either alone passes a wrong
implementation. A branch carrying only inherited content, updated from a
main that changed elsewhere, reports no conflicts and leaves an empty op
log. Main and the branch repainting the same shape still reports exactly
one conflict, naming that shape.
Verified: the whole backend suite, 919 tests and 5,525 assertions with 0
failures.
Three of the preview's paths reported no duration, and two of them were
the slowest a user meets. A trial can be asked what a merge cost and
could not be asked what the panel cost.
The listing now passes through `audited` with two counters beside the
duration, being how many open branches it compared and how many of those
the summary cache answered, because a listing's cost is entirely
explained by that ratio. The vector result was the one risk and it holds:
a Clojure vector carries metadata, and nothing between the handler and
the audit step rebuilds the value.
A branch open now reports the derive's own time, its op-log depth, and
its batch count, from `branch-file-data` itself. Depth is what predicts
the cost, so the record carries it. An ordinary read never enters that
function, so it gains no line, no prop, and no timer.
A branch save gains the log line the other five print, and its timer
moved rather than gaining a caveat. It now opens before `get-file`, so
the number covers the read and, for a branch, the derive inside it, which
is what compare, merge, update and materialize have always covered. The
six figures are comparable for the first time.
The concurrency limits were decoration. Thirteen ids were declared across
the branch and pull-request commands and `climit.edn` configured none, so
`climit/wrap` logged a warning at every start and ran each handler
unwrapped, leaving the root chain's forty global permits as the only
bound on an operation that holds three realized documents. Every declared
id now has an entry, the two readers that realize whole documents gained
a declaration, and each number is justified in a comment against
`create-file-snapshot` as its anchor and marked provisional until a
peak-heap measurement replaces it.
The test helper drives the private save path directly, so it opens the
timer the RPC method would have opened.
Verified: 73 tests, 694 assertions, 0 failures across the three branch
suites and `rpc-file-test`.
The preview's observability criterion is stated as every branch operation, and
two of them sat outside it. A save routed to a branch's op log carried only the
ordinary save props, and materialise carried none, so the two operations that
matter most to an enterprise trial were the two with no duration in the record.
Both attach `branch-operation`, `branch-outcome` and `branch-duration-ms` now:
`save-branch`/`saved` on the save path, `materialize`/`materialized` on the exit
door, including its idempotent no-op. The props travel in `::audit/replace-props`
on the save path because that is the key the ordinary save already replaces, so
the ordinary save's own props are untouched.
The test that asserts the audit props now covers a save and a materialise
alongside creation and merge. The three branch namespaces are 40 tests and 338
assertions, green.
The pull-request listing and the file-open info query each recomputed the
same [ahead behind conflicts] the branches listing had already cached, so
a file with ten open pull requests paid the whole cold cost the branch
panel no longer pays. All three now go through cached-diff-counts, keyed
on the same (base-snapshot-id, source-revn, branch-revn) tuple, and none
of them realizes main when every row it needs is cached.
A no-op merge, a conflict refusal and an unsupported-kind refusal are
outcomes the first enterprise trial will ask about, and only the applied
path was carrying its duration.
Three gates, each a measured cost divided into a 30 s budget and halved:
50,000 shapes (a merge on 21,169 shapes costs about 6 s), 500 pages (a
whole-file comparison of 48 pages costs 402 ms), and 100,000 op-log
changes (one value-derived squash already emits about 19,000). Each
refusal is a :restriction naming the limit, the actual value and what to
do instead, and none of them truncates anything.
The listing never refuses: it renders the branch panel, and its cost is
bounded by the summary cache instead.
Every branch operation also attaches its duration and outcome to its own
audit event, because duration is the number the first enterprise trial
will be asked about and the generic RPC event does not carry it.
The op log IS the delta between a branch and its merge base, so folding
validate/extract-affected-ids over it yields the exact set of pages the
branch can differ in. compute-merge takes :only-pages and bounds every
page pass to it, the presence pass included, which is where a whole-file
comparison spends its time.
The forward pass is bounded; the reverse one reports main's changes and
stays whole-file. The bound is dropped whenever the log holds a change
the reducer cannot scope to a page or a component (a deletion, page
order, a colour, a token), because for those the affected set is
incomplete and a bounded diff would hide work.
::materialize-file-branch persists a branch's derived state as the
file's own data payload, drops the op log, releases the pinned base
snapshot, closes any pull request over it and clears the branch marker,
so every read and write path takes the ordinary route from then on.
That is the preview's exit door: the flag can go off and the engine can
be replaced without stranding anybody with an unreadable file.
It keys on the branch file rather than the branch metadata, so calling
it twice is a reported no-op rather than an error.
The fetch the cache exists for is a file open, and expireAfterAccess of
ten minutes threw the entry away between two sessions on the same file:
a listing measured at 5 ms was 26 s again fifteen minutes later. An
entry is a uuid and five numbers, so the window costs nothing.
The listing's per-branch [ahead behind conflicts] summary is cached in
memory keyed by (base-snapshot-id, source-revn, branch-revn), which
names exactly the inputs the computation reads. A stale key cannot
exist, so the cache needs no invalidation logic: a repeat listing with
no intervening save performs no comparison work, and a cache miss is
the old behaviour. Failed computations are not stored, so a broken
branch keeps degrading to zeros with a warning.
Swap the storage and materialization layer of file branching to remove
the per-branch data duplication, keeping the RPC contracts and the
frontend unchanged.
- file_branch_change (0154): one row per branch save, the blob-encoded
change vector, ordered by revn; branch files store no data payload.
- Read path: app.binfile.common/get-file* derives the branch file data
by replaying the op log over the pinned merge-base snapshot, carrying
the base's version and migration set so need-migration? stays quiet.
- Save path: update-file routes branch files to persist-branch-file!,
which appends the vector to the op log and updates the file row
without any file_data write.
- Creation: create-file-branch no longer duplicates main's data; it
inserts an empty branch file row, grants the creator, and copies the
library rels.
- update-branch-from-main squashes the op log after integrating main's
changes: the repositioned base already carries main's side, so the
log is replaced with the net branch-only changes.
- Kept (keep-branch) branches retain the base pin: their state is
derived, so releasing the pin would leave them unreadable after GC.
Tests: backend-tests.rpc-file-branch-oplog-test asserts the storage and
derivation invariants; the existing branch, file, binfile, snapshot,
thumbnail, and pull request suites pass unchanged against the swapped
engine (84 tests, 1487 assertions, 0 failures).
This commit carries the design-branching preview by Juan de la Cruz:
elhombretecla/penpot:branching at 0ac265a3f5, which holds 45 commits
and 3 develop merges and was last synced with develop on 2026-09-16.
The branch is squashed together with a merge of origin/develop at
7c039231f7, so the tree is the merged result.
The merge resolved conflicts in six files:
- backend/src/app/migrations.clj: develop's 0155 and 0156 steps run
first, then the branching steps; the names differ.
- backend/src/app/rpc/commands/viewer.clj: both requires stay.
- common/src/app/common/flags.cljc: both flags stay.
- frontend/src/app/main/ui/dashboard/file_menu.cljs: "Create branch"
becomes a menu-item* with the same id, guard and handler.
- frontend/src/app/main/ui/dashboard/grid.cljs: the portal goes with
develop's menu rewrite; the branches popover stays in the trigger.
- frontend/translations/en.po: both sides' entries stay, with
develop's source references.
Co-authored-by: Juan de la Cruz <delacruzgarciajuan@gmail.com>
AI-assisted-by: mixed models
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Gate error reporters behind :error-reporting flag
Error reporters used to start on every boot with no way to
switch them off. They now only start when the new
:error-reporting flag is on (off by default, enable with
PENPOT_FLAGS=enable-error-reporting). The database reporter
needs the flag alone; mattermost needs the flag plus its
webhook url. Shutdown with the flag off is safe, and the
database loop now honors its runtime switch.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Add audit-log reference memory and update dev tooling
Adds the top-level audit-log reference memory with all event
flows and turns backend/audit-log into a redirect stub.
Also includes the staged dev-tooling updates (playwright
deps, mdts args, gitignore).
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Assign server-side initiator to frontend audit events
Frontend audit events now carry a server-assigned initiator
derived from the x-client header (penpot-frontend goes to
app, penpot-admin-console goes to admin-console, legacy
penpot-nitrate stays on admin-console, all else goes to
app). Client-sent values are always overwritten, and the
key survives on telemetry shadow rows.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep initiator on frontend telemetry context
The strips-pii-keys test still asserted the old contract
(initiator stripped). The initiator only names the sending
app, so it is not personal data and must survive the
telemetry filter like on the backend channel.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Document initiator on PostHog and browser origins
Complete the audit-log memory for the latest changes: the
initiator property forwarded to PostHog on both channels
and the eventOrigin rule for admin-console browser events.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix unscaped argument in script
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Run the audit archive cron when :nexus or :admin-console is on. Ship
allowlisted events to Admin Console first (with row ids for
idempotency), then the full chunk to Nexus when :nexus is set. Mark
archived_at for the whole chunk on success, including nitrate-only
mode. Rename the gate flag from :audit-log-archive to :nexus.
AI-assisted-by: Composer
Co-authored-by: Cursor <cursoragent@cursor.com>
* ✨ Add size limits to profile props and plugin registry
Bound the total serialized size of profile settings to 2 MiB
(:profile-props-max-size), checked on the merged result before
persisting, with a controlled :props-too-large error. Profiles
that already exceed the limit can still shrink but cannot grow.
Cap plugin registry entries in the shared schema (code 1 MiB, 50
plugins max, bounded name/host/description/icon) and restore rate
limiting on the plugin RPCs (profile-mutations bucket, one write
at a time per profile). The plugin manager now asks for
confirmation before removal and ignores repeated clicks while a
persist request is in flight.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Enforce plugin count cap, byte sizes and removal guard
Enforce the declared 50-plugin cap in add-profile-plugin with a
specific :too-many-plugins error (updates of existing entries
still pass); the cap lives in a shared max-plugins constant.
Measure profile props size in UTF-8 bytes instead of chars so
multibyte content cannot slip past the limit.
Cover install/remove persist logic with mocked-RPC frontend tests
(release semantics, in-flight dedupe, validation vs rollback
split) and add the missing boundary tests in common.
Expose the in-flight persist set from the plugin registry and
disable the remove button of entries being saved.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix rollback loops and restore paths in plugin registry
Restore the previous plugin version instead of dropping the entry
when a validation error rejects an update of an installed plugin.
Make compensating writes one-shot with terminal callbacks so a
persistent failure cannot ping-pong between install and remove.
Restores keep the original list position; the unused public
plugin-persisting? predicate is removed.
Pin count-before-size precedence with a dedicated test and fix
translation source refs to their canonical lines.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Guard notifications write and fix restore ordering
Route update-profile-notifications through check-props-size! so
oversized profiles cannot grow through that path; document the
exempt system writers. Remove the duplicated stale entries in
en.po, keeping the canonical translation refs.
Restore rejected plugin updates at their original list position
instead of leaving the optimistic move in place.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Skip no-op plugin removal and clarify size comments
Return early from remove-profile-plugin when the id is absent:
no wasted write, no size check, and no manufactured :plugins key
that could spuriously fail on oversized profiles.
Clarify that per-field string caps count chars while the byte
budget is enforced by profile-props-max-size.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 📎 Fix formatting in rlimit.edn for profile operations
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Fix formatting of import-binfile/global entry
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Simplify props size check and tighten plugin entry caps
Measure props with transit bytes directly instead of the
PGobject string roundtrip.
Rename check-props-size! to check-props-size: single hard limit
on the merged props, no growth comparison, and return props so
writers thread the check into the update.
Move the 2 MiB default into default-props-max-size on the
profile namespace, still overridable with the optional
:profile-props-max-size config entry.
Tighten registry-entry :code and :icon to 500 chars: they hold
manifest paths, not content.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix compatibility problems
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ♻️ Scope organization notifications to specific WebSocket topics
Publish team/org notifications to team-id and organization-id topics
instead of broadcasting to all connections via uuid/zero. Dashboard and
workspace now subscribe to their current team and organization on
initialization, receiving only relevant events.
Backend: added subscribe-organization/unsubscribe-organization WebSocket
handlers and updated :close to clean up organization subscriptions.
Modified notify-team-change, notify-organization-deletion, and
notify-organization-change-sso to publish to specific topics.
Frontend: dashboard and workspace now subscribe to team-id and
organization-id (when applicable) on initialization, with nil-guard
in topic filters.
Closes#11455
AI-assisted-by: longcat-2.0
* 🔧 Remove unused session-id binding in unsubscribe-organization
Clj-kondo lint fix.
AI-assisted-by: longcat-2.0
* 🐛 Fix permission check on team ws connection
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Enforce idle and absolute session expiration
Sessions now expire on two server-side conditions: an idle window
(PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE, default 7d) and an absolute cap
from creation (PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE_ABSOLUTE, default
30d, enforced by the token :exp claim). The daily session-gc task
deletes rows that exceed either window, so idle sessions can no
longer be replayed and active sessions are not deleted at the idle
window.
Also remove the legacy v1 HTTP sessions: the http_session table and
the string-id / :ver 0 token code paths are gone. Any v1 cookie now
requires a fresh login.
Document the session expiration configuration in the technical guide
and add a backend memory describing the token, renewal and GC model.
Closes#11646
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Address session-expiration review findings F1-F4
Fix the unreadable test (a stray paren broke whole-suite
discovery), enforce idle expiration on every request in
wrap-authz, fail boot fast when the absolute cap sits below
the idle window, and align config defaults with the memory
rule while fixing its migration number and stale reference.
Closes#11646
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add Malli schema for storage metadata with dual decode
Phase 1 of the storage_object.metadata migration: reads accept both
Transit and plain JSON (sniffed by the marker) and always return the
normalized shape; writes validate against a closed per-bucket Malli
schema and still serialize as Transit unless the new
:storage-metadata-as-json config flag is set.
The 0155 migration normalizes existing rows inside Transit (reference
to bucket, default bucket, drop of chunk leftovers) and is
idempotent; large instances should fake it and run the batched
script instead.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address review findings on storage metadata Phase 1
Collapse the dead :reference leg of the gc-touched bucket fallback
(the decode always sets :bucket on non-nil metadata, so it is only
reachable with a NULL column) and fix its comment.
Pin the write flag off in the transit-assuming metadata tests so the
suite proves the same with the flag set, and add coverage for the
flag rollback contract, JSON hash survival, NULL metadata in
gc-touched, and the 0155 normalization statements.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Backfill NULLs, canonical buckets, comment fix
Backfill NULL metadata columns in 0155 via coalesce (the key-missing
rule already matches them), derive valid-buckets from the Malli schema
dispatch entries so the list lives in one place, and correct the
lookup-bucket fallback comment to NULL columns.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Defer corrupt metadata rows in storage gc-touched
Decode touched rows individually so one non-map metadata value no
longer aborts the whole chunk: corrupt rows are logged and deferred
exactly one day in the same transaction, keeping their metadata
intact for a later repair, while healthy rows process normally.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address storage metadata phase 1 review findings
Address the review findings on the storage metadata phase 1 branch:
- Fix put-and-delete-object: it stored the object with
::sto/expired-at, so the row was already deleted and del-object!
returned false. Add delete-expired-object-returns-false to keep
the expired-delete case covered.
- Cache the Malli decoder and encoder per process. Building them
compiles the closed multi-dispatch schema, and decode-metadata
runs on every read path (get-object, dedup probes, GC batches).
- Catch Exception instead of Throwable in try-decode-row so JVM
Errors are not deferred as corrupt metadata.
- Add penpot_storage_gc_poison_total, emitted from
storage-gc-touched; wire ::mtx/metrics into its handler.
- Anchor the encoding sniff to the start of the document so a
plain JSON value that begins with a Transit-looking prefix is
not read as Transit.
- Cover every bucket on both encodings, a JSON roundtrip through
the jsonb column, nil metadata, the canonical bucket set and a
poison-only GC chunk.
- Rename private check-metadata! to check-metadata.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Simplify the storage metadata schema to a single map
Replace the per-bucket :multi dispatch with a single closed map: the
bucket is validated with ::sm/one-of over metadata-buckets (now a plain
set) and the remaining keys are typed optional fields. Per-bucket
enforcement shrinks to a one-line :fn guard requiring :file-id and :id
for file-data, whose ids the GC reads to resolve references.
- Drop the dead (sm/register! ::metadata ...): nothing references the
schema by keyword.
- Define tempfile-bucket and upload-session-bucket in the schema and
alias them from app.storage, removing duplicated literals.
- Keep content-type required and the map closed, so an unknown bucket
or key still fails fast on write.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop input coercion from encode-metadata
encode-metadata no longer runs the json-transformer decoder before
validation. On the write path its only effect was coercing string
UUIDs to UUID, and every producer already passes native UUIDs (the
RPC profile-id, uuid/random, or binfile ids decoded as ::sm/uuid).
Reads keep decoding, so stored Transit or JSON values still come
back as native types.
- Replace encode-accepts-string-uuids with
encode-rejects-string-uuids, pinning the stricter contract.
- Pass native UUIDs in encode-writes-plain-json-with-flag.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Document each statement in the storage metadata migration
Move the per-statement rules out of the header and add a comment to each
UPDATE explaining what it does: drop chunk leftovers, promote the legacy
"~:reference" to "~:bucket", drop residual "~:reference", and backfill the
default bucket. The header keeps the scope, the encoding note, the `->`
vs `?` note and the large-instance warning.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap wrapped lines in the backend storage memory
One line per bullet or paragraph, as mem:memory-maintenance requires.
Only formatting; no content change.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Defer storage GC poison rows in their own transaction
process-chunk! no longer takes poison-ids; it only processes the healthy
chunk. The deferral moves to defer-poison! and process-touched! runs it in
its own transaction, separate from the freeze/delete work. The loop still
drains while there is chunk or poison, so a batch made only of poison rows
does not leave healthy rows behind the LIMIT 10 waiting for the next run.
Add a regression test: ten poison rows plus one healthy row with a later
touched_at are all handled in the same run.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Declare per-bucket metadata requirements in one map
Replace the file-data-specific predicate with bucket-requirements, a map
from bucket to the extra keys it must carry. metadata-buckets is derived
from its keys and a single generic :fn enforces presence, so a new bucket
and its contract are one entry. organization now requires
:organization-id; file-data keeps requiring :file-id and :id.
Update the http-assets test helper to set organization-id for its
organization objects.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC helpers
Rename the internal helpers in app.storage.gc-touched (process-chunk,
defer-poison, mark-freeze-in-bulk, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC deleted helpers
Rename the internal helpers in app.storage.gc-deleted (clean-deleted,
delete-sobjects, delete-give-up, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix dedup lookup for JSON-encoded storage metadata
get-database-object-by-hash only matched the Transit keys, so once the
:storage-metadata-as-json flag wrote plain JSON rows the dedup stopped
finding them and duplicated blobs. Match both encodings with a UNION ALL
of two indexable branches.
- Add migration 0156 with the plain-key dedup index; the legacy 0068
index stays until Transit support is removed.
- Cover it with a JSON dedup test and a Transit -> JSON cross test.
AI-assisted-by: deepseek-v4.1-flash
The event was written twice per accepted organization invitation. The
backend submitted it, and the browser then re-submitted a copy of the
props that the backend had already put in the response under
`:organization-invitation-audit` (`handle-token :team-invitation` in
`verify-token.cljs`). Both rows carried the same name with different prop
vocabularies, and the browser copy only existed when the browser finished
the flow.
Emit the event from the backend only. It now also carries the three props
that lived in the browser copy: the organization member count before the
add, the add source, and whether the invitee also joined a team. The
origin moves to the event context as `:event-origin`. The response no
longer includes `:organization-invitation-audit`, so the browser stops
emitting the event and `verify-token.cljs` drops its
`app.main.data.event` require.
The `accept-*` events of this command now share one prop vocabulary:
`:profile-id` for the accepting profile, `:invited-by` for the inviter
and `:profile-email` for the email, replacing the mix of
`:user-id`/`:user-who-send-invitation` and `:email`.
Audit consumers of `accept-organization-invitation` now see one row per
acceptance instead of two, and must read the new prop names.
AI-assisted-by: space-bunny-free
The json reader of the binfile v3 import rewrites every key of every
entry to kebab-case, nested maps included. Shape `:svg-attrs` is the one
map whose keys are camelCase react prop names, because the svg import
path runs them through `attrs->props`, so an attribute exported as
`fillRule` came back as `:fill-rule` and was stored that way.
The renderer looks the attribute up by its camelCase name, does not find
it and falls back to the default fill rule, so a shape exported with
`fillRule: evenodd` was painted without its hole, and the attributes
panel showed `fill-rule`.
`clean-shape-post-decode` already runs on every shape right after the
schema decode, for page shapes and component objects alike, so the
repair goes there: run `:svg-attrs` back through `attrs->props`, the
same transform that built the keys. It is idempotent, so shapes that
arrive correct are left untouched.
The new tests import a real export that carries the attribute, for
page shapes and component shapes.
Closes#11954
AI-assisted-by: space-bunny-free
* 🐛 Attribute audit events to the caller, not the response
prepare-rpc-event took the event profile-id from the result map
whenever it carried one, before falling back to the caller. Any
command returning a response with a :profile-id key silently
credited the action to somebody else.
get-error-report returns the report with its decoded content
merged in, and that content holds the profile that owned the
report, so privileged reads were logged against the users whose
crashes were being inspected. verify-token on a team invitation
returns the inviter's profile-id, so accepting an invitation was
logged against the inviter.
Resolution is now ::audit/profile-id metadata, then
::rpc/profile-id, then the zero uuid; the response is never
consulted. The two verify-token branches that relied on it now
declare the profile in the result metadata. Every other command
either already declared it or returns no :profile-id; all 30
registered command namespaces were checked.
The tests that pinned the old behavior are replaced by ones
covering the new contract.
AI-assisted-by: space-bunny-free
* 🐛 Coerce the audit profile-id override to a uuid
The only sanctioned way for a command to override the profile of an
audit event is the ::audit/profile-id metadata, and the value is set
by hand in a dozen commands, some of them reading it from token
claims or other sources we do not type.
schema:event requires a uuid and submit* swallows the validation
error, so a string did not fail loudly: the row was dropped silently.
Values that cannot become a uuid are now discarded with a warning
and the event falls back to the caller, which is always a valid uuid.
A uuid, the common case, exits on the first check.
AI-assisted-by: space-bunny-free
* 🐛 Migrate openUIApi schema to Zod v4 function syntax
Zod 4 removed z.function().args(), which broke the
plugins-runtime build with implicit-any errors on every
openUIApi parameter and knock-on possibly-null errors on
the modal in plugin-manager.
Declare the inputs with z.function({ input: [...] }) so the
parameter and return types infer again; behavior is unchanged.
Add a regression spec covering delegation, optional args and
rejection of invalid theme and title values.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Fix deprecated markdown-it-anchor permalink option in docs
Migrate docs Eleventy config to the markdown-it-anchor v10 API.
Replace the deprecated boolean permalink option with
linkInsideHeader, keeping the same symbol and class.
Bump markdown-it-anchor to v10 and related docs deps.
AI-assisted-by: muse-spark-1.3-contributor
* ⬆️ Update deps
* ⬆️ Update base docker images
* 📎 Fix mcp tests
* ✨ Add renderer option on create-demo-profile command
Allow create-demo-profile to take an optional renderer (svg or
wasm) and store it on profile props. When omitted, no renderer is
written so the Penpot default still applies.
Closes#11893
AI-assisted-by: Muse Spark 1.3 Free
* 📚 Fix renderer doc version on create-demo-profile
Correct the ::doc/changes version for the new renderer param from
2.20 to 2.18.1.
AI-assisted-by: Muse Spark 1.3 Free
* 🐛 Allow invitation-based registration when disable-registration is set (#5178)
Per documentation, disable-registration 'disables registration
(still enabled for invitations only)'. Two bugs prevented this:
1. verify_token.clj: when processing an invitation token for a
non-logged-in user with no member-id, the redirect included
registration-disabled? in its condition, sending invited users
to the login page instead of the register page.
2. auth.clj validate-register-attempt!: the registration-disabled
check fired unconditionally before the invitation-token check,
rejecting the actual register RPC even with a valid invitation.
Fix: in verify_token.clj remove registration-disabled? from the
redirect condition for new-user invitations. In auth.clj restructure
the check as an if/else: with an invitation token, validate the token
and allow registration; without one, enforce the flag as before.
* 🐛 Allow registration with disabled public registration
Allow valid team invitations to create new profiles when public
registration is disabled, while keeping password login and invitation
validation required.
Add backend regression coverage for flag combinations and verify-token
redirects, frontend route coverage, and configuration documentation.
Closes#5178
AI-assisted-by: space-bunny-free
* 🐛 Revalidate active invitation during registration
Require a live, unexpired team invitation before using the
registration exception, and recheck it before creating a profile.
Reuse the same lookup in invitation token verification.
Add regression tests for canceled and expired invitations, the
registration race, and explicit redirect contracts. Update docs
and backend auth guidance.
AI-assisted-by: Space Bunny Free
* 🐛 Lock and normalize invitation registration checks
Lock active invitation rows during transactional registration and
acceptance so cancellations cannot race with profile or membership
creation.
Normalize invitation emails before comparisons and database lookups.
Add concurrency, email casing, and final flag regression tests.
AI-assisted-by: Space Bunny Free
---------
Co-authored-by: Sumit Ridhal <sridhal@redhat.com>
* ✨ Add deployment info to events
* ✨ Add get-environment-data RPC method
Add a single public RPC method returning the deployment type and
the enabled environment flags. It replaces get-deployment on the
management API and get-enabled-flags on the main API.
get-enabled-flags stays as a deprecated alias returning only the
flags, so existing callers keep working until it is removed.
The frontend event initialization now reads the flags from the new
method. The exposed flags stay limited to audit-log and telemetry
to avoid leaking internal backend flags.
AI-assisted-by: deepseek-v4.1-flash
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix organization/team switcher issues from UX review
* 🐛 Let members leave an organization without SSO credentials
* 🐛 Fix style from previewed organization in the team switcher
* 🐛 Fix leave-organization modal test stubs
Inject the storage component into the delete-object task handler and
validate it on initialization, so library absorption no longer fails with
"expected valid storage to be provided" for storage-backed file data.
Also strengthen the regression test: it now creates a real component in
the shared library and an instance in the referencing file, and verifies
the component is absorbed after deletion instead of only checking the
revision bump. The test disables the fdata/pointer-map feature to stay
focused on the storage fix.
Signed-off-by: Yuito Akatsuki (Tani Yutaka) <yuito@yuito-it.jp>
AI-assisted-by: deepseek-v4.1-flash
Requesting a binfile export with both `include-libraries` and
`embed-assets` enabled threw a raw `IllegalArgumentException`, which the
RPC layer surfaced to API clients as a generic
`:server-error`/`:unexpected` (HTTP 500) with no actionable code -- the
opaque failure reported in #7649 (perceived as an "empty" export).
The two options are genuinely mutually exclusive, so the request is
invalid input, not a server fault. Replace the raw throw in both the v1
and v3 export paths with `ex/raise :type :validation :code
:incompatible-options`, matching how the rest of the binfile code
reports bad input. Also fix the "mutally" -> "mutually" typo in the hint.
Verified on a local devenv: the export RPC now returns
`{:type :validation :code :incompatible-options}` instead of a 500, and
valid single-option exports are unaffected. Adds a regression test.
Fixes#7649
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Build organization invitation audit event in frontend
* ♻️ Align invitation token profile-id with created-by
The invitation token carried the minter in :profile-id while the
invitation row tracks the creator in :created-by. Both mean the
inviter, so re-sends or re-requested links made them disagree and
forced a second response key, :user-who-send-invitation.
Mint :profile-id from :created-by in both token creators, backfill
it from the row on accept (covers stale in-flight tokens), and drop
the duplicate response key. The frontend maps :profile-id to the
unchanged :user-who-send-invitation audit prop.
AI-assisted-by: Muse Spark 1.3 Free
* ♻️ Reuse token ids in invitation accept response
Backfill :member-id with the accepting profile and drop the
:user-id duplicate from the verify-token response, mirroring the
:profile-id/:user-who-send-invitation cleanup. The frontend maps
:member-id to the unchanged :user-id audit prop.
AI-assisted-by: Muse Spark 1.3 Free
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
