* 🐛 Refuse saves to a branch kept after its merge
A branch kept after its merge stays an ordinary file with status
"merged", so every save passed the edition check and appended ops to
an op log that no command integrates any more. Refuse such saves with
a :branch-merged validation error instead of stranding them. An
archived branch still takes saves: restoring it brings them back.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Tell merged and archived branches apart
The workspace treated "merged" and "archived" as one state: an
archived branch read "Merged branch" and lost every action, and a
merged row offered Restore, which the backend always refused.
Each status now has its own presentation. A kept merged branch keeps
"Merged branch" and View in main, and opens read-only: the workspace
permissions drop :can-edit while it is the open file, the way a
viewer gets a read-only workspace. An archived branch says so in the
banner and offers Compare and Restore. The header notes each status,
the switcher lists merged and archived branches in two collapsible
groups, the all-branches dialog gets a Merged tab, and the branch info
dialog offers Restore and Compare for an archived branch.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Refuse saves to an archived branch
An archived branch is now read-only, like a merged one: the user
restores it, or turns it into an ordinary file, to keep editing.
persist-branch-file! refuses its saves with a :branch-archived
validation error, distinct from :branch-merged so the client can say
which way out applies. A restored branch takes saves again, and so
does an archived branch turned into an ordinary file.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Open archived branches read-only with their ways out
An archived branch now opens read-only, like a merged one. Its banner,
its branch info dialog and its row in the all-branches dialog offer
the two ways back to editing, Restore and Turn into an ordinary file,
next to Compare. The header and switcher note reads "Archived, read
only".
A save the server refuses because the branch is merged or archived
now shows a warning that names the way out, instead of the generic
autosave error with a report link.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Add a branch count query for the workspace header
The header shows its branch row only when the file has branches, and
it learnt that from get-file-branches, which compares every diverged
branch against main. get-file-branch-count answers the same question
with one indexed count: archived and merged branches included, counted
from main when the file is itself a branch, with no comparison and no
climit, like the branches_count subquery of the dashboard listings.
The listing keeps its climit, and :get-file-branches/global now sets
the 60 s timeout that the header of climit.edn promises, so a waiter
behind six cold listings gets an error instead of waiting without
bound. Its rationale comment names the callers it serves now.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ List branches only when the switcher opens
Opening a file ran the counted branch listing for every user, because
the header decided whether to show its branch row from that listing.
A cold listing runs a three-way comparison per diverged branch, and
one save to main makes every branch of the file a cache miss again.
The header now loads the branch count and the branch context with the
file. The listing loads when the branch switcher or the all-branches
dialog opens, and both show "Loading branches…" until it arrives
instead of a list from an earlier opening.
A failed listing used to read as an empty list, which on main hid the
branch row and sent the shortcut to the create dialog. It now ends in
an error state: the switcher and the dialog say "Couldn't load
branches" with a Retry action, and the row stays, since its visibility
comes from the count. A successful listing refreshes the count, so
deleting the last branch from the list still hides the row.
Creating a pull request from the branch banner looked the branch up
in the listing to decide whether to save pending edits first. Without
the listing loaded at open it would skip the save, so it now reads the
branch context of the open file.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Show the branch row when the branch count fails
A failed get-file-branch-count left no count, so on main the header
hid its branch row and the shortcut opened the create dialog on a file
that may have branches. The failure is now kept as an error state
under :workspace-branch-count, and has-branches? reads it as true.
The row then shows the Main chip and, where the status note goes,
"Couldn't load branches" in the warning colour. Opening the switcher
loads the listing through the existing loading and Retry path, and a
successful listing settles the count, which replaces the note.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 📚 Name the callers of the branch listing limit again
The merge of iop-vc-preview took its climit.edn, whose comment on
:get-file-branches/global still described a panel open. Restore the
comment this branch wrote: the listing runs when the switcher, the
all-branches dialog or the dashboard popover opens, and a file open
only counts branches.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Mock the branch count in the library copy tests
The header now shows its branch row from get-file-branch-count, not
from the listing, so the library copy tests mock the count as well.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
A branch whose only change was file-level plugin data or the base font
size merged with a success response, and the change was lost. The merge
engine compared colours, typographies, components, media, pages, and
tokens only, so such a branch produced an empty summary and the merge
took the empty-changes shortcut.
The engine now classifies two more kinds three-way: file plugin data,
flattened to namespace and key as page plugin data already is, and the
file options, where an unset base font size reads as the default. Both
kinds count towards ahead, behind, and conflicts, appear in the compare
dialog, and translate into set-plugin-data and set-base-font-size
changes for merge and update from main. Both sides changing the same
key to different values is a modify-modify conflict.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Give the branch concurrency limits a timeout
Every global entry for branch and pull-request commands lacked a
:timeout, so a saturated limiter parked its waiters forever and
climit-test/example-config-bounds-every-wait failed on seven ids.
Each entry now waits at most 60 s, the value of the other slow
operations. The size gates in files_branch.clj refuse files whose
operation would exceed 30 s, so the wait covers one in-flight
operation with as much again in margin.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Make the quota racer a member of the file team
File access requires membership of the file's team since the
permission check started joining team_profile_rel. The racer in
create-checks-the-branch-quota-inside-the-lock held only a file role,
so its create failed with :object-not-found and the test never raced.
The racer now joins the file's team as an editor, so both creates
again compete for the last branch slot.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Every gated branch operation flattened the whole op log only to count
its changes, and the derive in the same request decoded those rows
again. The count is not recoverable from the bytes: a row holds one
compressed change vector whose length says nothing about the number of
changes beside a squash that emits tens of thousands of them for a
one-page delta.
Each `file_branch_change` row now stamps `changes_count` at write time,
in `persist-branch-file!` and in the squash `persist-branch-update!`
writes, and `binfile/common.clj::get-branch-oplog-depth` sums the live
rows. Migrations 0804 and 0805 add the column, fill it from the stored
vectors, and make it NOT NULL, so a writer that omits it fails instead
of recording an empty log.
The gate keeps its place ahead of the derive in `get-branch-diff`,
`merge-file-branch`, and `update-branch-from-main`, and a test pins
that it counts the changes a row holds rather than the rows.
Entry: pp:vcs:the-op-log-depth-check-pays-a-derive open -> done
AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Both conflicts are tail appends onto regions develop rewrote. In
migrations.clj keep both appended blocks, develop's two entries
first so the branch's four stay the tail of the vector. In
climit.edn take develop's rewritten entries and re-append the
branch's provisional block, moving the map's closing brace to the
last line.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The merge of main into staging brought the closed storage metadata
schema, which requires file-id and id on file-data objects. The
restore test built one without those keys, so it failed with an
invalid metadata error. Its helper now takes extra metadata and
the test passes the ids of the file and of its file-data row, as
production does.
The republish test asserted the exact list of waiting-gauge
publications, but the component's sampler starts its first tick right
away and under load lands inside the recorded window, publishing the
same count twice. The test now builds the counters alone, so the
republish call is the only publisher.
AI-assisted-by: deepseek-v4.1-flash
The queue gauge was written by whichever limiter of a name ran last, so a
per-profile pile-up never showed: a healthy profile writing 0 overwrote the
15 of the profile that was queueing (4 of 1441 samples in production), and
a limiter whose requests never come back read 0 while 72 threads stayed
parked.
A queue is an entry and an exit, so the new gauge is accumulated: one
counter per limiter name and instance, up when a call starts waiting for a
permit, down when it stops, so a call that never leaves stays visible. It
is always written with `:val`, and a daemon sampler republishes every count
every 15s so a name whose traffic died does not go stale in the scrape;
halting the component stops that thread.
Rejections had no metric at all: penpot_rpc_climit_rejected_total counts
them by name and reason (queue-full or timeout), and the log lines now
carry queue-max, permits, code and reason, so an operator can tell whether
the limiter did its job or was saturated.
The gauges that could not tell the truth are gone: queue and permits (same
last-write-wins problem, and waiting plus the wait histogram replace them)
and http_worker_busy_threads (reads 0 on an instance with 72 blocked
threads; pool_size is the signal). No other dashboard or alert rule reads
them.
AI-assisted-by: deepseek-v4.1-flash
Add a per-process availability flag (app.health) that /readyz
reports as 503 OFFLINE while offline, so the load balancer
drains the instance and in-flight requests finish untouched.
Operator surface is app.srepl.main/set-offline and
app.srepl.main/set-online; a restart comes back online.
Covered by backend-tests.health-test; HA docs updated.
AI-assisted-by: muse-spark-1.3-contributor
Restoring a file only un-marked the database rows (`file`,
`file_media_object`, `file_change`, `file_data`, `file_thumbnail`,
`file_tagged_object_thumbnail`), not the storage objects those rows
reference. An object that was already marked for deletion stayed
marked while a live row pointed at it, and those foreign keys are `NO
ACTION`: the storage GC cannot delete it and skips it forever (and
before the GC learned to skip, it cost the whole nightly run).
Restoring a team had the same gap with its font variants.
Both restores now unmark the storage objects of the rows they bring
back: the media and the thumbnail of the file media objects, the media
of the file and object thumbnails, the `storage-ref-id` of the file
data, and the four files of a font variant. Objects referenced by other
rows are left alone.
AI-assisted-by: deepseek-v4.1-flash
objects-gc deleted every kind of object inside one transaction per proc,
looping until empty, so a single row that a foreign key still references
aborted the whole run: the task died on its first file every night, after
the earlier steps had already spent around twenty minutes. The per-file
savepoint only saved the files proc and left the same shape everywhere
else.
Now every proc streams its pending keys with db/plan inside a read-only
transaction of its own and deletes them chunk by chunk, each chunk in a
transaction of its own with FOR UPDATE SKIP LOCKED. A chunk that fails is
reported with l/err and skipped, so the run reaches the end and the next
run retries the skipped rows.
AI-assisted-by: muse-spark-1.3-contributor
`storage-gc-deleted` ran its whole batch inside one transaction: the
handler opened it and `delete-in-bulk` reused that connection, so the
"separate transaction" its comment promised did not exist. A single
row that a foreign key still references aborted that transaction, the
cursor then died on its next fetch ("current transaction is aborted")
and the task ended up throwing `unhandled exception on running task`
after rolling back the whole night of work. That is what has kept
production from deleting a single object since late April.
Now `clean-deleted` walks the pending objects with the cursor in its
own read-only transaction and every chunk runs in its own one, so a
blocked chunk is logged with its ids and skipped while the rest is
committed. `task finished` is reachable again.
The private functions also drop the `!` suffix: the namespace is all
mutation, so the mark adds nothing and reads as noise.
The test builds a blocked object (a `file_media_object` pointing at it
as its thumbnail) plus a full chunk of pending objects and one more in
the next chunk, and checks that the run finishes and that the object of
the next chunk is deleted.
AI-assisted-by: deepseek-v4.1-flash
The four `process-deleted-*-cascade` helpers of `app.srepl.main`
loaded every soft-deleted row of a table into memory (`db/exec!`)
and ran `:delete-object` inline (`wrk/invoke!`), so they could not
be used against production: `file` alone has 2.1M deleted rows.
They now share a single `submit-deleted-cascade!` that reads the
rows with a server-side cursor (`db/plan`, in a read-only
transaction) and submits one task per row, so the work goes
through the workers like any other task. The row id goes as
`label` with `dedupe` on, and the helpers accept `:limit`, `:ids`
and `:dry-run?` to run bounded batches. Public names and their
zero-arity form are unchanged.
Tests cover the task params, the row's own date (not `now`), the
`limit`/`ids`/`dry-run?` options and that each helper reads its own
table.
AI-assisted-by: deepseek-v4.1-flash
An instance left the audit events pipeline dead: every push-audit-events
answered with 429 and 69 worker threads parked for good inside a limiter.
Two code paths of one profile acquired the same two limiters in opposite
order: a command declaring its own limits (upload-file-media-object) took
process-image/by-profile and then root/by-profile, while a command with no
limits of its own took the roots first and then re-entered climit from
inside its handler with the same ids (create-file-media-object-from-url).
Each path held what the other one waited for, and no limiter had a timeout
to break the wait.
Now wrap puts the root limiters after the command's own, so they end up
outermost and every path meets them first: the nested chains carry only the
command's own limits and run inside them, which puts both paths in the same
order. impl-invoke also keeps, per thread, the set of limiters that thread
is inside of and runs the handler inline when a call asks for one it
already holds, so a handler can no longer queue behind itself.
The shipped climit.edn sets a timeout on every limiter (60s for operations
that can be slow, 30s otherwise), so no wait can park a thread for ever.
The production file lives outside the repository and has to be mirrored.
Tests cover the acquisition order, the deadlock, the reentrancy and the
timeouts of the shipped config.
AI-assisted-by: deepseek-v4.1-flash
The command declared its limit as a plain keyword next to
`::climit/key-fn`, metadata that `climit/wrap` does not read: it
takes the key only from the positional `key-fn` of the vector form.
The limit was a single shared one, so when one call got stuck it
held the only permit and every audit batch of that instance was
rejected with 429 from then on.
The test builds the limiter from the command metadata and checks
that two profiles end up with different limiters.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Add skip-if-active option to jobs submit
Producers that need at most one live job per key can now pass
::jobs/skip-if-active: the submit counts live jobs for the same
tenant, name, queue and label in the same transaction and skips
the insert (returning nil) when one lives. The option requires an
explicit label and documents the same best-effort race caveat as
the dedupe it complements.
Closes#12265
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Record skipped job submissions as a metric
Skip-if-active submissions withheld nothing observable: the skip
now records penpot_jobs_skipped_total (name, queue) instead of the
submitted counter, so dashboards can tell withheld work apart from
new work. The insert path is untouched.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Drop jobs counters duplicated by histogram counts
The completed and requests counters counted exactly the same events
as the _count of their timing histograms, so they go away: terminal
totals now come from penpot_jobs_total_ms_count by outcome and
request totals from penpot_jobs_request_ms_count by outcome. One
event, one family from here on: counters only cover events without
latency, and the dispatched vs queue-wait pair stays because pushes
and claims are different events.
AI-assisted-by: muse-spark-1.3-contributor
Clients that only need a file's design tokens, such as a CI job that
syncs them to a git repository after an update-file webhook, had to
download the whole file with get-file.
get-file-tokens returns the file's tokens as a DTCG document, the same
format the token export produces, or nil when the file has none. Active
themes and sets come from the file's tokens status, so they reflect
what is currently active.
The document is returned as a plain map. A bare TokensLib is not a
collection, so the response middleware would hand it to ring unencoded.
Relates to #7091
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The Plugin API can write arbitrary key/value plugin data onto the file
root, but it could only be read back by downloading and decoding the
whole file. Add a `get-file-plugin-data` read command that returns the
file-level plugin data map, optionally narrowed to a single namespace,
so external tooling can fetch metadata such as issue or commit
references cheaply.
Read-only: reuses `bfc/get-file` and `check-read-permissions!`, and
adds no data-model or change-type changes. Object/page level reads are
intentionally left out of this first command.
Relates to #10169
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
`resolve-and-link-libraries` asserts `::bfc/team-id`, and
`enable-backend-asserts` is now enabled in `scripts/ci` and
`tests-backend.yml`. Without a team-id, the v3 import tests
error instead of passing. Production always provides the
team (RPC import, management and debug routes); mirror that
in the tests.
Update `mem:backend/testing` and `mem:scripts/ci` with the
`enable-backend-asserts` behavior.
Verified with `./scripts/ci backend`: 1235 tests, 6384
assertions, 0 failures.
AI-assisted-by: deepseek-v4.1-flash
A plugin with "scope": "global" keeps running across the dashboard
and files and can find and open the user's projects and files. Any
manifest could ask for this without the user being told.
Add the allow:global permission. The plugin registry adds it to
every manifest with global scope, so the install dialog lists it,
and a plugin that becomes global in a later version shows the
permissions update dialog again. The permission is valid in the
shared schema, which the backend checks when saving, and in the
runtime manifest schema. The built-in MCP plugin declares it too.
Document the scope property and the new permission for plugin
developers.
* ✨ Run binfile import and export as durable jobs
Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.
Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.
How:
- The substrate gains the pieces a user-facing job needs: optional
job-def metadata (family, resource-role), an expiry on submit, the
owner in the runner context, and job resources in their own bucket,
written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
business params, the registry checks the family, the job-def decodes
and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
vocabulary with self-contained counters, throttled heartbeats, and an
interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
the package descriptor; the import re-checks the edition permission,
runs the core in a transaction so a cancellation rolls it back, and
releases the consumed package and its temporary copy.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Handle externally-cancelled binfile jobs end to end
The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.
- The export emits `{:cancelled true}` per file and the modal paints a
neutral cancelled row instead of success; a message without an
artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
they replace.
Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Pre-translate the cancelled import message
The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Move cancelled strings under generic labels keys
The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add minor improvements
* ♻️ Rename export job command to create-export-binfile-job
Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.
submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.
The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Polish the import and export status rows
The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.
In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Show the cancelled export and fix the import error
The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).
The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.
The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Reflect the real job params in the creation commands
The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.
Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.
A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Stop the test system from running job runners
The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.
That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.
Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Upload the package one chunk at a time and as a percentage
The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.
The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.
AI-assisted-by: deepseek-v4.1-flash
* 🌐 Complete the Spanish catalogue
es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).
sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Move the job status keys into the jobs namespace
The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.
Rename them so the whole job vocabulary is one namespace:
labels.queued -> jobs.queued
labels.export-cancelled -> jobs.export-cancelled
labels.import-cancelled -> jobs.import-cancelled
No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.
AI-assisted-by: deepseek-v4.1-flash
* 🔥 Remove the translation keys nothing references
Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:
labels.uploading-file a duplicate of jobs.progress.stage.upload
since the upload stage says "Uploading file…"
labels.downloading-file dead since the durable jobs replaced the old
import and export progress rows
inspect.attributes.stroke.alignment.center
inspect.attributes.stroke.alignment.inner
inspect.attributes.stroke.alignment.outer
workspace.toolbar.mcp-connect-here
The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.
AI-assisted-by: deepseek-v4.1-flash
The branch compare panel now offers per-change selection, grouped by
the component each shape belongs to. Every summary entry carries a
stable change key (kind/page/id, the design-branches PoC format), and
the panel's checkboxes key on it.
merge-file-branch and update-branch-from-main take the deselected keys
as an excluded set. An excluded change is dropped on the user's word:
a conflicting entity resolves to the target side (the recorded choice,
per the no-silent-winner decision), and an excluded addition cascades
to its added descendants, so no emitted change references a parent the
merge does not create. A merge whose every change is excluded refuses
with :nothing-to-merge.
The update squash needs no filter: whatever excluded main work is
missing from the branch lands in the squashed net as a branch-side
delta, so the branch state still replays from its repositioned base.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The comparison pairs the branch's media rows with main's through the
content identity heuristic (media-pairs), which approximates the
pairing from the rows present on both files. The writes that actually
establish the pairing know it exactly, so now they record it and the
comparison reads the record.
Every file_branch_change row stamps media_id_map jsonb: the media id
map the save or the update from main that wrote the row applied, as
applied ({old-id -> new-id}, the branch's own row is the new one). A
save stamps the remap its media fix-up applied, filtered to the copies
whose source row the branch's source file owns; the squash row of an
update from main stamps the map applied to main's data before the
squash (the pairs plus the fresh copies).
files_branch.clj::branch-media-pairs unions the inverted stamps when
the log is fully stamped and falls back to the heuristic when any row
carries none (rows written before the column existed), so old logs
keep working and the heuristic never runs for stamped ones. Every
former media-pairs consumer goes through it: the comparison
(branch-id-map), both base normalizations, the merge and the update.
Signed-off-by: Alvaro Tejero Cantero <alvorithm@teje.ro>
AI-assisted-by: opus-5-5
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
develop replaced the legacy one-shot task system with app.jobs
(#11542). Two branching call sites still used the removed API, so the
backend no longer compiled on top of it:
- files-branch/delete-branch! schedules the branch file's
:delete-object job with jobs/submit, joined to the caller's
transaction through the cfg's ::db/conn, as files.clj does.
- files-pull-request/send-review-request-emails! queues the review
request with eml/send and ::eml/reuse-conn, as the comment emails
do, and now takes the transaction cfg instead of the connection.
The tests follow: the rollback test redefines eml/send, and the
file-gc scheduler test reads the scheduled work from the job table,
whose params are plain JSON.
AI-assisted-by: claude-opus-5-5
Branches and databases from the preview's earlier shapes are not
supported, and such branches are recreated. The earlier shapes are the
copy model, where a branch file kept its own data payload, and the
tables registered as 0150-add-file-branch-table,
0152-add-file-branch-table and 0153-add-file-pull-request-tables. This
commit removes the code that existed only for them. A fresh database
ends with the same schema as before.
- 0155-add-file-branch-table.sql loses its header about the old
registrations, every IF NOT EXISTS, and the base_branch_revn upgrade
and backfill. CREATE TABLE file_branch already declares the column.
- 0156 and 0157 lose every IF NOT EXISTS.
- 0158-add-file-branch-change-data-version folds into 0157, and its
file and registration go. CREATE TABLE file_branch_change declares
data_version NOT NULL, because both writers of the op log, a branch
save and the squash of an update from main, stamp every row.
- need-migration-to? loses its nil case, which replayed an unstamped
op-log row at the base's version, and the branch derive loses the
comment that described it. Stamped rows replay as before.
- The tests for a re-run of the branch table migration and for the
replay of an unstamped row go, with the require the first one used.
A database that holds the preview's older shapes must drop its
branching tables and the file.is_branch column, and delete the
branching steps from its migrations table, before it boots this code.
A database that ran 0157 without 0158 lacks data_version and counts as
an older shape. A database that already ran 0155 to 0158 under these
names keeps working, because a migration runs once per name.
AI-assisted-by: claude-opus-5-5
Depth gate: the compare reads the op log and applies the depth gate
before it derives the branch, the same order merge and update from
main use, so a log too deep to replay refuses before the derive runs.
The refusal names the materialise and the delete, the two remedies the
gates do not refuse, in the hint and in the notification copy.
Resolutions: the conflict answers of merge-file-branch and
update-branch-from-main carry the main revision the conflicts were
computed against, and the client sends it as expected-main-revn when
resolutions are applied before the dialog's diff fetch returns.
Libraries: absorbing skips branch files, so unpublishing or deleting a
shared library writes no payload on a branch and the branch keeps its
own link; a soft-deleted library resolves nothing there until the
branch's next update from main brings main's absorbed copies. The
absorbed write passes the file's features to update-file!, which the
hand-built params dropped, so the pointer-map fragments the absorption
regenerates are persisted and no payload references a fragment no
write produced.
Queries: the project file count, the team file count, the list of
files that use a library, and the socket-REPL team duplication skip
branch files.
Snapshots: the branch refusal lives in check-restorable!, which the
restore command calls before its backup snapshot and the two
socket-REPL restore helpers reach through restore!.
Migration 0155: the base_branch_revn backfill repositions only the
rows whose branch file carries a data payload, so a re-run changes
nothing on the branches the op-log model creates.
Tests: the branch merge tests run in JavaScript, which the common test
runner lists; new backend tests pin the gates, the absorption skip,
the query filters, the restore refusals, the migration re-run, and the
conflict answers' revision; a new frontend test pins the token the
senders carry with resolutions.
AI-assisted-by: mixed models
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The branch comments called the set of ids a document's references are
written in (main's file id and media rows, or the branch's) a frame.
Penpot's code uses frame for a board, through `:frame-id`, so a reader
could not tell which sense a comment meant. The comments, the
docstrings of `get-base-data`, `branch-base-data`, and `remap-changes`,
and one test name now say id space. Every board-sense use of frame is
unchanged.
AI-assisted-by: mixed models
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Four comments cited docket entry ids that resolve only on the machine
the set lives on, and the reviewer this branch is written for cannot
follow them. Each citation stood beside the number or the rule it
records, so the sentences stand without it.
AI-assisted-by: mixed models
Merge engine: a same-parent reorder, an addition under a parent main
deleted, and a branch deleting a container main edited each lost work
silently. Each now follows a policy read from
backend/resources/app/branch-merge-policies.edn, re-read when the file
changes, with the defaults: merge the reorder, and raise a conflict on
the deleted parent or container. Added siblings keep the branch's order.
A :mov-page index counts the target's full page list, a cleared page
background reaches main, a per-attribute resolution can remove an
attribute, and a token set reorder survives a rename on the other side.
Conflict payloads name :main and :branch by document in both
directions. Reorder detection costs nothing measurable on a page of
20,201 objects.
Data versions: each op-log row records the data version it was written
at, the branch replays its log in segments and migrates the document
at each boundary, and the comparison base is migrated like main and the
branch, so an upgrade no longer shows as false conflicts.
Writers: a binfile import, an admin repair, file GC and a snapshot
restore refuse a branch file, whose payload nothing reads, and the
branch base pin cannot be renamed into a user snapshot. A branch save
runs the ordinary media fix-up and logs the rewritten references. An
update from main deletes the branch's copy of media main deleted and
squashes no phantom media changes.
Commands and UI: the pull-request commands publish their messages after
the commit, the listing takes one connection per comparison and shows a
warning instead of "in sync" when a comparison fails, materialize takes
:branch-file-id, and a dashboard fetch that never answers ends in an
error state instead of a spinner, with a refetch on reconnect.
backend/dev/branch_merge_cases.clj builds one example file per merge
policy case for demonstrating the alternatives.
Two models wrote this change, claude-opus-5-5 and mimo-v2.6-pro, and
the work switched between them, so the trailer reads mixed models.
AI-assisted-by: mixed models
The source notes on files_branch.clj listed 32 review findings. This
commit fixes the 21 that are defects of the code; each fix carries a
regression test that failed before it and passes after.
Concurrency: merge, update from main, rename, archive and delete now
re-read the branch row under the advisory lock and work off that row,
so a concurrent merge or update can no longer leave a ten-year base
pin behind or release a superseded base twice. Delete and archive take
the branch file lock, create and reopen of a pull request refuse a
branch that a concurrent merge closed, and create checks the branch
quota under the lock. Merge, update and delete publish their msgbus
messages only after the transaction commits.
Merge, update and materialize re-validate after the repair: merge and
update refuse a result that stays invalid, materialize logs it. The
change log row carries the repair changes, and the three paths drop
the cached file summary under :redis-cache.
Contracts: a :conflict refusal answers 409 instead of 500, resolution
values must be :main or :branch, and update from main accepts the
expected-main-revn token the frontend now sends from a loaded diff.
Smaller fixes: create refuses a blank name, writes no file_data row,
and a rename reaches the branch file's own name. Deleting a kept
merged branch keeps its merged status. get-branch-diff reports the
creation time of the current base snapshot. Rename, archive and delete
report their duration and outcome. An image used only by a library
colour is copied into the target file. Dead fallbacks and dead
deleted-at checks are gone.
AI-assisted-by: claude-opus-5-5
Data loss, reproduced twice rather than inferred. A branch stores no
data of its own, so its whole work lives in `file_branch_change`, and
`update-branch-from-main` had a fast path that deleted every one of
those rows whenever the update found nothing of main's to apply. The
comment there assumed the branch was in sync with main, which holds only
when the branch has no changes of its own, and `branch-diverged?` was
already computed two lines above and consulted only inside
`reposition-base!`.
A branch with one fill edit of `#bb0000` and an untouched main, updated
with no resolutions, returned `:updated` and came back `#000000` with an
empty log. The user asked to catch up with a main that had not moved and
lost their edit. Through the modal the same defect made the resolution
that says "keep mine" the one that discards it.
The fast path is now guarded by `(not branch-diverged?)`, so a diverged
branch falls through to the squash that re-derives its net against the
new base. The predicate is the right one: it compares the branch file's
own revision counter against the revision the base was last positioned
at, and a log row and a revision bump are one event, because
`persist-branch-file!` is the log's only appender. The comment at the
guard states the invariant, which is that the log may be emptied only
when what it encodes already sits in the base.
One correction to how the fast path was described while this was being
chased: `(empty? changes)` does not mean the branch has nothing of its
own. In the update direction the change set is MAIN's contribution, so a
branch that is behind main in content already took the squash path and
still does. The fast path is reached when main has nothing to contribute,
which is what the third new test pins.
The three sibling paths that also empty or replace a log were each
judged and are sound, because each one writes the state elsewhere inside
the same transaction before emptying.
Three regression tests, one per reproduction plus the case the fast path
exists for, asserting through the derived document rather than through
log row counts, because a legitimate squash rewrites the log while
preserving the state.
Verified: the whole backend suite, 923 tests and 5,550 assertions with 0
failures. The two live demo branches were checked read-only and still
hold their one op-log row each, so the hand-test fixture was never
damaged.
Two defects with one cause, which is that a comparison and a write want
different identifier frames and the code used one frame for both.
Bringing main into a branch compared in the branch's frame. It remapped
main and the base into that frame, which puts the branch's inherited
content where it cannot agree with the base: everything the branch
inherited still names main, so every reference the remap rewrote read as
a difference. On a demo branch that was 364 of 368 reported conflicts,
and none of them was an edit anybody made. The comparison now runs in
main's frame, exactly as the compare view and the listing summary
already do, and the changes it produces are translated into the branch's
frame before they are applied and logged. That translation is the new
`branch_merge.cljc::remap-changes`, which is a separate function rather
than a call to `remap-refs` for a reason its docstring gives: `relink-refs`
finds a reference where the attribute and the value travel together, and
a change vector carries the identifier alone. The write map is renamed to
`apply-map`, so no later reader can reach for it as the comparison map.
The compare also decoded its merge base twice. The derive resolves and
decodes the pinned snapshot in order to fold the op log over it, and the
caller then decoded the same bytes again for the comparison's third
version, which is about 406 ms on the design-system file and is paid by
every uncached branch summary in a listing as well. The derive now
offers the pre-fold value it already holds, behind an explicit opt-in, so
an ordinary read gains neither a key nor a cost, and a branch read that
forgets the flag raises rather than comparing against nothing. The
invariant a comment now states is that the comparison's base is the
pre-fold value, which is sound because the fold is pure: `process-changes`
threads a value, every method builds persistently, and `PointerMap`'s
`assoc` and `without` return new objects, so the pre-fold base shares
structure with the folded result and costs no extra memory for the pages
the log did not touch.
The new test asserts both halves, because either alone passes a wrong
implementation. A branch carrying only inherited content, updated from a
main that changed elsewhere, reports no conflicts and leaves an empty op
log. Main and the branch repainting the same shape still reports exactly
one conflict, naming that shape.
Verified: the whole backend suite, 919 tests and 5,525 assertions with 0
failures.
Three of the preview's paths reported no duration, and two of them were
the slowest a user meets. A trial can be asked what a merge cost and
could not be asked what the panel cost.
The listing now passes through `audited` with two counters beside the
duration, being how many open branches it compared and how many of those
the summary cache answered, because a listing's cost is entirely
explained by that ratio. The vector result was the one risk and it holds:
a Clojure vector carries metadata, and nothing between the handler and
the audit step rebuilds the value.
A branch open now reports the derive's own time, its op-log depth, and
its batch count, from `branch-file-data` itself. Depth is what predicts
the cost, so the record carries it. An ordinary read never enters that
function, so it gains no line, no prop, and no timer.
A branch save gains the log line the other five print, and its timer
moved rather than gaining a caveat. It now opens before `get-file`, so
the number covers the read and, for a branch, the derive inside it, which
is what compare, merge, update and materialize have always covered. The
six figures are comparable for the first time.
The concurrency limits were decoration. Thirteen ids were declared across
the branch and pull-request commands and `climit.edn` configured none, so
`climit/wrap` logged a warning at every start and ran each handler
unwrapped, leaving the root chain's forty global permits as the only
bound on an operation that holds three realized documents. Every declared
id now has an entry, the two readers that realize whole documents gained
a declaration, and each number is justified in a comment against
`create-file-snapshot` as its anchor and marked provisional until a
peak-heap measurement replaces it.
The test helper drives the private save path directly, so it opens the
timer the RPC method would have opened.
Verified: 73 tests, 694 assertions, 0 failures across the three branch
suites and `rpc-file-test`.
A branch copy re-points its local references through the duplication pipeline
and leaves keys like :fill-color-ref-file and :typography-ref-file present with
nil where the source omitted them. Comparing the two forms as different made an
untouched entity look edited on one side, which becomes a modify-modify
conflict as soon as the other side moves the same shape.
The comparison now drops nil-valued keys deeply, for all three sides, in
three-way-entities and so for every pass that goes through it. The new engine
test failed before the change on both orderings and passes after; the three
branch suites report 30 tests, 245 assertions, 0 failures.
The preview's observability criterion is stated as every branch operation, and
two of them sat outside it. A save routed to a branch's op log carried only the
ordinary save props, and materialise carried none, so the two operations that
matter most to an enterprise trial were the two with no duration in the record.
Both attach `branch-operation`, `branch-outcome` and `branch-duration-ms` now:
`save-branch`/`saved` on the save path, `materialize`/`materialized` on the exit
door, including its idempotent no-op. The props travel in `::audit/replace-props`
on the save path because that is the key the ordinary save already replaces, so
the ordinary save's own props are untouched.
The test that asserts the audit props now covers a save and a materialise
alongside creation and merge. The three branch namespaces are 40 tests and 338
assertions, green.
