* 🐛 Refuse saves to a branch kept after its merge
A branch kept after its merge stays an ordinary file with status
"merged", so every save passed the edition check and appended ops to
an op log that no command integrates any more. Refuse such saves with
a :branch-merged validation error instead of stranding them. An
archived branch still takes saves: restoring it brings them back.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Tell merged and archived branches apart
The workspace treated "merged" and "archived" as one state: an
archived branch read "Merged branch" and lost every action, and a
merged row offered Restore, which the backend always refused.
Each status now has its own presentation. A kept merged branch keeps
"Merged branch" and View in main, and opens read-only: the workspace
permissions drop :can-edit while it is the open file, the way a
viewer gets a read-only workspace. An archived branch says so in the
banner and offers Compare and Restore. The header notes each status,
the switcher lists merged and archived branches in two collapsible
groups, the all-branches dialog gets a Merged tab, and the branch info
dialog offers Restore and Compare for an archived branch.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Refuse saves to an archived branch
An archived branch is now read-only, like a merged one: the user
restores it, or turns it into an ordinary file, to keep editing.
persist-branch-file! refuses its saves with a :branch-archived
validation error, distinct from :branch-merged so the client can say
which way out applies. A restored branch takes saves again, and so
does an archived branch turned into an ordinary file.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Open archived branches read-only with their ways out
An archived branch now opens read-only, like a merged one. Its banner,
its branch info dialog and its row in the all-branches dialog offer
the two ways back to editing, Restore and Turn into an ordinary file,
next to Compare. The header and switcher note reads "Archived, read
only".
A save the server refuses because the branch is merged or archived
now shows a warning that names the way out, instead of the generic
autosave error with a report link.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Add a branch count query for the workspace header
The header shows its branch row only when the file has branches, and
it learnt that from get-file-branches, which compares every diverged
branch against main. get-file-branch-count answers the same question
with one indexed count: archived and merged branches included, counted
from main when the file is itself a branch, with no comparison and no
climit, like the branches_count subquery of the dashboard listings.
The listing keeps its climit, and :get-file-branches/global now sets
the 60 s timeout that the header of climit.edn promises, so a waiter
behind six cold listings gets an error instead of waiting without
bound. Its rationale comment names the callers it serves now.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ List branches only when the switcher opens
Opening a file ran the counted branch listing for every user, because
the header decided whether to show its branch row from that listing.
A cold listing runs a three-way comparison per diverged branch, and
one save to main makes every branch of the file a cache miss again.
The header now loads the branch count and the branch context with the
file. The listing loads when the branch switcher or the all-branches
dialog opens, and both show "Loading branches…" until it arrives
instead of a list from an earlier opening.
A failed listing used to read as an empty list, which on main hid the
branch row and sent the shortcut to the create dialog. It now ends in
an error state: the switcher and the dialog say "Couldn't load
branches" with a Retry action, and the row stays, since its visibility
comes from the count. A successful listing refreshes the count, so
deleting the last branch from the list still hides the row.
Creating a pull request from the branch banner looked the branch up
in the listing to decide whether to save pending edits first. Without
the listing loaded at open it would skip the save, so it now reads the
branch context of the open file.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Show the branch row when the branch count fails
A failed get-file-branch-count left no count, so on main the header
hid its branch row and the shortcut opened the create dialog on a file
that may have branches. The failure is now kept as an error state
under :workspace-branch-count, and has-branches? reads it as true.
The row then shows the Main chip and, where the status note goes,
"Couldn't load branches" in the warning colour. Opening the switcher
loads the listing through the existing loading and Retry path, and a
successful listing settles the count, which replaces the note.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 📚 Name the callers of the branch listing limit again
The merge of iop-vc-preview took its climit.edn, whose comment on
:get-file-branches/global still described a panel open. Restore the
comment this branch wrote: the listing runs when the switcher, the
all-branches dialog or the dashboard popover opens, and a file open
only counts branches.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Mock the branch count in the library copy tests
The header now shows its branch row from get-file-branch-count, not
from the listing, so the library copy tests mock the count as well.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Files that use a shared library read its main, never a branch, so a
branch of a library is not isolated in the sense the create dialog
promised. On a shared library the create dialog now says that files
using the library can adopt changes from main only, the chip of a
library branch says the same, and the note beside main reads
"Default branch (read by library consumers)". Other files keep their
copy.
get-file-branch-info returns source-is-shared so a branch knows its
main is a library.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
A branch whose only change was file-level plugin data or the base font
size merged with a success response, and the change was lost. The merge
engine compared colours, typographies, components, media, pages, and
tokens only, so such a branch produced an empty summary and the merge
took the empty-changes shortcut.
The engine now classifies two more kinds three-way: file plugin data,
flattened to namespace and key as page plugin data already is, and the
file options, where an unset base font size reads as the default. Both
kinds count towards ahead, behind, and conflicts, appear in the compare
dialog, and translate into set-plugin-data and set-base-font-size
changes for merge and update from main. Both sides changing the same
key to different values is a modify-modify conflict.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Give the branch concurrency limits a timeout
Every global entry for branch and pull-request commands lacked a
:timeout, so a saturated limiter parked its waiters forever and
climit-test/example-config-bounds-every-wait failed on seven ids.
Each entry now waits at most 60 s, the value of the other slow
operations. The size gates in files_branch.clj refuse files whose
operation would exceed 30 s, so the wait covers one in-flight
operation with as much again in margin.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Make the quota racer a member of the file team
File access requires membership of the file's team since the
permission check started joining team_profile_rel. The racer in
create-checks-the-branch-quota-inside-the-lock held only a file role,
so its create failed with :object-not-found and the test never raced.
The racer now joins the file's team as an editor, so both creates
again compete for the last branch slot.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Shorten the rationale above the size gates and the docstring of
get-branching-limits, and say where the numbers were measured.
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Every gated branch operation flattened the whole op log only to count
its changes, and the derive in the same request decoded those rows
again. The count is not recoverable from the bytes: a row holds one
compressed change vector whose length says nothing about the number of
changes beside a squash that emits tens of thousands of them for a
one-page delta.
Each `file_branch_change` row now stamps `changes_count` at write time,
in `persist-branch-file!` and in the squash `persist-branch-update!`
writes, and `binfile/common.clj::get-branch-oplog-depth` sums the live
rows. Migrations 0804 and 0805 add the column, fill it from the stored
vectors, and make it NOT NULL, so a writer that omits it fails instead
of recording an empty log.
The gate keeps its place ahead of the derive in `get-branch-diff`,
`merge-file-branch`, and `update-branch-from-main`, and a test pins
that it counts the changes a row holds rather than the rows.
Entry: pp:vcs:the-op-log-depth-check-pays-a-derive open -> done
AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The four preview migrations leave the 0155-0159 range, where
develop keeps registering new entries around them, and take
0800-0803. Upstream cannot force another mid-development renumber
now; the migrations take their final autoincrement numbers in one
rename when the feature merges into Penpot.
Renaming a migration registers it under a new name, so any
database that ran the old names drops the branching tables, or
deletes their migrations rows, before it boots this build. The
shared devenv database and the ws1 test database are in that
state on this machine.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Both conflicts are tail appends onto regions develop rewrote. In
migrations.clj keep both appended blocks, develop's two entries
first so the branch's four stay the tail of the vector. In
climit.edn take develop's rewritten entries and re-append the
branch's provisional block, moving the map's closing brace to the
last line.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The merge of main into staging brought the closed storage metadata
schema, which requires file-id and id on file-data objects. The
restore test built one without those keys, so it failed with an
invalid metadata error. Its helper now takes extra metadata and
the test passes the ids of the file and of its file-data row, as
production does.
The republish test asserted the exact list of waiting-gauge
publications, but the component's sampler starts its first tick right
away and under load lands inside the recorded window, publishing the
same count twice. The test now builds the counters alone, so the
republish call is the only publisher.
AI-assisted-by: deepseek-v4.1-flash
The queue gauge was written by whichever limiter of a name ran last, so a
per-profile pile-up never showed: a healthy profile writing 0 overwrote the
15 of the profile that was queueing (4 of 1441 samples in production), and
a limiter whose requests never come back read 0 while 72 threads stayed
parked.
A queue is an entry and an exit, so the new gauge is accumulated: one
counter per limiter name and instance, up when a call starts waiting for a
permit, down when it stops, so a call that never leaves stays visible. It
is always written with `:val`, and a daemon sampler republishes every count
every 15s so a name whose traffic died does not go stale in the scrape;
halting the component stops that thread.
Rejections had no metric at all: penpot_rpc_climit_rejected_total counts
them by name and reason (queue-full or timeout), and the log lines now
carry queue-max, permits, code and reason, so an operator can tell whether
the limiter did its job or was saturated.
The gauges that could not tell the truth are gone: queue and permits (same
last-write-wins problem, and waiting plus the wait histogram replace them)
and http_worker_busy_threads (reads 0 on an instance with 72 blocked
threads; pool_size is the signal). No other dashboard or alert rule reads
them.
AI-assisted-by: deepseek-v4.1-flash
Add a per-process availability flag (app.health) that /readyz
reports as 503 OFFLINE while offline, so the load balancer
drains the instance and in-flight requests finish untouched.
Operator surface is app.srepl.main/set-offline and
app.srepl.main/set-online; a restart comes back online.
Covered by backend-tests.health-test; HA docs updated.
AI-assisted-by: muse-spark-1.3-contributor
`file_object_thumbnail` was replaced by `file_tagged_object_thumbnail`
and nothing refers to it anymore: no query in the backend touches it,
no table has a foreign key pointing at it, no view depends on it, and
it is empty in production. It also lacked `deleted_at`, which is why
restoring files had to leave it out. Only legacy naming survives in two
helpers (`get-file-object-thumbnails` and
`delete-file-object-thumbnails!`, which operate on
`file_tagged_object_thumbnail`).
The storage bucket named "file-object-thumbnail", the one that
`file_tagged_object_thumbnail` uses, is a different thing and stays.
AI-assisted-by: deepseek-v4.1-flash
Restoring a file only un-marked the database rows (`file`,
`file_media_object`, `file_change`, `file_data`, `file_thumbnail`,
`file_tagged_object_thumbnail`), not the storage objects those rows
reference. An object that was already marked for deletion stayed
marked while a live row pointed at it, and those foreign keys are `NO
ACTION`: the storage GC cannot delete it and skips it forever (and
before the GC learned to skip, it cost the whole nightly run).
Restoring a team had the same gap with its font variants.
Both restores now unmark the storage objects of the rows they bring
back: the media and the thumbnail of the file media objects, the media
of the file and object thumbnails, the `storage-ref-id` of the file
data, and the four files of a font variant. Objects referenced by other
rows are left alone.
AI-assisted-by: deepseek-v4.1-flash
objects-gc deleted every kind of object inside one transaction per proc,
looping until empty, so a single row that a foreign key still references
aborted the whole run: the task died on its first file every night, after
the earlier steps had already spent around twenty minutes. The per-file
savepoint only saved the files proc and left the same shape everywhere
else.
Now every proc streams its pending keys with db/plan inside a read-only
transaction of its own and deletes them chunk by chunk, each chunk in a
transaction of its own with FOR UPDATE SKIP LOCKED. A chunk that fails is
reported with l/err and skipped, so the run reaches the end and the next
run retries the skipped rows.
AI-assisted-by: muse-spark-1.3-contributor
`storage-gc-deleted` ran its whole batch inside one transaction: the
handler opened it and `delete-in-bulk` reused that connection, so the
"separate transaction" its comment promised did not exist. A single
row that a foreign key still references aborted that transaction, the
cursor then died on its next fetch ("current transaction is aborted")
and the task ended up throwing `unhandled exception on running task`
after rolling back the whole night of work. That is what has kept
production from deleting a single object since late April.
Now `clean-deleted` walks the pending objects with the cursor in its
own read-only transaction and every chunk runs in its own one, so a
blocked chunk is logged with its ids and skipped while the rest is
committed. `task finished` is reachable again.
The private functions also drop the `!` suffix: the namespace is all
mutation, so the mark adds nothing and reads as noise.
The test builds a blocked object (a `file_media_object` pointing at it
as its thumbnail) plus a full chunk of pending objects and one more in
the next chunk, and checks that the run finishes and that the object of
the next chunk is deleted.
AI-assisted-by: deepseek-v4.1-flash
The four `process-deleted-*-cascade` helpers of `app.srepl.main`
loaded every soft-deleted row of a table into memory (`db/exec!`)
and ran `:delete-object` inline (`wrk/invoke!`), so they could not
be used against production: `file` alone has 2.1M deleted rows.
They now share a single `submit-deleted-cascade!` that reads the
rows with a server-side cursor (`db/plan`, in a read-only
transaction) and submits one task per row, so the work goes
through the workers like any other task. The row id goes as
`label` with `dedupe` on, and the helpers accept `:limit`, `:ids`
and `:dry-run?` to run bounded batches. Public names and their
zero-arity form are unchanged.
Tests cover the task params, the row's own date (not `now`), the
`limit`/`ids`/`dry-run?` options and that each helper reads its own
table.
AI-assisted-by: deepseek-v4.1-flash
An instance left the audit events pipeline dead: every push-audit-events
answered with 429 and 69 worker threads parked for good inside a limiter.
Two code paths of one profile acquired the same two limiters in opposite
order: a command declaring its own limits (upload-file-media-object) took
process-image/by-profile and then root/by-profile, while a command with no
limits of its own took the roots first and then re-entered climit from
inside its handler with the same ids (create-file-media-object-from-url).
Each path held what the other one waited for, and no limiter had a timeout
to break the wait.
Now wrap puts the root limiters after the command's own, so they end up
outermost and every path meets them first: the nested chains carry only the
command's own limits and run inside them, which puts both paths in the same
order. impl-invoke also keeps, per thread, the set of limiters that thread
is inside of and runs the handler inline when a call asks for one it
already holds, so a handler can no longer queue behind itself.
The shipped climit.edn sets a timeout on every limiter (60s for operations
that can be slow, 30s otherwise), so no wait can park a thread for ever.
The production file lives outside the repository and has to be mirrored.
Tests cover the acquisition order, the deadlock, the reentrancy and the
timeouts of the shipped config.
AI-assisted-by: deepseek-v4.1-flash
The command declared its limit as a plain keyword next to
`::climit/key-fn`, metadata that `climit/wrap` does not read: it
takes the key only from the positional `key-fn` of the vector form.
The limit was a single shared one, so when one call got stuck it
held the only permit and every audit batch of that instance was
rejected with 429 from then on.
The test builds the limiter from the command metadata and checks
that two profiles end up with different limiters.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Add skip-if-active option to jobs submit
Producers that need at most one live job per key can now pass
::jobs/skip-if-active: the submit counts live jobs for the same
tenant, name, queue and label in the same transaction and skips
the insert (returning nil) when one lives. The option requires an
explicit label and documents the same best-effort race caveat as
the dedupe it complements.
Closes#12265
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Record skipped job submissions as a metric
Skip-if-active submissions withheld nothing observable: the skip
now records penpot_jobs_skipped_total (name, queue) instead of the
submitted counter, so dashboards can tell withheld work apart from
new work. The insert path is untouched.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Drop jobs counters duplicated by histogram counts
The completed and requests counters counted exactly the same events
as the _count of their timing histograms, so they go away: terminal
totals now come from penpot_jobs_total_ms_count by outcome and
request totals from penpot_jobs_request_ms_count by outcome. One
event, one family from here on: counters only cover events without
latency, and the dispatched vs queue-wait pair stays because pushes
and claims are different events.
AI-assisted-by: muse-spark-1.3-contributor
Clients that only need a file's design tokens, such as a CI job that
syncs them to a git repository after an update-file webhook, had to
download the whole file with get-file.
get-file-tokens returns the file's tokens as a DTCG document, the same
format the token export produces, or nil when the file has none. Active
themes and sets come from the file's tokens status, so they reflect
what is currently active.
The document is returned as a plain map. A bare TokensLib is not a
collection, so the response middleware would hand it to ring unencoded.
Relates to #7091
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The Plugin API can write arbitrary key/value plugin data onto the file
root, but it could only be read back by downloading and decoding the
whole file. Add a `get-file-plugin-data` read command that returns the
file-level plugin data map, optionally narrowed to a single namespace,
so external tooling can fetch metadata such as issue or commit
references cheaply.
Read-only: reuses `bfc/get-file` and `check-read-permissions!`, and
adds no data-model or change-type changes. Object/page level reads are
intentionally left out of this first command.
Relates to #10169
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
`resolve-and-link-libraries` asserts `::bfc/team-id`, and
`enable-backend-asserts` is now enabled in `scripts/ci` and
`tests-backend.yml`. Without a team-id, the v3 import tests
error instead of passing. Production always provides the
team (RPC import, management and debug routes); mirror that
in the tests.
Update `mem:backend/testing` and `mem:scripts/ci` with the
`enable-backend-asserts` behavior.
Verified with `./scripts/ci backend`: 1235 tests, 6384
assertions, 0 failures.
AI-assisted-by: deepseek-v4.1-flash
A plugin with "scope": "global" keeps running across the dashboard
and files and can find and open the user's projects and files. Any
manifest could ask for this without the user being told.
Add the allow:global permission. The plugin registry adds it to
every manifest with global scope, so the install dialog lists it,
and a plugin that becomes global in a later version shows the
permissions update dialog again. The permission is valid in the
shared schema, which the backend checks when saving, and in the
runtime manifest schema. The built-in MCP plugin declares it too.
Document the scope property and the new permission for plugin
developers.
Two comments cited an internal work-tracker id that a code reader
cannot resolve and that the tracker may rename. Both now say what the
rule says: one side keeping its value in a merge is a recorded choice,
never a silent win.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* ✨ Run binfile import and export as durable jobs
Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.
Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.
How:
- The substrate gains the pieces a user-facing job needs: optional
job-def metadata (family, resource-role), an expiry on submit, the
owner in the runner context, and job resources in their own bucket,
written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
business params, the registry checks the family, the job-def decodes
and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
vocabulary with self-contained counters, throttled heartbeats, and an
interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
the package descriptor; the import re-checks the edition permission,
runs the core in a transaction so a cancellation rolls it back, and
releases the consumed package and its temporary copy.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Handle externally-cancelled binfile jobs end to end
The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.
- The export emits `{:cancelled true}` per file and the modal paints a
neutral cancelled row instead of success; a message without an
artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
they replace.
Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Pre-translate the cancelled import message
The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Move cancelled strings under generic labels keys
The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add minor improvements
* ♻️ Rename export job command to create-export-binfile-job
Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.
submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.
The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Polish the import and export status rows
The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.
In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Show the cancelled export and fix the import error
The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).
The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.
The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Reflect the real job params in the creation commands
The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.
Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.
A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Stop the test system from running job runners
The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.
That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.
Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Upload the package one chunk at a time and as a percentage
The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.
The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.
AI-assisted-by: deepseek-v4.1-flash
* 🌐 Complete the Spanish catalogue
es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).
sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Move the job status keys into the jobs namespace
The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.
Rename them so the whole job vocabulary is one namespace:
labels.queued -> jobs.queued
labels.export-cancelled -> jobs.export-cancelled
labels.import-cancelled -> jobs.import-cancelled
No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.
AI-assisted-by: deepseek-v4.1-flash
* 🔥 Remove the translation keys nothing references
Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:
labels.uploading-file a duplicate of jobs.progress.stage.upload
since the upload stage says "Uploading file…"
labels.downloading-file dead since the durable jobs replaced the old
import and export progress rows
inspect.attributes.stroke.alignment.center
inspect.attributes.stroke.alignment.inner
inspect.attributes.stroke.alignment.outer
workspace.toolbar.mcp-connect-here
The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.
AI-assisted-by: deepseek-v4.1-flash
The branch compare panel now offers per-change selection, grouped by
the component each shape belongs to. Every summary entry carries a
stable change key (kind/page/id, the design-branches PoC format), and
the panel's checkboxes key on it.
merge-file-branch and update-branch-from-main take the deselected keys
as an excluded set. An excluded change is dropped on the user's word:
a conflicting entity resolves to the target side (the recorded choice,
per the no-silent-winner decision), and an excluded addition cascades
to its added descendants, so no emitted change references a parent the
merge does not create. A merge whose every change is excluded refuses
with :nothing-to-merge.
The update squash needs no filter: whatever excluded main work is
missing from the branch lands in the squashed net as a branch-side
delta, so the branch state still replays from its repositioned base.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The comparison pairs the branch's media rows with main's through the
content identity heuristic (media-pairs), which approximates the
pairing from the rows present on both files. The writes that actually
establish the pairing know it exactly, so now they record it and the
comparison reads the record.
Every file_branch_change row stamps media_id_map jsonb: the media id
map the save or the update from main that wrote the row applied, as
applied ({old-id -> new-id}, the branch's own row is the new one). A
save stamps the remap its media fix-up applied, filtered to the copies
whose source row the branch's source file owns; the squash row of an
update from main stamps the map applied to main's data before the
squash (the pairs plus the fresh copies).
files_branch.clj::branch-media-pairs unions the inverted stamps when
the log is fully stamped and falls back to the heuristic when any row
carries none (rows written before the column existed), so old logs
keep working and the heuristic never runs for stamped ones. Every
former media-pairs consumer goes through it: the comparison
(branch-id-map), both base normalizations, the merge and the update.
Signed-off-by: Alvaro Tejero Cantero <alvorithm@teje.ro>
AI-assisted-by: opus-5-5
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
develop replaced the legacy one-shot task system with app.jobs
(#11542). Two branching call sites still used the removed API, so the
backend no longer compiled on top of it:
- files-branch/delete-branch! schedules the branch file's
:delete-object job with jobs/submit, joined to the caller's
transaction through the cfg's ::db/conn, as files.clj does.
- files-pull-request/send-review-request-emails! queues the review
request with eml/send and ::eml/reuse-conn, as the comment emails
do, and now takes the transaction cfg instead of the connection.
The tests follow: the rollback test redefines eml/send, and the
file-gc scheduler test reads the scheduled work from the job table,
whose params are plain JSON.
AI-assisted-by: claude-opus-5-5
The viewer test that develop carries, share-link-bundle-trims-linked-
libraries, pins the envelope keys a trimmed library may expose, and the
branching column :is-branch is not one of them: whether the source file
of a library is a branch is not something a share-link viewer may read.
The column exists only on this stack, so develop's test could not see
it and the collision appears only on the rebase.
trim-library-data drops it. A library a branch links is materialised,
so the column is false there anyway; dropping it is about the envelope,
not about hiding a value.
backend-tests.rpc-viewer-test: 11 tests, 95 assertions, 0 failures.
AI-assisted-by: claude-opus-5-5
Branches and databases from the preview's earlier shapes are not
supported, and such branches are recreated. The earlier shapes are the
copy model, where a branch file kept its own data payload, and the
tables registered as 0150-add-file-branch-table,
0152-add-file-branch-table and 0153-add-file-pull-request-tables. This
commit removes the code that existed only for them. A fresh database
ends with the same schema as before.
- 0155-add-file-branch-table.sql loses its header about the old
registrations, every IF NOT EXISTS, and the base_branch_revn upgrade
and backfill. CREATE TABLE file_branch already declares the column.
- 0156 and 0157 lose every IF NOT EXISTS.
- 0158-add-file-branch-change-data-version folds into 0157, and its
file and registration go. CREATE TABLE file_branch_change declares
data_version NOT NULL, because both writers of the op log, a branch
save and the squash of an update from main, stamp every row.
- need-migration-to? loses its nil case, which replayed an unstamped
op-log row at the base's version, and the branch derive loses the
comment that described it. Stamped rows replay as before.
- The tests for a re-run of the branch table migration and for the
replay of an unstamped row go, with the require the first one used.
A database that holds the preview's older shapes must drop its
branching tables and the file.is_branch column, and delete the
branching steps from its migrations table, before it boots this code.
A database that ran 0157 without 0158 lacks data_version and counts as
an older shape. A database that already ran 0155 to 0158 under these
names keeps working, because a migration runs once per name.
AI-assisted-by: claude-opus-5-5
Depth gate: the compare reads the op log and applies the depth gate
before it derives the branch, the same order merge and update from
main use, so a log too deep to replay refuses before the derive runs.
The refusal names the materialise and the delete, the two remedies the
gates do not refuse, in the hint and in the notification copy.
Resolutions: the conflict answers of merge-file-branch and
update-branch-from-main carry the main revision the conflicts were
computed against, and the client sends it as expected-main-revn when
resolutions are applied before the dialog's diff fetch returns.
Libraries: absorbing skips branch files, so unpublishing or deleting a
shared library writes no payload on a branch and the branch keeps its
own link; a soft-deleted library resolves nothing there until the
branch's next update from main brings main's absorbed copies. The
absorbed write passes the file's features to update-file!, which the
hand-built params dropped, so the pointer-map fragments the absorption
regenerates are persisted and no payload references a fragment no
write produced.
Queries: the project file count, the team file count, the list of
files that use a library, and the socket-REPL team duplication skip
branch files.
Snapshots: the branch refusal lives in check-restorable!, which the
restore command calls before its backup snapshot and the two
socket-REPL restore helpers reach through restore!.
Migration 0155: the base_branch_revn backfill repositions only the
rows whose branch file carries a data payload, so a re-run changes
nothing on the branches the op-log model creates.
Tests: the branch merge tests run in JavaScript, which the common test
runner lists; new backend tests pin the gates, the absorption skip,
the query filters, the restore refusals, the migration re-run, and the
conflict answers' revision; a new frontend test pins the token the
senders carry with resolutions.
AI-assisted-by: mixed models
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
