Comments written for the duplicate-file prototype still described a
branch as a copy of its file, a merge as clean-only, token diffing as
missing, a review as a read-only snapshot, and file_change rows as
living for the team deletion delay. Each now states what the code at
this revision does: a branch is a file row derived from a pinned base
plus its op log, every change kind except :page-attrs is translated,
merges apply resolutions, a review opens the live editable branch, an
update squashes the log and repositions the base, and file_change rows
expire after one hour. The audited docstring names the props it sets,
the step 6 comment says each revn counter starts at its own side, and
handle-file-merged names both topics it arrives on.
Comment, docstring, and SQL-comment lines only. clj-kondo reports no
errors on the touched Clojure files. Translations are untouched.
AI-assisted-by: claude-opus-5-5
The guide still opened by calling a branch an isolated copy of a file,
called the merge base a snapshot frozen at creation and the op log
append-only, and gave the creation transaction a reason for deferring
constraints that does not hold. It now says a branch copies no data,
that an update from main repositions the base and squashes or empties
the log, and that every row referencing the new file is inserted after
it.
The same pass corrects five more statements: base_branch_revn holds the
branch file's own revision; a branch kept after a merge keeps its pin;
op log rows survive a logical deletion and go with the cascade when the
file row is hard-deleted; a resolution may be a per-attribute map; and
the listing now runs through `audited`, while a refusal raised as an
error writes no audit event. The RPC table gains get-branching-limits,
the test lists name all five test namespaces, and the UI and event
lists name the limits, merge, and info dialogs.
AI-assisted-by: claude-opus-5-5
Data loss, reproduced twice rather than inferred. A branch stores no
data of its own, so its whole work lives in `file_branch_change`, and
`update-branch-from-main` had a fast path that deleted every one of
those rows whenever the update found nothing of main's to apply. The
comment there assumed the branch was in sync with main, which holds only
when the branch has no changes of its own, and `branch-diverged?` was
already computed two lines above and consulted only inside
`reposition-base!`.
A branch with one fill edit of `#bb0000` and an untouched main, updated
with no resolutions, returned `:updated` and came back `#000000` with an
empty log. The user asked to catch up with a main that had not moved and
lost their edit. Through the modal the same defect made the resolution
that says "keep mine" the one that discards it.
The fast path is now guarded by `(not branch-diverged?)`, so a diverged
branch falls through to the squash that re-derives its net against the
new base. The predicate is the right one: it compares the branch file's
own revision counter against the revision the base was last positioned
at, and a log row and a revision bump are one event, because
`persist-branch-file!` is the log's only appender. The comment at the
guard states the invariant, which is that the log may be emptied only
when what it encodes already sits in the base.
One correction to how the fast path was described while this was being
chased: `(empty? changes)` does not mean the branch has nothing of its
own. In the update direction the change set is MAIN's contribution, so a
branch that is behind main in content already took the squash path and
still does. The fast path is reached when main has nothing to contribute,
which is what the third new test pins.
The three sibling paths that also empty or replace a log were each
judged and are sound, because each one writes the state elsewhere inside
the same transaction before emptying.
Three regression tests, one per reproduction plus the case the fast path
exists for, asserting through the derived document rather than through
log row counts, because a legitimate squash rewrites the log while
preserving the state.
Verified: the whole backend suite, 923 tests and 5,550 assertions with 0
failures. The two live demo branches were checked read-only and still
hold their one op-log row each, so the hand-test fixture was never
damaged.
Two defects with one cause, which is that a comparison and a write want
different identifier frames and the code used one frame for both.
Bringing main into a branch compared in the branch's frame. It remapped
main and the base into that frame, which puts the branch's inherited
content where it cannot agree with the base: everything the branch
inherited still names main, so every reference the remap rewrote read as
a difference. On a demo branch that was 364 of 368 reported conflicts,
and none of them was an edit anybody made. The comparison now runs in
main's frame, exactly as the compare view and the listing summary
already do, and the changes it produces are translated into the branch's
frame before they are applied and logged. That translation is the new
`branch_merge.cljc::remap-changes`, which is a separate function rather
than a call to `remap-refs` for a reason its docstring gives: `relink-refs`
finds a reference where the attribute and the value travel together, and
a change vector carries the identifier alone. The write map is renamed to
`apply-map`, so no later reader can reach for it as the comparison map.
The compare also decoded its merge base twice. The derive resolves and
decodes the pinned snapshot in order to fold the op log over it, and the
caller then decoded the same bytes again for the comparison's third
version, which is about 406 ms on the design-system file and is paid by
every uncached branch summary in a listing as well. The derive now
offers the pre-fold value it already holds, behind an explicit opt-in, so
an ordinary read gains neither a key nor a cost, and a branch read that
forgets the flag raises rather than comparing against nothing. The
invariant a comment now states is that the comparison's base is the
pre-fold value, which is sound because the fold is pure: `process-changes`
threads a value, every method builds persistently, and `PointerMap`'s
`assoc` and `without` return new objects, so the pre-fold base shares
structure with the folded result and costs no extra memory for the pages
the log did not touch.
The new test asserts both halves, because either alone passes a wrong
implementation. A branch carrying only inherited content, updated from a
main that changed elsewhere, reports no conflicts and leaves an empty op
log. Main and the branch repainting the same shape still reports exactly
one conflict, naming that shape.
Verified: the whole backend suite, 919 tests and 5,525 assertions with 0
failures.
Four defects the first by-hand pass found, and one of them was a lie the
copy told.
The exit door has a button. The limits dialog and three refusal messages
told a user that any branch can become an ordinary file, and nothing
under `frontend/src` called the command that does it. The kebab now
carries the action between Archive and Delete for an open branch, behind
a confirmation whose copy says what is kept and what is lost: the
content stays, and the file stops being a branch, so it can no longer be
compared with main or merged into main.
The counters say what they count. Both spans carry a title naming the
direction and the unit, which is entity-level changes across shapes,
pages, components, colours, typographies, media, and tokens, and stating
that saves are not counted. That reading is what made a jump from 7 to
420 look like a regression when the number was right.
The update resolver showed each side's values under the other side's
heading, and the cause was not a swapped label. The conflict payload is
role-keyed rather than document-keyed: `compute-merge*` runs the update
direction with the branch as `theirs`, and `three-way-entities` writes
`:main` for `theirs`, so an update hands the branch under the key named
`:main`. One binding re-keys each conflict's two sides, and the values
inside `:changed-attrs` with them, in update mode only. The headings,
the chips, and the resolution keywords then all speak documents, the
bulk buttons select the side they name, and the merge mode is unchanged
by construction rather than by inspection. The modal's fallback diff
fetch now also runs in the mode's own direction, so its key space
matches.
The two bars keep off each other. The banner publishes the room it takes
from the bottom edge as one custom property, measured from its live box,
and the palette lifts itself by that value. So the offset follows the
collapsed and the expanded state, and no literal is repeated in two
stylesheets.
Verified in the running frontend rather than by inspection: both
namespaces reload with no warnings, the re-keying is an involution on a
synthetic update conflict with base, id, reason, and label intact, the
real row component renders the new menu for open, archived, and merged
rows, and the confirm's accept path reaches the backend command with the
branch file id.
Three of the preview's paths reported no duration, and two of them were
the slowest a user meets. A trial can be asked what a merge cost and
could not be asked what the panel cost.
The listing now passes through `audited` with two counters beside the
duration, being how many open branches it compared and how many of those
the summary cache answered, because a listing's cost is entirely
explained by that ratio. The vector result was the one risk and it holds:
a Clojure vector carries metadata, and nothing between the handler and
the audit step rebuilds the value.
A branch open now reports the derive's own time, its op-log depth, and
its batch count, from `branch-file-data` itself. Depth is what predicts
the cost, so the record carries it. An ordinary read never enters that
function, so it gains no line, no prop, and no timer.
A branch save gains the log line the other five print, and its timer
moved rather than gaining a caveat. It now opens before `get-file`, so
the number covers the read and, for a branch, the derive inside it, which
is what compare, merge, update and materialize have always covered. The
six figures are comparable for the first time.
The concurrency limits were decoration. Thirteen ids were declared across
the branch and pull-request commands and `climit.edn` configured none, so
`climit/wrap` logged a warning at every start and ran each handler
unwrapped, leaving the root chain's forty global permits as the only
bound on an operation that holds three realized documents. Every declared
id now has an entry, the two readers that realize whole documents gained
a declaration, and each number is justified in a comment against
`create-file-snapshot` as its anchor and marked provisional until a
peak-heap measurement replaces it.
The test helper drives the private save path directly, so it opens the
timer the RPC method would have opened.
Verified: 73 tests, 694 assertions, 0 failures across the three branch
suites and `rpc-file-test`.
A branch copy re-points its local references through the duplication pipeline
and leaves keys like :fill-color-ref-file and :typography-ref-file present with
nil where the source omitted them. Comparing the two forms as different made an
untouched entity look edited on one side, which becomes a modify-modify
conflict as soon as the other side moves the same shape.
The comparison now drops nil-valued keys deeply, for all three sides, in
three-way-entities and so for every pass that goes through it. The new engine
test failed before the change on both orderings and passes after; the three
branch suites report 30 tests, 245 assertions, 0 failures.
The modal rebuilt its conflict list from a diff fetched separately, which is
computed in another id frame than the one the update command resolves against,
so an update could open a resolver showing 0 of 0 with an Apply that could
never enable. The conflicts a command returns are authoritative, because they
are what its own resolved? matches against: the modal now prefers them, keeps
them in the store, and the bulk actions resolve that same set.
Verified in a browser against preview-demo-loud without applying: 0 of 368
resolved with Apply disabled, then 368 of 368 with Apply enabled after Use
main for all. Main stayed at revn 28453 and the branch row did not move.
The sentence opened with every mutation and then listed three that are the
exception, which a reader has to reconcile. It now opens with the set that
does attach the three keys: the mutations through `audited` and a save routed
to a branch's op log.
The guide claimed the three keys for every branch operation. The three metadata
mutations and the three reads carry the generic RPC event alone, which records
who called what and no duration, so the section now says which is which.
The guide claimed the three audit keys for every branch operation through
`audited`, which was true of creation, compare, merge, update-from-main and
materialise. A save routed to a branch's op log attaches them on the
`update-file` event instead, because that is the props map the ordinary save
already replaces.
The preview's observability criterion is stated as every branch operation, and
two of them sat outside it. A save routed to a branch's op log carried only the
ordinary save props, and materialise carried none, so the two operations that
matter most to an enterprise trial were the two with no duration in the record.
Both attach `branch-operation`, `branch-outcome` and `branch-duration-ms` now:
`save-branch`/`saved` on the save path, `materialize`/`materialized` on the exit
door, including its idempotent no-op. The props travel in `::audit/replace-props`
on the save path because that is the key the ordinary save already replaces, so
the ordinary save's own props are untouched.
The test that asserts the audit props now covers a save and a materialise
alongside creation and merge. The three branch namespaces are 40 tests and 338
assertions, green.
A helper returning [:li ...] hands React a raw CLJS vector, because rumext
compiles hiccup at macro-expansion time; React then renders the leading
keyword as a child and the whole workspace tree falls into the error
boundary (React #31). Found by opening the dialog in a browser.
Every refusal the branching commands can emit now has a line a user can
read: the three size gates name the limit they hit with the number the
backend sent, the scope refusals (no branch of a branch, a released merge
base, a change kind an update cannot replay, a branch whose state moved)
each get their own message, and a Branching limits dialog reachable from
the panel footer states them before anybody meets one. The three numbers
come from ::get-branching-limits, so the copy keeps no second copy of a
limit it could drift from.
The pull-request listing and the file-open info query each recomputed the
same [ahead behind conflicts] the branches listing had already cached, so
a file with ten open pull requests paid the whole cold cost the branch
panel no longer pays. All three now go through cached-diff-counts, keyed
on the same (base-snapshot-id, source-revn, branch-revn) tuple, and none
of them realizes main when every row it needs is cached.
A no-op merge, a conflict refusal and an unsupported-kind refusal are
outcomes the first enterprise trial will ask about, and only the applied
path was carrying its duration.
Three gates, each a measured cost divided into a 30 s budget and halved:
50,000 shapes (a merge on 21,169 shapes costs about 6 s), 500 pages (a
whole-file comparison of 48 pages costs 402 ms), and 100,000 op-log
changes (one value-derived squash already emits about 19,000). Each
refusal is a :restriction naming the limit, the actual value and what to
do instead, and none of them truncates anything.
The listing never refuses: it renders the branch panel, and its cost is
bounded by the summary cache instead.
Every branch operation also attaches its duration and outcome to its own
audit event, because duration is the number the first enterprise trial
will be asked about and the generic RPC event does not carry it.
A 4-arity that delegates through the var counts as two calls to any test
that redefines it, which made a call-counting assertion measure dispatch
instead of comparisons.
The op log IS the delta between a branch and its merge base, so folding
validate/extract-affected-ids over it yields the exact set of pages the
branch can differ in. compute-merge takes :only-pages and bounds every
page pass to it, the presence pass included, which is where a whole-file
comparison spends its time.
The forward pass is bounded; the reverse one reports main's changes and
stays whole-file. The bound is dropped whenever the log holds a change
the reducer cannot scope to a page or a component (a deletion, page
order, a colour, a token), because for those the affected set is
incomplete and a bounded diff would hide work.
::materialize-file-branch persists a branch's derived state as the
file's own data payload, drops the op log, releases the pinned base
snapshot, closes any pull request over it and clears the branch marker,
so every read and write path takes the ordinary route from then on.
That is the preview's exit door: the flag can go off and the engine can
be replaced without stranding anybody with an unreadable file.
It keys on the branch file rather than the branch metadata, so calling
it twice is a reported no-op rather than an error.
The fetch the cache exists for is a file open, and expireAfterAccess of
ten minutes threw the entry away between two sessions on the same file:
a listing measured at 5 ms was 26 s again fifteen minutes later. An
entry is a uuid and five numbers, so the window costs nothing.
The listing's per-branch [ahead behind conflicts] summary is cached in
memory keyed by (base-snapshot-id, source-revn, branch-revn), which
names exactly the inputs the computation reads. A stale key cannot
exist, so the cache needs no invalidation logic: a repeat listing with
no intervening save performs no comparison work, and a cache miss is
the old behaviour. Failed computations are not stored, so a broken
branch keeps degrading to zeros with a warning.
The technical guide documented the full-copy branch model this change
replaces. Update the conceptual model, data model, creation steps,
ahead/behind computation, update-from-main (squash), and the limitations
section to describe the derived-state model.
Swap the storage and materialization layer of file branching to remove
the per-branch data duplication, keeping the RPC contracts and the
frontend unchanged.
- file_branch_change (0154): one row per branch save, the blob-encoded
change vector, ordered by revn; branch files store no data payload.
- Read path: app.binfile.common/get-file* derives the branch file data
by replaying the op log over the pinned merge-base snapshot, carrying
the base's version and migration set so need-migration? stays quiet.
- Save path: update-file routes branch files to persist-branch-file!,
which appends the vector to the op log and updates the file row
without any file_data write.
- Creation: create-file-branch no longer duplicates main's data; it
inserts an empty branch file row, grants the creator, and copies the
library rels.
- update-branch-from-main squashes the op log after integrating main's
changes: the repositioned base already carries main's side, so the
log is replaced with the net branch-only changes.
- Kept (keep-branch) branches retain the base pin: their state is
derived, so releasing the pin would leave them unreadable after GC.
Tests: backend-tests.rpc-file-branch-oplog-test asserts the storage and
derivation invariants; the existing branch, file, binfile, snapshot,
thumbnail, and pull request suites pass unchanged against the swapped
engine (84 tests, 1487 assertions, 0 failures).
This commit carries the design-branching preview by Juan de la Cruz:
elhombretecla/penpot:branching at 0ac265a3f5, which holds 45 commits
and 3 develop merges and was last synced with develop on 2026-09-16.
The branch is squashed together with a merge of origin/develop at
7c039231f7, so the tree is the merged result.
The merge resolved conflicts in six files:
- backend/src/app/migrations.clj: develop's 0155 and 0156 steps run
first, then the branching steps; the names differ.
- backend/src/app/rpc/commands/viewer.clj: both requires stay.
- common/src/app/common/flags.cljc: both flags stay.
- frontend/src/app/main/ui/dashboard/file_menu.cljs: "Create branch"
becomes a menu-item* with the same id, guard and handler.
- frontend/src/app/main/ui/dashboard/grid.cljs: the portal goes with
develop's menu rewrite; the branches popover stays in the trigger.
- frontend/translations/en.po: both sides' entries stay, with
develop's source references.
Co-authored-by: Juan de la Cruz <delacruzgarciajuan@gmail.com>
AI-assisted-by: mixed models
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
actions/create-github-app-token v3 deprecates the app-id input in
favour of client-id. Switch to the new TRIAGE_APP_CLIENT_ID secret.
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
Default is now no `!` for new functions. Existing `!`
stays as legacy until its gradual refactor.
Allowed cases live in clojure/idioms: mutating public
app.db API and functions working over transients. Any
further case must be registered there first.
AI-assisted-by: muse-spark-1.3-contributor
