3448 Commits

Author SHA1 Message Date
alonso.torres
26b2e30a97 ♻️ Move the placement step of a transform to common
Which shapes of a copy a transform leaves untouched (the ignore tree)
was frontend code, written twice: calculate-ignore-tree for the
modifiers of the SVG renderer and calculate-ignore-tree-wasm for the
matrices of the WASM renderer. The second is the single-pass version
of the first: each shape still compares itself with the copy root of
the latest transformed shape above it, as the repeated walks did when
the last one won.

app.common.logic.placement now holds it once: calculate-ignore-tree
takes the transformed ids in order and a function that transforms a
shape, built with modifiers-transform (SVG) or matrices-transform
(WASM), both cached per shape. check-delta and transform-attrs moved
with it. The frontend calls the common version, and its tests moved
to common-tests.logic.placement-test, which runs on the JVM and in JS,
with a rotation through modifiers added.

No behavior change: the golden masters and the suites are unchanged.
The benchmark does not run this code.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:16 +01:00
alonso.torres
591cdda9ed ♻️ Split app.common.logic.libraries by concern
The namespace held 3,300 lines: component sync, instance creation and
detach, swap and the variant switch's attribute carry, duplication,
and the sync of library colors and typographies. Each now has its own
namespace:

- app.common.logic.libraries (cll): component sync, reset, update
  main, attribute and placement sync, and the sync of a file with its
  libraries;
- app.common.logic.instances (cli): create, instantiate and restore a
  component, detach a copy;
- app.common.logic.swap (clsw): swap a copy head, carry overrides
  across a variant switch;
- app.common.logic.duplicate (cld): duplicate components and shapes;
- app.common.logic.library-assets (clla): sync of library colors and
  typographies, which the asset multimethods delegate to.

Each new namespace may require libraries, never the reverse; the
helpers they share became public there (attr-operations,
reposition-shape, orient-like, swap-content-root-id, ...). Callers use
the new aliases. The functions move as they were.

No behavior change: the golden masters and the suites are unchanged.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  37.3 / 1.06
  edit-through-a-chain-of-dependents   68.9 /  69.7 / 1.01
  library-update-into-copies           83.1 /  79.8 / 0.96
  main-edit-into-10-copies              1.8 /   1.6 / 0.92
  main-edit-into-100-copies            15.4 /  15.4 / 1.00
  main-edit-into-1000-copies          153.1 / 156.0 / 1.02
  variant-switch-into-copies           32.1 /  27.0 / 0.84

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:16 +01:00
alonso.torres
11f354a57e ♻️ Share one walk and one attribute loop in component sync
Sync and the variant switch each copied attributes with a loop of
their own (update-attrs, update-attrs-on-switch), and direct and
inverse sync each walked the shape tree with a function of their own.

- One attribute loop, attr-operations, builds the redo and undo :set
  operations. It takes a policy per attribute, sync-attr-operations or
  switch-attr-operations, which gives the [attr value] pairs to set:
  the attribute, after the derived ones it resets (position data
  before a text content, the transforms before a rotation). The two
  callers keep only what differs: which attributes they carry and how
  they end (the switch sets the copy's new touched groups).
  text-change-value and switch-text-change-value are one function,
  text-content-with-overrides; each caller passes the override groups
  it carries.
- One walk, sync-shape-recursive: sync the pair, pair the children
  with compare-children, recurse, finish. Two direction maps hold what
  differs: direct-sync (main to copy: a normal sync or a reset) and
  inverse-sync (copy to main, update main). Each gives the pair's
  sync, the four compare-children callbacks, the work after the
  children, and the container of the main shape. What stays the same
  along a walk travels in one context map instead of eleven arguments.

No behavior change: the golden master and the suites are unchanged.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  36.0 / 1.02
  edit-through-a-chain-of-dependents   68.9 /  66.4 / 0.96
  library-update-into-copies           83.1 /  74.9 / 0.90
  main-edit-into-10-copies              1.8 /   1.5 / 0.82
  main-edit-into-100-copies            15.4 /  14.7 / 0.95
  main-edit-into-1000-copies          153.1 / 144.8 / 0.95
  variant-switch-into-copies           32.1 /  26.7 / 0.83

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
9dd1b356ca ✨ Record a golden master of the component code
Before refactoring the component sync code, record the exact files the
component tests produce, so a refactor that changes any of them fails.

The JVM test common-tests.logic.golden-master-test runs every variant of
the composable cases the pure runner runs and every benchmark world. It
records the file after each user, undo and redo step (the runner's new
:observe option), the final files with the signatures of the undo/redo
round trip failures, and each world's file after its timed work. The
frontend interpreter records the store's files of every variant of its
cases and of the completion contract the same way.

app.common.test-helpers.golden-master writes a state as the normalized
files in sorted "path value" lines: ids become test labels, or numbers
in tree order when unlabelled; grid cells are named after their shape
and position; timestamps are left out. So a state is the same in every
run whatever ids it draws. Only a digest per state is committed; the
full lines go to tmp/golden-master/ and tmp/golden-master-frontend/
(recorded/, and actual/ for a state that differs), so diff -r shows a
change. PENPOT_GOLDEN_MASTER=record records them again.

Both sides repeat exactly across runs. Dropping :opacity from the
attributes sync copies fails the case C variants that edit it.

The interpreter's fixture also turns cts/wasm-enabled? off for each
composable test and restores it after, as it already restores the
store. A test that starts the WASM renderer left the flag on, so in a
full frontend run the cases ran on WASM shape proxies, which leave out
the attributes that are nil or false, and the frontend golden master
differed from a run of the cases alone (:flip-x and :flip-y).

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
7b9ddf5b19 ✨ Give a synced copy text variants of its own font families
Sync and the variant switch can take a copy text's family from one side
and its weight from the other. The variant id goes with the weight, so
it can name a variant of the other family: the WASM renderer then picks
the closest variant, but the SVG renderer falls back to the regular one.

The frontend now gives each text content that a sync or a component swap
commits variants of its own families: a run whose variant is not one of
its family's takes the family's closest to its weight and style
(cttx/update-font-variants, fonts/find-closest-variant). The pure runner
has no fonts, so case BA leaves variant ids out of its comparison.

The merge also leaves alone a run that already shows the placed style,
so a main edit no longer rewrites the variant ids of a copy whose
family is its own; with a new family, the run still takes the placed
variant id.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  35.7 / 1.01
  edit-through-a-chain-of-dependents   68.9 /  65.2 / 0.95
  library-update-into-copies           83.1 /  75.1 / 0.90
  main-edit-into-10-copies              1.8 /   1.5 / 0.86
  main-edit-into-100-copies            15.4 /  15.1 / 0.98
  main-edit-into-1000-copies          153.1 / 151.4 / 0.99
  variant-switch-into-copies           32.1 /  30.4 / 0.95

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
2488b0fc0f ✨ Migrate copy texts to the per-style text overrides
Files older than the feature components/text-overrides mark a copy text
with :text-content-attribute (any style) or :text-content-structure
(letters and styles). Migration 0037 compares each such copy with its
near main and gives it the groups its old flags allowed where it
differs; an old flag where nothing differs sets nothing. A copy whose
near main cannot be reached keeps every group its flags allowed, so it
loses no override. The migration adds the feature, so a client on the
old code cannot write the old groups into a migrated file.

Binfile import leaves the copies whose near main is in a library not
linked yet, as it does with swap slots, and converts them once the
libraries are linked: the deferred pass of the import now runs both
conversions (convert-deferred!, *defer-unreachable*).

Validation reports the old groups (:legacy-text-touched), and its
repair recomputes them against the near main; a text override group
without :content-group is still :invalid-text-touched. A shape with no
letters in it and its main no longer counts as differing in its styles.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  34.8 / 0.99
  edit-through-a-chain-of-dependents   68.9 /  65.0 / 0.94
  library-update-into-copies           83.1 /  86.9 / 1.05
  main-edit-into-10-copies              1.8 /   1.5 / 0.86
  main-edit-into-100-copies            15.4 /  15.0 / 0.97
  main-edit-into-1000-copies          153.1 / 150.6 / 0.98
  variant-switch-into-copies           32.1 /  26.8 / 0.84

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
5890b8d67d ✨ Give each style of a copy text its own override
A copy text had three override subgroups: its letters, any style, and
its structure (the number of paragraphs or runs). One style override
kept every other style of the main out (F72), letters and a style
together kept everything out (F73), a structure change kept the main's
styles out unless both texts had one style (F74), which the editors'
content never had (F79), and alignment, direction and vertical
alignment overrides were lost at the next sync (F76).

Now a copy text has :text-content-text for its letters and line breaks
and one group per style unit: family, weight (with style and variant),
size, line height, spacing, decoration, case, fills, typography,
alignment, direction and vertical alignment (cttx/style-units). An edit
sets the groups of what it changes: a letter both texts share that
changes its style, or a style that was not in the text before, so typed
letters touch no style. A token in the content sets the groups of the
styles it sets, and token sync of a text follows them.

Sync and the keep-touched variant switch share one merge
(cttx/merge-overrides): the letters of one side, and each unit from its
side placed on them. A unit with one value applies everywhere; else
letters both texts share (stretches of three or more) take the source's
style, and the rest follow the letter before them. The switch carries
an override only when the old and new mains agree on it (F77, F81), and
the copy then holds the groups its result differs in. A typography
reference stays only when every unit it sets comes from its side. The
variant id goes with the weight; where the family comes from the other
side, the frontend has to pick the family's closest variant.

app.common.types.text reads a content as letters, each with its run and
paragraph, and builds the content back from them; it gives the style of
one unit for a letter, compares styles by what they show, and finds the
letters two texts share with a letter diff (Myers). The component
contract memory describes the model.

Files older than this keep :text-content-attribute and
:text-content-structure, read as every group they may cover until the
migration recomputes them.

Tests whose expected values the model changes: text-sync-test (the
copy's weight, letters or new run no longer keep the main's size out;
the letters read whole), text-touched-test (its edits went to the
paragraph node, which shows nothing; they now edit the run),
variants-switch-test (the copies with two runs keep their letters
across a switch, and the size override the mains agree on), the
editor output table, case Y and the notation test (new groups). The 1j
pins of F72-F79 and F81 go; F79's direct test of equal-attrs? becomes a
sync of editor content. A new check holds the production override rule
to the 1j reference rule. New tests: the letter view, and a detached,
duplicated or reset copy text keeping or clearing its override groups.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  36.6 / 1.04
  edit-through-a-chain-of-dependents   68.9 /  67.2 / 0.98
  library-update-into-copies           83.1 /  72.5 / 0.87
  main-edit-into-10-copies              1.8 /   1.5 / 0.82
  main-edit-into-100-copies            15.4 /  14.9 / 0.97
  main-edit-into-1000-copies          153.1 / 153.4 / 1.00
  variant-switch-into-copies           32.1 /  27.1 / 0.85
The worlds hold no text. A main text edit synced into 100 copy texts
that changed their letters takes 40 ms, against 29 ms before: the merge
aligns the letters of each copy with the main's.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
a65a02bc02 🐛 Keep library color and typography syncs from touching copies
A library color sync wrote a copy's fills through update-shapes, and a
library typography sync wrote a copy text's content with a plain :set,
so both marked the copy as overridden (:fill-group, or :content-group
with :text-content-attribute). A later edit of that fill or text style
in the main then never reached the copy. Both syncs are derived writes:
they now write with ignore-touched.

The typography sync also merged the whole typography into each text node
that uses it, so its :modified-at (and :path) ended in the content of
the main and of every copy. It now writes only the text styles of the
typography.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  34.7 / 0.98
  edit-through-a-chain-of-dependents   68.9 /  67.6 / 0.98
  library-update-into-copies           83.1 /  76.0 / 0.91
  main-edit-into-10-copies              1.8 /   1.5 / 0.86
  main-edit-into-100-copies            15.4 /  15.2 / 0.99
  main-edit-into-1000-copies          153.1 / 157.1 / 1.03
  variant-switch-into-copies           32.1 /  27.0 / 0.84

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
e6940589f8 🐛 Count the shared copy heads in get-nesting-level-delta
get-nesting-level-delta counted the copy heads that a shape and its new
parent share with a two-argument predicate over a seq of pairs, so it
threw an arity error whenever both had copy heads. Its callers,
duplicate and the advance of copied objects, only pass parents outside
every copy today, so nobody reached it. The predicate now takes the
pair and compares the heads by id.

The advance of copied objects passed uuid/zero where the function
expects a shape; it worked only because the id of a uuid is nil. It now
passes the root shape.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
635e04b204 🐛 Sync a reordered copy child into its main on update main
In an inverse sync, when a copy child paired with a main child further
along the main's children, compare-children asked whether the copy
child was the main of the main child, with the arguments of
ctk/is-main-of? reversed. The answer was always no, so the pair took
the swapped callback, which does nothing in an inverse sync, and only
the move ran: the overrides of that copy child never reached the main.
The UI does not let a copy reorder its children, so it only showed
when the order differed for another reason, such as the undo of a
sync reorder.

The inverse branch now asks synced-pair? with the main child first,
as the direct branch does.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  35.5 / 1.01
  edit-through-a-chain-of-dependents   68.9 /  66.1 / 0.96
  library-update-into-copies           83.1 /  77.0 / 0.93
  main-edit-into-10-copies              1.8 /   1.6 / 0.92
  main-edit-into-100-copies            15.4 /  16.2 / 1.05
  main-edit-into-1000-copies          153.1 / 156.4 / 1.02
  variant-switch-into-copies           32.1 /  28.3 / 0.88

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
1249547413 ✨ Pin that a deleted component keeps passing changes on
When a component nested in others is deleted (inner in mid in outer,
mid deleted), a change to inner still reaches mid's stored shapes,
mid's copies, outer and outer's copies, and restoring mid brings back
an up-to-date main. Keep this behavior: outer is a live component, and
its users cannot see a deleted mid between it and inner.

Extend the deleted-component propagation test with a copy of the
deleted component and its restore, and reword the contract, which said
copies of a deleted component do not sync.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
07d03e30cd 🐛 Make a copy head that takes a position refer to its children
When a copy head took the place of a main shape and its component was
the one of that place, it stopped being a swap, but its children kept
referring to the main of its own component instead of to the children
of that place. Validation did not see it, and the next sync deleted
those children and added them again, losing their overrides, or the
head did not follow later changes to that place. It happened in:
- a variant switch that carries a swapped child into a variant whose
  child there has the same component;
- update main of a copy that swapped a nested copy back to the
  component a deeper main puts in that place, after the main in between
  swapped it for another one; the clone in the main also kept
  :swap-group;
- the repair of :ref-not-in-near-main, which also never cleared
  :swap-group.

The swap in a copy, the switch, update main and the repair now share
ctf/take-position: the head refers to the main shape of its position,
is swapped while its component differs from that shape's, and else its
descendants refer to the descendants of that shape.

The switch also read the swapped child from the objects of the swap,
where it was already gone with its old parent. It now reads it once it
is moved into the new copy, so its placement turns its children too,
and the undo takes it back to the root as the move to the root left it.
The swap finds the new shapes in its own objects, not in its changes.
The component memory says how a head takes a position.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  37.1 / 1.05
  edit-through-a-chain-of-dependents   68.9 /  70.5 / 1.02
  library-update-into-copies           83.1 /  77.3 / 0.93
  main-edit-into-10-copies              1.8 /   1.7 / 0.94
  main-edit-into-100-copies            15.4 /  16.0 / 1.04
  main-edit-into-1000-copies          153.1 / 160.6 / 1.05
  variant-switch-into-copies           32.1 /  28.7 / 0.89

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
f3e5dca1c0 ✨ Build library files with current features and migrations
The file builder behind @penpot/library pinned the features and
migrations it writes to the ones Penpot 2.7 accepted, so its files
could open there. It now writes the current defaults: the features a
new file gets and every available migration. Generated files were
already in the current format (paths, for one, are stored as path
data), so they now declare fdata/path-data and components/swap-group
as they should, and the import runs no migration on them.

Files built this way need the Penpot version the library is built with,
or a later one.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
402f4ac57d ✨ Turn a swapped copy head with the child it replaces
A swapped copy head followed the position, constraints and layout item
attributes of the main child it replaces, but not its rotation or
flips. When the main rotated or flipped that child, the copies that
kept it turned and the swapped ones did not. Sync, and a variant switch
that carries a swap, now give the head and its subtree the flips and
then the rotation of that child, around the head's own center, unless
the user changed the head's geometry. The swap itself uses the same
step.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  34.1 / 0.97
  edit-through-a-chain-of-dependents   68.9 /  75.5 / 1.10
  library-update-into-copies           83.1 /  81.3 / 0.98
  main-edit-into-10-copies              1.8 /   1.6 / 0.88
  main-edit-into-100-copies            15.4 /  15.6 / 1.01
  main-edit-into-1000-copies          153.1 / 153.2 / 1.00
  variant-switch-into-copies           32.1 /  31.4 / 0.98

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
c4006a3ad9 🐛 Keep the rotation and flips of a swapped shape
A swap placed the new copy at the position of the shape it replaces,
but not with its rotation or flips. When that shape was rotated or
flipped with its parent, the new copy came out straight. The new copy
and its subtree now take the flips and then the rotation of the shape
it replaces, around their own center, as geometry that comes from the
parent, not as an override. A variant switch goes through the same
swap.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  38.3 / 1.09
  edit-through-a-chain-of-dependents   68.9 /  70.7 / 1.03
  library-update-into-copies           83.1 /  82.2 / 0.99
  main-edit-into-10-copies              1.8 /   1.7 / 0.95
  main-edit-into-100-copies            15.4 /  16.7 / 1.08
  main-edit-into-1000-copies          153.1 / 167.4 / 1.09
  variant-switch-into-copies           32.1 /  29.6 / 0.92

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
f5e4cdcbd4 🐛 Keep a swapped child when a variant switch is undone
A variant switch with keep-touched moves a swapped child out of the old
copy to the page root, then into the new copy, which takes the same id
as the old one. The undo of the second move took the child back to
that id, so it landed in the new copy and went away with it when the
undo deleted the new copy. The undo now takes it back to the root,
and from there into the old copy.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
15add5038c ✨ Place a swapped child where the new variant has it
A variant switch with keep-touched carries a swapped child into the
new variant, but left it where it was in the old one. The next sync
then moved it to the place of the child it replaces, at a moment
unrelated to the switch. The switch now gives it that placement at
once, through the same step sync uses: position, constraints and
layout item attributes the user did not change.

Benchmark, against the same code without this change under the same
load (lower quantile, ms; without / with):
  deepest-edit-through-three-levels    36.0 /  36.0
  edit-through-a-chain-of-dependents   65.2 /  70.2
  library-update-into-copies           74.3 /  75.9
  main-edit-into-10-copies              1.6 /   1.5
  main-edit-into-100-copies            16.3 /  15.7
  main-edit-into-1000-copies          162.6 / 165.8
  variant-switch-into-copies           27.2 /  27.8

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
11ac2f8989 ✨ Migrate swap slots to override groups and remove them
The migration 0033 stores every swap of a file as an override group.
A swap made in a copy is paired with the child of the near main that
holds the same position, as sync paired it; any other slot is dropped.
All pairings are found on the data as it was, so swaps nested in swaps
pair against the content of the outer one, and libraries pair the same
whether they are migrated or not. A head that refers to its own slot
only inherited it, and drops it even when its near main cannot be
reached. Any other slot that matches no child, or whose near main is in
a library that is not loaded, keeps the slot as its ref, so validation
reports it. An import, which migrates its files before it links their
libraries, leaves those slots and converts them once the libraries are
linked. A migrated head that sits elsewhere than the child it replaces
would place it gets its geometry marked as touched, so the first sync
does not move it. Migrated and new files carry the feature
components/swap-group, so a client that still writes slots cannot open
them.

Validation checks the new form: :misplaced-slot reports any slot,
:misplaced-swap-group the group anywhere but on a nested copy head in a
copy, :ref-not-in-near-main a nested copy head whose ref is not a child
of the near main of its parent, and :duplicate-ref two children of a
copy that refer to the same main shape. Each has a repair.

Nothing but the migrations reads slots any more: the slot branches go
from sync, swap, switch, reset, detach, paste, duplicate and move. The
helpers that read and write a slot on a shape move to files/swap-slots,
and the ones that follow refs to files/comp-processors. The graph
projection drops the FillsSwapSlot edge, the components debugger shows
the component a swapped head takes its content from, and the plugin API
documents componentRefShape for a swapped copy. The swap tests are
named after what they check.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  34.6 / 0.98
  edit-through-a-chain-of-dependents   68.9 /  63.5 / 0.92
  library-update-into-copies           83.1 /  76.4 / 0.92
  main-edit-into-10-copies              1.8 /   1.6 / 0.87
  main-edit-into-100-copies            15.4 /  15.1 / 0.98
  main-edit-into-1000-copies          153.1 / 148.8 / 0.97
  variant-switch-into-copies           32.1 /  26.4 / 0.82

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
234fb7564a ✨ Store a swap inside a copy as an override group
A swapped copy sub-head now keeps the shape-ref of the main child it
replaces (its position), takes its content from its own component, and
carries the touched group :swap-group. It used to point at the root of
the new component and store a swap slot naming the replaced child.

Every path reads and writes that form:
- sync pairs a swapped head with the child it replaces and syncs it
  from its own component; it takes from the replaced child only its
  placement: position, constraints and layout item attributes;
- reset undoes a swap in place, keeping the head's id and the other
  overrides of the copy;
- detach, paste and duplicate make a head that leaves every copy level
  a plain copy of its own component, and advance a deeper one by
  position; the children of a swapped head never advance;
- a variant switch keeps a swap stored this way, and counts only the
  copy's own swap, not one made in a main of the chain;
- update main replaces the main child with a clone of the swapped one,
  and from a swapped head updates its own component;
- a swap inside a copy keeps the old shape-ref and adds :swap-group;
  swapping back to the component of that position gives a plain nested
  copy whose children refer to the position's children; a swap inside
  a main is a plain nested copy.
Files that hold swap slots keep working until a migration converts
them. The component memories describe the model.

This fixes what came from the slot: reset of a swapped child cleared
the other overrides of the copy (F12) and gave copies of a main new
ids (F66); update main left a swap mark (F16); a second variant switch
lost the overrides (F29); grid cells did not pair a swapped child
(F44); a swapped child did not follow its position (F65). F49 keeps
failing, in another form.

The tests that asserted slots now assert the override group; the tests
of the slot code build the old form with :as-slot?.

Benchmark (lower quantile, ms; baseline / now / ratio):
  deepest-edit-through-three-levels    35.2 /  35.4 / 1.01
  edit-through-a-chain-of-dependents   68.9 /  68.7 / 1.00
  library-update-into-copies           83.1 /  77.5 / 0.93
  main-edit-into-10-copies              1.8 /   1.6 / 0.92
  main-edit-into-100-copies            15.4 /  15.4 / 1.00
  main-edit-into-1000-copies          153.1 / 151.1 / 0.99
  variant-switch-into-copies           32.1 /  32.4 / 1.01

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
9acb64370c 🔥 Remove the components v1 leftovers
No components v1 file reaches the code any more: opening one fails the
feature check, and imports reject it or give it the feature. What only
served v1 goes:

- validate-file and validate-file-affected no longer skip a file
  without components/v2;
- the root of a deleted component is found by its main instance id
  only, in the type helper and the component sprite;
- generate-sync-library, which synced shapes inside live components,
  and the live components in containers-seq, so asset usages walk the
  pages only (deleted components keep their objects and still sync);
- the unused :components-v2 file option, from the schema, the file
  data and the test fixtures; stored files that carry it still
  validate;
- the team migration-in-progress check and its frontend alert, which
  leaves ephimeral features out of the team comparison;
- the components v2 introduction dialog, its styles and icons, and the
  :v2-info-shown profile prop;
- the graphic asset events and builders and the reads of graphics in
  the selection (the :add-media, :mod-media and :del-media changes stay,
  so stored changes still apply), and the uncalled render-page and
  render-components helpers.

AI-assisted-by: claude-opus-5-5
2026-11-10 20:21:15 +01:00
alonso.torres
256d6001a0 ✨ Test what copy texts keep through sync and switches
Add a battery of tests for text overrides in copies.
- A short notation for text content in tests (thtx): paragraphs of
  runs with short style names, read back with equal runs joined,
  edits in the notation, set-text and set-content through the change
  path, and the override rule for texts (override-units), which says
  which overrides a copy must hold against its main.
- Sync matrix: every whole-text copy edit against every main edit,
  plus the cards of the worked examples, the "Hello world" table, a
  token on a copy, typographies and a library color sync. Where the
  code differs from decisions (k) to (o), the result is pinned
  against F72 to F80.
- Switch matrix: copy edits against variants that differ in letters,
  size, color, a new line or a bolded word; F76, F77, F79 and F81
  pinned.
- Composable case BA: EditText also edits fills, font family and a
  whole text; the copy and the main change different styles, then
  reset or update main, in both runners; F72 pinned.
- Editor output: the content the WASM and v1 editors commit when a
  copy text is typed into, retyped, split with Enter and has a word
  bolded, recorded in the workspace and replayed on a copy. Typing
  sets only the text groups and a style change no text override;
  F79 pinned on both editors' content.
2026-11-10 20:21:15 +01:00
alonso.torres
62f99e0493 ✨ Make the component test suites fail when their claims break
A review found tests that would pass with what they test broken.
- Runners: both judge one variant at a time, fail a variant with no
  assertion and a known-failure mark that applies nowhere, and give
  the asserter the state the operations produced. The pure runner
  validates the final file and rejects operations of unknown kind.
  The frontend interpreter fails a step when the store reports an
  event error (Potok only logs it), reports asserter errors against
  their case, and stops the watchers that install! starts.
- Nodes drive the real code: a main root rename goes through
  end-rename-shape, and the clipboard's advance of nested copies is
  one common function, cll/advance-copied-objects, that copy-selected
  and both test paths call.
- Cases N, AH, AI, AK, AD, AE, K, C, H, S, AY, I, Y, AX and F check
  what they claim; O, R and T get real sizes.
- Stricter common unit tests found F60 (a repair detaches healthy
  copies), F61 (a switch drops a fixed-sizing size), F62 (the ref
  chain never stops at its target) and F63 (latent); all pinned.
- Frontend library tests: the RPC double records on subscribe, the
  dropped commits check that nothing syncs, and the link and share
  tests check the events sent.
- Plugin suite: OpSwap and OpDetach throw when the API call did
  nothing, and case AB checks that the reset head is an instance of
  the original component.
- The composable suite memory records the new rules.
2026-11-10 20:21:15 +01:00
alonso.torres
680b08dcaf ✨ Make composable interpreter steps wait for pending work
The composable interpreter treated a step as done after a 60 ms gap
with no commits, and treated its 2 s timeout as done too. Slow sync or
reflow work could finish after the step, so a case could assert on a
half-synced file.

Each event step now waits, as the plugin waitForLayoutUpdate does,
until every pending work tracked by the reflow registry has drained:
the watcher's sync wait, the sync it starts, and layout and text
reflow. A step that does not drain within 2 s fails the test and names
the work still pending, and the next variant starts with nothing
pending.

The tests cover an edit with a deliberately slow sync, undo and redo
steps that must wait for the watcher's touch-component, translation
and derived commits, a sync that never finishes, and a restart of the
edit watchers, which must keep the undo entries of an edit the same.

Also in this commit:

- Keep page-less changes from adding a nil page. Applying a commit locally
  rebuilds the object indices of every page its changes name. Changes that touch
  no page, such as :mod-component, name none, and their nil page id was rebuilt
  too: the file data in the store gained a page under a nil key holding
  {:objects nil}. Every component touch after an edit to a main added it, and
  undo never removed it.
- Check undo and redo of every step in the composable cases. The composable
  cases tested undo only where a case asked for it, so a change generator with a
  wrong inverse, or a step that splits into several undo entries, went
  unnoticed.
- Add operations and cases to the composable component suites. The composable
  runners could edit, add, remove and move children, swap and switch, but not
  run the other component operations.
- Test component validation, repair and library asset sync.
- Add a component sync benchmark with a recorded baseline. The coming refactors
  and fixes change sync code, and a slowdown must be seen and explained in the
  commit that causes it.
- Tidy the composable suite and make its cases stricter.
- Test the component sync paths no test reached. A coverage run found sync paths
  that no test ran.
- Test variant events, token sync and switch overrides. The variant events had
  almost no frontend tests, and several switch, paste, token and restore paths
  never ran.
- Test the file and container helpers and the library events. The component
  helpers of types/file and types/container and the library events of the
  workspace had the lowest coverage left.
2026-11-10 20:21:15 +01:00
alonso.torres
106572a2d6 ✨ Run the composable component cases on the JVM
The composable test engine, nodes, setups and cases lived in the
frontend test tree, so only the frontend interpreter could run them.

Move them to CLJC test helpers under app.common.test-helpers.composable
and add a pure runner (comp.runner) that does what the workspace does
around each operation, without the frontend store:
- after a user operation it syncs the changed components until none is
  left, as the frontend watcher does with one pass per sync commit;
- the operation and its syncs form one undo group, and the undo node
  reverts the latest one;
- it validates the file once the sync or the undo has settled.

Each node declares its kind: assembly, user, undo or frontend-only.
User nodes record the production changes they apply, and the runner
throws when a node breaks that. The swap and switch nodes build their
changes with the production clv/generate-component-swap-in-place.
Rotate and ChangeHeight need the frontend placement step, so the pure
runner rejects them.

The cases are zero-arg functions in comp.cases, shared by the frontend
interpreter and common-tests.logic.composable-sync-test. The latter
runs cases B to M and the new case O, a switch of an overridden copy,
on the JVM and in JS. Case N and the rotation case stay frontend-only.
2026-11-10 20:21:06 +01:00
alonso.torres
ef3e7f6f00 ♻️ Move variant switch and swap setup to common
The variant switch event picked its target variant, and the component
swap event worked out where the new head goes, inside the frontend
events. Code that runs without the frontend store, such as the pure
composable runner, could not do the same switch or swap. The swap and
the switch also checked for nesting loops with two different
functions.

Move them to app.common.logic.variants:
- find-switch-target picks the nearest variant with the requested
  value.
- generate-component-swap-in-place builds the swap changes keeping the
  old head's index, grid cell and swap-keep-attrs, plus the
  keep-touched step when asked.
- swap-nesting-loop? checks that a swap or switch does not nest a
  component inside itself. The swap menu, the plugin swapComponent and
  the variants-switch event all use it.

The frontend events behave as before, with two exceptions. Switching
to a value no other variant has does nothing, where it threw in
min-key. A swap also updates the layout of the shapes the keep-touched
step adds.
2026-11-10 20:21:06 +01:00
Andrey Antukh
671ff4a05f Merge remote-tracking branch 'origin/staging' into develop 2026-11-10 20:06:46 +01:00
Alonso Torres
4fb0ea9eee
🐛 Give unique values to same-name components combined as variants (#12881)
When components whose name matches the variant container are moved into
it, removing the container name leaves an empty value. Combining several
components named the same produced "Value 1" for the first variant and
empty values for the rest, so variants shared the same combination.

Assign the first unused "Value N" to the last property when all computed
values are blank, tracking values given within the same batch.

Combine moves the remaining mains into the container one at a time, in
the combine order, so the generated values follow that order instead of
the layer order.

Closes #12878
2026-11-10 09:16:08 +01:00
Luke Lau
6846afd08c
⚡ Send objects-map pages to the worker without a second encode (#12811)
* ⚡ Send objects-map pages to the worker without a second encode

When switching between pages on a large file e.g Material Design Icons, there's
some UI freeze from :index/initialize encoding object map shapes to send to the
worker. An objects map already stores the encoded JSON for each shape, and
`encode-str` was wrapping those strings in a second layer of JSON on the main
thread.

We can just transmit the already encoded JSON in a js/Map alongisde the
payload, similar to :blob, so structured clone just copies the strings verbaitm
without re-encoding. To do this this PR adds two functions to ObjectsMap that
converts to and from a js/Map.

The "Main components" page of the Material Design Icons has ~30k objects, so
this reduces the freeze when switching pages from 306ms -> 189ms (1.63x) on a
release build.

AI-assisted-by: claude-sonnet-5-5

* Split property tests out into new deftest

---------

Signed-off-by: Luke Lau <luke_lau@icloud.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:35:20 +01:00
Luke Lau
55360c52f5
✨ Cache the transit reader when decoding strings (#12704)
* ✨ Cache the transit reader when decoding strings

On the cljs side, we create a new transit reader on every call to decode-str.
This turns out to have a fair bit overhead in both transit-cljs and transit-js,
since it has to merge the handlers, call clj->js, allocate a new ReadCache,
etc.

By reusing it the measured decode time goes from ~58us -> ~41us, so about 1.4x
faster. This particularly helps for some operations like whenever a component
is edited and every object needs to be decoded, see
https://github.com/penpot/penpot/issues/9344, although they could probably be
made more efficient in other ways.

The reader usually clears its read cache after a successful read, but whenever
it throws it hangs around. So in this case we need to reset the cache
ourselves, otherwise the next read will be from a clobbered cache.

The tests were written by Claude Sonnet 5.5, and should cover the clobbering
behaviour above.

Signed-off-by: Luke Lau <luke@igalia.com>

* Keep one transit reader per type and rebuild it on add-handlers!

---------

Signed-off-by: Luke Lau <luke@igalia.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-11-09 17:34:24 +01:00
penpot-renovate[bot]
75b3da5290
⬆️ Update dependency pnpm to v12.18.0 (#12876)
- pnpm 12.18.0 (docker/images/Dockerfile.media-processor)
- pnpm 12.18.0 (docker/images/Dockerfile.mcp)
- pnpm 12.18.0 (docker/images/Dockerfile.exporter)
- pnpm 12.18.0 (docker/devenv/Dockerfile)
- pnpm 12.18.0 (render-wasm/package.json)
- pnpm 12.18.0 (plugins/package.json)
- pnpm 12.18.0 (plugins/libs/plugins-styles/package.json)
- pnpm 12.18.0 (plugins/libs/plugins-runtime/package.json)
- pnpm 12.18.0 (plugins/libs/plugin-types/package.json)
- pnpm 12.18.0 (plugins/apps/table-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/rename-layers-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/poc-tokens-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/poc-state-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/plugin-api-test-suite/package.json)
- pnpm 12.18.0 (plugins/apps/lorem-ipsum-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/icons-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/example-styles/package.json)
- pnpm 12.18.0 (plugins/apps/e2e/package.json)
- pnpm 12.18.0 (plugins/apps/create-palette-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/contrast-plugin/package.json)
- pnpm 12.18.0 (plugins/apps/composable-test-suite/package.json)
- pnpm 12.18.0 (plugins/apps/colors-to-tokens-plugin/package.json)
- pnpm 12.18.0 (package.json)
- pnpm 12.18.0 (media-processor/package.json)
- pnpm 12.18.0 (mcp/packages/server/package.json)
- pnpm 12.18.0 (mcp/packages/plugin/package.json)
- pnpm 12.18.0 (mcp/packages/common/package.json)
- pnpm 12.18.0 (mcp/packages/browser-extension/package.json)
- pnpm 12.18.0 (mcp/package.json)
- pnpm 12.18.0 (library/package.json)
- pnpm 12.18.0 (frontend/text-editor/package.json)
- pnpm 12.18.0 (frontend/packages/ui/package.json)
- pnpm 12.18.0 (frontend/packages/mousetrap/package.json)
- pnpm 12.18.0 (frontend/packages/draft-js/package.json)
- pnpm 12.18.0 (frontend/package.json)
- pnpm 12.18.0 (exporter/package.json)
- pnpm 12.18.0 (docs/package.json)
- pnpm 12.18.0 (common/package.json)
- pnpm 12.18.0 (backend/package.json)

Signed-off-by: penpot-renovate[bot] <338182062+penpot-renovate[bot]@users.noreply.github.com>
2026-11-07 22:52:43 +01:00
Elena Torró
7ecf48c52c
⚡ Speed up uploading page shapes to wasm (#12854)
* ⚡ Compute text fonts once per shape when uploading to wasm

* ⚡ Write the shapes batch in place and coerce fills once

* 🐛 Free the upload buffer when a batch record write fails
2026-11-06 15:45:34 +01:00
Andrey Antukh
a77cb2608e
🐛 Normalize stored SVG attribute keys to kebab-case (#12006)
* 🐛 Normalize stored SVG attribute keys to kebab-case

The v3 reader already emits kebab-case keys, but birth stored the
camelCase React prop names, so import and export never converged:
fill rules painted wrongly and filter defs lost their blur.

Birth now normalizes every stored attribute key to kebab-case
through the `svg-props` whitelist (`attrs->kebab-props`), applied on
all element paths of the svg importer, on the file builder and on
defs. svg-raw `:content` is kebab-ized in spelling only and keeps
unknown keys, because the sidebar reads them. Non-spec attrs Penpot
must keep live in `penpot-extra-attrs`, seeded with `:stroke-style`.
Render boundaries translate back to camelCase for React and for the
wasm props, and the shape schema declares the closed kebab contract
derived from that same whitelist, so birth and validation cannot
drift.

Migration `0035-fix-shape-svg-attrs-and-defs` converts old files
once with the birth transforms, and carries back the svg-viewbox
rect normalization the withdrawn `0021` had. That old id leaves
`available-migrations`, so files recording it migrate again through
this one, and the binfile cleaner drops its own svg repair.

Review fixes included here: `setup-opacity` and the blend mode read
the stored keys, gradient stops accept `:style` as a CSS string or a
map, `inherit-attributes` reads `:style` so a child's inline style
beats the inherited group attr per CSS, and the svg-attrs menu shows
spec spelling (`viewBox`, `class`) through `stored-attr-display-name`.

Tests cover the migration, the closed svg-attr schemas and the
import/export round trip; the svg-content schema test pins its
sample seed, which dealt no svg-raw shape on unlucky runs.

Closes #11947

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Keep legacy svg node tags and transforms valid in migration 0035

The closed svg node schemas require keyword tags and map attrs, but
stored files hold def and svg-raw content tags as strings, and some
node `:attrs` as nil. Migration 0035 left them as they were, so the
file data check that runs after migrating rejected those files and
they could not be opened. `normalize-node-tree` now turns string tags
into keywords and nil attrs into an empty map, at birth and in 0035.

With keyword tags, the svg defs renderer applies the shape transform
to user-space defs. Legacy `:svg-transform` values are plain maps,
which multiply into NaN matrices. `gmt/coerce` turns them into
matrices, in 0035 and in the binfile cleaner, because every v3 import
decodes the transform as a plain map again. Legacy masks and gradients
now render as designed in the svg renderer.

The binfile test helper calls `export-files!` with the current config
keys.

AI-assisted-by: claude-opus-5-5

---------
2026-11-06 12:52:53 +01:00
María Valderrama
2df4d10bd5
✨ Enforce attribute-based organization permissions (#12868)
* ✨ Enforce attribute-based organization permissions

* 📎 Code review
2026-11-06 11:40:50 +01:00
Elena Torró
7fa225115e
⚡ Reuse uuid instances when decoding transit on the frontend (#12792) 2026-11-06 11:33:37 +01:00
Andrés Moya
018b857de1
🐛 Improve sync of layout item attributes in components (#12851)
* 🐛 Preserve layout item touched groups when switching variant

* 🔧 Add systematic tests to check layout items component sync

* 🔧 Add small updates after comments

* 🔧 Add tests for partial overrides
2026-11-06 09:51:12 +01:00
alonso.torres
eadd1cce70 🐛 Keep a copy linked when the repair finds its component's main
A component whose :main-instance-id points to a missing shape fails
validation on its copies with :ref-shape-not-found. A repair scoped to
the copy's page never sees the main's own error, so it detached the
copy, even when the main was still on the component's page.

The repair of :ref-shape-not-found now looks for the component's main
on its page first and points :main-instance-id back to it, which keeps
the copy linked. A copy is still detached when no main is found.
2026-11-05 23:59:32 +01:00
alonso.torres
2f3b928a60 ⚡ Check the schema of only the pages a save touches
update-file checked the schema of the whole file on every save, which
loads and decodes every page even when the changes touch one. On files
with several pages that was most of the time of a save.

validate-file-schema-affected! checks the file data with the pages index
reduced to the pages the changes modify; the data outside pages is still
checked whole, and a change type it does not know makes it check the
whole file. update-file uses it for the strict and the soft schema check.

A save of 10 commits on a file of 10 pages of 2k shapes goes from ~1.1 s
to ~240 ms; a single page file costs the same.
2026-11-05 23:59:32 +01:00
alonso.torres
4f42ab1d2d ✨ Keep the valid commits of a save that fails validation
A save whose changes failed validation was rejected as a whole: autosave
stopped, the edits made after it piled up unsent, and a reload lost
them all, the valid ones included.

The frontend now sends each commit of a save as its own group with its
commit id (:changes-with-metadata). When applying and validating all of
them at once fails, the backend searches by bisection for the first
commit that fails, keeps the commits before it and answers with
:rejected (that commit and every later one). Only the kept changes are
stored, snapshotted and broadcast, so other users never see the rejected
ones. Flat :changes and groups without ids stay all-or-nothing; database
and network errors still fail the whole request. A rejection without an
error type is reported as an internal error.

The frontend rolls the file back to what the server holds: it undoes
every pending commit of that file, applies again the remote changes that
arrived meanwhile, and redoes the kept commits. Undoing only the
rejected commits would write stale values over a peer's later change to
the same attribute. The rejected and later commits leave the queue, the
buffer and the undo history, and a warning that hides after ten seconds
tells the user. The pure bookkeeping lives in
app.main.data.persistence.commits.

The pending notice of a commit the rollback dropped no longer leaves the
save status at pending, which made "Pin this version" hang and fail.

Debug tools drive these cases by hand: debug.commitInvalidChange() makes a
save fail validation, holdSaves() delays the answers, failNextSave() and
failNextRollback() fail them on purpose (backend flag debug-save-faults),
and persistenceState() shows the queue.
2026-11-05 23:59:32 +01:00
Andrey Antukh
97fe51e08b Merge remote-tracking branch 'origin/staging' into develop 2026-11-05 22:43:31 +01:00
Elena Torró
afc40a3309
🐛 Refresh wasm heap views after allocations and wasm calls (#12833)
Growing the heap detaches typed views taken before an alloc or a call.
Take them after, so reads and writes never hit a detached buffer.
2026-11-05 18:09:59 +01:00
Belén Albeza
fa1b3d4708
⚡ Keep only the shape data the worker reads (#12840)
* ⚡ Keep only the shape data the worker reads

The worker kept a full copy of the current page for its selection and
snap indexes. It now keeps only the attributes the indexes and the
change replay read, and re-indexes only the shapes that changed.

On a 52k-shape page the worker heap goes from 698 MB to 286 MB.

Closes #12740

AI-assisted-by: claude-opus-5-5

* ♻️ Share the worker re-trim with its replay test

The worker and its replay test now share `retrim-objects`.

AI-assisted-by: claude-opus-5-5
2026-11-05 16:25:29 +01:00
Pablo Alba
7666d7d500 🐛 Fix Creating or combining a variant set renames the component first 2026-11-04 21:44:44 +01:00
Pablo Alba
eeda9c1d18 🐛 Fix renaming a variant set saves half-finished steps 2026-11-04 21:44:44 +01:00
Pablo Alba
69fa682217 🐛 Keep variant sets intact under group, mask, board, and layout 2026-11-04 16:45:23 +01:00
Luke Lau
2573908315
⚡ Drop unused used-names sets from component duplication (#12809)
unames or update-unames! are no longer used after a1a3d09998, and each call to
cfh/get-used-names is a scan across all objects
2026-11-04 13:00:24 +01:00
David Barragán Merino
0e9a9a8950
💥 Enable admin console by default (#12475)
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
2026-11-03 23:34:16 +01:00
Elena Torró
51cbae6ea5
⚡ Improve memory consumption on shape decoding (#12790)
* ⚡ Avoid decoding the opened page twice for its frame index

The frame index was built over a with-meta copy, so pages got decoded again.
Read it from the page objects themselves and fix the with-meta identity check.

* ⚡ Intern keywords when decoding transit on the frontend

Each shape is decoded on its own, so every shape had its own key keywords.
Cache them in the read handler so all decoded data shares one instance.

* ⚡ Use an array map as the wasm shape proxy delegate

A hash map with ~35 keys costs ~3 KB of nodes per shape on large pages.
An array map holds the same entries in one array, at a small lookup cost.
2026-11-03 17:41:36 +01:00
Andrey Antukh
be37341f36 Merge remote-tracking branch 'origin/staging' into develop 2026-11-03 15:07:18 +01:00
Alonso Torres
410bde8103
🐛 Move displaced path content back onto its box (#12754)
Some files have component copies whose path drawing (:content) sits away
from its box (:selrect and :points). The renderer draws the content, while
the sidebar, layouts and syncs use the box, so these icons render outside
their boards and are clipped or drawn over other shapes. Moving the shape
keeps the gap, and sync never repairs it because both groups are touched.

Add the migration 0034-fix-path-content-offset. For every path in pages
and in component objects, deleted components included, it derives the box
from the content as update-geometry does and, when the sizes agree but the
position does not, moves the content onto the stored box. The box, the
transform and the touched groups stay as they are; paths whose drawing has
another size are left untouched.

Closes #12752
2026-11-03 12:12:08 +01:00
Andrey Antukh
93d36d04d8
✨ Stabilize fragmented file data across snapshots and GC (#12420)
* ✨ Stabilize fragmented file data across snapshots and GC

Snapshots, automatic and user, now store pointer-based rows that share
fragments with the live file instead of duplicating page content.
`get-file-snapshot` resolves those pointers on the server so version
previews keep rendering plain data, and `restore!` reuses the shared
rows. Export still materializes (the package carries no fragments) and
import regenerates them.

`file-gc` treats the main file and every unpurged snapshot as liveness
roots, reads the file unrealized, and reattaches the original pointer
wherever the cleaned value is unchanged, so a pass over an untouched
file writes no fragments and a pass over an edited file keeps the ids
of the pages it did not touch. Snapshot creation and the restore
backup take the file row `FOR SHARE`, which serializes them with the
pass and closes the race that let a new snapshot reference fragments
the pass was about to mark.

All backend pointer bindings go through the `fdata/with-pointers`,
`with-tracked`, `with-tracked-pointers` and `with-untracked` macros.
The loader reads fragments a previous pass marked until `objects-gc`
purges them, and the frontend fails loudly with retry instead of
dropping an unresolved fragment.

Review fixes in this branch: the automatic snapshot attaches before
pending migrations, `realize-pointers` is the single verb,
`reattach-unchanged` takes the identity fast path, the restore guard
takes its revn from the backup read, and `PointerMap.withMeta`
generates a fresh id when metadata changes.

Tests cover both snapshot writers (red without the lock), snapshot
sharing and pin release through the real delete and purge paths,
reattach against orphan marking, preview after offload, the pointer
binding contracts and the with-meta id change.

Closes #12410

AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: deepseek-v4.1-flash

♻️ Pin no-churn behavior for untouched pages

Pin the behavior the churn verification measured: a save keeps the
fragment id of every page it does not touch, with a page of eleven keys
(PersistentHashMap), a 46-shape objects map and a nine-page index (hash
map). Also pin the mixed case: a data-level objects-map flip rewrites
the old pointer pages once and the next save keeps the new ids.

The memory records the identity rule behind it: re-assoc with the same
value returns the same map in array and hash maps alike.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

⚡ Stop loading untouched pages when enabling objects-map

`enable-objects-map` wrapped every page on every save, and reading a
page's objects forced its PointerMap to load the fragment. A save of
one page in a seven-page file therefore made seven fragment reads.

Skip unloaded pointers: their fragment was already wrapped by the
encode-file that wrote it, so loading it only to wrap it again is a
no-op. The save now loads only the pages it edits. A page with a plain
fragment (a data-level feature flip) keeps its plain fragment until it
is first loaded or edited; a migration that enables the feature must
rewrite the data itself.

Cover the change with load-count tests for save and restore, update the
flip test to the new contract, and drop the stale restore comment.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

⚡ Keep page fragment ids through pending migrations

Applying a pending migration on read realized the file, ran the
migration and persisted it again; the persist wrapped every plain page
in a new PointerMap, so every page got a new fragment id even when the
migration did not change it. That leaves the fragment cache cold and
rewrites every fragment on the first read after a deploy.

Capture the original pointers before realizing, reattach the unchanged
positions after migrating (same value comparison file-gc already uses),
persist the reattached file and return the realized one: the read
contract is plain data. Read-only connections skip the persist and the
reattach. A migration that adds objects-map skips the reattach so the
pages get wrapped.

Move the three helpers to fdata so file-gc and the migration path share
them, and cover the path with a real 0031 migration test: the unchanged
page keeps its id, the changed page gets a new one, the caller gets
realized data and a second read does not migrate again.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Realize library pointers with each library's own loader

Fragment rows are keyed by file id, and every caller binds the
fragment loader of the file it works on. get-resolved-file-libraries
returned libraries with their pages still as pointers, so the first
read of a library page under fdata/pointer-map asked the consuming
file for the fragment and failed with :fragment-not-found. That
broke migrations and file validation for every file using a
pointer-mapped library.

The weak-map loader now realizes each library's pointers with the
library's own id. The test fails without the change.

AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>

* 🐛 Restore the pointer-map require the repair path still needs

The branch dropped `app.util.pointer-map` from the namespace while the
auto-repair path (added to develop before this branch) still rebinds
`pmap/*tracked*` around validation and repair in
`repair-preexisting-damage` and `process-changes-and-validate`. The
namespace did not compile, so the whole backend failed to load.

Those toggles do not fit the fdata macros: `with-untracked` binds nil
and `with-tracked` creates a fresh atom, while the repair path has to
turn tracking off and restore the existing atom (and read it before the
repair starts). Restore the require and keep the bindings explicit.

Also pin the feature-off path with a test: with `fdata/pointer-map` off
(the default), a save creates no fragments, file-gc writes nothing and
the snapshot preview still returns plain data. Drop a stray `\n` char
literal left at the end of one deftest.

Update the memories with the exception to the macro rule, the snapshot
FOR SHARE locking and the pre-migration automatic snapshot.

Closes #12410

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Validate debug files with their own pointer loader

The `/dbg/actions` file validation passed the file id where
`get-resolved-file-libraries` expects the file map, so the library map
was seeded with a nil key. It also validated a file read without
realizing it, so the first page lookup of a pointer-mapped file raised
"load is not supported when *load-fn* is not bind" and the endpoint
could not validate exactly the files the feature touches.

Read the file realized (validation walks every page and component
anyway) and build the library map from the file. The library loader
already resolves each library's pointers with its own id, so the
realized file and its libraries validate as plain data. Covered by a
handler-level test that fails before the change.

AI-assisted-by: deepseek-v4.1-flash

* ♻️ Bind the snapshot loader through the fdata helper

`reduce-snapshots` hand-rolled the same `binding` that
`fdata/with-pointers` builds (it also loaded the function's own
pointer loader before entering the reduction). Use the macro so the
rule holds there too and the namespace no longer needs
`app.util.pointer-map`; the loader is identical (same cfg and file-id)
and snapshot walking behaves the same.

AI-assisted-by: deepseek-v4.1-flash

* 📚 Note the pointer-binding exceptions and snapshot ordering

Record what the feature review settled:

- Two direct `binding`s stay on purpose: the fine-grained `*tracked*`
  toggles of `app.rpc.commands.files-update` (validation/repair turn
  tracking off and restore the existing atom) and
  `binfile.common/encode-file` (fresh tracked set inheriting the
  current one). Atom-scoped macros would only be worth it if the set
  grows.
- The automatic snapshot is taken before the pending migration, and
  the pre-migration window is narrow: the `get-file` RPC migrates on
  read and its ETag hashes the available migrations, so a post-deploy
  read cannot be a 304 from the pre-deploy state. The old-format
  snapshot only appears when the save is the first request to touch
  the file after the deploy, and `get-file-snapshot` previews it as
  stored.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Apply pending migrations when previewing a version

Automatic snapshots are attached before the save applies pending file
migrations, so a version can store the old format. The preview RPC
handed that data to the UI as stored, and the viewport renders it with
the current code.

Realize the snapshot with the live file id (fragments are keyed by
file, not by snapshot), normalize its migrations to a set (the decoder
keeps the pgarray as a vector, and `set/difference` would compare
indexes) and apply the pending migrations with the pure
`fmg/migrate-file` and the live libraries. Nothing is persisted: the
stored version keeps its data and restoring it re-migrates on the next
read. The response now carries the snapshot's migrations.

Use `:features` the way `get-file` does for `fdata/objects-map`: a
client that does not declare it gets plain objects. The historical
feature set is not checked against the enabled one on purpose: a
version may predate a feature change.

Cover the preview of a snapshot with a pending migration and the
objects realization; both pin that the stored row is untouched. Update
the file-data memory with the preview contract and the vector/set
trap.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Decode snapshot migrations as an ordered set

`file-snapshots/decode-snapshot` handed the `file_change.migrations`
pgarray as a vector. `set/difference` falls back to index lookups when
the first collection is shorter than the second, so the pending set was
right only by accident: a stored subset happens to work, while a
stored set larger than `available-migrations` (a snapshot taken by a
newer build) marks every available migration as pending and re-applies
it to already migrated data.

Decode it as a `d/ordered-set`, the type the rest of the code and
`schema:file` expect. This also covers `restore!` and the snapshot
list, and it lets `get-file-snapshot` drop the `(into #{} ...)`
normalization it was carrying. Re-wrap the union in `fmg/migrate` so
the invariant holds even when a caller hands a plain set.

A snapshot with NULL `migrations` predates SQL migration 0140 and
keeps the format version of its time. Pin that legacy path: the
preview regenerates the applied set from that version, migrates in
memory and leaves the stored row untouched. Update the file-data
memory with the invariant and the legacy path.

AI-assisted-by: deepseek-v4.1-flash

* 🐛 Fix flaky waiting-gauge assertion in climit test

The throwable-while-waiting test asserted the exact list of
waiting-gauge publications, but the component sampler republishes
the same count when its first tick lands inside the recorded
window under load. Assert only the last value, which is the
contract the helper documents.

AI-assisted-by: muse-spark-1.3

---------

Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Co-authored-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
2026-11-02 15:02:55 +01:00