While saves fail the workspace turns read-only, but only some tools
checked it. Edits from the asset, color and token panels, and a drag or
a drawing in progress when the lock came, still made commits that
queued behind the failed save.
commit-changes refuses commits while the save-failure lock holds,
except those that are not user edits: the undo of rolled-back commits,
text position data and component touches. Undo and redo do nothing
while locked. When the lock comes, moves, resizes, rotations, drawings
and the drags of flex, guide and grid handles in progress are cancelled
and commit nothing, and the design panel shows the Inspect view.
Reloading the open file (a peer's restore, the fallback of a rollback)
runs initialize-workspace again without leaving the workspace. The
streams the first run started kept running next to the new ones, so a
commit added one undo entry per reload and each notification was
handled twice. The old subscription also sent its unsubscribe after the
new subscribe, which ended the new one on the server.
A new initialize-workspace, text reflow, shape layout or notification
subscription stops the one before it, and a subscription replaced by
one to the same file sends no unsubscribe.
The MCP plugin sends the rollbacks it saw with its next task response
and then forgets them. When that response came after the server had
given up on the task, the server dropped it and the agent never heard
about the rollback. A reload of the tab or of the plugin lost them too.
The server now sends the rollbacks of such a late response back to the
plugin, which reports them again with its next response. The plugin
keeps the rollbacks not reported yet in its local storage, one entry
per file, so a reload does not lose them. Both MCP plugin manifests ask
for the allow:localstorage permission for that.
Plugin writes failed whenever the workspace read-only flag was set. The
Inspect tab sets it too, although it only hides the editing tools, so an
MCP agent could not edit while the user was inspecting. A user who
became a viewer had the flag cleared, so plugin writes did not throw and
silently did nothing.
Writes now fail on what makes them unsafe, not on the flag: failing
saves, a user who cannot edit, a version preview or a lost render
context. The error says which one, and the MCP instructions tell the
agent which of them pass by waiting.
The backend rejects a save after the call that made the changes has
returned, and the workspace then undoes them. A plugin, or the MCP
agent, went on working with shapes that were no longer there and never
learned why.
The rollback event now names the files that lost commits and the pages
and page shapes their changes touched, and plugins get it as the new
`contentrollback` event (`ContentRollback` type) through a subscription
to the store stream, so no rollback is dropped by debouncing.
The MCP plugin keeps the rollbacks it is told about and sends them with
its next task response, failed or not. The server collects them per tool
call and puts a warning first in that tool's response, naming what was
affected; the instructions tell the agent to read the pages again and
redo what is missing.
A read-only workspace (saves failing, a user who cannot edit, a version
preview) only hid the editing tools: plugins, the MCP among them, still
changed shapes, and those edits were never saved. Every plugin write now
throws a read-only error, whatever the throwValidationErrors flag, the
same way a write to a removed shape does.
`check-permission` with a write permission runs the new `check-writable`,
which covers most writers. Writers without a permission check (most
context methods such as create*, group, align or flatten, the z-order
methods, applyTypography on a text range, removing a ruler guide) call it
themselves, and shape setters get it through `check-shape-writable`.
Reads, comments and undo blocks keep working, so a block opened before a
lock still closes.
The MCP instructions tell the agent what the read-only error means.
Closing the v2 text editor with Escape could crash the workspace. The
text options panel renders once more with the editor it saw last, after
closing has disposed it, and reading the style of a disposed editor
throws.
The panel now takes the values from the shape when the editor is gone.
- Ending a drag inside the frame the shapes already belong to sent an
empty reparent save and bumped the revision. The move commits only
when some shape changes parent.
- When a user restored a version, other sessions reloaded the file with
no notice. They now see a dialog naming who restored which version
(by its name, else its date), saying that later changes are no longer
in the file and, when they had changes not saved yet, that those were
lost.
- Exports drawn in the browser (PDF, SVG, and PNG when the WASM renderer
draws it) and duplicating a file from the dashboard started at once,
so an export could show changes a pending save then rejected, and a
copy could miss queued edits. Both wait for the file's pending saves.
A client export warns that it includes unsaved changes; a copy warns
that it misses them.
A rollback can remove or change shapes the UI still refers to.
- The selection drops shapes that are gone and the color picker that
edited them closes.
- The undo history is cut at the reverted action; it stays when no
entry holds a reverted change. Undo and redo commits carry real undo
changes and the history index they moved from, so a reverted Ctrl+Z
puts the change and the index back.
- An open path edition whose shape the rollback changed closes without
saving the editor's stale copy. The v2 text editor loads the reverted
text again; the WASM editor already showed it.
- The Alt measurements look the hovered shape up in the current data.
- Plugin setters, resize() and rotate() on a removed shape throw
"Shape not found", whatever the plugin's throwValidationErrors flag.
- A save that failed on a short platform failure (a lost or refused
database connection, lack of resources, a server shutting down, a
serialization failure or deadlock, a socket or HTTP timeout, a host
that did not resolve, an interruption) answered 500, which stops
autosave. It now answers 503, which the frontend retries. Other I/O
and database errors keep answering 500, timeouts 504.
- A save answered not-found no longer replaces the workspace with the
access page. It is a terminal save error: the workspace stays open
with the failed save notice, report and reload link. When edit rights
are lost, the notice says how many changes could not be saved.
- A library sync marked the library synced three seconds later,
whatever happened to its save. A rejected sync was undone but stayed
marked: copies stayed out of date and the "updates available" notice
never came back. The mark now waits for the save, a rollback of the
sync brings the notice back, and a failed mark (e.g. the user just
became a viewer) is logged instead of leaving the workspace.
- While saves fail the workspace is read-only. The view-mode banner
says editing is disabled because changes can't be saved, with the
Inspect banner's size and bold lead and no Done button. Users can
still switch pages, pan, zoom and inspect. The lock lifts when a save
lands and brings back the mode the user had or picked meanwhile; the
WebGL context lock keeps the workspace read-only if it lasts longer.
The lock lives in app.main.data.persistence.lock.
- A viewer is never locked. A role change that leaves the user a viewer
keeps the edits that lost permission, queued and on screen, and shows
the count notice again. The role change that brings edit rights back
releases them and rolls them back on screen, as a rejected save would.
Any other terminal failure is released by the next role change or file
opened.
The debug faults `io` (socket timeout, 503) and `eof` (500) cover both
sides of the I/O rule.
A rollback reverts whole undo groups, across files and saves, so that
no user action is left half done. Some actions still split:
- An action that commits several times inside one undo transaction
(e.g. applying a token to several shapes) gave each commit its own
group, so a rejection could revert part of it: shapes ended up linked
to a token whose value was never applied. An undo transaction now
opens with a commit group that every commit made inside it takes.
- The commit that undoes saved commits of a rejected action joined an
open transaction's group, so a later rejection there reverted the
undo too. It gets a group of its own.
- Emptying the open transaction during a rollback dropped its group, so
its later commits were no longer one unit. It keeps its group.
- Waiting for a save ended as soon as the queue emptied, while later
commits could still wait in the buffer. Waiters now also wait for an
empty buffer.
A test covers a token action of three commits in one save whose third
commit is rejected.
A save whose changes failed validation was rejected as a whole: autosave
stopped, the edits made after it piled up unsent, and a reload lost
them all, the valid ones included.
The frontend now sends each commit of a save as its own group with its
commit id (:changes-with-metadata). When applying and validating all of
them at once fails, the backend searches by bisection for the first
commit that fails, keeps the commits before it and answers with
:rejected (that commit and every later one). Only the kept changes are
stored, snapshotted and broadcast, so other users never see the rejected
ones. Flat :changes and groups without ids stay all-or-nothing; database
and network errors still fail the whole request. A rejection without an
error type is reported as an internal error.
The frontend rolls the file back to what the server holds: it undoes
every pending commit of that file, applies again the remote changes that
arrived meanwhile, and redoes the kept commits. Undoing only the
rejected commits would write stale values over a peer's later change to
the same attribute. The rejected and later commits leave the queue, the
buffer and the undo history, and a warning that hides after ten seconds
tells the user. The pure bookkeeping lives in
app.main.data.persistence.commits.
The pending notice of a commit the rollback dropped no longer leaves the
save status at pending, which made "Pin this version" hang and fail.
Debug tools drive these cases by hand: debug.commitInvalidChange() makes a
save fail validation, holdSaves() delays the answers, failNextSave() and
failNextRollback() fail them on purpose (backend flag debug-save-faults),
and persistenceState() shows the queue.
A remote commit undoes the pending local commits, applies the remote
changes and replays the local ones, so the local file ends like the
server will: remote first, local last. Only queued commits took part;
the ones still waiting in the persistence buffer (up to ~3s of edits)
stayed under the remote change. When both touched the same attribute,
the remote value won locally but lost on the server.
The buffer now lives in the store ([:persistence :buffer]) instead of
an rx buffer, so the rebase can reach it. A flush merges it per file
into queue entries that keep the list of commits they carry, and the
leave-page guard reads buffer and queue from the store. A buffer left
by a previous session is queued when persistence starts.
* ♻️ Move the dashboard header styles to the DS
The page header of the dashboard and settings pages took its styles from
common/refactor/common-dashboard.scss. Move the rules still in use to
dashboard/common/header.scss, written with DS tokens:
- The page titles become heading* (title-large) with a class instead of
a bare h1; the team section links get a class, and the active class
moves from the li to the link.
- Drop the rules that matched nothing: .element-name, .context-menu,
.icon and .btn-secondary inside the header, and the global .btn-primary
and .btn-secondary (only the fonts page uses them; it keeps the old
import until its open change lands).
- The sidebar and subscription styles imported the file without using
it; drop the import.
The header looks the same (0 px in the new visual test). The title line
height goes from 1.15 to 1.4, which does not move the text.
AI-assisted-by: claude-opus-5-5
* ♻️ Move the dashboard libraries and search styles to the DS
Replace the deprecated spacing, size and border variables with DS
tokens and logical properties, and drop the refactor/ import.
- The search placeholder icon gets a class for its color instead of a
.icon svg rule; its xl size already gives the 32px box.
- Delete the selectors no markup uses: .dashboard-projects in both
files and .dashboard-shared in search.
No visual change (0 px in the new libraries and search visual test).
AI-assisted-by: claude-opus-5-5
* 🔥 Move the fonts page header to the DS and delete common-dashboard
The fonts page was the last user of common/refactor/common-dashboard.scss:
its header now uses the shared DS header styles and a DS heading, like
the other dashboard pages. Nothing else imports common-dashboard, so it
is deleted.
No visual change.
AI-assisted-by: claude-opus-5-5
* 📎 Remove a double blank line in the dashboard sidebar styles
Prettier rejects the two blank lines left after the imports, which made
the frontend lint fail in CI.
AI-assisted-by: claude-opus-5-5
* 📎 Remove a blank line between the subscription style imports
Stylelint (at-rule-empty-line-before) rejects the blank line left between
the @use rules after the common-dashboard import was dropped.
AI-assisted-by: claude-opus-5-5
* 🐛 Keep multi-select variant switches refreshing every instance
Multi variants-switch ended each swap with select-shape, which replaced
the selection, interrupted layout/WASM work, and navigated once per copy
so only one instance looked updated under flex. Skip per-swap select and
layout in the batch, re-read live geometry, coalesce layout ids via a
companion DataEvent, then restore selection once. The same path covers
Assets component-multi-swap. Frontend tests lock select-shape count,
layout coalesce, and mixed no-op selections.
Closes#12829
AI-assisted-by: Composer
* ♻️ Use DS icon buttons in the path editing bar
The path editing bar now uses DS icon buttons (ghost, tooltip below) and
DS icons, like the top toolbar. The toggles (draw/move mode, handler
types, snap) use aria-pressed, and the handler type trigger uses the DS
flyout indicator, with no tooltip because its menu opens on hover.
This also shows the handler and snap icons again: since #12659 their
SVGs draw with currentColor, and the legacy %button-icon set color to
transparent, so they were invisible.
Add a visual test of the bar, its hover and the handler menu.
AI-assisted-by: claude-opus-5-5
* ♻️ Move the path editing bar styles to the DS
Drop the deprecated styles import, resolve --panel-border-color to
--color-background-quaternary and use logical properties, except top and
height of the bar, which its transition names.
Delete the selectors the DS buttons left unused: .handler-trigger and
.flyout-indicator (the DS icon button draws its own indicator).
No visual change.
AI-assisted-by: claude-opus-5-5
* 📎 Add the blank line stylelint expects in the path bar styles
Stylelint (scss/double-slash-comment-empty-line-before) rejects the
comment right after a declaration, which made the frontend lint fail in
CI.
AI-assisted-by: claude-opus-5-5
* ⚡ Keep only the shape data the worker reads
The worker kept a full copy of the current page for its selection and
snap indexes. It now keeps only the attributes the indexes and the
change replay read, and re-indexes only the shapes that changed.
On a 52k-shape page the worker heap goes from 698 MB to 286 MB.
Closes#12740
AI-assisted-by: claude-opus-5-5
* ♻️ Share the worker re-trim with its replay test
The worker and its replay test now share `retrim-objects`.
AI-assisted-by: claude-opus-5-5
Migrate the remaining legacy components in the inspect module to the
modern rumext syntax: typography-block, text-block, svg-attr, svg-block,
selection-rect, selection-feedback, layer-item and left-sidebar.
Rename the ?-suffixed props of layer-item (component-child? to
is-component-child, hide-toggle? to hide-toggle) and drop the unused
sortable? and filtered? props passed from left-sidebar.
Relates to #9260
AI-assisted-by: claude-opus-5-5
Signed-off-by: Nimesh Kulkarni <167794819+GitNimay@users.noreply.github.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Give the workspace logo a real link and the DS logo asset
The Penpot logo in the workspace header was an <a> without href, so the
keyboard could not reach it and it could not open in a new tab. It now
links to the dashboard Recent page; a plain click still runs the in-app
navigation (prevent default), and Ctrl/Cmd-click opens the link in a
new tab.
The logo uses the DS raw-svg* asset instead of the legacy icon (same
drawing), with its own 32px class. The legacy svg kept its 500px height
attribute, so it overflowed the header by 222px above and below. This
was the last legacy icon in left_header.cljs.
New visual test "Shows the Penpot logo link": 0 px of difference at rest
and on hover.
AI-assisted-by: claude-opus-5-5
* ♻️ Make the workspace project name a real link
The project name in the workspace header was a div with on-click that
opened the project files in a new tab, so the keyboard could not reach
it. It is now a link to the same page with target "_blank": the browser
opens the new tab, and Tab, Enter and "Open link" work as for any link.
The left header visual test gives 0 px of difference.
AI-assisted-by: claude-opus-5-5
The Penpot logo in the viewer header was an <a> without href. For team
members it now links to the dashboard Recent page of the file's team: a
plain click still runs the in-app navigation (prevent default), and
Ctrl/Cmd-click opens the link in a new tab. Without team permission it
stays as before: no href and no click.
The logo uses the DS raw-svg* asset instead of the legacy icon (same
drawing, same 28px class). This was the last legacy icon in
viewer/header.cljs.
The "Viewer header" visual test gives 0 px of difference.
AI-assisted-by: claude-opus-5-5
A template card was an <a> without href inside a focusable div: Tab
reached the div, and neither Enter nor Space imported the template, so
the cards could not be used from the keyboard. Importing is an action,
so the card is now a <button> with the same look; it takes the tab
index the div had, and the manual Enter handler goes away.
Keyboard focus looks like hover (accent border and "Add to your
project").
The templates visual test gives 0 px of difference at rest, on hover and
for the scroll and collapse buttons; a new shot covers the focus.
AI-assisted-by: claude-opus-5-5
"Change email" and "Want to remove your account?" were <a> without href
that open modals: the keyboard could not reach them. They are now
<button type="button"> that keep their text-link look (accent color,
underline on hover), so Tab reaches them and Enter or Space opens the
modal.
The profile visual test gives 0 px of difference, now also with a shot
of the hover on "Want to remove your account?".
AI-assisted-by: claude-opus-5-5
On the internal error page and the feedback page, "Download report.txt"
was an <a> without href, so Tab skipped it. It is an action, so it becomes
a <button type="button"> with the same look, underline on hover included.
The type keeps it from sending the feedback form.
A new visual test reaches both pages through a mocked server error and
checks 0 px of difference and the download with Enter.
AI-assisted-by: claude-opus-5-5
"Forgot password?", "Create an account" and "Login here." switch the
section of the login dialog (viewer and MCP authorize) and of the dialog
on the not found page. They were <a> without href, so Tab skipped them.
They become <button type="button"> with the same look; a class replaces
the bare a selector in static.scss.
A new visual test opens both dialogs, uses the keyboard to switch
sections and checks 0 px of difference at rest and on hover.
AI-assisted-by: claude-opus-5-5
* 🐛 Create Plugin API shapes at the page root
Create component instances and SVG shapes at the page root so a board at
the page origin cannot capture them. Continue using the existing forced
frame option for component instances and an explicit parent frame for
SVG imports. Interactive placement retains its current behavior.
Cover component instances, SVG imports and SVG imports with images on
ordinary, flex and grid origin boards, including undo, redo, position
changes and explicit reparenting.
Closes#12086
AI-assisted-by: deepseek-v4.1-flash
Signed-off-by: choi junsoo <junsoo1172@gmail.com>
* 🐛 Preserve interactive creation placement
Cover component and SVG placement inside and outside an origin board.
Exercise the sidebar move lifecycle and verify parent preservation.
Correct the Plugin API changelog category.
AI-assisted-by: gpt-6
Signed-off-by: choi junsoo <junsoo1172@gmail.com>
* 🔧 Cover page-root creation in the Plugin API test suite
Check that LibraryComponent.instance(), createShapeFromSvg() and
createShapeFromSvgWithImages() create shapes at the page root when a
board covers the page origin. Each test removes the shape it creates,
since it lives outside the scratch board.
AI-assisted-by: claude-opus-5-5
---------
Signed-off-by: choi junsoo <junsoo1172@gmail.com>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
The Integrations settings page labelled the MCP server as Beta, while
the MCP docs give no maturity status. Drop the badge, its style and the
unused translation key so the UI no longer presents it as beta.
Closes#12800
AI-assisted-by: claude-opus-5-5
The token pill worked out inside the component whether its token is
applied to the selection, whether it can apply to it at all and why
the token is invalid, so no test could reach that logic. Move it into
`pill-state`, and split the tooltip into `tooltip-info`, which decides
what it says, and `render-tooltip`, which renders it. Both are pure
and get unit tests for the applied, partial, disabled and error states
and each tooltip case.
The two chained `on-click` handlers become one. Nothing else changes
in behavior.
Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
At some point the code around this changed, I believe this used to reduce over
file ids. However today it reduces over entire library maps, so it passes a map
instead of an id to `dm/get-in files`. This then hashes the library which
forces it to decode all of its shapes, and then it fails to lookup the data. So
the component/colors/typography counts were always returning 0.
Using reduce-kv fixes this, and we can just get the library data directly with
`(:data library)`.
On the backstage file in penpot-files this brings the time to the telemetry
snapshot down from 3.8s to 1.5s
AI-assisted-by: claude-5.5-sonnet
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* ⚡ Avoid decoding the opened page twice for its frame index
The frame index was built over a with-meta copy, so pages got decoded again.
Read it from the page objects themselves and fix the with-meta identity check.
* ⚡ Intern keywords when decoding transit on the frontend
Each shape is decoded on its own, so every shape had its own key keywords.
Cache them in the read handler so all decoded data shares one instance.
* ⚡ Use an array map as the wasm shape proxy delegate
A hash map with ~35 keys costs ~3 KB of nodes per shape on large pages.
An array map holds the same entries in one array, at a small lookup cost.
Most browsers return "ja" rather than "ja-JP" in navigator.language.
Because parse-locale expands "ja-JP" to ["ja_jp" "ja"] but keeps "ja"
as ["ja"], using "ja_jp" causes autodetection to fall back to English.
Normalize the Japanese locale identifier to "ja" and map jpn_JP.po to
translation.ja.js, matching other single-region locales like Ukrainian.
AI-assisted-by: gemini-3.8-flash
A token cannot contain tokens: next to a token `a`, the tokens `a.b`
and `a.b.c` are now invalid with tokenscript, whatever the order of
their sets. As with StyleDictionary, their pills and the token form
show "Group name of a.b conflicts with a token of the same name in
another active set." and `{a.b}` is a missing reference.
The interpreter reports them as :processor/token-name-collision, which
the adapter maps to :error.token/name-collision. frontend/deps.edn
updates penpot/tokenscript-interpreter to v0.2.0, which adds the check.
Since v0.1.0 its errors carry codes and data only, which the adapter
already relies on.
Signed-off-by: Florian Schroedl <flo.schroedl@gmail.com>
* 🐛 Fail MCP tasks at once when their plugin connection closes
A response only counts when it comes from the connection the task went
to, so a task sent over a connection that then closes can never finish.
It used to wait for its full timeout. The plugin bridge now fails such
tasks as soon as their connection closes.
Connections also log their close code and reason, to tell a page
closing its socket from a browser or network failure.
* ✨ Announce the MCP session to a browser extension
Let a browser extension learn which tab holds an MCP session, so it can
keep the browser from discarding that tab. While the MCP plugin runs,
the page listens for the extension's hello message, and says it is
ready once it listens, so that a content script that said hello too
early says it again. After a hello, the page posts its session ID and
connection status, and posts them again on every status change. When
the plugin stops, it reports the session as disconnected.
The plugin also reports "disconnected" while it reconnects on its own,
for instance after a page resume, so the page announces "connecting"
then and "disconnected" only once the user has stopped the connection.
Otherwise the extension would forget the session at every reconnect.
The page only accepts messages posted on its own window by its own
origin, posts to its own origin only, and sends nothing before the
extension asks. It reaches the window through app.util.globals, whose
stand-in for the unit tests accepts postMessage.
* ✨ Keep Penpot tabs connected to MCP from being frozen
Chrome freezes some background tabs, for instance busy ones while
Battery Saver is on, and a frozen page cannot run the MCP plugin.
Chrome does not freeze a page that holds a Web Lock, so the page holds
a lock named penpot-mcp-session from the moment the user asks to
connect until the connection stops: on disconnect, logout, MCP being
turned off or the MCP runtime starting again. The lock is shared, so
several tabs can hold it at once. Browsers without Web Locks skip it,
and when the lock request fails, the next connection request tries
again.
Firefox does not freeze background pages, so the lock changes nothing
there.
* ✨ Add a browser extension that keeps MCP tabs from being discarded
A browser may discard a background tab to save memory. A discarded
Penpot tab reloads when the user returns to it and loses its MCP
session.
Add mcp/packages/browser-extension, a Manifest V3 extension for Chrome
and Firefox, built from one source into dist/chrome and dist/firefox.
It is written in TypeScript, bundled with esbuild and imports the
protocol types from mcp-common. The package joins the mcp workspace, so
the build, test and fmt scripts of mcp/ cover it; the server release
build skips it.
Its content script runs on design.penpot.app, on the test environments
under penpot.dev and on local development instances. It asks the page
for its MCP session on load, again when the page says it is ready and
when the page returns from the back/forward cache, and forwards each
valid session announcement to the background. It accepts only messages
the page posted on its own window from its own origin, and posts its
hello to that origin only. It also tells the background when the page
is hidden.
The background records which tab holds each MCP session. A tab holds
the session it last announced as connected, until its page announces a
disconnect, is hidden, or the tab is closed or replaced. The map lives
in chrome.storage.session, so it survives restarts of the background
and clears when the browser closes. While a tab holds a session, the
browser may not discard it automatically.
The extension asks only for the storage permission. Chrome runs the
background as a service worker; Firefox runs it as an event page and
needs the add-on ID, mcp-extension@penpot.app, and Firefox 140. The
toolbar popup lists the connected tabs. The icon is the Penpot logo,
white on a black rounded square, with the SVG source next to the PNGs.
A README explains how the extension works and how to build and load
it.
* 🐛 Ask MCP plugins for heartbeats so hidden tabs do not look frozen
The server refuses to send a task to a plugin whose last heartbeat is
older than 30 s, taking its tab for frozen. The plugin sent heartbeats
only from a 10 s timer, and Chrome runs the timers of a hidden page as
rarely as once a minute: without Energy Saver after about a minute
hidden, and at once with it. A Penpot tab hidden for a minute therefore
failed that check about half the time, although it could run tasks.
Chrome handles socket messages of a hidden page at once, and a frozen
page cannot handle them at all. The server now sends each plugin a
heartbeat request every 10 s, and the plugin answers it with a
heartbeat, so the heartbeat age again tells a frozen tab from a hidden
one. The plugin keeps its timer, so it still sends heartbeats to a
server that does not ask. A plugin that does not know the request
passes it to the plugin code, which ignores it.
* ✨ Stabilize fragmented file data across snapshots and GC
Snapshots, automatic and user, now store pointer-based rows that share
fragments with the live file instead of duplicating page content.
`get-file-snapshot` resolves those pointers on the server so version
previews keep rendering plain data, and `restore!` reuses the shared
rows. Export still materializes (the package carries no fragments) and
import regenerates them.
`file-gc` treats the main file and every unpurged snapshot as liveness
roots, reads the file unrealized, and reattaches the original pointer
wherever the cleaned value is unchanged, so a pass over an untouched
file writes no fragments and a pass over an edited file keeps the ids
of the pages it did not touch. Snapshot creation and the restore
backup take the file row `FOR SHARE`, which serializes them with the
pass and closes the race that let a new snapshot reference fragments
the pass was about to mark.
All backend pointer bindings go through the `fdata/with-pointers`,
`with-tracked`, `with-tracked-pointers` and `with-untracked` macros.
The loader reads fragments a previous pass marked until `objects-gc`
purges them, and the frontend fails loudly with retry instead of
dropping an unresolved fragment.
Review fixes in this branch: the automatic snapshot attaches before
pending migrations, `realize-pointers` is the single verb,
`reattach-unchanged` takes the identity fast path, the restore guard
takes its revn from the backup read, and `PointerMap.withMeta`
generates a fresh id when metadata changes.
Tests cover both snapshot writers (red without the lock), snapshot
sharing and pin release through the real delete and purge paths,
reattach against orphan marking, preview after offload, the pointer
binding contracts and the with-meta id change.
Closes#12410
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: deepseek-v4.1-flash
♻️ Pin no-churn behavior for untouched pages
Pin the behavior the churn verification measured: a save keeps the
fragment id of every page it does not touch, with a page of eleven keys
(PersistentHashMap), a 46-shape objects map and a nine-page index (hash
map). Also pin the mixed case: a data-level objects-map flip rewrites
the old pointer pages once and the next save keeps the new ids.
The memory records the identity rule behind it: re-assoc with the same
value returns the same map in array and hash maps alike.
Closes#12410
AI-assisted-by: deepseek-v4.1-flash
⚡ Stop loading untouched pages when enabling objects-map
`enable-objects-map` wrapped every page on every save, and reading a
page's objects forced its PointerMap to load the fragment. A save of
one page in a seven-page file therefore made seven fragment reads.
Skip unloaded pointers: their fragment was already wrapped by the
encode-file that wrote it, so loading it only to wrap it again is a
no-op. The save now loads only the pages it edits. A page with a plain
fragment (a data-level feature flip) keeps its plain fragment until it
is first loaded or edited; a migration that enables the feature must
rewrite the data itself.
Cover the change with load-count tests for save and restore, update the
flip test to the new contract, and drop the stale restore comment.
Closes#12410
AI-assisted-by: deepseek-v4.1-flash
⚡ Keep page fragment ids through pending migrations
Applying a pending migration on read realized the file, ran the
migration and persisted it again; the persist wrapped every plain page
in a new PointerMap, so every page got a new fragment id even when the
migration did not change it. That leaves the fragment cache cold and
rewrites every fragment on the first read after a deploy.
Capture the original pointers before realizing, reattach the unchanged
positions after migrating (same value comparison file-gc already uses),
persist the reattached file and return the realized one: the read
contract is plain data. Read-only connections skip the persist and the
reattach. A migration that adds objects-map skips the reattach so the
pages get wrapped.
Move the three helpers to fdata so file-gc and the migration path share
them, and cover the path with a real 0031 migration test: the unchanged
page keeps its id, the changed page gets a new one, the caller gets
realized data and a second read does not migrate again.
Closes#12410
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Realize library pointers with each library's own loader
Fragment rows are keyed by file id, and every caller binds the
fragment loader of the file it works on. get-resolved-file-libraries
returned libraries with their pages still as pointers, so the first
read of a library page under fdata/pointer-map asked the consuming
file for the fragment and failed with :fragment-not-found. That
broke migrations and file validation for every file using a
pointer-mapped library.
The weak-map loader now realizes each library's pointers with the
library's own id. The test fails without the change.
AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Restore the pointer-map require the repair path still needs
The branch dropped `app.util.pointer-map` from the namespace while the
auto-repair path (added to develop before this branch) still rebinds
`pmap/*tracked*` around validation and repair in
`repair-preexisting-damage` and `process-changes-and-validate`. The
namespace did not compile, so the whole backend failed to load.
Those toggles do not fit the fdata macros: `with-untracked` binds nil
and `with-tracked` creates a fresh atom, while the repair path has to
turn tracking off and restore the existing atom (and read it before the
repair starts). Restore the require and keep the bindings explicit.
Also pin the feature-off path with a test: with `fdata/pointer-map` off
(the default), a save creates no fragments, file-gc writes nothing and
the snapshot preview still returns plain data. Drop a stray `\n` char
literal left at the end of one deftest.
Update the memories with the exception to the macro rule, the snapshot
FOR SHARE locking and the pre-migration automatic snapshot.
Closes#12410
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Validate debug files with their own pointer loader
The `/dbg/actions` file validation passed the file id where
`get-resolved-file-libraries` expects the file map, so the library map
was seeded with a nil key. It also validated a file read without
realizing it, so the first page lookup of a pointer-mapped file raised
"load is not supported when *load-fn* is not bind" and the endpoint
could not validate exactly the files the feature touches.
Read the file realized (validation walks every page and component
anyway) and build the library map from the file. The library loader
already resolves each library's pointers with its own id, so the
realized file and its libraries validate as plain data. Covered by a
handler-level test that fails before the change.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Bind the snapshot loader through the fdata helper
`reduce-snapshots` hand-rolled the same `binding` that
`fdata/with-pointers` builds (it also loaded the function's own
pointer loader before entering the reduction). Use the macro so the
rule holds there too and the namespace no longer needs
`app.util.pointer-map`; the loader is identical (same cfg and file-id)
and snapshot walking behaves the same.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Note the pointer-binding exceptions and snapshot ordering
Record what the feature review settled:
- Two direct `binding`s stay on purpose: the fine-grained `*tracked*`
toggles of `app.rpc.commands.files-update` (validation/repair turn
tracking off and restore the existing atom) and
`binfile.common/encode-file` (fresh tracked set inheriting the
current one). Atom-scoped macros would only be worth it if the set
grows.
- The automatic snapshot is taken before the pending migration, and
the pre-migration window is narrow: the `get-file` RPC migrates on
read and its ETag hashes the available migrations, so a post-deploy
read cannot be a 304 from the pre-deploy state. The old-format
snapshot only appears when the save is the first request to touch
the file after the deploy, and `get-file-snapshot` previews it as
stored.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Apply pending migrations when previewing a version
Automatic snapshots are attached before the save applies pending file
migrations, so a version can store the old format. The preview RPC
handed that data to the UI as stored, and the viewport renders it with
the current code.
Realize the snapshot with the live file id (fragments are keyed by
file, not by snapshot), normalize its migrations to a set (the decoder
keeps the pgarray as a vector, and `set/difference` would compare
indexes) and apply the pending migrations with the pure
`fmg/migrate-file` and the live libraries. Nothing is persisted: the
stored version keeps its data and restoring it re-migrates on the next
read. The response now carries the snapshot's migrations.
Use `:features` the way `get-file` does for `fdata/objects-map`: a
client that does not declare it gets plain objects. The historical
feature set is not checked against the enabled one on purpose: a
version may predate a feature change.
Cover the preview of a snapshot with a pending migration and the
objects realization; both pin that the stored row is untouched. Update
the file-data memory with the preview contract and the vector/set
trap.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Decode snapshot migrations as an ordered set
`file-snapshots/decode-snapshot` handed the `file_change.migrations`
pgarray as a vector. `set/difference` falls back to index lookups when
the first collection is shorter than the second, so the pending set was
right only by accident: a stored subset happens to work, while a
stored set larger than `available-migrations` (a snapshot taken by a
newer build) marks every available migration as pending and re-applies
it to already migrated data.
Decode it as a `d/ordered-set`, the type the rest of the code and
`schema:file` expect. This also covers `restore!` and the snapshot
list, and it lets `get-file-snapshot` drop the `(into #{} ...)`
normalization it was carrying. Re-wrap the union in `fmg/migrate` so
the invariant holds even when a caller hands a plain set.
A snapshot with NULL `migrations` predates SQL migration 0140 and
keeps the format version of its time. Pin that legacy path: the
preview regenerates the applied set from that version, migrates in
memory and leaves the stored row untouched. Update the file-data
memory with the invariant and the legacy path.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix flaky waiting-gauge assertion in climit test
The throwable-while-waiting test asserted the exact list of
waiting-gauge publications, but the component sampler republishes
the same count when its first tick lands inside the recorded
window under load. Assert only the last value, which is the
contract the helper documents.
AI-assisted-by: muse-spark-1.3
---------
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
Co-authored-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
* 🐛 Stop worker index updates from adding nil-keyed entries
When a commit touched a page the worker had not indexed, or deleted the
indexed page, `:index/update` ran the selection and snap updates with a
nil page. That left entries under a nil key (and a stale selection index
for a deleted page) for the rest of the session.
Skip updates for pages the worker does not hold, since a later
`:index/initialize` sends the whole page again, and drop the selection
and snap indexes when the changes delete the page.
AI-assisted-by: claude-opus-5-5
* 🐛 Free the worker's copy of a page when leaving it
The worker kept every visited page, with its selection and snap
indexes, for the whole session. On large files each visited page added
hundreds of MB to the worker heap.
Add an `:index/finalize` worker command that drops a page and its
indexes, and send it from `finalize-page`, which runs on every page
switch and when leaving the workspace. Coming back to a page indexes it
again from the full page, as before.
Closes#12738
AI-assisted-by: claude-opus-5-5
