`file_change` is the biggest relation of the database: 683 GB in
production, of which almost all the TOAST is dead space (the legacy
data is already migrated to `file_data`; what is left is the xlog of
the revisions, 4.4 kB on average). It grows with every save of a file
and every version, and its rows are spread over the whole table, so
every query that filters by a file has to walk all of it.
The table is now partitioned by `HASH (file_id)` with 16 partitions, the
same bounds as `file_data`, so the queries that filter by a file can
prune to one partition and the join between the two tables is resolved
partition by partition. The primary key has to include the partition
key, so it becomes `(file_id, id)`, and every access path of the table
filters by `file_id` accordingly: the snapshot commands require it
(2.19), the frontend sends it, and the garbage collector locks the
pending rows by their `(file_id, id)` pair.
The migration creates the partitioned table with the final schema and
copies the rows, keeping the old one aside as `file_change_legacy` for
the rollback. The copy happens inside the transaction of the migration,
so it does not fit a big instance: there the migration is marked as
applied and the procedure of `operations/` does it by hand, in phases,
driven by the `updated_at` watermark.
Along the way:
- every mutation of the table moves `updated_at` to the moment of the
mutation, so the phased copy does not lose the rows changed while it
runs. It is never set to the `deleted_at` of the row, which for the
system snapshots is a date in the future.
- a new `kind` column tells the snapshots (1) and the revisions (2) of
the table apart, and the two paths that insert in it fill it. It has
no default on purpose, so an insert that forgets it fails instead of
storing the wrong kind silently. It is a reserve: the queries still
tell a snapshot from a revision by its label, and no constraint ties
the two together.
- the revision does not repeat the features of the file anymore: they
were 348 bytes written on every save and nobody read them from there.
- the legacy file data backend is gone, and with it the `data`,
`data_ref_id` and `data_backend` columns, which nobody wrote anymore.
- the migration copies the legacy blobs of the snapshots to `file_data`
before the column goes away, and stops when a snapshot with legacy
data is left without its blob: the backend is better down than losing
content.
- the foreign key of the legacy table on `file_id` goes away with the
copy: it is `NO ACTION`, so while the legacy table has rows, the
objects GC cannot delete the files they point to.
- the quota of the snapshots of a team resolves the files of the team
first and counts their versions with `= ANY`, instead of joining the
team with every version of the table.
- the pool sets `enable_partitionwise_join` on every connection, so the
join between the two partitioned tables is resolved partition by
partition.
- the snapshot lookup reports an unknown snapshot with its own error
instead of the generic "object not found" of the database helper, and
the decode script fails instead of printing an empty JSON.
- the procedure of `operations/` stops the backend and the workers
during the cutover: the pool keeps server-side prepared statements,
and a prepared statement holds the OID of the table it was prepared
with, so a connection would keep writing to the legacy table after
the rename.
- the procedure creates the partitions with their final name, so the
result is the schema a new installation gets, and its verification
checks the names and not only how many there are.
- the index of the system snapshots is gone: it filtered them by
`deleted_at IS NULL`, and they always carry a deletion date in the
future, so it never matched a row.
- the configuration guide, the memories and the docstring of the decode
script say that the `legacy-db` backend is gone, instead of offering
a value that takes the service down.
- a test fixes the schema the migration leaves: the 16 partitions with
their name, the primary key, the foreign keys, the check of the kind,
the six indexes, the absence of the legacy columns, and that an insert
without a kind or with an unknown one fails.
Closes#12416
AI-assisted-by: glm-5.3-flash
AI-assisted-by: minimax-m3.1-flash
* 🐛 Add content:write permission checks to Design Tokens plugin API
The Design Tokens API (tokens.cljs) had zero permission checks, allowing
any plugin to create, modify, and delete tokens, sets, and themes
regardless of granted permissions.
Add r/check-permission checks to all 22 write operations across:
- token-proxy: name, value, description, duplicate, remove, applyToken
- token-set-proxy: name, active, toggleActive, addToken, duplicate, remove
- token-theme-proxy: group, name, active, toggleActive, addSet, removeSet,
duplicate, remove
- tokens-catalog: addTheme, addSet
Follows the established pattern from comments.cljs, file.cljs, page.cljs.
Closes#11137
AI-assisted-by: qwen3.7-plus
* 🐛 Add permission checks to shape proxy interactions, detach, export, and variants
The shape proxy (shape.cljs) had multiple operations missing permission
checks, plus a cond ordering bug that bypassed the existing content:write
check for text shapes in commit-fills!.
Fix commit-fills! cond ordering: move permission check before the
text-shape branch so text shapes are also protected.
Add content:write permission checks to:
- interaction-proxy: :trigger, :delay, :action setters, :remove method
- shape-proxy: :addInteraction, :removeInteraction, :detach
- shape-proxy: :applyToken, :switchVariant, :combineAsVariants
Add content:read permission check to:
- shape-proxy: :export (read/extraction operation)
Follows the established pattern from :resize, :rotate, :blocked setters.
Relates to #11137
AI-assisted-by: qwen3.7-plus
* 🐛 Add library:write permission checks to variant plugin API
The library.cljs variant operations (variant-proxy and
lib-component-proxy) had seven mutating operations that
did not check the library:write permission, allowing
any plugin to create, modify, and delete component
variants regardless of granted permissions.
Add r/check-permission checks to all 7 operations:
- variant-proxy: addVariant, addProperty,
removeProperty, renameProperty
- lib-component-proxy: transformInVariant, addVariant,
setVariantProperty
Follows the established pattern from the :name and
:path setters in the same file.
Relates to #11137
AI-assisted-by: qwen3.7-plus
* 🐛 Add content:write permission checks to flow and flex layout plugin API
Add permission checks to prototype flow and flex layout operations
that were missing them, allowing plugins to modify flows and layout
structure without explicit user permission.
Changes:
- page.cljs: Add content:write checks to flow-proxy (name,
startingBoard setters, remove) and page-proxy (createFlow,
removeFlow)
- flex.cljs: Add content:write checks to flex-layout-proxy
(remove, appendChild)
Follows the established pattern from tokens.cljs, shape.cljs,
and library.cljs.
Relates to #11137
AI-assisted-by: qwen3.7-plus
* 🐛 Add user:read permission checks to plugin API
Add permission checks to user identity accessors that were bypassing
the consent model, allowing plugins to access user data regardless
of whether the user granted user:read permission.
Changes:
- api.cljs: Add user:read checks to getCurrentUser and getActiveUsers
- comments.cljs: Add user:read checks to comment-proxy and
comment-thread-proxy owner/user getters
- file.cljs: Add user:read check to file-version-proxy createdBy getter
When user:read permission is not granted:
- getCurrentUser() returns null
- getActiveUsers() returns empty array
- owner/user/createdBy getters return null
Follows the established pattern from other permission checks in the
plugin API.
Relates to #11137
AI-assisted-by: qwen3.7-plus
* 🐛 Fix problem with token API
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Auto-link libraries during import based on slugified name
When a Penpot file is exported without bundled libraries and then
imported into a different environment, external library links are
broken because library UUIDs differ across environments.
This feature adds a heuristic to auto-relink libraries by matching
slugified library names against shared files in the target team:
- Export: embed external library metadata (id, name, slug, used-by)
in the manifest when libraries are not included in the export.
- Import: resolve external libraries by slugifying shared file names
in the destination team and matching against manifest slugs.
- Single match: auto-link silently (creates file-library-rel row).
- Multiple matches: emit SSE event so the frontend shows a selection
dialog for the user to pick the correct library.
- No match: import continues without linking (current behavior).
Backend changes:
- Extended manifest schema with optional :external-libraries field
- Added slugify-name, get-files-names, get-shared-files-for-team,
find-shared-files-by-slug helpers in app.binfile.common
- Threaded team-id into import cfg from RPC layer
- Added resolve-external-libraries and auto-link-libraries in v3
- Emit :library-candidates SSE event for multi-match cases
Frontend changes:
- Worker captures library-candidates SSE events and forwards them
- Import dialog shows auto-link notification and multi-match
selection UI with select dropdowns
- Added link-files-to-library! RPC helper for user selections
- Added en/es translations for new UI strings
Closes#9263
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add UI for the auto-link plumbing
* ♻️ Consolidate auto-link libraries with unified export-type and fix ref integrity
Consolidates the auto-link libraries feature into a single coherent implementation:
- Unify boolean flags (embed-assets, include-libraries, link-later)
into single ::bfc/export-type parameter
- Fix critical reference-integrity bug: pre-resolution no longer remaps
:component-file refs when no link is created (multi-match / no-permission)
- compute-link-decisions as single source of truth for auto-link logic
- 80+ backend tests covering round-trip, cross-team, permissions,
edge cases, and reference integrity
AI-assisted-by: longcat-2.0
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Eva Marco <evamarcod@gmail.com>
* ✨ Adds static dispatch safe stubs in tests
* 🐛 Fix shapesColors metadata key to match ColorShapeInfo
* 🐛 Fix CommentThread.remove rejecting the owner's own threads
* 🐛 Fix page.removeCommentThread throwing on a spurious Promise
* ✨ Implement ShapeBase.swapComponent in the plugin API
* ✨ Expose File.revn in the plugin API
* 🐛 Fix FileVersion.createdAt calling Luxon method on a js/Date
* 🐛 Fix plugin font/typography application to text and ranges
* 🐛 Default plugin overlay interaction position for non-manual types
* 🐛 Fix plugin interaction setters passing an id-only shape
* 🐛 Fix grid addColumnAtIndex rejecting valid track types
* 🐛 Expose libraryId on library color/typography/component proxies
* ✨ Implement LibraryTypography.setFont in the plugin API
* 🐛 Fix typography.applyToTextRange reading unexposed range bounds
* 🐛 Fix utils.geometry.center argument mismatch
* 🐛 Fix localStorage.removeItem calling getItem
* 🐛 Fix shape backgroundBlur proxy key casing
* 🐛 Report boolean shape type as 'boolean' in the plugin API
* 🐛 Return the resulting paths from plugin flatten
* 🐛 Make plugin z-order methods act on the target shape
* 🐛 Make is-variant-container? return a boolean
* ✨ Implement Group.isMask in the plugin API
* 🐛 Return a shape proxy from TextRange.shape
* 🐛 Return the duplicated set from TokenSet.duplicate
* 🐛 Fix theme addSet/removeSet reading set name with a keyword
* 🐛 Accept string fontFamilies token value in the plugin API
* 🐛 Fix combineAsVariants ignoring the passed component ids
* 🐛 Fix board removeRulerGuide ignoring its argument
* 🐛 Fix board guides setter schema and parser
* 🐛 Avoid 0-byte allocation when syncing empty grid tracks
* 🐛 Validate grid track indices in the plugin API
* 🐛 Return null for empty input in group() and centerShapes()
* 🐛 Return TokenTypographyValue[] from a typography token's resolvedValue
* 🐛 Return TokenShadowValue[] from a shadow token's resolvedValue
* 🐛 Return string[] from a fontFamilies token's resolvedValue
* 🐛 Clear mutually-exclusive reps when setting LibraryColor gradient/image
* 🐛 Add readonly tags to types, deprecate Image type
* 📚 Update plugins changelog
* ♻️ Split history workspace.cljs to workspace/pages.cljs - rename file to target-name
* ♻️ Split history workspace.cljs to workspace/pages.cljs - rename source-file to git-split-temp
* ♻️ Split history workspace.cljs to workspace/pages.cljs - restore name of source-file
* ♻️ Cleanup after adding new ns, add exports
* ♻️ Move set-plugin-data to main.data.plugins
* 🐛 Possible bugfix, cherry-picked from commit 8f7c63d6e2c (conflict during refactor)
---------
Co-authored-by: Eva Marco <eva.marco@kaleidos.net>
