When render-wasm is active, shape exports (single file, ZIP, and Export
boards multi-page PDF) run on the shared worker WASM module so the UI
stays responsive. A progressive PDF session (begin/add-page/finish)
updates progress per page; cancel settles on the client. The server
exporter path remains for the SVG renderer.
* 🐛 Pass version tag to worker so it loads the current render wasm
* 🐛 Keep wasm thumbnail queue alive when one render fails
* 🐛 Fail wasm thumbnail request when canvas context cannot init
* ✨ Run binfile import and export as durable jobs
Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.
Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.
How:
- The substrate gains the pieces a user-facing job needs: optional
job-def metadata (family, resource-role), an expiry on submit, the
owner in the runner context, and job resources in their own bucket,
written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
business params, the registry checks the family, the job-def decodes
and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
vocabulary with self-contained counters, throttled heartbeats, and an
interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
the package descriptor; the import re-checks the edition permission,
runs the core in a transaction so a cancellation rolls it back, and
releases the consumed package and its temporary copy.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Handle externally-cancelled binfile jobs end to end
The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.
- The export emits `{:cancelled true}` per file and the modal paints a
neutral cancelled row instead of success; a message without an
artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
they replace.
Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Pre-translate the cancelled import message
The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Move cancelled strings under generic labels keys
The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add minor improvements
* ♻️ Rename export job command to create-export-binfile-job
Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.
submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.
The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Polish the import and export status rows
The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.
In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Show the cancelled export and fix the import error
The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).
The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.
The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Reflect the real job params in the creation commands
The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.
Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.
A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Stop the test system from running job runners
The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.
That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.
Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Upload the package one chunk at a time and as a percentage
The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.
The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.
AI-assisted-by: deepseek-v4.1-flash
* 🌐 Complete the Spanish catalogue
es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).
sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Move the job status keys into the jobs namespace
The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.
Rename them so the whole job vocabulary is one namespace:
labels.queued -> jobs.queued
labels.export-cancelled -> jobs.export-cancelled
labels.import-cancelled -> jobs.import-cancelled
No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.
AI-assisted-by: deepseek-v4.1-flash
* 🔥 Remove the translation keys nothing references
Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:
labels.uploading-file a duplicate of jobs.progress.stage.upload
since the upload stage says "Uploading file…"
labels.downloading-file dead since the durable jobs replaced the old
import and export progress rows
inspect.attributes.stroke.alignment.center
inspect.attributes.stroke.alignment.inner
inspect.attributes.stroke.alignment.outer
workspace.toolbar.mcp-connect-here
The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix nested board drop target ignoring ancestor clip bounds
Frame hit-testing (get-frame-by-position, get-frames-by-position and
top-nested-frame) only checked a candidate board's own rectangle,
without accounting for an ancestor board with clip content enabled.
A nested board wider/taller than its clipping ancestor could still be
picked as the drop target in its invisible, clipped-away area, so a
dragged shape would get reparented there and disappear from view.
Add clipped-by-ancestor? to reject a point when it falls outside the
bounds of any ancestor board that has clip content enabled, so the
lookup now stops at the correct visible ancestor instead of
descending into the hidden region.
* 🐛 Fix Ctrl+click deep-select reaching into clipped board area
The clip-aware quadtree query (query-index) filters candidate shapes
by whether they overlap every clip-parent ancestor, but the whole
filter was skipped whenever clip-children? was false. That flag is
turned off while a modifier key (Ctrl/Cmd) is held for deep/penetrate
selection, which was meant to let it reach past boolean/mask clip
boundaries, but it also disabled enforcement for board "Clip content"
ancestors, letting a modifier-held click select a shape sitting in a
board's invisible, clipped-away region.
overlaps-parent? now only relaxes the check for non-frame clip-parents
(bool shapes / mask children) when clip-children? is false; board clip
ancestors are always enforced regardless of the modifier key.
* ✨ Auto-link libraries during import based on slugified name
When a Penpot file is exported without bundled libraries and then
imported into a different environment, external library links are
broken because library UUIDs differ across environments.
This feature adds a heuristic to auto-relink libraries by matching
slugified library names against shared files in the target team:
- Export: embed external library metadata (id, name, slug, used-by)
in the manifest when libraries are not included in the export.
- Import: resolve external libraries by slugifying shared file names
in the destination team and matching against manifest slugs.
- Single match: auto-link silently (creates file-library-rel row).
- Multiple matches: emit SSE event so the frontend shows a selection
dialog for the user to pick the correct library.
- No match: import continues without linking (current behavior).
Backend changes:
- Extended manifest schema with optional :external-libraries field
- Added slugify-name, get-files-names, get-shared-files-for-team,
find-shared-files-by-slug helpers in app.binfile.common
- Threaded team-id into import cfg from RPC layer
- Added resolve-external-libraries and auto-link-libraries in v3
- Emit :library-candidates SSE event for multi-match cases
Frontend changes:
- Worker captures library-candidates SSE events and forwards them
- Import dialog shows auto-link notification and multi-match
selection UI with select dropdowns
- Added link-files-to-library! RPC helper for user selections
- Added en/es translations for new UI strings
Closes#9263
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add UI for the auto-link plumbing
* ♻️ Consolidate auto-link libraries with unified export-type and fix ref integrity
Consolidates the auto-link libraries feature into a single coherent implementation:
- Unify boolean flags (embed-assets, include-libraries, link-later)
into single ::bfc/export-type parameter
- Fix critical reference-integrity bug: pre-resolution no longer remaps
:component-file refs when no link is created (multi-match / no-permission)
- compute-link-decisions as single source of truth for auto-link logic
- 80+ backend tests covering round-trip, cross-team, permissions,
edge cases, and reference integrity
AI-assisted-by: longcat-2.0
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Eva Marco <evamarcod@gmail.com>
* ✨ Add headless wasm render backend to the exporter
* ♻️ Move render-wasm bridge to common and split wasm builds
* 🔧 Upload builtin font variants in the wasm exporter
* ♻️ Move shared font and resources utils out of render_wasm
* ⚡ Fetch only the exported roots in the wasm exporter
* ⚡ Bound save_layer rects in the vector export path
* 🐛 Fix nil getData crash dropping ZIP without manifest.json
Add nil-guard in read-as-text to raise typed :invalid-entry error instead of calling (.getData nil writer) which produced a raw TypeError.
Made read-zip-manifest public (was defn-) with explicit detection of missing manifest.json, raising typed :invalid-penpot-file validation error. The existing catch path surfaces this hint as a friendly user error instead of the raw TypeError text.
Add regression tests for both paths. 374 users were affected, 704 occurrences across 2.17.0-RC2/RC3/RC4.
Fixes#10709.
AI-assisted-by: minimax-m3
* 🐛 Harden dnd/get-data against missing dataTransfer
When the sortable hook or any caller passes a synthetic event
without a dataTransfer property (e.g. a dragend fired after a drop
that has already cleared the transfer), the previous implementation
called .getData directly on the nil/undefined result and threw
"Cannot read properties of undefined (reading 'getData')".
Wrap the body in when-let so get-data returns nil cleanly when
dataTransfer is missing. All three current callers
(hooks.cljs:164, viewport/actions.cljs:531 and :562) already treat
the return value as optional via when-let / when, so no caller
breaks.
Add a regression test covering both the missing-dataTransfer case
and a real dataTransfer roundtrip.
AI-assisted-by: minimax-m3
* 🐛 Harden paste handler against missing clipboardData in forms
When a paste event arrives without a clipboardData property (e.g.
a programmatically dispatched ClipboardEvent in some browsers, or
edge cases like dragging a file with no text content), the previous
implementation called .getData directly on the nil/undefined
clipboardData and threw "Cannot read properties of undefined
(reading 'getData')".
Wrap the body in when-let so the paste logic is skipped entirely
when clipboardData is missing. The existing (string? paste-data)
guard in the inner when already tolerates nil; no other caller
behavior changes.
AI-assisted-by: minimax-m3
* 🐛 Harden paste handler against missing clipboardData in components/forms
Same defensive pattern as the main/ui/forms.cljs paste handler: wrap
the body in when-let so the .getData call is skipped when the
clipboardData property is missing on the paste event. Prevents the
raw "Cannot read properties of undefined (reading 'getData')"
TypeError for programmatic / edge-case paste events.
AI-assisted-by: minimax-m3
* 🐛 Harden v3 text editor paste and styles-fn against undefined receivers
Two related fixes for the "Cannot read properties of undefined
(reading 'getData')" family of bugs in the workspace text editor:
- v3_editor.cljs: wrap the paste body in when-let on clipboardData
so .getData("text/plain") is never called on a nil receiver. The
existing (when (and text (seq text))) guard already tolerates nil
text; only the outer .getData call was unprotected.
- editor.cljs: add (and content ...) to the if branch in styles-fn
so .getText and .getData are never called on a nil content. The
else branch (legacy.txt/styles-to-attrs) is already the correct
fallback for missing content.
Add a regression test that mirrors the fixed patterns and verifies
they no longer throw on synthetic events with no clipboardData or
nil content.
AI-assisted-by: minimax-m3
* 🐛 Harden get-editor-block-data and get-editor-block-type against nil block
getCurrentBlock from Draft.js can return undefined for an empty
selection (e.g. before any block is created). The previous
implementations called .getData / .getType directly on the result
and threw "Cannot read properties of undefined (reading
'getData')" / "...reading 'getType')".
Wrap both functions in (when (some? block) ...) so they return nil
cleanly. Callers in editor.cljs and text_editor.cljs already handle
nil results (render-block short-circuits via the case on type; the
text-data caller in text_editor.cljs lets nil flow up), so no
upstream change is required.
Add a regression test covering both functions with nil and
js/undefined input.
AI-assisted-by: minimax-m3
* 🐛 Harden draft-js block-data helpers against nil block
Three related fixes in the vendored draft-js package:
- mergeBlockData: early-return undefined when block is falsy.
Without this, the first line (block.getData()) throws for callers
that pass a nil block.
- splitBlockPreservingData: guard the blockMap.get(...) lookup. If
the start key is stale (e.g. after a Modifier.splitBlock that
doesn't actually produce the expected key), .get() returns
undefined and the subsequent .getData() throws. Fall back to an
empty Immutable Map for the block data.
- updateBlockData: short-circuit (return state unchanged) when
mergeBlockData returns undefined. Without this, the chain
newBlock.getData() would throw on the same nil-block case that
mergeBlockData now guards.
These match the defensive nil-handling pattern used elsewhere in
the frontend (.getData callers) and protect against stale
selection keys in the Draft.js content state.
AI-assisted-by: minimax-m3
Non-export zip files were tagged as :legacy-zip with the body attached,
causing downstream parsing to crash on unrecognized zip content. Now
they are marked :unknown, matching how other unrecognized formats are
handled, so the import fails gracefully.
AI-assisted-by: deepseek-v4-flash
Introduce a purpose-agnostic three-step session-based upload API that
allows uploading large binary blobs (media files and .penpot imports)
without hitting multipart size limits.
Backend:
- Migration 0147: new `upload_session` table (profile_id, total_chunks,
created_at) with indexes on profile_id and created_at.
- Three new RPC commands in media.clj:
* `create-upload-session` – allocates a session row; enforces
`upload-sessions-per-profile` and `upload-chunks-per-session`
quota limits (configurable in config.clj, defaults 5 / 20).
* `upload-chunk` – stores each slice as a storage object;
validates chunk index bounds and profile ownership.
* `assemble-file-media-object` – reassembles chunks via the shared
`assemble-chunks!` helper and creates the final media object.
- `assemble-chunks!` is a public helper in media.clj shared by both
`assemble-file-media-object` and `import-binfile`.
- `import-binfile` (binfile.clj): accepts an optional `upload-id` param;
when provided, materialises the temp file from chunks instead of
expecting an inline multipart body, removing the 200 MiB body limit
on .penpot imports. Schema updated with an `:and` validator requiring
either `:file` or `:upload-id`.
- quotes.clj: new `upload-sessions-per-profile` quota check.
- Background GC task (`tasks/upload_session_gc.clj`): deletes stalled
(never-completed) sessions older than 1 hour; scheduled daily at
midnight via the cron system in main.clj.
- backend/AGENTS.md: document the background-task wiring pattern.
Frontend:
- New `app.main.data.uploads` namespace: generic `upload-blob-chunked`
helper drives steps 1–2 (create session + upload all chunks with a
concurrency cap of 2) and emits `{:session-id uuid}` for callers.
- `config.cljs`: expose `upload-chunk-size` (default 25 MiB, overridable
via `penpotUploadChunkSize` global).
- `workspace/media.cljs`: blobs ≥ chunk-size go through the chunked path
(`upload-blob-chunked` → `assemble-file-media-object`); smaller blobs
use the existing direct `upload-file-media-object` path.
`handle-media-error` simplified; `on-error` callback removed.
- `worker/import.cljs`: new `import-blob-via-upload` helper replaces the
inline multipart approach for both binfile-v1 and binfile-v3 imports.
- `repo.cljs`: `:upload-chunk` derived as a `::multipart-upload`;
`form-data?` removed from `import-binfile` (JSON params only).
Tests:
- Backend (rpc_media_test.clj): happy path, idempotency, permission
isolation, invalid media type, missing chunks, session-not-found,
chunk-index out-of-range, and quota-limit scenarios.
- Frontend (uploads_test.cljs): session creation and chunk-count
correctness for `upload-blob-chunked`.
- Frontend (workspace_media_test.cljs): direct-upload path for small
blobs, chunked path for large blobs, and chunk-count correctness for
`process-blobs`.
- `helpers/http.cljs`: shared fetch-mock helpers (`install-fetch-mock!`,
`make-json-response`, `make-transit-response`, `url->cmd`).
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Introduce a purpose-agnostic three-step session-based upload API that
allows uploading large binary blobs (media files and .penpot imports)
without hitting multipart size limits.
Backend:
- Migration 0147: new `upload_session` table (profile_id, total_chunks,
created_at) with indexes on profile_id and created_at.
- Three new RPC commands in media.clj:
* `create-upload-session` – allocates a session row; enforces
`upload-sessions-per-profile` and `upload-chunks-per-session`
quota limits (configurable in config.clj, defaults 5 / 20).
* `upload-chunk` – stores each slice as a storage object;
validates chunk index bounds and profile ownership.
* `assemble-file-media-object` – reassembles chunks via the shared
`assemble-chunks!` helper and creates the final media object.
- `assemble-chunks!` is a public helper in media.clj shared by both
`assemble-file-media-object` and `import-binfile`.
- `import-binfile` (binfile.clj): accepts an optional `upload-id` param;
when provided, materialises the temp file from chunks instead of
expecting an inline multipart body, removing the 200 MiB body limit
on .penpot imports. Schema updated with an `:and` validator requiring
either `:file` or `:upload-id`.
- quotes.clj: new `upload-sessions-per-profile` quota check.
- Background GC task (`tasks/upload_session_gc.clj`): deletes stalled
(never-completed) sessions older than 1 hour; scheduled daily at
midnight via the cron system in main.clj.
- backend/AGENTS.md: document the background-task wiring pattern.
Frontend:
- New `app.main.data.uploads` namespace: generic `upload-blob-chunked`
helper drives steps 1–2 (create session + upload all chunks with a
concurrency cap of 2) and emits `{:session-id uuid}` for callers.
- `config.cljs`: expose `upload-chunk-size` (default 25 MiB, overridable
via `penpotUploadChunkSize` global).
- `workspace/media.cljs`: blobs ≥ chunk-size go through the chunked path
(`upload-blob-chunked` → `assemble-file-media-object`); smaller blobs
use the existing direct `upload-file-media-object` path.
`handle-media-error` simplified; `on-error` callback removed.
- `worker/import.cljs`: new `import-blob-via-upload` helper replaces the
inline multipart approach for both binfile-v1 and binfile-v3 imports.
- `repo.cljs`: `:upload-chunk` derived as a `::multipart-upload`;
`form-data?` removed from `import-binfile` (JSON params only).
Tests:
- Backend (rpc_media_test.clj): happy path, idempotency, permission
isolation, invalid media type, missing chunks, session-not-found,
chunk-index out-of-range, and quota-limit scenarios.
- Frontend (uploads_test.cljs): session creation and chunk-count
correctness for `upload-blob-chunked`.
- Frontend (workspace_media_test.cljs): direct-upload path for small
blobs, chunked path for large blobs, and chunk-count correctness for
`process-blobs`.
- `helpers/http.cljs`: shared fetch-mock helpers (`install-fetch-mock!`,
`make-json-response`, `make-transit-response`, `url->cmd`).
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Allow viewers to select locked elements in canvas
* ✨ Add ability to lock guides to prevent accidental movement
---------
Signed-off-by: Dexterity104 <hatanokanjiro@gmail.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The current binfile export process uses a streaming technique. The
major problem with the streaming approach is the case when an error
happens on the middle of generation, because we have no way to
notify the user about the error (because the response is already
is sent and contents are streaming directly to the user
client/browser).
This commit replaces the streaming with temporal files and SSE
encoded response for emit the export progress events; once the
exportation is finished, a temporal uri to the exported artifact
is emited to the user via "end" event and the frontend code
will automatically trigger the download.
Using the SSE approach removes possible transport timeouts on export
large files by sending progress data over the open connection.
This commit also removes obsolete code related to old binfile
formats.
