Clients that only need a file's design tokens, such as a CI job that
syncs them to a git repository after an update-file webhook, had to
download the whole file with get-file.
get-file-tokens returns the file's tokens as a DTCG document, the same
format the token export produces, or nil when the file has none. Active
themes and sets come from the file's tokens status, so they reflect
what is currently active.
The document is returned as a plain map. A bare TokensLib is not a
collection, so the response middleware would hand it to ring unencoded.
Relates to #7091
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The Plugin API can write arbitrary key/value plugin data onto the file
root, but it could only be read back by downloading and decoding the
whole file. Add a `get-file-plugin-data` read command that returns the
file-level plugin data map, optionally narrowed to a single namespace,
so external tooling can fetch metadata such as issue or commit
references cheaply.
Read-only: reuses `bfc/get-file` and `check-read-permissions!`, and
adds no data-model or change-type changes. Object/page level reads are
intentionally left out of this first command.
Relates to #10169
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
Inject the storage component into the delete-object task handler and
validate it on initialization, so library absorption no longer fails with
"expected valid storage to be provided" for storage-backed file data.
Also strengthen the regression test: it now creates a real component in
the shared library and an instance in the referencing file, and verifies
the component is absorbed after deletion instead of only checking the
revision bump. The test disables the fdata/pointer-map feature to stay
focused on the storage fix.
Signed-off-by: Yuito Akatsuki (Tani Yutaka) <yuito@yuito-it.jp>
AI-assisted-by: deepseek-v4.1-flash
The seven creation commands no longer accept an optional client
id: create-file, create-project, create-team,
create-team-with-invitations, upload-file-media-object,
create-file-media-object-from-url and assemble-file-media-object.
The server always generates the identifier; a sent id is ignored.
Malli maps are open and the RPC layer never strips unknown params,
so the handlers that would still honor an id (create-file,
create-project) now drop it explicitly. Internal callers that pass
remapped ids (project duplicate, binfile import) keep working.
Closes#11783
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Restrict optional RPC ids to user-provided UUIDs
Add ::sm/user-provided-uuid, backed by a version and variant
aware regex that only accepts v4, v7 and v8 instances. Use it
for the optional :id of the creation RPC commands so reserved
versions such as v3 are rejected at validation time. Reads
such as get-team keep the lax ::sm/uuid. Cover the predicate
and the schema on both JVM and JS runtimes.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Cover id version restriction at the RPC boundary
Add backend regression tests proving the seven creation commands
reject reserved-version ids (v3) with :params-validation and
accept v4 ids (plus v7/v8 on create-team) through the real
decode and validate path. Also drop two duplicated assertions
and document the version and variant of every fixture UUID in
user-provided-test.
AI-assisted-by: muse-spark-1.3-contributor
The get-page RPC command did not validate that the requested page-id was within the share-link's authorized :pages set, allowing share-link holders to read out-of-scope pages. The get-file-fragment command had the same issue.
This fix adds page scope validation to get-page, rejecting requests for pages not in the share-link's :pages set with a :not-found error. For get-file-fragment, share-link access is denied entirely as fragments lack direct page-id mapping.
The fix aligns these commands with the existing correct behavior in get-view-only-bundle, which already filters pages based on share-link scope.
Closes#11281
AI-assisted-by: qwen3.7-plus
* 🐛 Add ownership check to share-link deletion
The delete-share-link RPC command only verified file-level edit
permission but did not check if the caller owned the share-link.
This allowed any file editor to delete share-links created by
other users, disrupting collaborative workflows.
The fix adds an ownership check that allows deletion only by:
- The share-link creator (owner-id matches profile-id)
- File admins (is-admin permission)
- File owners (is-owner permission)
Implemented using TDD:
- RED: Test demonstrates IDOR vulnerability (editor can delete)
- GREEN: Ownership check prevents unauthorized deletion
- All existing tests continue to pass
Closes#11289
AI-assisted-by: qwen3.7-plus
* 🐛 Add test coverage for share-link deletion escape hatches
Address code review feedback for PR #11290:
- Add test for editor deleting their own share-link
- Add test for admin deleting editor's share-link
- Add test for owner deleting editor's share-link
- Remove redundant :is-owner check (already included in :is-admin)
- Add clarifying comment about :is-admin including :is-owner
Closes#11289
AI-assisted-by: qwen3.7-plus
* 🐛 Add permission checks to WebSocket subscription handlers
Check file and team read permissions before allowing WebSocket
subscriptions to prevent resource enumeration via presence
notifications.
AI-assisted-by: mimo-v2.5-pro
* 🐛 Fix random backend test failure
Capture unique constraint violation in insert-file! and return
generic :not-found error instead of propagating raw PostgreSQL
exception, preventing file existence oracle.
AI-assisted-by: mimo-v2.5-pro
Add check-library-team-ownership! helper that verifies both the file
and library share the same team before creating or modifying library
relations. This prevents cross-team library injection where a user
with edit permissions on files in different teams could link them
across team boundaries.
Applied to link-file-to-library, unlink-file-from-library, and
update-file-library-sync-status handlers.
AI-assisted-by: mimo-v2.5
* 🐛 Fix several issues in RPC command handlers
- Reject circular library references in link-file-to-library
- Add explicit team permission check in search-files
- Constrain search-term max length to 250 chars
- Include :deleted-at in file ETag for COND caching
- Move storage I/O outside DB transaction in create-file-thumbnail
AI-assisted-by: deepseek-v4-pro
* 📎 Check perms before circular link checks
* 🐛 Handle circular library reference error
Catch :circular-library-reference error from backend when linking
files to libraries. Show user-friendly toast notification instead of
propagating unhandled error. Add English and Spanish translations.
AI-assisted-by: qwen3.7-plus
* 🐛 Fix library updates reappear after file is reloaded
Summary
Migrate synced_at timestamps to a standalone file_library_sync table to ensure sync state is tracked for both direct and transitive libraries.
Problem
Transitive libraries (libraries imported by other libraries) are not stored as direct rows in file_library_rel. Because the system previously coupled synced_at directly to the file_library_rel schema, transitive libraries lacked a persistent location for their sync timestamps. This caused sync states to be lost or incorrectly reported for nested dependencies.
Changes
Schema Migration: Created file_library_sync and migrated existing synced_at values from file_library_rel.
Decoupling: Removed tight Foreign Key coupling to allow sync rows to exist independently of specific relationship records.
Persistent Writes: Added upsert-file-library-sync! helper. Updated all import, duplication, and RPC write paths (v1/v2/v3 importers, link-file-library) to ensure every write persists a sync row.
Unified Reads: Updated both direct and recursive/transitive library queries to fetch synced_at from the new table.
Testing: Added regression tests to verify that sync rows are correctly created/updated even when a transitive relation is absent in file_library_rel.
Impact
This fix ensures that the system accurately records and retrieves sync states for the entire library dependency tree, resolving the bug where nested libraries appeared out of sync.
* ✨ MR review
* 🎉 Add get-file-stats RPC command
Introduce a new lightweight RPC query that returns aggregate statistics
for a single file: page count, shape counts by type, component/color/
typography counts, and inbound and outbound library reference counts.
Mirrors the existing get-file-summary permission and decoding pattern.
Useful for plugin authors enforcing per-file budgets, the
@penpot/library npm SDK, and future admin dashboards. Purely additive
— no migrations, no UI, no breaking changes.
Signed-off-by: edwin-rivera-dev <bytelogic772@gmail.com>
* 🐛 Bind *load-fn* around file data walk in get-file-stats
The binding previously wrapped only — a plain key
lookup that does not realize any pointers — so by the time
walked and accessed on
each page, was unbound and every PointerMap
dereference threw , failing the three new tests.
Move inside the form so the walk runs
with available, matching the existing pattern used in
.
Signed-off-by: Edwin Rivera <bytelogic772@gmail.com>
---------
Signed-off-by: edwin-rivera-dev <bytelogic772@gmail.com>
Signed-off-by: Edwin Rivera <bytelogic772@gmail.com>
Skip storage objects touched less than 2 hours ago, matching the pattern
used by upload-session-gc. Update all affected tests to advance the clock
past the threshold using ct/*clock* bindings.
Skip storage objects touched less than 2 hours ago, matching the pattern
used by upload-session-gc. Update all affected tests to advance the clock
past the threshold using ct/*clock* bindings.
This simplifes the mental model on how it works and simplifies testing
of the related code.
This also normalizes storage object deletion in the same way as the
rest of objects in penpot (now future deletion date on storage object
also means storage object to be deleted).
Mainly always resolve the public uri at frontend, making the
PENPOT_PUBLIC_URI less necessary to be changed. This improves
the experience of on-premise configuration.
Also removes unnecesary calls for thumbnail generation
on components.
this allows almost all api operations to success usin application/json
encoding with the exception of the update-file, which we need to
approach a bit differently;
the reason update-file is different, is because the operations vector
is right now defined without the context of shape type, so we are just
unable to properly parse the value to correct type using the schema
decoding mechanism
The main issue was the long running gc operation that
affects storage objects with deduplication. The long running
transacion ends locking some storage object rows which collaterally
made operations like import-binfile become blocked indefinitelly
because of the same rows (because of deduplication).
The solution used in this commit is split operations on small
chunks so we no longer use long running transactions that holds
too many locks. With this approach we will make a window to work
concurrently all operarate the distinct operations that requires
locks on the same rows.
