schema-version is the future cache-invalidation key of the
relation overlay and lives only in in-memory session metadata,
so the literal takes the holder name while nothing persisted
compares it. The Ladybug tier's persisted penpot-graph-slice-4
in GraphMeta is untouched.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The three test namespaces still carried graph-overlay in their
names and files. They are backend-tests.relation-overlay-test,
-sync-test, and -console-test now, so the branch carries no old
holder name outside dated records and the storage literal.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The holder vocabulary names the persisted relation write an
export, so the batch tier takes app.graph.export, its file entry
point becomes export-file!, and the two on-connection helpers
follow inside the namespace. print-ingest! and the srepl
ingest-file-to-graph! keep their names, with their references
repointed; their own renames can ride a reviewer's ask.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The datascript dependency's comment in frontend/deps.edn still
named the graph-overlay and the retired derived-index worker, and
the Ladybug-mirror set's docstring still named the graph-worker
branch. Both now use the holder names.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The holder vocabulary names a graph by scope and a holder by
location, scope, and form, so the overlay namespace takes the
holder's name: app.graph.relation-overlay, with its console,
queries, and sync children. Aliases keep their short forms.
schema-version keeps its "penpot-graph-overlay-2" literal: it is
a storage version, and renaming it would rekey stored artefacts
without a schema change.
AI-assisted-by: glm-5.3-flash
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>
The cascade's verify gate lints backend, common, and frontend/src/bench,
and clj-kondo exits non-zero on a warning, so the redundant let in
apply-set-token-set held every rebuilt candidate of this branch back.
The two lets become one, with no change in behaviour. An unused pid
binding in graph-overlay-test gets its underscore.
cljfmt fix over the three files ./scripts/ci reported as formatted
incorrectly, so the backend and frontend fmt checks pass.
clj-kondo over the verify scope: 0 errors, 0 warnings. The overlay
suites: 56 tests, 788 assertions, 0 failures.
AI-assisted-by: claude-opus-5-5
develop added ingest-on-connection-requires-allocator to
graph_sync_parity_test.clj, beside the round trip it shares no fixture
with. This branch retires that file with app.graph.sync, while
app.graph.ingest stays as the batch-export tier, so the test moves to a
namespace of its own rather than leaving with the suite.
AI-assisted-by: claude-opus-5-5
develop replaced :set-active-token-themes with :set-tokens-status, which
writes the active theme and set ids into :tokens-status, and added
:set-tokens-source, which points a file at an external tokens library.
the-classification-table-names-every-change-type fails by name on both,
which is what it is for.
Both are no-ops for the overlay. The builder indexes token sets and
tokens from the file's own :tokens-lib and stores no active status, and
it reads that lib whatever :tokens-source says, so neither change can
reach a datom a rebuild would write.
AI-assisted-by: claude-opus-5-5
The move mirror re-pointed a parent edge for every shape a :mov-objects
change named, so the fold followed a move the document refused. Two
measured consequences: moving a shape into its own subtree left the fold
disagreeing with a rebuild, and moving a shape under its own descendant
closed a cycle in the fold's parent edges, after which the fold had not
returned eight seconds later with app.graph.overlay.sync/subtree-eids on
the stack.
The gate is all-or-nothing for the whole change, which is the half a
per-shape mirror cannot see: process-change :mov-objects proceeds only
when the change names a shape, the parent exists, and every named shape
passes its own is-valid-move?, so one invalid shape in a batch of ten
moves none of them. The mirror now asks the same question of the running
state, in the change's own order.
It asks rather than answers, because a predicate written again in the
applier would be a second definition of which moves are legal, and the
two would drift the first time Penpot changes a nesting rule. So
calculate-invalid-targets and the predicate move out of the change
method into valid-move? at the namespace level, taking the objects view
and the two flags, with the method's own call site delegating to them and
no behaviour changed. The objects view is what the answer takes rather
than a container: the decision walks the moved shape's subtree down
:shapes and the destination's chain up :parent-id, reading :component-id
and :shape-ref off every shape it meets, and the overlay holds all of
those, which is why the same predicate answers for an index.
subtree-eids gains the guard the cycle case needs, so a cyclic index
fails loudly instead of spinning: the walk tracks the shapes it has
entered and returns.
Six new tests, and each mutation of the repair fails by name rather than
in an unrelated test: dropping the gate fails seven assertions across the
three gate tests, weakening every? to some fails the three that pin the
change-level reading, asking the predicate about the wrong destination
fails the same seven, and removing the walk guard fails the two that pin
the cycle.
backend-tests.graph-overlay-sync-test: 29 tests, 228 assertions, 0
failures.
Entry: pp:graph:decide-a-move-the-way-the-document-does open -> done
AI-assisted-by: deepseek-flash
Child order lives in the document as each container's :shapes vector, and
the projection recorded it only as the global Euler intervals, so the
index had no attribute a consumer could read to learn that a shape is the
third child of its parent rather than the fourth. :shape/order is that
attribute now: a plain 0-based integer, dense within one parent, declared
in the schema beside :shape/id and written by the builder per shape from
the parent's vector, exactly as the intervals are derived from it. The
intervals keep their purpose, which is containment, and the ordinal does
not answer a containment query.
With an order in the index, :reorder-children stops being a skip.
process-change carries :parent-id and :shapes, so the new arm writes the
container's ordinals from that vector and retracts the ordinal of a child
the vector no longer holds. The classification table's row moves to
:covered, and the tier supports twenty-five of the thirty-nine change
types rather than twenty-four.
The applier's other gaps close beside it. The raw colour and typography
reference ids ride in the index beside the resolved refs, so an asset
added after the shape that named it repairs every shape that was waiting
for it. A token name written onto a taken name displaces the holder as
one retraction rather than refusing. Every declared shape attribute names
its writer, so an attribute cannot be added to the schema and to no
writer. And a batch that skipped or dropped a change marks the session
stale, so both read gates refuse rather than answer from an index that
disagrees with the document.
Measured on a 5,101-object page: 20,411 datoms before one insertion and
20,412 after, of which one is the new shape's ordinal and none touches a
shape already present, so an append rewrites no sibling while a deletion
rewrites the ordinals after it (9,997 of them for the hundredth child)
and a move pays the interval renumbering it already paid.
backend-tests.graph-overlay-sync-test: 23 tests, 203 assertions, 0
failures.
Entry: pp:graph:sibling-order-needs-an-authored-attribute open -> done
Entry: pp:graph:mirror-the-reorder-children-change open -> done
Entry: pp:graph:store-the-raw-asset-ref open -> done
Entry: pp:graph:mirror-the-token-name-displacements open -> done
Entry: pp:graph:pin-the-attribute-vocabulary open -> done
Entry: pp:graph:does-a-set-token-carry-its-id open -> answered
Entry: pp:graph:stale-index-gate open -> done
Entry: pp:graph:close-applier-coverage-gap open -> done
AI-assisted-by: deepseek-flash
Review found two defects in the bulk build.
A reference may name a tempid the transaction never defines, because
ref-resolver answers with whatever ctf/find-ref-shape names and that
shape can sit in a container the builder skipped. The flattener resolved
such a target to nil, which puts a nil-valued reference in the index.
d/db-with refuses the whole transaction there, with :transact/syntax and
"Tempids used only as value", so the bulk path refuses it too and names
the defect: :projection/dangling-reference. A unit test pins both
refusals, and pins that the same transaction works on both paths once the
target is defined. The flattener is now tx->datoms, taking a
transaction, so the case is testable without a document that reaches it.
The timings in the docstring were the prototype's, taken on a JVM that
had served the session. Re-taken thirty seconds after boot, with the
first pass reported apart: a build is 800 ms cold and 471 to 513 ms warm
against 2,357 ms and 2,147 to 2,181 ms, so 4.2 times rather than 5.3.
The walk alone is 314 to 325 ms and steady from the first pass, which
puts the flatten and the load together at about 185 ms of a warm build.
Entity ids are an artefact of the interning order, and the docstring now
says so: deleting one leaf shape shifts every later id. Two builds of one
document are comparable by id and two builds of two documents are not,
which is a constraint on whatever compares them.
backend suite unchanged at 161 pre-existing failures, 826 tests and 5499
assertions against 824 and 5491 before.
The transact path spends its time resolving one tempid per entity, not
inserting datoms: the 2026-08-21 retrofit added 48 per cent more datoms
for no extra time. datascript's init-db skips that resolution and builds
the B-trees from sorted arrays, and build-tx already mints deterministic
string tempids from document ids, so assigning final entity ids is one
interning pass over the transaction.
On the 21,218-shape design system, measured back to back in one warm
JVM: 410 ms against 2,150 ms for the same 217,598 datoms. The graphs are
the same graph. Entity ids differ and carry no meaning, and the datom
sets are identical once every id is read back through the tempid that
minted it, which the new test asserts along with the stats and the edge
counts.
This is sound only while every build transform reads the document rather
than the half-built index, because init-db has no index to read. The
existing transforms all do: euler-numbering walks :objects, ref-resolver
goes through ctf/find-ref-shape, and the asset maps come from :data.
backend suite unchanged at 161 pre-existing failures (media and storage,
unrelated), 825 tests and 5495 assertions against 824 and 5491 before.
The experiment entry records the rule as (uses-token ?s ?tok ?attr),
with the applying property a rule variable rather than a per-shape
attribute walk; the two-place form was this port's own invention.
shapes-using-token now reads the property from the rule's third place.
AI-assisted-by: claude-fable-5
Four architectural results of the 2026-08-20 study become structure
(docs.local/graph/20260820-report-graph-overlay-datascript.md):
The relation vocabulary is a rule set, not raw clauses: child-of,
descendant-of, instance-of, refers-to, fills-swap-slot and uses-token
are named once in app.graph.overlay.queries, and every consumer query
and backported beadpot assertion traverses relations through them, so
an encoding can change under a rule without a consumer moving.
The subtle helpers run in the builder, never in a query:
ctf/find-ref-shape is called once per copy shape at build time and its
answer is the plain :shape/refers-to reference. The sync path
re-resolves the affected set when resolution inputs change: the shape
on :shape-ref, the subtree on head attributes and reparents, every
instance subtree of a component whose record or copy moved.
Containment carries a second form: one global Euler-tour numbering,
one DFS per container from a single global counter, so "inside subtree
X" is two interval comparisons. Per-container counters are the bug the
suite pins against: ranges would overlap numerically across pages.
The sync path renumbers a container from beyond the global maximum;
interval containment, not the numbers, is the invariant.
Token applications fold the property into the attribute name:
{:token/fill "layerTwo.background"} over the closed vocabulary of
app.common.types.token/all-keys, behind the uses-token rule so the
choice stays reversible. The relation-entity encoding and its two
extra datoms per application are gone.
31 tests, 650 assertions, 0 failures across the three suites.
AI-assisted-by: claude-fable-5
app.graph.sync existed to keep a live Ladybug session in step with the
editor's change vectors; with sessions on the overlay it has no caller.
Its parity suite and the binder gate move with it: the overlay sync
parity test replays the same change vocabulary against
app.graph.overlay.sync, and the console gate test guards the Datalog
surface where validate-on-connection! guarded Cypher. The validator
itself stays in app.graph.ladybug with the batch tier.
AI-assisted-by: claude-fable-5
app.graph.debug sessions now hold a datascript overlay in an atom:
readers never lock, the msgbus sync loop is the only writer, and the
Ladybug engine no longer appears on the session path. The console
query box speaks Datalog; the default query lists component instances
by page, with filter_* columns carrying entity ids for the graph-view
result filter exactly as before. The node inspector asks
[:find ?a ?v :where [<eid> ?a ?v]] and renders datom pairs. Node ids
in the graph-view export are datascript eids, which keeps two shapes
sharing one uuid in different containers distinct; the Cypher export
merged them.
export-session-database! is retired with the session tier: an overlay
has no database file. The reproducible .lbug artifact stays on
source=file through app.graph.ingest, which keeps the batch-export
tier and the beadpot parity route alive.
AI-assisted-by: claude-fable-5
An index of identity, type, and topology in datascript, replacing the
per-file Ladybug database for reverse and cross-container questions.
Entity identity is the (container, shape) pair, never the shape id
alone: uuid/zero keys every page's root frame, duplicated pages share
non-root ids, and deleted components keep their own copy of the
main-instance subtree under the same ids.
app.graph.overlay projects a file's data into the index: pages and
component containers, shapes, components, and the asset node kinds the
Ladybug projection never gained (colors, typographies, token sets,
tokens), with their link transforms from beadpot's ledger (UsesColor,
UsesTypography, UsesToken) as reference attributes and relation
entities. Color references are stored per source (fill, stroke, text
content) so the sync path can rebuild each contribution from a single
:set operation.
app.graph.overlay.queries holds the shared rule set and the standing
question set. The four Cypher edge kinds are rules over indexed
attributes, so they cannot drift from what they restate. The helpers in
app.common.types.component and friends stay the oracle: instance-head?
is the [?s :shape/component-id _] clause, never a reimplementation, and
the local upward walk stays with ctn/get-parent-heads when the caller
holds the document, because the pointer chain beats the rule walk by
three orders of magnitude.
app.graph.overlay.sync applies Penpot change vectors to the index,
pure db-to-db, mirroring app.common.files.changes/process-change per
clause, including load-component-objects on soft component deletion.
app.graph.overlay.console parses, gates, and formats free-form Datalog:
datascript resolves namespace-qualified symbols in call position via
resolve, so every call symbol must be a built-in, an aggregate, or a
shared rule.
The suites pin: overlay answers identical to the helpers on constructed
fixtures (177 assertions, beadpot's graph tests backported with each
claim verified against a helper), synced index equal to a rebuilt one
after every prefix of a 29-change replay covering all 12 supported
change types, and the console gate.
Measured on the Pencil design system (49 pages, 21 256 shapes, 730
components): build 1.63 s, 157 893 datoms, about 7.5 MB retained,
answers identical to the helpers across 724 components, 1 000 sampled
parent-head chains, and 5 549 frame subtrees; sync p95 0.28 ms per
commit. The Ladybug session tier measured 33 to 43 s and 144 MB for a
20k-shape file the projection can ingest; the projection still rejects
every real file at the Path node.
AI-assisted-by: claude-fable-5
First step of the graph-overlay port: index identity, type and topology
in datascript over the document, instead of projecting every shape
attribute into a Ladybug database. Backend and frontend both declare it,
because the index is meant to end up in the derived-index worker.
Measured motivation (devenv, JVM datascript 1.8.1): a 20k-shape file
indexes in 1.1s and 5.7MB, against 33-43s and 144MB for the equivalent
Ladybug session.
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Gate error reporters behind :error-reporting flag
Error reporters used to start on every boot with no way to
switch them off. They now only start when the new
:error-reporting flag is on (off by default, enable with
PENPOT_FLAGS=enable-error-reporting). The database reporter
needs the flag alone; mattermost needs the flag plus its
webhook url. Shutdown with the flag off is safe, and the
database loop now honors its runtime switch.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Add audit-log reference memory and update dev tooling
Adds the top-level audit-log reference memory with all event
flows and turns backend/audit-log into a redirect stub.
Also includes the staged dev-tooling updates (playwright
deps, mdts args, gitignore).
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Assign server-side initiator to frontend audit events
Frontend audit events now carry a server-assigned initiator
derived from the x-client header (penpot-frontend goes to
app, penpot-admin-console goes to admin-console, legacy
penpot-nitrate stays on admin-console, all else goes to
app). Client-sent values are always overwritten, and the
key survives on telemetry shadow rows.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep initiator on frontend telemetry context
The strips-pii-keys test still asserted the old contract
(initiator stripped). The initiator only names the sending
app, so it is not personal data and must survive the
telemetry filter like on the backend channel.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Document initiator on PostHog and browser origins
Complete the audit-log memory for the latest changes: the
initiator property forwarded to PostHog on both channels
and the eventOrigin rule for admin-console browser events.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix unscaped argument in script
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Isolate backend-test database per devenv instance
Each wsN now gets its own backend-test database
(penpot_test_wsN) and Valkey DB (6+N) on the shared
infra, so parallel test runs no longer wipe each
other. The main database stays shared on purpose.
manage.sh creates the test database on bring-up and
passes PENPOT_TEST_* into the main container. Test
helpers already read those vars, so no runtime code
changes. scripts/psql and db-schema gain --ws.
AI-assisted-by: muse-spark-1.3-contributor-free
* 🐛 Fix per-instance test database creation on bring-up
psql -c does not reliably mix SQL with psql-only
commands, so the CREATE DATABASE ... WHERE NOT
EXISTS ... \gexec one-liner never ran. Use two
plain SQL round-trips instead (check pg_database,
then CREATE DATABASE) and print what happens, so a
future failure shows up in the bring-up output.
AI-assisted-by: muse-spark-1.3-contributor-free
Run the audit archive cron when :nexus or :admin-console is on. Ship
allowlisted events to Admin Console first (with row ids for
idempotency), then the full chunk to Nexus when :nexus is set. Mark
archived_at for the whole chunk on success, including nitrate-only
mode. Rename the gate flag from :audit-log-archive to :nexus.
AI-assisted-by: Composer
Co-authored-by: Cursor <cursoragent@cursor.com>
* ✨ Add size limits to profile props and plugin registry
Bound the total serialized size of profile settings to 2 MiB
(:profile-props-max-size), checked on the merged result before
persisting, with a controlled :props-too-large error. Profiles
that already exceed the limit can still shrink but cannot grow.
Cap plugin registry entries in the shared schema (code 1 MiB, 50
plugins max, bounded name/host/description/icon) and restore rate
limiting on the plugin RPCs (profile-mutations bucket, one write
at a time per profile). The plugin manager now asks for
confirmation before removal and ignores repeated clicks while a
persist request is in flight.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Enforce plugin count cap, byte sizes and removal guard
Enforce the declared 50-plugin cap in add-profile-plugin with a
specific :too-many-plugins error (updates of existing entries
still pass); the cap lives in a shared max-plugins constant.
Measure profile props size in UTF-8 bytes instead of chars so
multibyte content cannot slip past the limit.
Cover install/remove persist logic with mocked-RPC frontend tests
(release semantics, in-flight dedupe, validation vs rollback
split) and add the missing boundary tests in common.
Expose the in-flight persist set from the plugin registry and
disable the remove button of entries being saved.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix rollback loops and restore paths in plugin registry
Restore the previous plugin version instead of dropping the entry
when a validation error rejects an update of an installed plugin.
Make compensating writes one-shot with terminal callbacks so a
persistent failure cannot ping-pong between install and remove.
Restores keep the original list position; the unused public
plugin-persisting? predicate is removed.
Pin count-before-size precedence with a dedicated test and fix
translation source refs to their canonical lines.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Guard notifications write and fix restore ordering
Route update-profile-notifications through check-props-size! so
oversized profiles cannot grow through that path; document the
exempt system writers. Remove the duplicated stale entries in
en.po, keeping the canonical translation refs.
Restore rejected plugin updates at their original list position
instead of leaving the optimistic move in place.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Skip no-op plugin removal and clarify size comments
Return early from remove-profile-plugin when the id is absent:
no wasted write, no size check, and no manufactured :plugins key
that could spuriously fail on oversized profiles.
Clarify that per-field string caps count chars while the byte
budget is enforced by profile-props-max-size.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 📎 Fix formatting in rlimit.edn for profile operations
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Fix formatting of import-binfile/global entry
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Simplify props size check and tighten plugin entry caps
Measure props with transit bytes directly instead of the
PGobject string roundtrip.
Rename check-props-size! to check-props-size: single hard limit
on the merged props, no growth comparison, and return props so
writers thread the check into the update.
Move the 2 MiB default into default-props-max-size on the
profile namespace, still overridable with the optional
:profile-props-max-size config entry.
Tighten registry-entry :code and :icon to 500 chars: they hold
manifest paths, not content.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix compatibility problems
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ♻️ Scope organization notifications to specific WebSocket topics
Publish team/org notifications to team-id and organization-id topics
instead of broadcasting to all connections via uuid/zero. Dashboard and
workspace now subscribe to their current team and organization on
initialization, receiving only relevant events.
Backend: added subscribe-organization/unsubscribe-organization WebSocket
handlers and updated :close to clean up organization subscriptions.
Modified notify-team-change, notify-organization-deletion, and
notify-organization-change-sso to publish to specific topics.
Frontend: dashboard and workspace now subscribe to team-id and
organization-id (when applicable) on initialization, with nil-guard
in topic filters.
Closes#11455
AI-assisted-by: longcat-2.0
* 🔧 Remove unused session-id binding in unsubscribe-organization
Clj-kondo lint fix.
AI-assisted-by: longcat-2.0
* 🐛 Fix permission check on team ws connection
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Enforce idle and absolute session expiration
Sessions now expire on two server-side conditions: an idle window
(PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE, default 7d) and an absolute cap
from creation (PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE_ABSOLUTE, default
30d, enforced by the token :exp claim). The daily session-gc task
deletes rows that exceed either window, so idle sessions can no
longer be replayed and active sessions are not deleted at the idle
window.
Also remove the legacy v1 HTTP sessions: the http_session table and
the string-id / :ver 0 token code paths are gone. Any v1 cookie now
requires a fresh login.
Document the session expiration configuration in the technical guide
and add a backend memory describing the token, renewal and GC model.
Closes#11646
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Address session-expiration review findings F1-F4
Fix the unreadable test (a stray paren broke whole-suite
discovery), enforce idle expiration on every request in
wrap-authz, fail boot fast when the absolute cap sits below
the idle window, and align config defaults with the memory
rule while fixing its migration number and stale reference.
Closes#11646
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add Malli schema for storage metadata with dual decode
Phase 1 of the storage_object.metadata migration: reads accept both
Transit and plain JSON (sniffed by the marker) and always return the
normalized shape; writes validate against a closed per-bucket Malli
schema and still serialize as Transit unless the new
:storage-metadata-as-json config flag is set.
The 0155 migration normalizes existing rows inside Transit (reference
to bucket, default bucket, drop of chunk leftovers) and is
idempotent; large instances should fake it and run the batched
script instead.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address review findings on storage metadata Phase 1
Collapse the dead :reference leg of the gc-touched bucket fallback
(the decode always sets :bucket on non-nil metadata, so it is only
reachable with a NULL column) and fix its comment.
Pin the write flag off in the transit-assuming metadata tests so the
suite proves the same with the flag set, and add coverage for the
flag rollback contract, JSON hash survival, NULL metadata in
gc-touched, and the 0155 normalization statements.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Backfill NULLs, canonical buckets, comment fix
Backfill NULL metadata columns in 0155 via coalesce (the key-missing
rule already matches them), derive valid-buckets from the Malli schema
dispatch entries so the list lives in one place, and correct the
lookup-bucket fallback comment to NULL columns.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Defer corrupt metadata rows in storage gc-touched
Decode touched rows individually so one non-map metadata value no
longer aborts the whole chunk: corrupt rows are logged and deferred
exactly one day in the same transaction, keeping their metadata
intact for a later repair, while healthy rows process normally.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address storage metadata phase 1 review findings
Address the review findings on the storage metadata phase 1 branch:
- Fix put-and-delete-object: it stored the object with
::sto/expired-at, so the row was already deleted and del-object!
returned false. Add delete-expired-object-returns-false to keep
the expired-delete case covered.
- Cache the Malli decoder and encoder per process. Building them
compiles the closed multi-dispatch schema, and decode-metadata
runs on every read path (get-object, dedup probes, GC batches).
- Catch Exception instead of Throwable in try-decode-row so JVM
Errors are not deferred as corrupt metadata.
- Add penpot_storage_gc_poison_total, emitted from
storage-gc-touched; wire ::mtx/metrics into its handler.
- Anchor the encoding sniff to the start of the document so a
plain JSON value that begins with a Transit-looking prefix is
not read as Transit.
- Cover every bucket on both encodings, a JSON roundtrip through
the jsonb column, nil metadata, the canonical bucket set and a
poison-only GC chunk.
- Rename private check-metadata! to check-metadata.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Simplify the storage metadata schema to a single map
Replace the per-bucket :multi dispatch with a single closed map: the
bucket is validated with ::sm/one-of over metadata-buckets (now a plain
set) and the remaining keys are typed optional fields. Per-bucket
enforcement shrinks to a one-line :fn guard requiring :file-id and :id
for file-data, whose ids the GC reads to resolve references.
- Drop the dead (sm/register! ::metadata ...): nothing references the
schema by keyword.
- Define tempfile-bucket and upload-session-bucket in the schema and
alias them from app.storage, removing duplicated literals.
- Keep content-type required and the map closed, so an unknown bucket
or key still fails fast on write.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop input coercion from encode-metadata
encode-metadata no longer runs the json-transformer decoder before
validation. On the write path its only effect was coercing string
UUIDs to UUID, and every producer already passes native UUIDs (the
RPC profile-id, uuid/random, or binfile ids decoded as ::sm/uuid).
Reads keep decoding, so stored Transit or JSON values still come
back as native types.
- Replace encode-accepts-string-uuids with
encode-rejects-string-uuids, pinning the stricter contract.
- Pass native UUIDs in encode-writes-plain-json-with-flag.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Document each statement in the storage metadata migration
Move the per-statement rules out of the header and add a comment to each
UPDATE explaining what it does: drop chunk leftovers, promote the legacy
"~:reference" to "~:bucket", drop residual "~:reference", and backfill the
default bucket. The header keeps the scope, the encoding note, the `->`
vs `?` note and the large-instance warning.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap wrapped lines in the backend storage memory
One line per bullet or paragraph, as mem:memory-maintenance requires.
Only formatting; no content change.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Defer storage GC poison rows in their own transaction
process-chunk! no longer takes poison-ids; it only processes the healthy
chunk. The deferral moves to defer-poison! and process-touched! runs it in
its own transaction, separate from the freeze/delete work. The loop still
drains while there is chunk or poison, so a batch made only of poison rows
does not leave healthy rows behind the LIMIT 10 waiting for the next run.
Add a regression test: ten poison rows plus one healthy row with a later
touched_at are all handled in the same run.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Declare per-bucket metadata requirements in one map
Replace the file-data-specific predicate with bucket-requirements, a map
from bucket to the extra keys it must carry. metadata-buckets is derived
from its keys and a single generic :fn enforces presence, so a new bucket
and its contract are one entry. organization now requires
:organization-id; file-data keeps requiring :file-id and :id.
Update the http-assets test helper to set organization-id for its
organization objects.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC helpers
Rename the internal helpers in app.storage.gc-touched (process-chunk,
defer-poison, mark-freeze-in-bulk, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC deleted helpers
Rename the internal helpers in app.storage.gc-deleted (clean-deleted,
delete-sobjects, delete-give-up, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix dedup lookup for JSON-encoded storage metadata
get-database-object-by-hash only matched the Transit keys, so once the
:storage-metadata-as-json flag wrote plain JSON rows the dedup stopped
finding them and duplicated blobs. Match both encodings with a UNION ALL
of two indexable branches.
- Add migration 0156 with the plain-key dedup index; the legacy 0068
index stays until Transit support is removed.
- Cover it with a JSON dedup test and a Transit -> JSON cross test.
AI-assisted-by: deepseek-v4.1-flash
The import-svg-attrs-asset helper called (first result) on the
map returned by import-files!, yielding a clojure.lang.MapEntry
instead of a file UUID. PostgreSQL rejected the MapEntry as a
query parameter with "Can't infer the SQL type".
Extract the id via (first (:file-ids result)) as the neighboring
test already does.
AI-assisted-by: longcat-2.5-preview-free
The event was written twice per accepted organization invitation. The
backend submitted it, and the browser then re-submitted a copy of the
props that the backend had already put in the response under
`:organization-invitation-audit` (`handle-token :team-invitation` in
`verify-token.cljs`). Both rows carried the same name with different prop
vocabularies, and the browser copy only existed when the browser finished
the flow.
Emit the event from the backend only. It now also carries the three props
that lived in the browser copy: the organization member count before the
add, the add source, and whether the invitee also joined a team. The
origin moves to the event context as `:event-origin`. The response no
longer includes `:organization-invitation-audit`, so the browser stops
emitting the event and `verify-token.cljs` drops its
`app.main.data.event` require.
The `accept-*` events of this command now share one prop vocabulary:
`:profile-id` for the accepting profile, `:invited-by` for the inviter
and `:profile-email` for the email, replacing the mix of
`:user-id`/`:user-who-send-invitation` and `:email`.
Audit consumers of `accept-organization-invitation` now see one row per
acceptance instead of two, and must read the new prop names.
AI-assisted-by: space-bunny-free
The json reader of the binfile v3 import rewrites every key of every
entry to kebab-case, nested maps included. Shape `:svg-attrs` is the one
map whose keys are camelCase react prop names, because the svg import
path runs them through `attrs->props`, so an attribute exported as
`fillRule` came back as `:fill-rule` and was stored that way.
The renderer looks the attribute up by its camelCase name, does not find
it and falls back to the default fill rule, so a shape exported with
`fillRule: evenodd` was painted without its hole, and the attributes
panel showed `fill-rule`.
`clean-shape-post-decode` already runs on every shape right after the
schema decode, for page shapes and component objects alike, so the
repair goes there: run `:svg-attrs` back through `attrs->props`, the
same transform that built the keys. It is idempotent, so shapes that
arrive correct are left untouched.
The new tests import a real export that carries the attribute, for
page shapes and component shapes.
Closes#11954
AI-assisted-by: space-bunny-free
* 🐛 Attribute audit events to the caller, not the response
prepare-rpc-event took the event profile-id from the result map
whenever it carried one, before falling back to the caller. Any
command returning a response with a :profile-id key silently
credited the action to somebody else.
get-error-report returns the report with its decoded content
merged in, and that content holds the profile that owned the
report, so privileged reads were logged against the users whose
crashes were being inspected. verify-token on a team invitation
returns the inviter's profile-id, so accepting an invitation was
logged against the inviter.
Resolution is now ::audit/profile-id metadata, then
::rpc/profile-id, then the zero uuid; the response is never
consulted. The two verify-token branches that relied on it now
declare the profile in the result metadata. Every other command
either already declared it or returns no :profile-id; all 30
registered command namespaces were checked.
The tests that pinned the old behavior are replaced by ones
covering the new contract.
AI-assisted-by: space-bunny-free
* 🐛 Coerce the audit profile-id override to a uuid
The only sanctioned way for a command to override the profile of an
audit event is the ::audit/profile-id metadata, and the value is set
by hand in a dozen commands, some of them reading it from token
claims or other sources we do not type.
schema:event requires a uuid and submit* swallows the validation
error, so a string did not fail loudly: the row was dropped silently.
Values that cannot become a uuid are now discarded with a warning
and the event falls back to the caller, which is always a valid uuid.
A uuid, the common case, exits on the first check.
AI-assisted-by: space-bunny-free
* 🐛 Migrate openUIApi schema to Zod v4 function syntax
Zod 4 removed z.function().args(), which broke the
plugins-runtime build with implicit-any errors on every
openUIApi parameter and knock-on possibly-null errors on
the modal in plugin-manager.
Declare the inputs with z.function({ input: [...] }) so the
parameter and return types infer again; behavior is unchanged.
Add a regression spec covering delegation, optional args and
rejection of invalid theme and title values.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Fix deprecated markdown-it-anchor permalink option in docs
Migrate docs Eleventy config to the markdown-it-anchor v10 API.
Replace the deprecated boolean permalink option with
linkInsideHeader, keeping the same symbol and class.
Bump markdown-it-anchor to v10 and related docs deps.
AI-assisted-by: muse-spark-1.3-contributor
* ⬆️ Update deps
* ⬆️ Update base docker images
* 📎 Fix mcp tests
Bump the devenv and image Dockerfiles to pnpm 12.6.0
(latest stable; 12.8.0 is still on the next tag) with
fresh SHA256 pins, and opencode to 1.18.33 plus
opencode2 to 2.0.18 with fresh checksums.
Stamp all 35 package.json files via
scripts/sync-pnpm-version and refresh the 11 lockfiles;
diffs are metadata-only, with no dependency re-resolution.
Drop the last corepack calls from the media-processor
build script and its generated image setup: pnpm now
ships as a system binary and self-heals version drift.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add renderer option on create-demo-profile command
Allow create-demo-profile to take an optional renderer (svg or
wasm) and store it on profile props. When omitted, no renderer is
written so the Penpot default still applies.
Closes#11893
AI-assisted-by: Muse Spark 1.3 Free
* 📚 Fix renderer doc version on create-demo-profile
Correct the ::doc/changes version for the new renderer param from
2.20 to 2.18.1.
AI-assisted-by: Muse Spark 1.3 Free
* 🐛 Allow invitation-based registration when disable-registration is set (#5178)
Per documentation, disable-registration 'disables registration
(still enabled for invitations only)'. Two bugs prevented this:
1. verify_token.clj: when processing an invitation token for a
non-logged-in user with no member-id, the redirect included
registration-disabled? in its condition, sending invited users
to the login page instead of the register page.
2. auth.clj validate-register-attempt!: the registration-disabled
check fired unconditionally before the invitation-token check,
rejecting the actual register RPC even with a valid invitation.
Fix: in verify_token.clj remove registration-disabled? from the
redirect condition for new-user invitations. In auth.clj restructure
the check as an if/else: with an invitation token, validate the token
and allow registration; without one, enforce the flag as before.
* 🐛 Allow registration with disabled public registration
Allow valid team invitations to create new profiles when public
registration is disabled, while keeping password login and invitation
validation required.
Add backend regression coverage for flag combinations and verify-token
redirects, frontend route coverage, and configuration documentation.
Closes#5178
AI-assisted-by: space-bunny-free
* 🐛 Revalidate active invitation during registration
Require a live, unexpired team invitation before using the
registration exception, and recheck it before creating a profile.
Reuse the same lookup in invitation token verification.
Add regression tests for canceled and expired invitations, the
registration race, and explicit redirect contracts. Update docs
and backend auth guidance.
AI-assisted-by: Space Bunny Free
* 🐛 Lock and normalize invitation registration checks
Lock active invitation rows during transactional registration and
acceptance so cancellations cannot race with profile or membership
creation.
Normalize invitation emails before comparisons and database lookups.
Add concurrency, email casing, and final flag regression tests.
AI-assisted-by: Space Bunny Free
---------
Co-authored-by: Sumit Ridhal <sridhal@redhat.com>
* ✨ Add deployment info to events
* ✨ Add get-environment-data RPC method
Add a single public RPC method returning the deployment type and
the enabled environment flags. It replaces get-deployment on the
management API and get-enabled-flags on the main API.
get-enabled-flags stays as a deprecated alias returning only the
flags, so existing callers keep working until it is removed.
The frontend event initialization now reads the flags from the new
method. The exposed flags stay limited to audit-log and telemetry
to avoid leaking internal backend flags.
AI-assisted-by: deepseek-v4.1-flash
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
Drop 277 keys nothing references from en.po (verified against
frontend/src and common/src) and let sync propagate the
deletions to every locale. Clear all 10 fuzzy entries: fill
the 5 empty translations, keep the 4 valid ones, and drop the
duplicated max-quote-reached in favor of max-quota-reached
(the backend code stays, the UI maps it to the quota text).
Recover 22 used-but-missing keys with translations: the 19
shortcuts section/subsection labels plus connected-to,
pixel-grid-color and tokens.add-set. Make the rest
statically visible to rehash instead: :label fns on shortcut
commands, sections and subsections (one debug-only and one
colorpicker-local id exempt); case branches in place of
dm/str-built keys (export modal, text decoration and
transform, undo history with raw-key fallback); hoist
conditionals out of tr calls; pre-translate modal props and
role labels; replace the lone (i18n/tr ...) site with tr.
Turn static :error/code data into eager :error/fn calls in
the common schemas and the auth/password forms. Rename the
two keys containing spaces and point team leave at
max-quota-reached. Backend-driven keys stay dynamic by
design, declared with (tr ...) comments: the five
weak-password details, team and organization notifications.
Tooling: rehash also scans common/src and no longer treats
a missing -l as no locale; new clj-kondo tr-dynamic warning
flags non-literal tr args (lint scripts use --fail-level
error so it never fails CI); tr docstring states the
literal-only rule. Tests cover the shortcut label wiring,
the undo-history fallback and the :error/fn schemas.
Translations memory rewritten to match; es check word list
gains three entries.
Rebased onto develop: adopt the register field-error UX
(the weak-password declarations move onto the :options
code), keep develop's newer keys (connection-error,
account-locked, save-retrying, tokens-source strings) with
fresh references, and reword the shortcuts.cljs prose
comment so rehash does not invent a "literal" key.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix organization/team switcher issues from UX review
* 🐛 Let members leave an organization without SSO credentials
* 🐛 Fix style from previewed organization in the team switcher
* 🐛 Fix leave-organization modal test stubs
Inject the storage component into the delete-object task handler and
validate it on initialization, so library absorption no longer fails with
"expected valid storage to be provided" for storage-backed file data.
Also strengthen the regression test: it now creates a real component in
the shared library and an instance in the referencing file, and verifies
the component is absorbed after deletion instead of only checking the
revision bump. The test disables the fdata/pointer-map feature to stay
focused on the storage fix.
Signed-off-by: Yuito Akatsuki (Tani Yutaka) <yuito@yuito-it.jp>
AI-assisted-by: deepseek-v4.1-flash
