Clients that only need a file's design tokens, such as a CI job that
syncs them to a git repository after an update-file webhook, had to
download the whole file with get-file.
get-file-tokens returns the file's tokens as a DTCG document, the same
format the token export produces, or nil when the file has none. Active
themes and sets come from the file's tokens status, so they reflect
what is currently active.
The document is returned as a plain map. A bare TokensLib is not a
collection, so the response middleware would hand it to ring unencoded.
Relates to #7091
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The Plugin API can write arbitrary key/value plugin data onto the file
root, but it could only be read back by downloading and decoding the
whole file. Add a `get-file-plugin-data` read command that returns the
file-level plugin data map, optionally narrowed to a single namespace,
so external tooling can fetch metadata such as issue or commit
references cheaply.
Read-only: reuses `bfc/get-file` and `check-read-permissions!`, and
adds no data-model or change-type changes. Object/page level reads are
intentionally left out of this first command.
Relates to #10169
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Run binfile import and export as durable jobs
Import and export of `.penpot` files become durable jobs: two commands
create them and answer at once with a job id the caller follows, while
a worker of the `binfile` queue does the work. Each job keeps its own
ledger (owner, params frozen at creation, result, progress history and
an expiry), and the legacy commands and their SSE keep working.
Why: both operations ran inside the request. A large import held it
open for minutes, progress only existed while the caller kept the
stream open, and a client that went away left work nobody recorded,
retried or could cancel. The unified `job` substrate already dispatched,
leased, retried and logged events, but it had no user-facing ledger.
How:
- The substrate gains the pieces a user-facing job needs: optional
job-def metadata (family, resource-role), an expiry on submit, the
owner in the runner context, and job resources in their own bucket,
written with the profile of the job and never deduplicated.
- One generic command per family: the caller names the job and sends its
business params, the registry checks the family, the job-def decodes
and validates, and permission and quota are checked first.
- Progress taps of the core become milestones of the job: a shared stage
vocabulary with self-contained counters, throttled heartbeats, and an
interrupt that stops the run inside its own transaction on cancel.
- Cancel reaches pending and running jobs from both modals, and closing
a modal mid-run cancels the job it was following.
- The export re-checks the read permission at run time and answers with
the package descriptor; the import re-checks the edition permission,
runs the core in a transaction so a cancellation rolls it back, and
releases the consumed package and its temporary copy.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Handle externally-cancelled binfile jobs end to end
The `cancelled` outcome of a job nobody followed no longer hangs its
screens: the data layers emit an explicit terminal message for it.
- The export emits `{:cancelled true}` per file and the modal paints a
neutral cancelled row instead of success; a message without an
artifact never reaches the download branch.
- The import closes every entry with a dedicated cancelled message
through the existing error rendering, so the wizard completes.
- Both flows are pinned by data-layer tests, including the red state
they replace.
Also in this change: the progress-contract comment of the management
API, early business-params validation in create-import-job (before
assembling the upload), a `release-resource` helper without fabricated
job ids, a note on the heartbeat write cost, and a memory cleanup.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Pre-translate the cancelled import message
The cancelled entry of the import wizard carried its message as a key
for the UI to translate, which needed a rehash marker comment. Like
its neighboring errors, it now translates eagerly in the data layer:
the literal call keeps rehash clean with no marker, and the tests
assert through the same call instead of the raw key.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Move cancelled strings under generic labels keys
The dashboard-specific cancelled key never shipped: before use it moves
under `labels.*` next to the generic strings both flows already use,
with Spanish translations following the established wording.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add minor improvements
* ♻️ Rename export job command to create-export-binfile-job
Export now has one command per job type, the shape the :export-assets
command needs: create-export-binfile-job freezes exactly the
:export-binfile job, so :name no longer travels in the body and only
:params does. Import keeps the generic envelope.
submit-job reads the queue from the job-def metadata ::jobs/queue-name
(falling back to :binfile), read by the command and never by the
substrate, so a job-def can route its work to another queue without a
change in the command.
The rename reaches the frontend and the tests: the binfile export is
asked for under its own name, and the exporter service keeps
create-export-job.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Polish the import and export status rows
The status row of a file already carries what the file is doing, so the
generic "Uploading file…"/"Downloading file…" lines are gone: the
import wizard drops its own (the export one went with the jobs
refactor), along with the now-unused style.
In the export, every file is queued up front, even the ones whose job
is created only after the previous one is over, and a file stays queued
until its first milestone: a worker picking the job up no longer ends
the queue. The queued text is a step smaller than the milestone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Show the cancelled export and fix the import error
The cancelled export was unreachable: mark-file-cancelled sets :loading
false and the label lived inside the loading branch, so a cancelled
file only painted a neutral notification. The terminal state now
carries the label, and the export flags lose the `?` suffix
(predicates keep it; a data prop is not a question).
The import wizard rendered (:error entry) through tr, but every
producer writes user-facing text (a hint or a translated label), never
a key: it renders the text as-is now, and the dynamic-key ignores and
the stale comment are gone.
The total-text-limit comment no longer references the 200MiB it
replaced; the value is 500MiB in develop already.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Reflect the real job params in the creation commands
The creation commands declared their params as an open empty map, so the
RPC contract and the generated docs said nothing about them. Each command
now imports the schema from the job namespace
(`export-binfile/schema:params`, `import-binfile/schema:create-params`),
and the RPC validates the business params before the command runs. The
job-def still owns the params and re-validates the assembled ones at run
time. The commands no longer decode the params again: the RPC did it.
Import joins export as one command per job type: `create-import-job`
becomes `create-import-binfile-job`, its body no longer carries `:name`
and the job-def is resolved by name like export's. Its creation schema
keeps the version optional, because the command reads it from the package
header when the caller does not say it, and fills the manifest metadata
kept for audit.
A version out of range is now refused by the schema (`:params-validation`)
instead of the command's own `:unsupported-version` check, which is gone.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Stop the test system from running job runners
The test system is main/system-config plus main/worker-config with a
few components dissoc'ed. The runners of default, webhook and cron were
listed by hand; the binfile runner added with the durable binfile jobs
was not, so a real penpot/job-runner/binfile/0 thread ran during the
whole suite.
That runner races the tests that drive the loop by hand (run-batch plus
run-worker-loop): it takes the job the test just dispatched and
completes it, and the test then reads a state it did not produce. That
is the reported flake of binfile-jobs-integration-test, whose middle
assertion waits for a runner of another queue to not take the job. It
only reproduced in CI: in the devenv PENPOT_TENANT makes the tenant
baked into main/worker-config differ from the one the test builds, so
the two used different queue keys locally.
Drop every [<profile> :app.worker/runner] by key shape instead of
listing the known queues, so a new queue cannot bring the race back.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Upload the package one chunk at a time and as a percentage
The upload milestone counts chunks, not units of work, so the row wrote
"Uploading the package 3/9" over a file the user had uploaded once: it
read as if nine packages were on their way. Write the stage as a
percentage of the file instead ("Uploading file… (33%)"), and make the
English text of the stage say file, not package.
The chunks also go one at a time now (concat-all instead of merge-all
with two in flight), so the percentage moves in a straight line instead
of jumping. upload-blob-chunked is the generic uploader, so media and
font uploads become sequential too.
AI-assisted-by: deepseek-v4.1-flash
* 🌐 Complete the Spanish catalogue
es.po was 25 keys short of en.po: the 15 that the durable binfile jobs
introduced (14 jobs.progress.stage.* and labels.queued) plus 10 gaps
that predate them (errors.connection-error, errors.save-retrying, the
find shortcuts, workspace.header.retrying,
workspace.tokens.stroke-width and the mcp session labels).
sync -l es then sorts the entries, syncs the #: comments from en and
drops the 7 dashboard.import.progress.* keys en no longer has. The
msgid sets of en and es are now identical (0 missing, 0 extra).
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Move the job status keys into the jobs namespace
The three status words the durable import and export rows render were
filed under labels.*, next to the generic labels bucket, while their
progress stages live under jobs.progress.stage.*: the strings of one
feature sat in two places.
Rename them so the whole job vocabulary is one namespace:
labels.queued -> jobs.queued
labels.export-cancelled -> jobs.export-cancelled
labels.import-cancelled -> jobs.import-cancelled
No other locale has these keys yet, so the rename costs no translation.
The convention (a key names the feature that renders it; labels.* holds
the generic reusable words) is written down in the frontend/translations
memory.
AI-assisted-by: deepseek-v4.1-flash
* 🔥 Remove the translation keys nothing references
Six keys of en.po (and of es.po, which now mirrors it) have no (tr ...)
call site left in frontend/src or common/src, and no literal reference
either:
labels.uploading-file a duplicate of jobs.progress.stage.upload
since the upload stage says "Uploading file…"
labels.downloading-file dead since the durable jobs replaced the old
import and export progress rows
inspect.attributes.stroke.alignment.center
inspect.attributes.stroke.alignment.inner
inspect.attributes.stroke.alignment.outer
workspace.toolbar.mcp-connect-here
The first two are leftovers of the durable binfile jobs, the other four
predate them. Both catalogues keep an identical msgid set afterwards.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
* 🎉 Move backend jobs to the unified job table substrate
Replace the generic 'task' table with a unified 'job' table as the
durable substrate for all backend jobs, and migrate every producer
and consumer to it. The legacy 'task' table stops receiving writes
but stays dormant (still cleaned by tasks-gc); upload-session
chunking and the other post-branch upstream changes are preserved.
New 0154 migration creates the 'job' table: dispatch and lifecycle
columns (name, queue, priority, scheduled_at, retries, status,
timestamps, props) plus optional user-facing ledger columns
(profile_id, target, progress, error, result, resource_id,
expires_at). No modified_at trigger: the application updates it on
every write for lease and orphan detection. Partial indexes cover
the dispatcher claim query, orphan scanning, and per-profile
listing.
New app.jobs public API: submit!/invoke!, heartbeats and progress
reporting, a precompiled job-def registry (name, schema, handler,
decoder, validator), an ephemeral request/response mode, and a
management API for external workers. The dispatcher claims
new/retry rows (FOR UPDATE SKIP LOCKED) with a plain JSON redis
payload and marks lease-expired running rows as orphaned; the
runner executes per-queue with the same retry conventions.
New jobs GC (daily): deletes expired rows and retains internal
terminal rows per :jobs-retention, touching referenced storage
objects so storage-gc-touched reclaims them. Job resources live in
a dedicated job-resource bucket with a reclaim branch in
storage-gc-touched.
All production call-sites (sendmail, delete-object, demo-purge,
file-gc, offload, webhooks, quotes, nitrate bulk delete,
snapshots, telemetry) and the ten cron tasks now submit through
app.jobs; cron is a pure scheduler and the legacy worker registry,
task-table inserts, and per-namespace handlers are removed.
Includes migrations, dispatcher, runner, jobs, request/response,
jobs GC, management API, and cron test suites.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: muse-spark-1.3-contributor
AI-assisted-by: kimi-k3
AI-assisted-by: mimo-v2.5
* ✨ Address review comments on jobs substrate
Resolves the four open review threads on the jobs PR. Orphaned jobs
are now marked aborted, a system-side terminal state with no retry:
the dispatcher sweep stores the end event with outcome aborted in a
single bulk insert in the same transaction, records the terminal
outcome next to the orphaned counter, and reports an error log;
orphans keep alerting through the log and the counter, with no
user notification.
The aborted state is terminal everywhere: the status CHECK, the
sweep, the lifecycle writers, the jobs GC retention, the backlog
gauges and the user status mapping, with tests covering each
transition. The claim comment, the 2.20 doc version and the legacy
task table notes are corrected, and objects-gc heartbeats per chunk.
AI-assisted-by: muse-spark-1.3-contributor
Giant legitimate exports hold hundreds of MB of JSON across
hundreds of thousands of KB-sized entries without tripping any
per-entry cap, so the 200MiB cumulative default rejected them out
of the box. Raise it to 500MiB; the 20MiB per-entry cap still
bounds single-entry inflation. Pin the floor with a regression
test.
AI-assisted-by: muse-spark-1.3-contributor
The S3 file download opens the temp file with CREATE_NEW, so it
fails when the path already exists. tempfile creates the empty
file on reserve, which made every large download depend on the
SDK deleting the placeholder between its own retries.
Add tmp/temp-path, which reserves a unique path and registers
cleanup without creating the file, and use it for S3 downloads.
The old retry on FileAlreadyExists stays as a safety net for a
name clash.
AI-assisted-by: muse-spark-1.3-contributor
Delete the team-scoped file, project and pin relations on leave-team
and delete-team-member, require a live team membership in the file,
project, search and listing permission queries, and clean
pre-existing orphan rows with a migration. Former members now get
not-found on file actions instead of keeping read and rename access
(GHSA-v9r9-h77c-55m2).
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Sanitize SVG namespace-prefixed elements and attributes
The SVG sanitizer compared qualified tag and attribute names against
its blocklists. The XML parser keeps the namespace prefix in the
keyword name (❌script), so a script element written with a prefix
bound to the SVG namespace passed sanitization untouched and was
later served as image/svg+xml, executing in the Penpot origin.
Match tags and attributes on the local name (the part after the last
colon) instead, so dangerous elements and attributes are removed no
matter which prefix they use. Legitimately prefixed attributes such
as xlink:href keep working.
Closes#12071
AI-assisted-by: glm-5.3-flash
* ♻️ Cover prefixed SVG vectors and hoist sanitizer blocklist
Apply the code review findings on top of the sanitizer fix:
- Add a test for the javascript: href vector using a prefix bound to
the xlink namespace, the other payload variant the old exact-key
check missed.
- Add a test asserting that legitimately prefixed attributes such as
xml:space are preserved by the local-name matching.
- Hoist the dangerous-tags blocklist next to the other sanitizer
policy defs and bind the keyword name once in local-name.
Relates to #12071
AI-assisted-by: glm-5.3-flash
* ✨ Gate error reporters behind :error-reporting flag
Error reporters used to start on every boot with no way to
switch them off. They now only start when the new
:error-reporting flag is on (off by default, enable with
PENPOT_FLAGS=enable-error-reporting). The database reporter
needs the flag alone; mattermost needs the flag plus its
webhook url. Shutdown with the flag off is safe, and the
database loop now honors its runtime switch.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Add audit-log reference memory and update dev tooling
Adds the top-level audit-log reference memory with all event
flows and turns backend/audit-log into a redirect stub.
Also includes the staged dev-tooling updates (playwright
deps, mdts args, gitignore).
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Assign server-side initiator to frontend audit events
Frontend audit events now carry a server-assigned initiator
derived from the x-client header (penpot-frontend goes to
app, penpot-admin-console goes to admin-console, legacy
penpot-nitrate stays on admin-console, all else goes to
app). Client-sent values are always overwritten, and the
key survives on telemetry shadow rows.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep initiator on frontend telemetry context
The strips-pii-keys test still asserted the old contract
(initiator stripped). The initiator only names the sending
app, so it is not personal data and must survive the
telemetry filter like on the backend channel.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Document initiator on PostHog and browser origins
Complete the audit-log memory for the latest changes: the
initiator property forwarded to PostHog on both channels
and the eventOrigin rule for admin-console browser events.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix unscaped argument in script
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Run the audit archive cron when :nexus or :admin-console is on. Ship
allowlisted events to Admin Console first (with row ids for
idempotency), then the full chunk to Nexus when :nexus is set. Mark
archived_at for the whole chunk on success, including nitrate-only
mode. Rename the gate flag from :audit-log-archive to :nexus.
AI-assisted-by: Composer
Co-authored-by: Cursor <cursoragent@cursor.com>
* ✨ Add size limits to profile props and plugin registry
Bound the total serialized size of profile settings to 2 MiB
(:profile-props-max-size), checked on the merged result before
persisting, with a controlled :props-too-large error. Profiles
that already exceed the limit can still shrink but cannot grow.
Cap plugin registry entries in the shared schema (code 1 MiB, 50
plugins max, bounded name/host/description/icon) and restore rate
limiting on the plugin RPCs (profile-mutations bucket, one write
at a time per profile). The plugin manager now asks for
confirmation before removal and ignores repeated clicks while a
persist request is in flight.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Enforce plugin count cap, byte sizes and removal guard
Enforce the declared 50-plugin cap in add-profile-plugin with a
specific :too-many-plugins error (updates of existing entries
still pass); the cap lives in a shared max-plugins constant.
Measure profile props size in UTF-8 bytes instead of chars so
multibyte content cannot slip past the limit.
Cover install/remove persist logic with mocked-RPC frontend tests
(release semantics, in-flight dedupe, validation vs rollback
split) and add the missing boundary tests in common.
Expose the in-flight persist set from the plugin registry and
disable the remove button of entries being saved.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix rollback loops and restore paths in plugin registry
Restore the previous plugin version instead of dropping the entry
when a validation error rejects an update of an installed plugin.
Make compensating writes one-shot with terminal callbacks so a
persistent failure cannot ping-pong between install and remove.
Restores keep the original list position; the unused public
plugin-persisting? predicate is removed.
Pin count-before-size precedence with a dedicated test and fix
translation source refs to their canonical lines.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Guard notifications write and fix restore ordering
Route update-profile-notifications through check-props-size! so
oversized profiles cannot grow through that path; document the
exempt system writers. Remove the duplicated stale entries in
en.po, keeping the canonical translation refs.
Restore rejected plugin updates at their original list position
instead of leaving the optimistic move in place.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Skip no-op plugin removal and clarify size comments
Return early from remove-profile-plugin when the id is absent:
no wasted write, no size check, and no manufactured :plugins key
that could spuriously fail on oversized profiles.
Clarify that per-field string caps count chars while the byte
budget is enforced by profile-props-max-size.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 📎 Fix formatting in rlimit.edn for profile operations
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Fix formatting of import-binfile/global entry
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Simplify props size check and tighten plugin entry caps
Measure props with transit bytes directly instead of the
PGobject string roundtrip.
Rename check-props-size! to check-props-size: single hard limit
on the merged props, no growth comparison, and return props so
writers thread the check into the update.
Move the 2 MiB default into default-props-max-size on the
profile namespace, still overridable with the optional
:profile-props-max-size config entry.
Tighten registry-entry :code and :icon to 500 chars: they hold
manifest paths, not content.
Closes#11592
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix compatibility problems
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ♻️ Scope organization notifications to specific WebSocket topics
Publish team/org notifications to team-id and organization-id topics
instead of broadcasting to all connections via uuid/zero. Dashboard and
workspace now subscribe to their current team and organization on
initialization, receiving only relevant events.
Backend: added subscribe-organization/unsubscribe-organization WebSocket
handlers and updated :close to clean up organization subscriptions.
Modified notify-team-change, notify-organization-deletion, and
notify-organization-change-sso to publish to specific topics.
Frontend: dashboard and workspace now subscribe to team-id and
organization-id (when applicable) on initialization, with nil-guard
in topic filters.
Closes#11455
AI-assisted-by: longcat-2.0
* 🔧 Remove unused session-id binding in unsubscribe-organization
Clj-kondo lint fix.
AI-assisted-by: longcat-2.0
* 🐛 Fix permission check on team ws connection
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
* ✨ Enforce idle and absolute session expiration
Sessions now expire on two server-side conditions: an idle window
(PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE, default 7d) and an absolute cap
from creation (PENPOT_AUTH_TOKEN_COOKIE_MAX_AGE_ABSOLUTE, default
30d, enforced by the token :exp claim). The daily session-gc task
deletes rows that exceed either window, so idle sessions can no
longer be replayed and active sessions are not deleted at the idle
window.
Also remove the legacy v1 HTTP sessions: the http_session table and
the string-id / :ver 0 token code paths are gone. Any v1 cookie now
requires a fresh login.
Document the session expiration configuration in the technical guide
and add a backend memory describing the token, renewal and GC model.
Closes#11646
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Address session-expiration review findings F1-F4
Fix the unreadable test (a stray paren broke whole-suite
discovery), enforce idle expiration on every request in
wrap-authz, fail boot fast when the absolute cap sits below
the idle window, and align config defaults with the memory
rule while fixing its migration number and stale reference.
Closes#11646
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add Malli schema for storage metadata with dual decode
Phase 1 of the storage_object.metadata migration: reads accept both
Transit and plain JSON (sniffed by the marker) and always return the
normalized shape; writes validate against a closed per-bucket Malli
schema and still serialize as Transit unless the new
:storage-metadata-as-json config flag is set.
The 0155 migration normalizes existing rows inside Transit (reference
to bucket, default bucket, drop of chunk leftovers) and is
idempotent; large instances should fake it and run the batched
script instead.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address review findings on storage metadata Phase 1
Collapse the dead :reference leg of the gc-touched bucket fallback
(the decode always sets :bucket on non-nil metadata, so it is only
reachable with a NULL column) and fix its comment.
Pin the write flag off in the transit-assuming metadata tests so the
suite proves the same with the flag set, and add coverage for the
flag rollback contract, JSON hash survival, NULL metadata in
gc-touched, and the 0155 normalization statements.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Backfill NULLs, canonical buckets, comment fix
Backfill NULL metadata columns in 0155 via coalesce (the key-missing
rule already matches them), derive valid-buckets from the Malli schema
dispatch entries so the list lives in one place, and correct the
lookup-bucket fallback comment to NULL columns.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Defer corrupt metadata rows in storage gc-touched
Decode touched rows individually so one non-map metadata value no
longer aborts the whole chunk: corrupt rows are logged and deferred
exactly one day in the same transaction, keeping their metadata
intact for a later repair, while healthy rows process normally.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Address storage metadata phase 1 review findings
Address the review findings on the storage metadata phase 1 branch:
- Fix put-and-delete-object: it stored the object with
::sto/expired-at, so the row was already deleted and del-object!
returned false. Add delete-expired-object-returns-false to keep
the expired-delete case covered.
- Cache the Malli decoder and encoder per process. Building them
compiles the closed multi-dispatch schema, and decode-metadata
runs on every read path (get-object, dedup probes, GC batches).
- Catch Exception instead of Throwable in try-decode-row so JVM
Errors are not deferred as corrupt metadata.
- Add penpot_storage_gc_poison_total, emitted from
storage-gc-touched; wire ::mtx/metrics into its handler.
- Anchor the encoding sniff to the start of the document so a
plain JSON value that begins with a Transit-looking prefix is
not read as Transit.
- Cover every bucket on both encodings, a JSON roundtrip through
the jsonb column, nil metadata, the canonical bucket set and a
poison-only GC chunk.
- Rename private check-metadata! to check-metadata.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Simplify the storage metadata schema to a single map
Replace the per-bucket :multi dispatch with a single closed map: the
bucket is validated with ::sm/one-of over metadata-buckets (now a plain
set) and the remaining keys are typed optional fields. Per-bucket
enforcement shrinks to a one-line :fn guard requiring :file-id and :id
for file-data, whose ids the GC reads to resolve references.
- Drop the dead (sm/register! ::metadata ...): nothing references the
schema by keyword.
- Define tempfile-bucket and upload-session-bucket in the schema and
alias them from app.storage, removing duplicated literals.
- Keep content-type required and the map closed, so an unknown bucket
or key still fails fast on write.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop input coercion from encode-metadata
encode-metadata no longer runs the json-transformer decoder before
validation. On the write path its only effect was coercing string
UUIDs to UUID, and every producer already passes native UUIDs (the
RPC profile-id, uuid/random, or binfile ids decoded as ::sm/uuid).
Reads keep decoding, so stored Transit or JSON values still come
back as native types.
- Replace encode-accepts-string-uuids with
encode-rejects-string-uuids, pinning the stricter contract.
- Pass native UUIDs in encode-writes-plain-json-with-flag.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Document each statement in the storage metadata migration
Move the per-statement rules out of the header and add a comment to each
UPDATE explaining what it does: drop chunk leftovers, promote the legacy
"~:reference" to "~:bucket", drop residual "~:reference", and backfill the
default bucket. The header keeps the scope, the encoding note, the `->`
vs `?` note and the large-instance warning.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap wrapped lines in the backend storage memory
One line per bullet or paragraph, as mem:memory-maintenance requires.
Only formatting; no content change.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Defer storage GC poison rows in their own transaction
process-chunk! no longer takes poison-ids; it only processes the healthy
chunk. The deferral moves to defer-poison! and process-touched! runs it in
its own transaction, separate from the freeze/delete work. The loop still
drains while there is chunk or poison, so a batch made only of poison rows
does not leave healthy rows behind the LIMIT 10 waiting for the next run.
Add a regression test: ten poison rows plus one healthy row with a later
touched_at are all handled in the same run.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Declare per-bucket metadata requirements in one map
Replace the file-data-specific predicate with bucket-requirements, a map
from bucket to the extra keys it must carry. metadata-buckets is derived
from its keys and a single generic :fn enforces presence, so a new bucket
and its contract are one entry. organization now requires
:organization-id; file-data keeps requiring :file-id and :id.
Update the http-assets test helper to set organization-id for its
organization objects.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC helpers
Rename the internal helpers in app.storage.gc-touched (process-chunk,
defer-poison, mark-freeze-in-bulk, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the ! suffix from storage GC deleted helpers
Rename the internal helpers in app.storage.gc-deleted (clean-deleted,
delete-sobjects, delete-give-up, ...) to drop the trailing !.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Fix dedup lookup for JSON-encoded storage metadata
get-database-object-by-hash only matched the Transit keys, so once the
:storage-metadata-as-json flag wrote plain JSON rows the dedup stopped
finding them and duplicated blobs. Match both encodings with a UNION ALL
of two indexable branches.
- Add migration 0156 with the plain-key dedup index; the legacy 0068
index stays until Transit support is removed.
- Cover it with a JSON dedup test and a Transit -> JSON cross test.
AI-assisted-by: deepseek-v4.1-flash
The event was written twice per accepted organization invitation. The
backend submitted it, and the browser then re-submitted a copy of the
props that the backend had already put in the response under
`:organization-invitation-audit` (`handle-token :team-invitation` in
`verify-token.cljs`). Both rows carried the same name with different prop
vocabularies, and the browser copy only existed when the browser finished
the flow.
Emit the event from the backend only. It now also carries the three props
that lived in the browser copy: the organization member count before the
add, the add source, and whether the invitee also joined a team. The
origin moves to the event context as `:event-origin`. The response no
longer includes `:organization-invitation-audit`, so the browser stops
emitting the event and `verify-token.cljs` drops its
`app.main.data.event` require.
The `accept-*` events of this command now share one prop vocabulary:
`:profile-id` for the accepting profile, `:invited-by` for the inviter
and `:profile-email` for the email, replacing the mix of
`:user-id`/`:user-who-send-invitation` and `:email`.
Audit consumers of `accept-organization-invitation` now see one row per
acceptance instead of two, and must read the new prop names.
AI-assisted-by: space-bunny-free
The json reader of the binfile v3 import rewrites every key of every
entry to kebab-case, nested maps included. Shape `:svg-attrs` is the one
map whose keys are camelCase react prop names, because the svg import
path runs them through `attrs->props`, so an attribute exported as
`fillRule` came back as `:fill-rule` and was stored that way.
The renderer looks the attribute up by its camelCase name, does not find
it and falls back to the default fill rule, so a shape exported with
`fillRule: evenodd` was painted without its hole, and the attributes
panel showed `fill-rule`.
`clean-shape-post-decode` already runs on every shape right after the
schema decode, for page shapes and component objects alike, so the
repair goes there: run `:svg-attrs` back through `attrs->props`, the
same transform that built the keys. It is idempotent, so shapes that
arrive correct are left untouched.
The new tests import a real export that carries the attribute, for
page shapes and component shapes.
Closes#11954
AI-assisted-by: space-bunny-free
* 🐛 Attribute audit events to the caller, not the response
prepare-rpc-event took the event profile-id from the result map
whenever it carried one, before falling back to the caller. Any
command returning a response with a :profile-id key silently
credited the action to somebody else.
get-error-report returns the report with its decoded content
merged in, and that content holds the profile that owned the
report, so privileged reads were logged against the users whose
crashes were being inspected. verify-token on a team invitation
returns the inviter's profile-id, so accepting an invitation was
logged against the inviter.
Resolution is now ::audit/profile-id metadata, then
::rpc/profile-id, then the zero uuid; the response is never
consulted. The two verify-token branches that relied on it now
declare the profile in the result metadata. Every other command
either already declared it or returns no :profile-id; all 30
registered command namespaces were checked.
The tests that pinned the old behavior are replaced by ones
covering the new contract.
AI-assisted-by: space-bunny-free
* 🐛 Coerce the audit profile-id override to a uuid
The only sanctioned way for a command to override the profile of an
audit event is the ::audit/profile-id metadata, and the value is set
by hand in a dozen commands, some of them reading it from token
claims or other sources we do not type.
schema:event requires a uuid and submit* swallows the validation
error, so a string did not fail loudly: the row was dropped silently.
Values that cannot become a uuid are now discarded with a warning
and the event falls back to the caller, which is always a valid uuid.
A uuid, the common case, exits on the first check.
AI-assisted-by: space-bunny-free
* 🐛 Migrate openUIApi schema to Zod v4 function syntax
Zod 4 removed z.function().args(), which broke the
plugins-runtime build with implicit-any errors on every
openUIApi parameter and knock-on possibly-null errors on
the modal in plugin-manager.
Declare the inputs with z.function({ input: [...] }) so the
parameter and return types infer again; behavior is unchanged.
Add a regression spec covering delegation, optional args and
rejection of invalid theme and title values.
AI-assisted-by: muse-spark-1.3-contributor
* 📚 Fix deprecated markdown-it-anchor permalink option in docs
Migrate docs Eleventy config to the markdown-it-anchor v10 API.
Replace the deprecated boolean permalink option with
linkInsideHeader, keeping the same symbol and class.
Bump markdown-it-anchor to v10 and related docs deps.
AI-assisted-by: muse-spark-1.3-contributor
* ⬆️ Update deps
* ⬆️ Update base docker images
* 📎 Fix mcp tests
* ✨ Add renderer option on create-demo-profile command
Allow create-demo-profile to take an optional renderer (svg or
wasm) and store it on profile props. When omitted, no renderer is
written so the Penpot default still applies.
Closes#11893
AI-assisted-by: Muse Spark 1.3 Free
* 📚 Fix renderer doc version on create-demo-profile
Correct the ::doc/changes version for the new renderer param from
2.20 to 2.18.1.
AI-assisted-by: Muse Spark 1.3 Free
* 🐛 Allow invitation-based registration when disable-registration is set (#5178)
Per documentation, disable-registration 'disables registration
(still enabled for invitations only)'. Two bugs prevented this:
1. verify_token.clj: when processing an invitation token for a
non-logged-in user with no member-id, the redirect included
registration-disabled? in its condition, sending invited users
to the login page instead of the register page.
2. auth.clj validate-register-attempt!: the registration-disabled
check fired unconditionally before the invitation-token check,
rejecting the actual register RPC even with a valid invitation.
Fix: in verify_token.clj remove registration-disabled? from the
redirect condition for new-user invitations. In auth.clj restructure
the check as an if/else: with an invitation token, validate the token
and allow registration; without one, enforce the flag as before.
* 🐛 Allow registration with disabled public registration
Allow valid team invitations to create new profiles when public
registration is disabled, while keeping password login and invitation
validation required.
Add backend regression coverage for flag combinations and verify-token
redirects, frontend route coverage, and configuration documentation.
Closes#5178
AI-assisted-by: space-bunny-free
* 🐛 Revalidate active invitation during registration
Require a live, unexpired team invitation before using the
registration exception, and recheck it before creating a profile.
Reuse the same lookup in invitation token verification.
Add regression tests for canceled and expired invitations, the
registration race, and explicit redirect contracts. Update docs
and backend auth guidance.
AI-assisted-by: Space Bunny Free
* 🐛 Lock and normalize invitation registration checks
Lock active invitation rows during transactional registration and
acceptance so cancellations cannot race with profile or membership
creation.
Normalize invitation emails before comparisons and database lookups.
Add concurrency, email casing, and final flag regression tests.
AI-assisted-by: Space Bunny Free
---------
Co-authored-by: Sumit Ridhal <sridhal@redhat.com>
* ✨ Add deployment info to events
* ✨ Add get-environment-data RPC method
Add a single public RPC method returning the deployment type and
the enabled environment flags. It replaces get-deployment on the
management API and get-enabled-flags on the main API.
get-enabled-flags stays as a deprecated alias returning only the
flags, so existing callers keep working until it is removed.
The frontend event initialization now reads the flags from the new
method. The exposed flags stay limited to audit-log and telemetry
to avoid leaking internal backend flags.
AI-assisted-by: deepseek-v4.1-flash
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix organization/team switcher issues from UX review
* 🐛 Let members leave an organization without SSO credentials
* 🐛 Fix style from previewed organization in the team switcher
* 🐛 Fix leave-organization modal test stubs
Inject the storage component into the delete-object task handler and
validate it on initialization, so library absorption no longer fails with
"expected valid storage to be provided" for storage-backed file data.
Also strengthen the regression test: it now creates a real component in
the shared library and an instance in the referencing file, and verifies
the component is absorbed after deletion instead of only checking the
revision bump. The test disables the fdata/pointer-map feature to stay
focused on the storage fix.
Signed-off-by: Yuito Akatsuki (Tani Yutaka) <yuito@yuito-it.jp>
AI-assisted-by: deepseek-v4.1-flash
Requesting a binfile export with both `include-libraries` and
`embed-assets` enabled threw a raw `IllegalArgumentException`, which the
RPC layer surfaced to API clients as a generic
`:server-error`/`:unexpected` (HTTP 500) with no actionable code -- the
opaque failure reported in #7649 (perceived as an "empty" export).
The two options are genuinely mutually exclusive, so the request is
invalid input, not a server fault. Replace the raw throw in both the v1
and v3 export paths with `ex/raise :type :validation :code
:incompatible-options`, matching how the rest of the binfile code
reports bad input. Also fix the "mutally" -> "mutually" typo in the hint.
Verified on a local devenv: the export RPC now returns
`{:type :validation :code :incompatible-options}` instead of a 500, and
valid single-option exports are unaffected. Adds a regression test.
Fixes#7649
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Build organization invitation audit event in frontend
* ♻️ Align invitation token profile-id with created-by
The invitation token carried the minter in :profile-id while the
invitation row tracks the creator in :created-by. Both mean the
inviter, so re-sends or re-requested links made them disagree and
forced a second response key, :user-who-send-invitation.
Mint :profile-id from :created-by in both token creators, backfill
it from the row on accept (covers stale in-flight tokens), and drop
the duplicate response key. The frontend maps :profile-id to the
unchanged :user-who-send-invitation audit prop.
AI-assisted-by: Muse Spark 1.3 Free
* ♻️ Reuse token ids in invitation accept response
Backfill :member-id with the accepting profile and drop the
:user-id duplicate from the verify-token response, mirroring the
:profile-id/:user-who-send-invitation cleanup. The frontend maps
:member-id to the unchanged :user-id audit prop.
AI-assisted-by: Muse Spark 1.3 Free
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Restore rate-limit headers and add Retry-After
The account-lockout change replaced the header-forwarding 429 handler
with a body-only one, so existing RPC rate-limit responses lost their
x-rate-limit-remaining and x-rate-limit-reset headers. Account lockout
never sent Retry-After either.
Make handle-error :rate-limit preserve ::http/headers and add a
retry-after header when the exception carries a non-nil :ttl in
seconds, keeping the current JSON body. Add focused tests for both the
lockout and the RPC limiter paths.
Document activation, defaults, password/LDAP scope, Redis fail-open
behavior, and the lockout risk, and record the final HTTP contract in
the backend auth memory.
Refs #11397
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Add Retry-After to RPC 429 and expose headers in CORS
Address review follow-ups on the account-lockout 429 contract:
- The RPC limiter now sets retry-after in its 429 headers (seconds
until the longest rejecting limit resets), so it matches the
account-lockout response and the HTTP standard.
- CORS exposes retry-after, x-rate-limit-remaining, and
x-rate-limit-reset so browser clients can read them.
- Use backticks for Retry-After and account-locked in the docs for
consistency with nearby sections.
Refs #11397
AI-assisted-by: deepseek-v4.1-flash
* 🐳 Add upstream diagnostics to nginx access log
Enrich every access-log line with the internal journey of the request:
the status the backend answered (us), the time spent connecting to it
(uct), the time spent waiting for its answer (urt) and the internal
address that served the request (ua).
A plain 502 line used to say nothing about where the request died. With
this format, the tail of the line classifies the failure: connection
rejected, backend accepted and hung (uct + urt under 1s), or backend
stuck until read timeout. This was the missing witness in the Sep 20
incident, where nginx received connection resets with zero timeouts and
zero rejections.
Applied both to the production image template and the devenv config.
With proxy_pass on variables there is no upstream keepalive, so uct
measures one real TCP connection per request.
Parsing the new fields (us, uct, urt, ua) on the log shipper is left to
ops, so they can be filtered in Loki.
AI-assisted-by: glm-5.3-flash
* 🐳 Add stub_status endpoint for nginx metrics
Add a dedicated localhost-only server (listen 127.0.0.1:8082) exposing
/stub_status next to every other location of the public server. Ops can
run the official nginx-prometheus-exporter as a sidecar against
http://127.0.0.1:8082/stub_status and get nginx_connections_active,
accepted vs handled, reading/writing/waiting and request rates in
Prometheus.
Binding it to localhost and its own server keeps it unreachable from
outside the host and out of the public surface, and access_log off
avoids polluting Loki with one line per Prometheus scrape. The base
image already ships stub_status compiled in, so no image rebuild is
needed.
Applied both to the production image template and the devenv config.
AI-assisted-by: glm-5.3-flash
* ✨ Expose http server gate metrics (worker and connector)
The backend already measured dispatch latency but nothing reported the
state of the "house door": the xnio worker queue and threads, and the
monitor-level listener counters. This was the exact blind spot of the
Sep 20 incident, where the server kept answering health checks while it
accepted connections and dropped them without response.
Add a periodic metrics sampler that lives and dies with the http
server (single daemon thread, 15s interval, each sample guarded so an
unexpected error does not cancel subsequent runs) and publishes:
- worker (xnio MXBean gauges): penpot_http_worker_queue_size,
busy_threads, pool_size and max_pool_size. Negative samples are
discarded: the MXBean transiently reports -1 on the busy thread
count (verified live), and a stale negative would read as zero.
- listener (Undertow connector statistics, enabled via the new
:server/statistics yetti option): penpot_http_connector_active*
_connections gauge and requests_total / errors_total counters.
Undertow exposes absolute totals, so the sampler keeps a watermark
atom and publishes deltas, skipping (and moving forward past) a
counter reset.
The connector-level part depends on yetti v11.11, which now accepts
a :server/statistics server option (patch authored and released
upstream; before it, ListenerInfo#getConnectorStatistics always
returned nil).
New tests cover the samplers with fake MXBean/collector statistics
against real prometheus collectors, including the negative-sample
filter, the delta/watermark logic and the sampler lifecycle.
AI-assisted-by: glm-5.3-flash
* 🐛 Include jdk.management in the backend runtime JRE
The production image builds a trimmed JRE with jlink and omitted
jdk.management. Without that module the OS MXBean is
sun.management.BaseOperatingSystemImpl, which has no
getProcessCpuTime, getOpenFileDescriptorCount nor
getMaxFileDescriptorCount. The prometheus client StandardExports
reads those getters reflectively and collect() swallows the
NoSuchMethodException, so process_open_fds, process_max_fds and
process_cpu_seconds_total silently disappeared from /metrics while
the other process_* families kept flowing.
Verified against Prometheus: the app job only ever exposed
process_start_time_seconds, process_virtual_memory_bytes and
process_resident_memory_bytes; the fd and cpu families were absent.
Reproduced locally by running the backend metrics registry on a JRE
built with the same jlink module list (false/false/false) and on one
with jdk.management added (true/true/true).
Add the module to --add-modules and pin the metric contract with
backend-tests.metrics-test.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Build the http metrics sampler on promesa.exec
Replace the hand-rolled ScheduledThreadPoolExecutor and ThreadFactory
with promesa.exec primitives: px/scheduled-executor with a daemon
thread factory, and a px/schedule chain that reschedules the next
sample when the current one finishes.
Beyond fitting the existing periodic-task pattern (worker/cron,
rpc/rlimit), the chained schedule makes the docstring promise real:
with scheduleAtFixedRate an exception escaping the runnable cancelled
the following executions, while the reschedule now happens in a
finally block.
The sampler shutdown uses px/shutdown-now (shutdown! is deprecated in
promesa 12.0.0) to cancel the pending sample, keeping the previous
halt semantics.
The lifecycle test moves to the promesa predicates and a new test
covers the error-resilience promise: the first sample runs, throws,
and the next one is still scheduled.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Tighten the http metrics samplers
The samplers are leaf functions: they receive what they need and
publish it. Drop the internal nil guards (if there is no metrics
instance or no mxbean there is nothing to call them for) and move the
checks to the boundary, where the optional data is resolved:
sample-http-metrics now short-circuits with some-> and when-let.
Write the four worker gauges as four static operations instead of a
vector of pairs walked by doseq: the set is fixed, so the collection
only adds an allocation and hides each operation.
Drop the ! suffix from the sample-*-metrics family: ! marks a function
whose contract is to mutate state, while these report, and the mutation
happens in the mtx/run! they call. The constant true return, which only
existed so the removed guard tests could assert it, goes away too.
Tests follow the move: the internal-guard tests are replaced by one
boundary test (a nil server publishes nothing).
AI-assisted-by: deepseek-v4.1-flash
* 📚 Add the function design rules memory
Document the rules that came out of the http metrics sampler review:
preconditions are checked at the boundary instead of re-checked in the
core, optional-by-design data is guarded where the optionality is born,
a fixed set of operations is written statically, ! marks mutation and
not reporting, and production code is not shaped for tests.
Also state in the memory maintenance guide that memories must not use
manual line wrapping.
Linked from critical-info so it is read when designing a solution or
an API, not only when touching the samplers.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap the critical-info memory lines
The memory maintenance guide forbids manual line wrapping, so rewrite
critical-info with one line per bullet and paragraph. A stray `*` at
the start of one continuation line is dropped.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the redundant guard in the http server halt
create-metrics-sampler always returns the scheduler, so the sampler is
always present when integrant calls halt-key!; the nil check was dead
code, same as the yt/stop! call next to it.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Add srepl helper to delete profiles by email
Add `delete-profiles-by-email!` to app.srepl.main. It accepts a
single email, a comma separated list of emails or a coll of emails,
resolves each profile, logs it to audit and enqueues the
delete-object task. The deleted-at is backdated with the configured
deletion-delay so profiles and their owned teams are purged on the
next gc pass.
Extract the per-email deletion logic into a private fn and reuse it
from `delete-profiles-in-bulk!`. Add tests for the new
`parse-emails` helper.
AI-assisted-by: glm-5.3-flash
* ✨ Add storage operation metrics for S3 and buckets
Expose Prometheus metrics for the object storage subsystem.
The S3 backend now attaches an AWS SDK MetricPublisher that counts
API calls, retries and latency per operation and target. The storage
layer counts logical operations and deduplication outcomes per Penpot
bucket, and the assets handlers count served requests per route.
Closes#11676
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Fix storage metrics labels, errors and test gaps
Address the review findings on the storage metrics commit.
Label reads with the object's own backend, count failed asset
serving as errors without swallowing them, and cover the failed
S3 call, S3 asset path and permission-denied branches with tests.
Also share the label helper and reuse the metrics test helper.
Closes#11676
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Harden storage metrics and fill test gaps
Address the second-round review findings on storage metrics.
Unknown backends now fail explicitly and count as errors, exists
stays paired with its dedup outcome, and the thumbnail, missing
storage, expired reads, unknown buckets and write failure paths
are covered by tests. Label coercion goes through the shared
metrics helper.
Closes#11676
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Harden storage metrics accuracy and coverage
Address the full-branch review findings on storage metrics.
Touch and delete emit only on changed rows, reads emit after the
backend fetch, unknown backends fail explicitly, and tempfile
mismatches count as unauthorized. Publisher nil policy, pairing
rules and attempt semantics are documented and covered by tests.
Closes#11676
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Address full-branch review findings on storage metrics
Touch and delete resolve labels from the row, reads stay paired,
failures are covered by tests, and logging, ranges and docs are
tightened. Includes the label helper unit tests and the retries
wording clarification.
Closes#11676
AI-assisted-by: muse-spark-1.3-contributor
* ⚡ Label touch and del metrics from UPDATE RETURNING
The storage metrics change resolved metric labels for touch-object!
and del-object! with an extra SELECT per id-based call. Since app.main
instruments storage unconditionally, every GC collector and binfile
import paid that extra round trip: deleting a team with 10k media
objects doubled the storage_object statements exactly on the paths
that already process the most rows.
touch-object! and del-object! now take only the object id (UUID) and
read the labels from the updated row itself via RETURNING id, backend,
metadata: one statement, no pre-read, and labels that always match the
row actually mutated. del-object! additionally guards on deleted_at
IS NULL, so a repeated delete returns false and emits no metric.
Also from the review of the full branch: extract the duplicated
serve/emit/rethrow block in app.http.assets into one helper; give
penpot_storage_s3_timing explicit histogram buckets up to 60s (the
default cap at 7.5s hid the slow S3 calls the metric exists for);
drop the unused ::target-id config key from the S3 backend and
hardcode the :default target label until per-bucket routing lands.
AI-assisted-by: glm-5.3-flash
* ✨ Harden storage metric recording and definitions
The metric definition schema is now closed and declares every key
the collectors read: buckets, quantiles, max-age and reg. A typo
such as a misspelled ::mdef/buckets used to compile and silently
fall back to the default histogram buckets; it now fails at
startup.
The asset result-label fallback coerced an absent status to 500,
so a future serve path without a status would have counted
successes as errors. The mapping is now explicit and documented:
served below 400, unauthorized for 401/403, not-found for 404,
and error for everything else, including an absent status.
The never-fail try/catch around metric recording existed four
times with drift. One app.metrics/run-safe! helper replaces them:
it no-ops on a nil metrics instance and logs the first failure
per hint at warn level, then at debug, so a broken setup surfaces
once without flooding the log. The S3 publisher keeps its outer
try/catch: it is the SDK MetricPublisher contract boundary.
AI-assisted-by: glm-5.3-flash
* ✨ Make metrics mandatory and run! safe by default
Recording a metric must never change the behavior of the operation
being measured, so `run!` now catches recording failures itself: the
first failure per metric id logs at warn, later ones at debug. This
replaces the `run-safe!` helper, whose four copies had drifted, and
applies the guarantee to every emit site instead of only storage.
The metrics instance precondition is a plain assert, and the collector
lookup stays outside the recording guard, so a missing instance fails
hard even when asserts are disabled. Metrics is therefore no longer
optional: the storage, s3-backend and db-pool schemas require
`::mtx/metrics`, and the assets handler cfg always carries it.
`wrap-publisher` no longer returns nil for a nil instance, and the db
pool wires the prometheus tracker unconditionally.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Send notification email on password change
Add a password-changed email template and send it after every
successful password change or first password set operation. This
gives users a visible signal when a new authentication factor is
established on their account.
Closes#11392
AI-assisted-by: longcat-2.0
* 🐛 Address code review feedback on password notification
- Send password-changed notification from recover-profile too
(forgot-password reset path was missing the email).
- Strengthen test assertions to verify email factory, recipient,
and name via :call-args-list instead of just call-count.
- Add negative test: no email sent when old-password is wrong.
- Wrap pre-existing update-profile-password test with send! mock
to keep its scope focused.
Ref: PR #11393
AI-assisted-by: longcat-2.0
