* ✨ Count the application logs as a metric, by logger and level
The error panels had no metric to read: they counted log lines in Loki,
because the backend published nothing about the errors of the application
(the only thing alive in Prometheus was the http listener error count,
which measures another thing). Now there is
`penpot_log_events_total{logger, level}`.
The counter comes from a new component, `app.loggers.metrics/reporter`,
which hooks a watch on the log record like the database and mattermost
reporters do, but without a thread or a channel: the increment is in
memory, so it runs in the watcher itself. Only `warn`, `error` and `fatal`
are counted, and only the records that go through `app.common.logging`:
the lines a library writes straight to log4j2 are not counted (one in a
thousand today).
The body of the watcher is wrapped in a `try/catch` that swallows the
error on purpose: the watcher fires from the `swap!` of the log record,
inside the logging path, so an exception there would reach whoever was
logging and would take the logging executor down with it. A test pins that
a failing publication does not propagate.
The suite removes the reporter (it would count the logs of the tests and
the recording of their `mtx/run!`), so a test of its own pins the wiring.
The memories carry the counter, its two limits and why the `try/catch` is
not noise; the grafana one also brings the notes of this afternoon about
the service account, the dashboard search and Cloudflare, which were
waiting for a commit.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Address the review findings on the log-events metric
The review of the counter found that the reason written for the
`try/catch` was not the real one, so the memory and the comments say now
what happens if it is removed: with `::l/sync?` the exception reaches
whoever was logging, and on the async path it dies in the uncaught
handler of the logging executor -- outside the application's logging --
where the pool replaces the worker, so logging continues and that line is
lost. They also say that a publication failing every time is silent: the
counter stays flat and nothing is written about it.
The rest are small things the review asked for: the test of the `fatal`
level (it was in the counted set and nothing verified it), the HELP of the
metric naming the three levels, the require of the test without the unused
alias that clj-kondo flagged, and the note that the `level` is lowercase
here and uppercase in Loki.
The grafana memory loses what was said twice: the two pairs of bullets
about reading the API (the service account 403s and the search that lies)
are merged, the CLIMIT notes are one bullet instead of two, and the four
bullets about editing dashboards are grouped in their own section. Its
host name is gone as well, which the first line of that file forbids.
AI-assisted-by: deepseek-v4.1-flash
Port staging c74d415 (#12252) to main, adapted to this branch.
The :export-shapes modal event carried the full :exports vector,
each export embedding its shape, which could reach megabytes and
wedge the audit archive queue past the proxy body limit. Build a
compact body instead: totals, distinct shapes and per-type counts,
plus the page name. Other modals keep their props.
Move the modal :origin prop to ::ev/origin so make-proto-event
records it as :event-origin; the UI still reads the plain prop.
Bound exception messages at the source: get-hint clamps to
max-message-length (1000 chars) and the CLJS format-throwable
clamps the trace message line and honors :trace-length. The
Context Hint header in generate-report prints get-hint instead
of the raw hint. Also fix first-line on the JVM, where
cuerdas/index-of returns nil and (pos? nil) throws.
Adaptations to main: skip audit-log.md (absent here), skip the
generate-report huge-message test (staging async helpers are
absent), and fit the errors.cljs hunk to main's generate-report.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Add async test helpers and the with-mocks* macro
Backport of the test scaffolding that the persistence retry work needs,
taken from staging. Purely additive: no existing helper changes behavior,
so the suite result is unchanged (740 tests, 0 failures).
- frontend-tests.helpers.async: ->promise, await-response, settle,
observe and wait-for. wait-for checks immediately and then polls one
macrotask at a time, bounded, so it fails instead of hanging.
- mock/with-mocks*: installs mocks with set! and evaluates its body in a
generated ^:async fn, so the test reads sequentially and there is no
done to thread. Evaluates to a promise.
- mock/asap and mock/run-mocked: run-mocked awaits the body and restores
the originals always, in that order, reporting a rejection or a
non-promise return as an :error.
The matching change that made with-mocks defer its body via asap is NOT
backported: it would force all 58 existing uses across 14 files into
done-chained contexts, which is a separate refactor.
AI-assisted-by: space-bunny-free
* ✨ Add an error report governor with compact payloads
One browser session could emit an unbounded number of identical error
reports, and a dropped connection was reported as an application defect
with the full payload. Both are fixed here, in one place, because the
persistence retry work that follows depends on the new submit-report
contract.
- A report governor in app.main.errors. Each report carries a
fingerprint (report-name|type|code|hint|first stack frame); the first
occurrence is always emitted, repeats inside a 2 minute window are
counted, and the next emitted report carries them as :occurrences. The
cache is bounded by evicting the oldest entry, so memory stays fixed.
- flash reserves the report before generating it, so a suppressed
occurrence never pays the report-building cost. Its whole body now
runs inside one ts/schedule callback, a reporting or notification
failure is logged instead of propagating, and it returns a total
promise so tests can await it.
- generate-report accepts :format :compact|:full. The compact payload
keeps the context header plus type, code and uri, and skips the
stack, the ex-data dump (it can carry request headers) and the
last-events list.
- environment-error-types is derived from rp/retryable-types, so the
transport classification has a single source of truth. Environment
failures are audit-only handled-exceptions and never reach the
exception page or the alerts. :offline gets its own handler instead
of falling through to :default.
:occurrences travels inside the audit event :props, which the backend
schema accepts as a free-form map, so no backend change is needed.
The dedup governor test file is new: 7 tests over the fingerprint, the
pure reservation decision, and the flash pipeline.
Relates to #11726, #11743
AI-assisted-by: space-bunny-free
* ♻️ Answer commit refusals with an empty stream
commit-changes used to return nil when it refused to build a commit,
which forced every caller that wanted to observe the outcome to
special-case nil. It now returns an empty stream, so a refusal and an
empty result look the same and callers can subscribe uniformly.
Only the refusal branch changes: the accepted branch already answered
with (rx/of commit). Potok treats a nil watch result as "nothing", so
both spellings were already correct at the store level.
The 57 call sites that wrap this event in rx/of are unaffected. They
hold the ::commit-changes event itself, not the value its watch
returns, so they keep processing exactly as before.
AI-assisted-by: space-bunny-free
* ✨ Retry transient saves with backoff and a reconnect notice
Any failed update-file landed on the same path: the queue halted in
:error, the retained changes were flushed away, and the user was told
to reload. A dropped connection and a revoked permission are not the
same thing, so they now take different paths.
transient-error? splits them, checking :type and :cause-type because
persistence-failed wraps the original cause. Transient failures enter
a :retrying episode: the head commit stays queued and resends with
backoff at 2s, 8s and 20s, then falls through to today's terminal path.
Everything else is terminal and keeps the exact :error path.
- Resends rotate the ::request-id stamp only once the old request has
left active-requests, carry the same :commit-id, and never double
send while one request is in flight.
- Retry timers carry an episode token; a timer whose episode was
superseded stays silent instead of sending or failing.
- One tagged reconnect notice per episode, hidden by tag on save and on
terminal failure. Recovery is silent.
- The browser online event re-enters the runner, but only for a live
:retrying episode, guarded by exists? js/window.
- wait-persisted-or-error waits through :retrying instead of cutting the
wait short.
The episode keeps its :run-id, so a new local edit only joins the queue.
Re-entering the runner on every edit would bypass the backoff and spend
one attempt per batched commit.
The retry test namespace is new: 13 tests over the classification, the
backoff schedule, stamp rotation, stale tokens and the notice lifecycle.
Relates to #11724
AI-assisted-by: space-bunny-free
* 🐛 Fix stalled saves that never answer and edits lost on leave
Two holes the retry work did not close.
An update-file request that never gets an answer kept its attempt
stamp forever, so the queue could not move and the user saw a saving
state that never resolved. The request now has a 7 minute timeout that
fails it as a transient network error, which hands it to the retry path
instead of blocking. The timeout is longer than the 5 minute stall
threshold on purpose: the stalled report must fire before the retry,
not instead of it.
The leave-page warning also had gaps. It now tracks unsaved edits
directly, through sync-unload-guard, which holds :persistence in
st/ongoing-tasks while edits sit in the buffer or the queue. Edits kept
in the queue after a failed save keep the guard on too, so closing the
tab warns instead of dropping them.
store.cljs guards its ongoing-tasks watcher with exists? js/window,
because the new tests exercise it without a window.
Four new tests: three for the guard across buffering, an in-flight save
and a retained edit, and one for a hung request that must resend and
save.
Closes#12223
AI-assisted-by: space-bunny-free
* 🐛 Skip the save when a missing font cannot be remapped
fix-deleted-fonts emitted text and typography changes even when no
replacement font existed, so opening a page that uses a deleted custom
font rewrote identical content and cleared position-data on every
visit. Every visit became a save with nothing to save.
Both loops now compare the remapped value with the original and emit
nothing when they match. The remapped path is unchanged.
Two new tests cover the no-replacement case and the remapped case.
Closes#12151
AI-assisted-by: space-bunny-free
* ✨ Add a debug helper that simulates an intermediary response
A save that never reaches the backend is hard to reproduce: you need a
proxy, a captive portal or a CDN in the way. This wraps fetch and
answers the named API method with a status and a body that carries no
Penpot error code, leaving every other request alone.
Without a duration the simulation is spent after answering one call.
With one it answers every matching call for that long, which is how to
watch the save retry, warn, keep retrying and finally recover.
debug.simulateHttpError('update-file')
debug.simulateHttpError('get-comment-threads', 403)
debug.simulateHttpError('update-file', 524, 120)
debug.clearHttpErrorSimulation()
Dev-only and self-contained: no production path reads it.
AI-assisted-by: space-bunny-free
* 🌐 Add the save retry and connection error strings
Three strings the new persistence and error paths need:
- errors.save-retrying, shown once per retry episode while the queue
resends a queued save after a connection loss.
- workspace.header.retrying, the save indicator state for an episode
in progress.
- errors.connection-error, shown by the network and offline handlers,
which no longer fall through to the generic message.
Added to en.po only. The translations script also normalizes every
other locale and drops 439 strings that were already stale before this
work; that belongs in its own maintenance pass, not in a backport.
AI-assisted-by: space-bunny-free
* 🐛 Report save failures as unhandled when they are our defect
flash-persistence reported every failure it did not delegate as
handled-exception, including the ones that are plainly our fault. A
validation failure means the client sent something the server refused,
yet it landed in the audit log with a name that skips the internal
reports and the alerts. Nobody was paged.
The report type now follows the cause. :validation, :server-error and
:internal report as unhandled-exception, so they alert. The rest are
unchanged.
Two exclusions are deliberate:
- Environment failures stay handled. They reach this point after the
retry budget is spent, and the user's connection is not our bug.
- :assertion stays handled, because its own handler already reports it
that way and the two paths should not disagree.
The cause type is read from :cause-type first, since persistence-failed
wraps the original under :type :persistence. Reading :type alone would
match the wrapper every time and change nothing.
The user still gets the same sticky toast and report link; only the
audit event name moves.
The test that asserted a validation failure is "still notified" now
asserts which event name it gets, across the bare and wrapped shapes,
the environment types that must stay handled, and an unclassified one.
AI-assisted-by: space-bunny-free
* 🐛 Unblock persistence when opening a file after a terminal failure
A terminal save failure set the state to :error and left the queue
alone. recover-persistence only resumed for :save-permission-denied, so
for every other terminal code both of its branches were false and it
emitted nothing. Leaving and reopening the same file therefore changed
nothing: the editor stayed halted until a reload, which rebuilds the
whole store and loses the queue anyway.
The path had two jobs under one name. initialize-persistence fires when
the workspace view mounts and tears down the previous subscription;
recover-persistence fires when a file is opened or the role changes.
They are now separate: initialization picks up a queue left behind by a
previous session, and deliberately does not lift a halt, so undeliverable
edits are released when the user acts rather than when the view mounts.
Recovery lifts the halt by releasing what can never be saved.
release-failed-file drops every queued commit of the file that failed,
then resume-persistence clears the error and settles an empty queue to
:saved. Retaining them until that point is deliberate: it is what keeps
wait-persisted-or-error rejecting, which is what stops
workspace/versions.cljs from building a snapshot out of unsaved edits.
Releasing at failure time would make waiters resolve instead.
Permission loss is treated as permanent too, which the previous
exception got wrong. The backend raises :not-found for a missing edit
permission, so a resend does not succeed — it routes the user to the
exception page. The old branch made that reachable: it resumed the save
whenever the cached can-edit flag said yes, and the server disagreed.
The only case where the flag can return to true is a team switch, where
the queue is gone anyway, so nothing real is lost by releasing.
Because the queue is shared, dropping the failed file is also what lets
the other files' pending edits drain. One file that cannot be saved no
longer blocks everything else.
Tests: the failure retains the queue and notifies once, recovery
releases it without replaying, a failed file does not block a second
one, permission loss is released rather than resumed, and initialization
leaves a halt alone.
AI-assisted-by: space-bunny-free
* ✨ Offer a reload on the save failure toast
A terminal save failure stops autosave for the whole session, but the
toast asked the user to reload without offering the reload. It listed
five implied actions, none of them reachable from the banner, and it
never said that the changes could not be saved.
flash now takes a :links vector of actions, appended after the download
link that :report-link? produces, so a caller can add recovery actions
without reaching into the report handling. flash-persistence passes a
reload built from labels.reload-page and location.reload, the pattern
already used elsewhere in the app for a full restart.
The message now says what happened, what is lost, and what to do about
it: autosave has stopped for this session, the changes since the last
successful save cannot be saved, and either reloading or leaving the
file and opening it again restarts it. Naming both matters, because the
toast links are easy to miss.
Saying the edits are lost is not optional: recovery releases them, so
without it the user finds out when they come back to the file.
Translated in en.po and es.po only. The translations script would also
normalize all 40 locales and drop entries that were already stale, which
is a maintenance pass of its own.
The reload callback cannot be invoked under the test runner, since
js/location does not exist there, so it is covered by its label and by
being callable. Whether the reload works is a manual check.
AI-assisted-by: space-bunny-free
* 🐛 Classify the responses the backend never wrote
A reply written by a proxy instead of the backend carries no Penpot
error code, so handle-response raised it as :internal. A 504 on a save
skipped the retry cycle and stopped autosave at once, and because
:internal reports as unhandled, it raised an alert. In a standard
deployment this is how a slow save ends: nginx answers 504 after 300s,
before the client's 7 minute timeout fires.
The status now decides the type:
- 504 and the Cloudflare 520-524 family raise :gateway-error, which
joins retryable-types. Saves retry with backoff like 502 and 503,
GETs retry, and a save that spends its retries reports as handled,
because the report rules derive from the same set.
- 429 raises :rate-limit, and any other error status
:unexpected-response. Neither retries; both show a toast instead of
the error page.
- :internal is kept for a response below 400 that does not decode.
Bodies we cannot read are cut to 500 characters, since proxy error
pages run to tens of kilobytes.
Tests: classification and retries per status, the transport handlers
keep the page, a 504 and a 524 built by the real handle-response go
through the save path and land on a retry, and the new types stay
handled once the retries are spent.
AI-assisted-by: claude-opus-5-5
* ✨ Keep retrying failed saves on a slow cycle
Once the three backoff retries were spent, a save that failed on the
connection stopped autosave like any terminal failure, and opening a
file again dropped its edits. A connection that comes back after a
minute lost the work all the same.
A transport failure that outlasts the backoff now arms a slow cycle:
one silent resend every 30s, until a save lands or 24h have passed
since the failure started. The cycle stops when persistence restarts,
the queue resumes another way, or the workspace closes.
A save that keeps failing slowly is not resent for a day. A gateway
that answers 504 after its own timeout may leave the backend applying
the save, so each resend can cost the whole save again, and the failing
file holds back every other file's queued edits. An attempt that fails
after 90s or more counts as a slow failure; at three, the slow cycle
stops sending the commit and the save fails as before. Fast failures,
such as being offline, never count.
- The first failure shows a toast that says the changes are kept and
keep being sent, with no reload link, since a reload loses them. A
save that lands hides it.
- Opening a file sends a queue still inside its retry window once
more instead of releasing it. Failures outside the window, and every
non-transport failure, are still released.
- A freshly loaded file gets back the queued edits of its version, so
reopening it shows the changes that are still waiting to be saved. An
edit that no longer applies to the loaded data leaves the file as the
server sent it, so the file still opens; the edits stay queued.
- A commit whose earlier request is no longer in flight is simply sent
again under the same commit id: changes are idempotent, so there is
no unknown-outcome guard any more. A request still in flight still
blocks a parallel send.
flash takes a :tag so the save failure toast can be hidden by tag.
Tests: the spent backoff arms the slow cycle, a 504 arms it, a terminal
failure does not, the cycle stops when the window closes or at the slow
failure limit, recovery resends instead of releasing, reopening restores
only the loaded file's edits and survives one that no longer applies,
and the retrying toast offers no reload.
* 🐛 Turn off the leave-page warning when failed edits are released
The leave-page warning only followed the queue on appends and discards.
Opening a file again releases the edits of a file that failed for good,
but that release changed the queue without either event, so the warning
stayed on with nothing left to save. Leaving the page asked about
unsaved changes until the next save went through.
The warning now also follows the release.
Tests: releasing the last failed edit turns the warning off.
* 🐛 Stop retrying failed saves outside the workspace
Leaving the workspace was meant to stop the slow retry cycle, but only
a cycle already armed stopped. A backoff timer kept firing after the
user left, and a backoff spent outside the workspace armed a slow cycle
that kept sending for up to a day from the dashboard. Those sends check
the edit permission of whatever team is open, so a switch to a team
where the user only views failed the queued edits for good, and the
next file they opened dropped them.
Leaving the workspace now suspends retries. A pending backoff timer
goes stale, and while suspended a transport failure schedules no
backoff, a spent backoff arms no slow cycle and the online event sends
nothing. The edits stay queued and a request in flight still lands.
Opening a file lifts the suspension and resends a retrying save whose
backoff was cut short; a role change does not lift it.
Tests: leaving stops a pending backoff, a failure after leaving
schedules no retry and ignores the online event, a backoff spent
outside the workspace arms no slow cycle, and opening a file saves the
held edit in each case.
---------
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Add a check-commit-guidelines job to commit-checker.yml
that runs scripts/check-commit on every new commit, so
the Co-authored-by ban holds in CI too. PR events check
out the base ref, keeping the executed script trusted.
AI-assisted-by: muse-spark-1.3-contributor
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The Co-authored-by trailer is now forbidden. The
commit checker fails any message holding it, whatever
the case, and the docs record the ban.
AI-assisted-by: muse-spark-1.3-contributor
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
State in the contributing guide that issues under `needs triage` and
`internal` are not open to contributions. Every new issue starts under
`needs triage` until the product team reviews it, so contributors could
mistake a fresh issue for an invitation and send a PR that we then turn
down.
A PR opened on one of those issues may be closed without review,
without an explanation and without a reply. The wording follows the
existing note about AI-assisted content.
The help center page keeps its current wording for now.
AI-assisted-by: deepseek-v4.1-flash
Repair scripts run against production with a plain SQL client, not the
devenv psql: no `\set`, `\if` or `\copy` there. The SQL datasource the
agent can query is a read-only replica that rejects writes (a temporary
table included) and cuts long queries, so its queries are plain SELECTs
bounded by a list of ids: an unbounded scan over tens of millions of rows
does not come back.
AI-assisted-by: deepseek-v4.1-flash
The queue gauge was written by whichever limiter of a name ran last, so a
per-profile pile-up never showed: a healthy profile writing 0 overwrote the
15 of the profile that was queueing (4 of 1441 samples in production), and
a limiter whose requests never come back read 0 while 72 threads stayed
parked.
A queue is an entry and an exit, so the new gauge is accumulated: one
counter per limiter name and instance, up when a call starts waiting for a
permit, down when it stops, so a call that never leaves stays visible. It
is always written with `:val`, and a daemon sampler republishes every count
every 15s so a name whose traffic died does not go stale in the scrape;
halting the component stops that thread.
Rejections had no metric at all: penpot_rpc_climit_rejected_total counts
them by name and reason (queue-full or timeout), and the log lines now
carry queue-max, permits, code and reason, so an operator can tell whether
the limiter did its job or was saturated.
The gauges that could not tell the truth are gone: queue and permits (same
last-write-wins problem, and waiting plus the wait histogram replace them)
and http_worker_busy_threads (reads 0 on an instance with 72 blocked
threads; pool_size is the signal). No other dashboard or alert rule reads
them.
AI-assisted-by: deepseek-v4.1-flash
Add a per-process availability flag (app.health) that /readyz
reports as 503 OFFLINE while offline, so the load balancer
drains the instance and in-flight requests finish untouched.
Operator surface is app.srepl.main/set-offline and
app.srepl.main/set-online; a restart comes back online.
Covered by backend-tests.health-test; HA docs updated.
AI-assisted-by: muse-spark-1.3-contributor
`file_object_thumbnail` was replaced by `file_tagged_object_thumbnail`
and nothing refers to it anymore: no query in the backend touches it,
no table has a foreign key pointing at it, no view depends on it, and
it is empty in production. It also lacked `deleted_at`, which is why
restoring files had to leave it out. Only legacy naming survives in two
helpers (`get-file-object-thumbnails` and
`delete-file-object-thumbnails!`, which operate on
`file_tagged_object_thumbnail`).
The storage bucket named "file-object-thumbnail", the one that
`file_tagged_object_thumbnail` uses, is a different thing and stays.
AI-assisted-by: deepseek-v4.1-flash
Restoring a file only un-marked the database rows (`file`,
`file_media_object`, `file_change`, `file_data`, `file_thumbnail`,
`file_tagged_object_thumbnail`), not the storage objects those rows
reference. An object that was already marked for deletion stayed
marked while a live row pointed at it, and those foreign keys are `NO
ACTION`: the storage GC cannot delete it and skips it forever (and
before the GC learned to skip, it cost the whole nightly run).
Restoring a team had the same gap with its font variants.
Both restores now unmark the storage objects of the rows they bring
back: the media and the thumbnail of the file media objects, the media
of the file and object thumbnails, the `storage-ref-id` of the file
data, and the four files of a font variant. Objects referenced by other
rows are left alone.
AI-assisted-by: deepseek-v4.1-flash
objects-gc deleted every kind of object inside one transaction per proc,
looping until empty, so a single row that a foreign key still references
aborted the whole run: the task died on its first file every night, after
the earlier steps had already spent around twenty minutes. The per-file
savepoint only saved the files proc and left the same shape everywhere
else.
Now every proc streams its pending keys with db/plan inside a read-only
transaction of its own and deletes them chunk by chunk, each chunk in a
transaction of its own with FOR UPDATE SKIP LOCKED. A chunk that fails is
reported with l/err and skipped, so the run reaches the end and the next
run retries the skipped rows.
AI-assisted-by: muse-spark-1.3-contributor
`storage-gc-deleted` ran its whole batch inside one transaction: the
handler opened it and `delete-in-bulk` reused that connection, so the
"separate transaction" its comment promised did not exist. A single
row that a foreign key still references aborted that transaction, the
cursor then died on its next fetch ("current transaction is aborted")
and the task ended up throwing `unhandled exception on running task`
after rolling back the whole night of work. That is what has kept
production from deleting a single object since late April.
Now `clean-deleted` walks the pending objects with the cursor in its
own read-only transaction and every chunk runs in its own one, so a
blocked chunk is logged with its ids and skipped while the rest is
committed. `task finished` is reachable again.
The private functions also drop the `!` suffix: the namespace is all
mutation, so the mark adds nothing and reads as noise.
The test builds a blocked object (a `file_media_object` pointing at it
as its thumbnail) plus a full chunk of pending objects and one more in
the next chunk, and checks that the run finishes and that the object of
the next chunk is deleted.
AI-assisted-by: deepseek-v4.1-flash
The four `process-deleted-*-cascade` helpers of `app.srepl.main`
loaded every soft-deleted row of a table into memory (`db/exec!`)
and ran `:delete-object` inline (`wrk/invoke!`), so they could not
be used against production: `file` alone has 2.1M deleted rows.
They now share a single `submit-deleted-cascade!` that reads the
rows with a server-side cursor (`db/plan`, in a read-only
transaction) and submits one task per row, so the work goes
through the workers like any other task. The row id goes as
`label` with `dedupe` on, and the helpers accept `:limit`, `:ids`
and `:dry-run?` to run bounded batches. Public names and their
zero-arity form are unchanged.
Tests cover the task params, the row's own date (not `now`), the
`limit`/`ids`/`dry-run?` options and that each helper reads its own
table.
AI-assisted-by: deepseek-v4.1-flash
An instance left the audit events pipeline dead: every push-audit-events
answered with 429 and 69 worker threads parked for good inside a limiter.
Two code paths of one profile acquired the same two limiters in opposite
order: a command declaring its own limits (upload-file-media-object) took
process-image/by-profile and then root/by-profile, while a command with no
limits of its own took the roots first and then re-entered climit from
inside its handler with the same ids (create-file-media-object-from-url).
Each path held what the other one waited for, and no limiter had a timeout
to break the wait.
Now wrap puts the root limiters after the command's own, so they end up
outermost and every path meets them first: the nested chains carry only the
command's own limits and run inside them, which puts both paths in the same
order. impl-invoke also keeps, per thread, the set of limiters that thread
is inside of and runs the handler inline when a call asks for one it
already holds, so a handler can no longer queue behind itself.
The shipped climit.edn sets a timeout on every limiter (60s for operations
that can be slow, 30s otherwise), so no wait can park a thread for ever.
The production file lives outside the repository and has to be mirrored.
Tests cover the acquisition order, the deadlock, the reentrancy and the
timeouts of the shipped config.
AI-assisted-by: deepseek-v4.1-flash
The command declared its limit as a plain keyword next to
`::climit/key-fn`, metadata that `climit/wrap` does not read: it
takes the key only from the positional `key-fn` of the vector form.
The limit was a single shared one, so when one call got stuck it
held the only permit and every audit batch of that instance was
rejected with 429 from then on.
The test builds the limiter from the command metadata and checks
that two profiles end up with different limiters.
AI-assisted-by: deepseek-v4.1-flash
The footer had a hardcoded 2025 copyright and links that no longer match
penpot.app. The year is now set at build time. Sections, links and
social icons follow the current penpot.app footer, and the grid wraps
into two rows to fit the new sections.
Closes#12132
AI-assisted-by: claude-opus-5-5
Check a milestone against the Main project board with
scripts/project-anomalies.py (check writes
tmp/<MILESTONE>-ANOMALIES.md): open issues with merged PRs,
milestone mismatches either way, needs triage labels and
unassigned non-community issues. Outsiders resolve in bulk;
only a missing milestone auto-fixes, on confirmation.
Teach scripts/gh.py batched issue lookup (issue subcommand),
milestone on prs output, plus assignees and projects fields,
and retry transient HTTP 504s.
AI-assisted-by: muse-spark-1.3
Move the inline programs out of the update-changelog skill into
scripts/changelog.py (check-merged, cross-ref, report) and shrink
the skill to the workflow alone.
Teach scripts/gh.py to retry transient HTTP 504s, resolve lookups
in batches of 50 (new issue subcommand, milestone on prs output),
and report 💥 entries without the breaking change label.
Add the nine missing 2.17.0 changelog entries found while
trying the new flow.
AI-assisted-by: muse-spark-1.3
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
The S3 file download opens the temp file with CREATE_NEW, so it
fails when the path already exists. tempfile creates the empty
file on reserve, which made every large download depend on the
SDK deleting the placeholder between its own retries.
Add tmp/temp-path, which reserves a unique path and registers
cleanup without creating the file, and use it for S3 downloads.
The old retry on FileAlreadyExists stays as a safety net for a
name clash.
AI-assisted-by: muse-spark-1.3-contributor
Delete the team-scoped file, project and pin relations on leave-team
and delete-team-member, require a live team membership in the file,
project, search and listing permission queries, and clean
pre-existing orphan rows with a migration. Former members now get
not-found on file actions instead of keeping read and rename access
(GHSA-v9r9-h77c-55m2).
AI-assisted-by: muse-spark-1.3-contributor
Drop the equals-sign banners from the CLJS throwable formatter,
so new frontend error reports are born without them. The JVM
branch never printed them. Adds a test asserting full reports
carry sections but no separator runs.
AI-assisted-by: Muse Spark 1.3 Free
* 🐛 Sanitize SVG namespace-prefixed elements and attributes
The SVG sanitizer compared qualified tag and attribute names against
its blocklists. The XML parser keeps the namespace prefix in the
keyword name (❌script), so a script element written with a prefix
bound to the SVG namespace passed sanitization untouched and was
later served as image/svg+xml, executing in the Penpot origin.
Match tags and attributes on the local name (the part after the last
colon) instead, so dangerous elements and attributes are removed no
matter which prefix they use. Legitimately prefixed attributes such
as xlink:href keep working.
Closes#12071
AI-assisted-by: glm-5.3-flash
* ♻️ Cover prefixed SVG vectors and hoist sanitizer blocklist
Apply the code review findings on top of the sanitizer fix:
- Add a test for the javascript: href vector using a prefix bound to
the xlink namespace, the other payload variant the old exact-key
check missed.
- Add a test asserting that legitimately prefixed attributes such as
xml:space are preserved by the local-name matching.
- Hoist the dangerous-tags blocklist next to the other sanitizer
policy defs and bind the keyword name once in local-name.
Relates to #12071
AI-assisted-by: glm-5.3-flash
The event was written twice per accepted organization invitation. The
backend submitted it, and the browser then re-submitted a copy of the
props that the backend had already put in the response under
`:organization-invitation-audit` (`handle-token :team-invitation` in
`verify-token.cljs`). Both rows carried the same name with different prop
vocabularies, and the browser copy only existed when the browser finished
the flow.
Emit the event from the backend only. It now also carries the three props
that lived in the browser copy: the organization member count before the
add, the add source, and whether the invitee also joined a team. The
origin moves to the event context as `:event-origin`. The response no
longer includes `:organization-invitation-audit`, so the browser stops
emitting the event and `verify-token.cljs` drops its
`app.main.data.event` require.
The `accept-*` events of this command now share one prop vocabulary:
`:profile-id` for the accepting profile, `:invited-by` for the inviter
and `:profile-email` for the email, replacing the mix of
`:user-id`/`:user-who-send-invitation` and `:email`.
Audit consumers of `accept-organization-invitation` now see one row per
acceptance instead of two, and must read the new prop names.
AI-assisted-by: space-bunny-free
The json reader of the binfile v3 import rewrites every key of every
entry to kebab-case, nested maps included. Shape `:svg-attrs` is the one
map whose keys are camelCase react prop names, because the svg import
path runs them through `attrs->props`, so an attribute exported as
`fillRule` came back as `:fill-rule` and was stored that way.
The renderer looks the attribute up by its camelCase name, does not find
it and falls back to the default fill rule, so a shape exported with
`fillRule: evenodd` was painted without its hole, and the attributes
panel showed `fill-rule`.
`clean-shape-post-decode` already runs on every shape right after the
schema decode, for page shapes and component objects alike, so the
repair goes there: run `:svg-attrs` back through `attrs->props`, the
same transform that built the keys. It is idempotent, so shapes that
arrive correct are left untouched.
The new tests import a real export that carries the attribute, for
page shapes and component shapes.
Closes#11954
AI-assisted-by: space-bunny-free
* 🐛 Attribute audit events to the caller, not the response
prepare-rpc-event took the event profile-id from the result map
whenever it carried one, before falling back to the caller. Any
command returning a response with a :profile-id key silently
credited the action to somebody else.
get-error-report returns the report with its decoded content
merged in, and that content holds the profile that owned the
report, so privileged reads were logged against the users whose
crashes were being inspected. verify-token on a team invitation
returns the inviter's profile-id, so accepting an invitation was
logged against the inviter.
Resolution is now ::audit/profile-id metadata, then
::rpc/profile-id, then the zero uuid; the response is never
consulted. The two verify-token branches that relied on it now
declare the profile in the result metadata. Every other command
either already declared it or returns no :profile-id; all 30
registered command namespaces were checked.
The tests that pinned the old behavior are replaced by ones
covering the new contract.
AI-assisted-by: space-bunny-free
* 🐛 Coerce the audit profile-id override to a uuid
The only sanctioned way for a command to override the profile of an
audit event is the ::audit/profile-id metadata, and the value is set
by hand in a dozen commands, some of them reading it from token
claims or other sources we do not type.
schema:event requires a uuid and submit* swallows the validation
error, so a string did not fail loudly: the row was dropped silently.
Values that cannot become a uuid are now discarded with a warning
and the event falls back to the caller, which is always a valid uuid.
A uuid, the common case, exits on the first check.
AI-assisted-by: space-bunny-free
Requesting a binfile export with both `include-libraries` and
`embed-assets` enabled threw a raw `IllegalArgumentException`, which the
RPC layer surfaced to API clients as a generic
`:server-error`/`:unexpected` (HTTP 500) with no actionable code -- the
opaque failure reported in #7649 (perceived as an "empty" export).
The two options are genuinely mutually exclusive, so the request is
invalid input, not a server fault. Replace the raw throw in both the v1
and v3 export paths with `ex/raise :type :validation :code
:incompatible-options`, matching how the rest of the binfile code
reports bad input. Also fix the "mutally" -> "mutually" typo in the hint.
Verified on a local devenv: the export RPC now returns
`{:type :validation :code :incompatible-options}` instead of a 500, and
valid single-option exports are unaffected. Adds a regression test.
Fixes#7649
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
Add a link-issue command that creates GitHub's Development reference
and verifies both sides. Keep Closes in descriptions for context, but
make the API link the source of truth, including for merged PRs.
Add tests for successful links, missing verification, output, and
failures. Update the PR workflow memories and create-pr skill to use
the command.
AI-assisted-by: space-bunny-free
Remove the legacy @opencode-ai/plugin import and V1 server export so
the local plugin loads under OpenCode V2 without project dependencies.
Add a focused smoke test for the V2 export and tool registration, and
update the related script memories.
AI-assisted-by: space-bunny-free
A compose port mapping delivers traffic to the container address,
never to loopback, so `listen 127.0.0.1:8082` made
`ports: <host>:8082` fail from the host. Both configs (image
template and devenv) now use `listen 8082`, which binds every
interface and matches the implicit bind of the public
`listen 8080 default_server`.
Rewrite both block comments to state the new bind and who decides
access from outside the host. The scrape URI stays on
127.0.0.1:8082: it still reaches the socket.
AI-assisted-by: mimo-v2.6-flash-free
* 🐳 Add upstream diagnostics to nginx access log
Enrich every access-log line with the internal journey of the request:
the status the backend answered (us), the time spent connecting to it
(uct), the time spent waiting for its answer (urt) and the internal
address that served the request (ua).
A plain 502 line used to say nothing about where the request died. With
this format, the tail of the line classifies the failure: connection
rejected, backend accepted and hung (uct + urt under 1s), or backend
stuck until read timeout. This was the missing witness in the Sep 20
incident, where nginx received connection resets with zero timeouts and
zero rejections.
Applied both to the production image template and the devenv config.
With proxy_pass on variables there is no upstream keepalive, so uct
measures one real TCP connection per request.
Parsing the new fields (us, uct, urt, ua) on the log shipper is left to
ops, so they can be filtered in Loki.
AI-assisted-by: glm-5.3-flash
* 🐳 Add stub_status endpoint for nginx metrics
Add a dedicated localhost-only server (listen 127.0.0.1:8082) exposing
/stub_status next to every other location of the public server. Ops can
run the official nginx-prometheus-exporter as a sidecar against
http://127.0.0.1:8082/stub_status and get nginx_connections_active,
accepted vs handled, reading/writing/waiting and request rates in
Prometheus.
Binding it to localhost and its own server keeps it unreachable from
outside the host and out of the public surface, and access_log off
avoids polluting Loki with one line per Prometheus scrape. The base
image already ships stub_status compiled in, so no image rebuild is
needed.
Applied both to the production image template and the devenv config.
AI-assisted-by: glm-5.3-flash
* ✨ Expose http server gate metrics (worker and connector)
The backend already measured dispatch latency but nothing reported the
state of the "house door": the xnio worker queue and threads, and the
monitor-level listener counters. This was the exact blind spot of the
Sep 20 incident, where the server kept answering health checks while it
accepted connections and dropped them without response.
Add a periodic metrics sampler that lives and dies with the http
server (single daemon thread, 15s interval, each sample guarded so an
unexpected error does not cancel subsequent runs) and publishes:
- worker (xnio MXBean gauges): penpot_http_worker_queue_size,
busy_threads, pool_size and max_pool_size. Negative samples are
discarded: the MXBean transiently reports -1 on the busy thread
count (verified live), and a stale negative would read as zero.
- listener (Undertow connector statistics, enabled via the new
:server/statistics yetti option): penpot_http_connector_active*
_connections gauge and requests_total / errors_total counters.
Undertow exposes absolute totals, so the sampler keeps a watermark
atom and publishes deltas, skipping (and moving forward past) a
counter reset.
The connector-level part depends on yetti v11.11, which now accepts
a :server/statistics server option (patch authored and released
upstream; before it, ListenerInfo#getConnectorStatistics always
returned nil).
New tests cover the samplers with fake MXBean/collector statistics
against real prometheus collectors, including the negative-sample
filter, the delta/watermark logic and the sampler lifecycle.
AI-assisted-by: glm-5.3-flash
* 🐛 Include jdk.management in the backend runtime JRE
The production image builds a trimmed JRE with jlink and omitted
jdk.management. Without that module the OS MXBean is
sun.management.BaseOperatingSystemImpl, which has no
getProcessCpuTime, getOpenFileDescriptorCount nor
getMaxFileDescriptorCount. The prometheus client StandardExports
reads those getters reflectively and collect() swallows the
NoSuchMethodException, so process_open_fds, process_max_fds and
process_cpu_seconds_total silently disappeared from /metrics while
the other process_* families kept flowing.
Verified against Prometheus: the app job only ever exposed
process_start_time_seconds, process_virtual_memory_bytes and
process_resident_memory_bytes; the fd and cpu families were absent.
Reproduced locally by running the backend metrics registry on a JRE
built with the same jlink module list (false/false/false) and on one
with jdk.management added (true/true/true).
Add the module to --add-modules and pin the metric contract with
backend-tests.metrics-test.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Build the http metrics sampler on promesa.exec
Replace the hand-rolled ScheduledThreadPoolExecutor and ThreadFactory
with promesa.exec primitives: px/scheduled-executor with a daemon
thread factory, and a px/schedule chain that reschedules the next
sample when the current one finishes.
Beyond fitting the existing periodic-task pattern (worker/cron,
rpc/rlimit), the chained schedule makes the docstring promise real:
with scheduleAtFixedRate an exception escaping the runnable cancelled
the following executions, while the reschedule now happens in a
finally block.
The sampler shutdown uses px/shutdown-now (shutdown! is deprecated in
promesa 12.0.0) to cancel the pending sample, keeping the previous
halt semantics.
The lifecycle test moves to the promesa predicates and a new test
covers the error-resilience promise: the first sample runs, throws,
and the next one is still scheduled.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Tighten the http metrics samplers
The samplers are leaf functions: they receive what they need and
publish it. Drop the internal nil guards (if there is no metrics
instance or no mxbean there is nothing to call them for) and move the
checks to the boundary, where the optional data is resolved:
sample-http-metrics now short-circuits with some-> and when-let.
Write the four worker gauges as four static operations instead of a
vector of pairs walked by doseq: the set is fixed, so the collection
only adds an allocation and hides each operation.
Drop the ! suffix from the sample-*-metrics family: ! marks a function
whose contract is to mutate state, while these report, and the mutation
happens in the mtx/run! they call. The constant true return, which only
existed so the removed guard tests could assert it, goes away too.
Tests follow the move: the internal-guard tests are replaced by one
boundary test (a nil server publishes nothing).
AI-assisted-by: deepseek-v4.1-flash
* 📚 Add the function design rules memory
Document the rules that came out of the http metrics sampler review:
preconditions are checked at the boundary instead of re-checked in the
core, optional-by-design data is guarded where the optionality is born,
a fixed set of operations is written statically, ! marks mutation and
not reporting, and production code is not shaped for tests.
Also state in the memory maintenance guide that memories must not use
manual line wrapping.
Linked from critical-info so it is read when designing a solution or
an API, not only when touching the samplers.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap the critical-info memory lines
The memory maintenance guide forbids manual line wrapping, so rewrite
critical-info with one line per bullet and paragraph. A stray `*` at
the start of one continuation line is dropped.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the redundant guard in the http server halt
create-metrics-sampler always returns the scheduler, so the sampler is
always present when integrant calls halt-key!; the nil check was dead
code, same as the yt/stop! call next to it.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Add srepl helper to delete profiles by email
Add `delete-profiles-by-email!` to app.srepl.main. It accepts a
single email, a comma separated list of emails or a coll of emails,
resolves each profile, logs it to audit and enqueues the
delete-object task. The deleted-at is backdated with the configured
deletion-delay so profiles and their owned teams are purged on the
next gc pass.
Extract the per-email deletion logic into a private fn and reuse it
from `delete-profiles-in-bulk!`. Add tests for the new
`parse-emails` helper.
AI-assisted-by: glm-5.3-flash
* 🐛 Fix error when reset overrides on a swapped copy
* 🐛 Add regression test for reset overrides inside group
Cover the case where a nested copy lives inside a group (not directly
under the instance root). After a swap, reset overrides must undo the
swap without error (#11656).
---------
Co-authored-by: Alejandro Alonso <alejandroalonsofernandez@gmail.com>
