* 🎉 Add telemetry anonymous event collection
Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:
Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
telemetry flag active, audit-log flag inactive
Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
Keeps UUID/boolean/number values; for login/identify events preserves
lang, auth-backend, email-domain; for navigate events preserves route,
file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
Backend: version, initiator, client-version, client-user-agent.
Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
access-token fields removed from context
Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option
Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
extraction from email addresses
Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)
RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation
RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API
SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys
Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
per iteration, encodes and sends each page, deletes on success,
stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id
Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
safe base64 for JSON-safe string transport
Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
efficient telemetry batch collection queries
Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support
Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
happy-path batch send and delete, failure retention, payload anonymity,
context stripping, timestamp day precision, batch encoding round-trip,
multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add pr feedback changes
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The recursive `read-items` function in `app.util.sse/read-stream`
caused a synchronous stack overflow when reading buffered stream
data. Each `rx/mapcat` call chained another recursive invocation
on the same call stack without yielding to the event loop.
Replace the recursive pattern with an `rx/create`-based async pump
that uses Promise `.then()` chaining, keeping the call stack depth
constant regardless of stream size.
Also add progress reporting with names and IDs during binfile
export and import, and bump `eventsource-parser` dependency.
Closes#9470
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix incorrect invitation token handling on register process
- Reject prepare-register-profile when an active profile already
exists for the requested email.
- Stop embedding an existing profile's :profile-id into the
prepared-register JWE. Profile resolution in register-profile is
now done exclusively by email lookup, never by a JWE claim.
- Add created? guard to the invitation-success branch in
register-profile, so existing profiles (active or not) cannot
reach session creation via anonymous registration.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Restructure invitation handling inside register-profile
Move the invitation-success branch into the created? sub-cond so it
sits alongside the other post-creation branches, making the control
flow consistent.
- Active new profile + matching invitation: mint session and return
:invitation-token (frontend redirects to :auth-verify-token).
- Not-yet-active new profile + matching invitation: embed the
invitation token inside the verify-email JWE and send the
verification email. When the user clicks the link, they get
logged in and the frontend completes the team-invitation flow.
- Extend send-email-verification! with an optional invitation-token
parameter propagated into the verify-email JWE claims.
- Update the frontend verify-email handler to navigate to
:auth-verify-token when the response carries :invitation-token.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Handle email-already-exists error on registration form
Add a specific handler for the [:validation :email-already-exists] error
code in the registration form's on-error callback. The backend raises
this error when an active profile already exists for the requested email,
but the frontend was falling through to the generic error message.
Now it shows the existing "Email already used" i18n message instead of
the generic "Something wrong has happened" toast.
* 🐛 Reset submitted state on registration form error
The on-error handler in the registration form was not resetting the
submitted? state, causing the submit button to remain disabled after
any error. The completion callback in rx/subs! only fires on success,
not on error.
Add (reset! submitted? false) at the beginning of the on-error handler
so the form becomes submittable again after any error, allowing the user
to fix their input and retry.
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Two coupled defects made shape.applyToken(), token.applyToShapes() and
token.applyToSelected() silently no-op when invoked from JavaScript with
an array of strings (e.g. token.applyToShapes([rect], ["fill"])):
1. token-attr-plugin->token-attr only consulted its alias map when the
input was already a keyword; string inputs fell through unchanged,
causing downstream token-attr? to return false.
2. The inner schemas used plain [:set ...] which lacks the :decode/json
transformer for JS array -> Clojure set coercion. Switching to
Penpot's custom [::sm/set ...] lets the standard JSON decoder
pipeline handle the conversion automatically.
This is a backport of commit 1eac3e2be5f973359ad2ec9bac4e80a9d5a9e022
which fixes GitHub #9162.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Remove the need to navigate to page for deletion operation
* 🐛 Fix multiple selection with applied-tokens on stroke-color
* 🐛 Fix button position on page header
---------
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
If the mcp key has expired, the switch that indicates the status in the dashboard will appear as disabled, and will show a modal for regenerate the key. It will also appear as disabled in the workspace, not allowing the plugin to connect
- Remove penpotWorkerURI from index.mustache and rasterizer.mustache templates
- Remove worker_main entry from the build manifest
- Construct worker URI in config.cljs by joining public-uri with worker path
- Fix global variable casing for plugins-list-uri and templates-uri
- Fix alignment in worker.cljs let bindings
Introduce a purpose-agnostic three-step session-based upload API that
allows uploading large binary blobs (media files and .penpot imports)
without hitting multipart size limits.
Backend:
- Migration 0147: new `upload_session` table (profile_id, total_chunks,
created_at) with indexes on profile_id and created_at.
- Three new RPC commands in media.clj:
* `create-upload-session` – allocates a session row; enforces
`upload-sessions-per-profile` and `upload-chunks-per-session`
quota limits (configurable in config.clj, defaults 5 / 20).
* `upload-chunk` – stores each slice as a storage object;
validates chunk index bounds and profile ownership.
* `assemble-file-media-object` – reassembles chunks via the shared
`assemble-chunks!` helper and creates the final media object.
- `assemble-chunks!` is a public helper in media.clj shared by both
`assemble-file-media-object` and `import-binfile`.
- `import-binfile` (binfile.clj): accepts an optional `upload-id` param;
when provided, materialises the temp file from chunks instead of
expecting an inline multipart body, removing the 200 MiB body limit
on .penpot imports. Schema updated with an `:and` validator requiring
either `:file` or `:upload-id`.
- quotes.clj: new `upload-sessions-per-profile` quota check.
- Background GC task (`tasks/upload_session_gc.clj`): deletes stalled
(never-completed) sessions older than 1 hour; scheduled daily at
midnight via the cron system in main.clj.
- backend/AGENTS.md: document the background-task wiring pattern.
Frontend:
- New `app.main.data.uploads` namespace: generic `upload-blob-chunked`
helper drives steps 1–2 (create session + upload all chunks with a
concurrency cap of 2) and emits `{:session-id uuid}` for callers.
- `config.cljs`: expose `upload-chunk-size` (default 25 MiB, overridable
via `penpotUploadChunkSize` global).
- `workspace/media.cljs`: blobs ≥ chunk-size go through the chunked path
(`upload-blob-chunked` → `assemble-file-media-object`); smaller blobs
use the existing direct `upload-file-media-object` path.
`handle-media-error` simplified; `on-error` callback removed.
- `worker/import.cljs`: new `import-blob-via-upload` helper replaces the
inline multipart approach for both binfile-v1 and binfile-v3 imports.
- `repo.cljs`: `:upload-chunk` derived as a `::multipart-upload`;
`form-data?` removed from `import-binfile` (JSON params only).
Tests:
- Backend (rpc_media_test.clj): happy path, idempotency, permission
isolation, invalid media type, missing chunks, session-not-found,
chunk-index out-of-range, and quota-limit scenarios.
- Frontend (uploads_test.cljs): session creation and chunk-count
correctness for `upload-blob-chunked`.
- Frontend (workspace_media_test.cljs): direct-upload path for small
blobs, chunked path for large blobs, and chunk-count correctness for
`process-blobs`.
- `helpers/http.cljs`: shared fetch-mock helpers (`install-fetch-mock!`,
`make-json-response`, `make-transit-response`, `url->cmd`).
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
- Add session ID tracking to RPC layer (backend and frontend)
- Send session ID header with RPC requests for request correlation
- Rename file-restore to file-restored for consistency
- Extract initialize-file function from initialize-workspace flow
- Improve file restoration initialization with wait-for-persistence
- Extract initialize-version event handler for version restoration
- Fix viewport key generation with file version numbers for proper re-renders
- Update layout item schema and constraints to use internal sizing state
- Add v-sizing state retrieval in layout-size-constraints component
- Refactor file-change notifications stream handling with rx/map
- Fix team-id lookup in restore-version-from-plugins
Improves request traceability across frontend/backend sessions and streamlines
the workspace initialization flow for file restoration scenarios.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Guard imperative DOM operations (removeChild, RAF callbacks) against
race conditions where React has already unmounted the target nodes.
- assets/common.cljs: add dom/child? guard before removeChild in RAF
- dynamic_modifiers.cljs: capture RAF IDs and cancel them on cleanup;
add null guards for DOM nodes that may no longer exist
- hooks.cljs: guard portal container removal with dom/child? check
- errors.cljs: extract is-ignorable-exception? to a top-level defn
and add NotFoundError/removeChild to ignorable exceptions, since
these are caused by browser extensions modifying React-managed DOM
- Add unit tests for is-ignorable-exception? predicate
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🚑 Fix RangeError from re-entrant error handling in errors.cljs
Two complementary changes to prevent 'RangeError: Maximum call stack
size exceeded' when an error fires while the potok store error pipeline
is still on the call stack:
1. Re-entrancy guard on on-error: a volatile flag (handling-error?)
is set true for the duration of each on-error invocation. Any
nested call (e.g. from a notification emit that itself throws) is
suppressed with a console.error instead of recursing indefinitely.
2. Async notification in flash: the st/emit!(ntf/show ...) call is
now wrapped in ts/schedule (setTimeout 0) so the notification event
is pushed to the store on the next event-loop tick, outside the
error-handler call stack. This matches the pattern already used by
the :worker-error, :svg-parser and :comment-error handlers.
* 🐛 Add unit tests for app.main.errors
Test coverage for the error-handling module:
- stale-asset-error?: 6 cases covering keyword-constant and
protocol-dispatch mismatch signatures, plus negative cases
- exception->error-data: plain JS Error, ex-info with/without :hint
- on-error dispatch: map errors routed via ptk/handle-error, JS
exceptions wrapped into error-data before dispatch
- Re-entrancy guard: verifies that a second on-error call issued
from within a handle-error method is suppressed (exactly one
handler invocation)
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The 'Move to' menu in the dashboard file context menu only filtered
out the first selected file's project from the available target list.
When multiple files from different projects were selected, the other
files' projects still appeared as valid targets, causing a 400
'cant-move-to-same-project' backend error.
Now all selected files' project IDs are collected and excluded from
the available target projects.
lambdaisland/uri's query-string->map uses :multikeys :duplicates by
default: a key that appears once yields a plain string, but the same
key repeated yields a vector. cljs.core/parse-long only accepts
strings and therefore threw "Expected string, got: object" whenever
a URL contained a duplicate 'index' parameter.
Add rt/get-query-param to app.main.router. The helper returns the
scalar value of a query param key, taking the last element when the
value is a sequential (i.e. the key was repeated). Use it at every
call site that feeds a query-param value into parse-long, in both
app.main.ui (page*) and app.main.data.viewer.
Refactor use-portal-container to allocate one persistent <div> per
logical category (:modal, :popup, :tooltip, :default) instead of
creating a new div for every component instance. This keeps the DOM
clean with at most four fixed portal containers and eliminates the
arbitrary growth of empty <div> elements on document.body while
preserving the removeChild race condition fix.
