* 🎉 Add telemetry anonymous event collection
Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:
Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
telemetry flag active, audit-log flag inactive
Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
Keeps UUID/boolean/number values; for login/identify events preserves
lang, auth-backend, email-domain; for navigate events preserves route,
file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
Backend: version, initiator, client-version, client-user-agent.
Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
access-token fields removed from context
Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option
Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
extraction from email addresses
Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)
RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation
RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API
SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys
Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
per iteration, encodes and sends each page, deletes on success,
stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id
Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
safe base64 for JSON-safe string transport
Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
efficient telemetry batch collection queries
Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support
Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
happy-path batch send and delete, failure retention, payload anonymity,
context stripping, timestamp day precision, batch encoding round-trip,
multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add pr feedback changes
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The recursive `read-items` function in `app.util.sse/read-stream`
caused a synchronous stack overflow when reading buffered stream
data. Each `rx/mapcat` call chained another recursive invocation
on the same call stack without yielding to the event loop.
Replace the recursive pattern with an `rx/create`-based async pump
that uses Promise `.then()` chaining, keeping the call stack depth
constant regardless of stream size.
Also add progress reporting with names and IDs during binfile
export and import, and bump `eventsource-parser` dependency.
Closes#9470
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The recursive `read-items` function in `app.util.sse/read-stream`
caused a synchronous stack overflow when reading buffered stream
data. Each `rx/mapcat` call chained another recursive invocation
on the same call stack without yielding to the event loop.
Replace the recursive pattern with an `rx/create`-based async pump
that uses Promise `.then()` chaining, keeping the call stack depth
constant regardless of stream size.
Also add progress reporting with names and IDs during binfile
export and import, and bump `eventsource-parser` dependency.
Closes#9470
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Closing the fill dialog while an image-fill upload is still in flight
(or while a gradient is mid-edit) leaves the colorpicker's
current-color with only :opacity — no :image, :gradient, or :color.
update-colorpicker-color's WatchEvent then constructed
`(add-recent-color partial)`, which runs the value through
`clr/check-color` and threw "expected valid color". The user saw an
Internal Assertion Error toast and lost the in-flight upload.
The existing `ignore-color?` guard reads `:type` from the *output* of
`get-color-from-colorpicker-state` — but that helper strips :type from
its result, so the guard never actually fires. Add a schema-based gate
(same validator add-recent-color itself uses) right before `rx/of`, so
a partial selection is silently dropped instead of crashing the
workspace. Behaviour for fully-valid colors is unchanged.
Tests cover three cases: (1) a partial image-tab state with only
:opacity returns nil from watch (was: throws); (2) the same partial
shape on the color tab also returns nil — pinning down that the prior
:type guard wouldn't have caught it; (3) a fully-populated plain color
still produces a watch observable so the guard isn't over-eager.
Closes#8443
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
Rename component from link-button to link-button* and remove the legacy
::mf/wrap-props false metadata. Update all callsites to use the modern
[:> lb/link-button* ...] syntax instead of [:& lb/link-button ...].
Part of the #9260 issue.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Step toward issue #9260 (incremental migration of legacy UI
components to the modern `*`-suffixed syntax, removing the per-render
JS-to-Clojure props conversion overhead).
Twin namespaces with parallel structure: each defines six components
that drive a recursive text rendering pass over the editor's content
tree (root -> paragraph-set -> paragraph -> node -> text). Both files
were uniformly legacy: every component carried `::mf/wrap-props
false` and read its props with `(obj/get props "key")`. None had
`::mf/register`, `unchecked-get` or `obj/merge!`, so they qualify as
clean Case-A migrations.
frontend/src/app/main/ui/shapes/text/fo_text.cljs (6 components)
----------------------------------------------------------------
- `render-text` -> `render-text*`
- `render-root` -> `render-root*`
- `render-paragraph-set` -> `render-paragraph-set*`
- `render-paragraph` -> `render-paragraph*`
- `render-node` -> `render-node*` (forward-props case,
see below)
- `text-shape` -> `text-shape*` (`::mf/forward-ref`
preserved)
The four leaf components switch from `[props]` + per-key
`(obj/get props "key")` to standard destructuring. `text-shape`
already used destructuring under `::mf/props :obj`; that legacy
metadata is dropped because the modern `*` form handles props
automatically. Its single `::mf/forward-ref true` is kept per the
prompt's "preserve forward-ref" rule.
`render-node` is the recursive driver. It needs to forward all of
its incoming props to the matched paragraph-* / text component and
then to a child `render-node*` after overriding `:node`, `:index`
and `:key`. The migrated form uses `::mf/props :obj` together with
`{:keys [node] :as props}` to keep the JS-object props symbol
available, and `(mf/spread-props props {…})` replaces the previous
`obj/clone` + `obj/set!` chain.
`app.util.object` is no longer required by this namespace and the
`(:require ... [app.util.object :as obj] ...)` line is removed.
frontend/src/app/main/ui/shapes/text/html_text.cljs (6 components)
-----------------------------------------------------------------
Identical six-component shape as `fo_text.cljs`, plus a `code?`
flag threaded through every component to switch the rendering path
between regular shapes and code-style shapes.
- `render-text` -> `render-text*`
- `render-root` -> `render-root*`
- `render-paragraph-set` -> `render-paragraph-set*`
- `render-paragraph` -> `render-paragraph*`
- `render-node` -> `render-node*` (same forward-props
treatment as above,
plus `is-code` in
the spread)
- `text-shape` -> `text-shape*` (`::mf/forward-ref`
preserved)
The `code?` boolean prop is renamed to `is-code` per the migration
prompt's "?-suffixed boolean -> `is-` prefix" rule. The rename is
applied at every read site (5 components) and at the `text-shape*`
internal call to `render-node*`, so the prop is consistent inside
the namespace.
`app.util.object` is no longer required by this namespace either
and the corresponding `:require` line is dropped.
External call sites (3 files, 4 sites)
--------------------------------------
- `frontend/src/app/main/ui/shapes/text.cljs` - the legacy
text-shape wrapper (intentionally kept legacy in this PR because
it dispatches to `svg/text-shape`, which is still being touched by
the in-flight PR #9016) now calls `[:> fo/text-shape* props]`.
The `props` symbol is the wrapper's incoming JS-object; modern
destructured components accept JS-object props at the call site
via `[:>` so this works unchanged.
- `frontend/src/app/util/code_gen/markup_html.cljs` -
`(mf/element text/text-shape #js {:shape shape :code? true})`
becomes
`(mf/element text/text-shape* #js {:shape shape :is-code true})`
(component renamed and the `code?` JS key updated to match the
renamed prop).
- `frontend/src/app/main/ui/workspace/shapes/text/viewport_texts_html.cljs`
- `[:& html/text-shape {…}]` -> `[:> html/text-shape* {…}]`.
Behavior preserved verbatim
---------------------------
Same render output, same forward-ref forwarding semantics, same
recursive children-by-index keying, same default `:dir "auto"` on
`render-paragraph*`. The visible-prop changes are only the `code?`
-> `is-code` rename, all driven from this namespace and its single
caller in `markup_html.cljs`.
Github #9260
Signed-off-by: FairyPigDev <luislee3108@gmail.com>
When the Stripe checkout fails to start, the subscription page now
shows an inline error in the Business Nitrate card under the CTA
instead of a toast. When the post-payment activation fails, the toast
message is updated to point users to support@penpot.app.
The nitrate-form modal also passed a URI object to
build-nitrate-callback-urls while the underlying append-query-param
relied on lambdaisland's u/parse, which only accepts strings. Switched
to the local u/uri helper so both strings and URI records work, so
failures opened from the modal land on the subscription page.
Adopts the rumext * suffix convention for the following components,
invoking them with the [:> JS-style syntax to match the rest of the
codebase (see e.g. rea*, single-selection* in viewport/selection):
- measurements: size-display, distance-display-pill, selection-rect,
distance-display, selection-guides, measurement
- shapes/svg-defs: svg-node, svg-defs (also drop the now-redundant
{::mf/wrap-props false} annotations)
Updates all call sites in inspect/selection_feedback, shapes/shape,
workspace/viewport, and workspace/viewport_wasm. Pure rename — no
behavioral change.
Signed-off-by: bitcompass <devwiz.sh@gmail.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
