* 🎉 Add telemetry anonymous event collection
Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:
Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
telemetry flag active, audit-log flag inactive
Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
Keeps UUID/boolean/number values; for login/identify events preserves
lang, auth-backend, email-domain; for navigate events preserves route,
file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
Backend: version, initiator, client-version, client-user-agent.
Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
access-token fields removed from context
Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option
Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
extraction from email addresses
Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)
RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation
RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API
SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys
Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
per iteration, encodes and sends each page, deletes on success,
stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id
Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
safe base64 for JSON-safe string transport
Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
efficient telemetry batch collection queries
Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support
Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
happy-path batch send and delete, failure retention, payload anonymity,
context stripping, timestamp day precision, batch encoding round-trip,
multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add pr feedback changes
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
- Add session ID tracking to RPC layer (backend and frontend)
- Send session ID header with RPC requests for request correlation
- Rename file-restore to file-restored for consistency
- Extract initialize-file function from initialize-workspace flow
- Improve file restoration initialization with wait-for-persistence
- Extract initialize-version event handler for version restoration
- Fix viewport key generation with file version numbers for proper re-renders
- Update layout item schema and constraints to use internal sizing state
- Add v-sizing state retrieval in layout-size-constraints component
- Refactor file-change notifications stream handling with rx/map
- Fix team-id lookup in restore-version-from-plugins
Improves request traceability across frontend/backend sessions and streamlines
the workspace initialization flow for file restoration scenarios.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* Revert "🐛 Fix plugin sandbox freezing CLJS Proxy constructor breaking Transit encoding"
This reverts commit 27a934dcfd579093b066c78d67eba782ba6229cb.
* 🐛 Fix unexpected corner case between SES hardening and transit
The cause of the issue is a race condition between plugin loading
and the first time js/Date objects are encoded using transit. Transit
encoder populates the prototype of the Date object the first time a
Date instance is encoded, but if SES freezes the Date prototype before
transit, an strange exception will be raised on encoding any object
that contains Date instances.
Example of the exception:
Cannot define property transit$guid$4a57baf3-8824-4930-915a-fa905479a036,
object is not extensible
* ✨ Use update-when for update dashboard state
This make updates more consistent and reduces possible eventual
consistency issues in out of order events execution.
* 🐛 Detect stale JS modules at boot and force reload
When the browser serves cached JS files from a previous deployment
alongside a fresh index.html, code-split modules reference keyword
constants that do not exist in the stale shared.js, causing TypeError
crashes.
This adds a compile-time version tag (via goog-define / closure-defines)
that is baked into the JS bundle. At boot, it is compared against the
runtime version tag from index.html (which is always fresh due to
no-cache headers). If they differ, the app forces a hard page reload
before initializing, ensuring all JS modules come from the same build.
* 📎 Ensure consistent version across builds on github e2e test workflow
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Move devtools perf logging helpers to util.perf ns
* 💄 Move flag check to the entry point instead of initialize event
* ♻️ Make performance events consistent with other events
The rationale behind this change is to allow use of already
declared react context on modals; because with portals, react
propagates top context to the children, independently if they
are direct descendant on dom or not.
The frontend-only features are now ignored from files and from teams
and they do not autoassigns automatically to team and file on file
creation or update operations.
