When the :telemetry flag is ON and :audit-log is OFF, frontend and
backend events are stored anonymously in the audit_log table and
shipped in compressed batches by the existing telemetry task.
Stored rows strip props and ip-addr but preserve the profile-id, since
Penpot profile UUIDs are already anonymous random identifiers with no
PII attached. Timestamps are truncated to day precision to avoid leaking
exact event timing. Only a safe subset of context fields is preserved:
- Backend events: initiator, version, client-version, client-user-agent
- Frontend events: browser, os, locale, screen metrics and event-origin
Backend (app.loggers.audit):
- Store backend telemetry events with source='telemetry', the safe
context subset described above, and timestamps truncated to day
precision via ct/truncate.
Frontend RPC (app.rpc.commands.audit):
- Add filter-safe-context to retain only the allowed frontend context
fields.
- Add xf:map-telemetry-event-row transducer that anonymises frontend
events before inserting them.
- push-audit-events now accepts events when telemetry is active.
Telemetry task (app.tasks.telemetry):
- gc-telemetry-events: enforces a 100,000-row safety cap by dropping
the oldest rows first.
- collect-and-send-audit-events: loop that fetches up to 10,000 rows
per iteration, encodes and sends each page, deletes it on success,
and stops immediately on failure leaving remaining rows for retry.
- send-event-batch: POSTs a fressian+zstd batch (base64-encoded via
blob/encode-str) to the telemetry endpoint, including instance-id
and profile-id per event.
- delete-sent-events: deletes successfully shipped rows by id.
Blob utilities (app.util.blob):
- Add blob/encode-str and blob/decode-str: convenience wrappers that
combine blob encoding with base64 for JSON-safe string transport.
Database:
- Add index on audit_log (source, created_at ASC) to support efficient
queries for telemetry batch collection.
Tests (backend-tests.tasks-telemetry-test):
- 21 tests, 94 assertions covering all code paths: disabled/enabled
telemetry, no-events no-op, happy-path batch send and delete, failure
retention, payload anonymity, context stripping, timestamp day
precision, batch encoding round-trip, multi-page iteration, GC cap
enforcement.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add newsletter opt-in checkbox to registration validation form
Add accept-newsletter-updates support through the full registration
token flow. The newsletter checkbox is now available on the
registration validation form, allowing users to opt-in during the
email verification step.
Backend changes:
- Refactor prepare-register to consolidate UTM params and newsletter
preference into props at token creation time
- Add accept-newsletter-updates to prepare-register-profile and
register-profile schemas
- Handle newsletter-updates in register-profile by updating token
claims props on second step
Frontend changes:
- Add newsletter-options component to register-validate-form
- Add accept-newsletter-updates to validation schema
- Fix subscription finalize/error handling in register form
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Refactor auth register components to modern style
Migrate all components in app.main.ui.auth.register and
app.main.ui.auth.login/demo-warning to use the modern * suffix
convention, removing deprecated ::mf/props :obj metadata and
updating all invocations from [:& name] to [:> name*] syntax.
Components updated:
- terms-and-privacy -> terms-and-privacy*
- register-form -> register-form*
- register-methods -> register-methods*
- register-page -> register-page*
- register-success-page -> register-success-page*
- terms-register -> terms-register*
- register-validate-form -> register-validate-form*
- register-validate-page -> register-validate-page*
- demo-warning -> demo-warning*
Also remove unused old context-notification import in login.cljs.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🔥 Remove unused onboarding-newsletter component
The newsletter opt-in is now handled directly in the registration
form via the newsletter-options* component, making the standalone
onboarding-newsletter modal obsolete.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix register test for UTM params to use prepare-register step
UTM params are now extracted and stored in token props during the
prepare-register step, not at register-profile time. Move utm_campaign
and mtm_campaign from the register-profile call to the
prepare-register-profile call in the test.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Extend the telemetry payload with a sorted list of unique email domains
extracted from all registered profile email addresses. The new
:email-domains field is populated via a single SQL query using
split_part and DISTINCT, and is included in the stats sent when
telemetry is enabled.
Also update the tasks-telemetry-test to assert the new field is present
and contains the expected domain values.
The download-image function in app.media silently succeeded when the
remote image URL was unreachable or returned an error status code,
causing create-file-media-object-from-url to report success with no
actual image stored.
Add exception handling for connection refused, timeouts, and I/O errors
around the HTTP request, and validate the HTTP status code in
parse-and-validate before processing the response body.
Fixes#8499
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
This simplifes the mental model on how it works and simplifies testing
of the related code.
This also normalizes storage object deletion in the same way as the
rest of objects in penpot (now future deletion date on storage object
also means storage object to be deleted).
The main idea behind this refactor is make the
API less especialized for specific use of out internal
submidules and make it more general and usable
for more general purposes (per example cache)
Mainly make it receive the whol cfg/system instead only props. This
makes the api more flexible for a future extending without the need
to change the api again.
Replace general usage of virtual threads with platform threads
and use virtual threads for lightweight procs such that websocket
connections. This decision is made mainly because virtual threads
does not appear on thread dumps in an easy way so debugging issues
becomes very difficult.
The threads requirement of penpot for serving http requests
is not very big so having so this decision does not really affects
the resource usage.
