3123 Commits

Author SHA1 Message Date
Andrey Antukh
f17fbbf546 🎉 Add telemetry anonymous event collection
When the :telemetry flag is ON and :audit-log is OFF, frontend and
backend events are stored anonymously in the audit_log table and
shipped in compressed batches by the existing telemetry task.

Stored rows strip props and ip-addr but preserve the profile-id, since
Penpot profile UUIDs are already anonymous random identifiers with no
PII attached. Timestamps are truncated to day precision to avoid leaking
exact event timing. Only a safe subset of context fields is preserved:

- Backend events: initiator, version, client-version, client-user-agent
- Frontend events: browser, os, locale, screen metrics and event-origin

Backend (app.loggers.audit):
- Store backend telemetry events with source='telemetry', the safe
  context subset described above, and timestamps truncated to day
  precision via ct/truncate.

Frontend RPC (app.rpc.commands.audit):
- Add filter-safe-context to retain only the allowed frontend context
  fields.
- Add xf:map-telemetry-event-row transducer that anonymises frontend
  events before inserting them.
- push-audit-events now accepts events when telemetry is active.

Telemetry task (app.tasks.telemetry):
- gc-telemetry-events: enforces a 100,000-row safety cap by dropping
  the oldest rows first.
- collect-and-send-audit-events: loop that fetches up to 10,000 rows
  per iteration, encodes and sends each page, deletes it on success,
  and stops immediately on failure leaving remaining rows for retry.
- send-event-batch: POSTs a fressian+zstd batch (base64-encoded via
  blob/encode-str) to the telemetry endpoint, including instance-id
  and profile-id per event.
- delete-sent-events: deletes successfully shipped rows by id.

Blob utilities (app.util.blob):
- Add blob/encode-str and blob/decode-str: convenience wrappers that
  combine blob encoding with base64 for JSON-safe string transport.

Database:
- Add index on audit_log (source, created_at ASC) to support efficient
  queries for telemetry batch collection.

Tests (backend-tests.tasks-telemetry-test):
- 21 tests, 94 assertions covering all code paths: disabled/enabled
  telemetry, no-events no-op, happy-path batch send and delete, failure
  retention, payload anonymity, context stripping, timestamp day
  precision, batch encoding round-trip, multi-page iteration, GC cap
  enforcement.

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-04-21 14:47:43 +02:00
Andrey Antukh
de27ea904d
Add minor adjustments to the auth events (#9027) 2026-04-16 09:59:45 +02:00
Andrey Antukh
c8675c5b7e
♻️ Normalize newsletter-updates checbox on different register flows (#8839)
*  Add newsletter opt-in checkbox to registration validation form

Add accept-newsletter-updates support through the full registration
token flow. The newsletter checkbox is now available on the
registration validation form, allowing users to opt-in during the
email verification step.

Backend changes:
- Refactor prepare-register to consolidate UTM params and newsletter
  preference into props at token creation time
- Add accept-newsletter-updates to prepare-register-profile and
  register-profile schemas
- Handle newsletter-updates in register-profile by updating token
  claims props on second step

Frontend changes:
- Add newsletter-options component to register-validate-form
- Add accept-newsletter-updates to validation schema
- Fix subscription finalize/error handling in register form

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* ♻️ Refactor auth register components to modern style

Migrate all components in app.main.ui.auth.register and
app.main.ui.auth.login/demo-warning to use the modern * suffix
convention, removing deprecated ::mf/props :obj metadata and
updating all invocations from [:& name] to [:> name*] syntax.

Components updated:
- terms-and-privacy -> terms-and-privacy*
- register-form -> register-form*
- register-methods -> register-methods*
- register-page -> register-page*
- register-success-page -> register-success-page*
- terms-register -> terms-register*
- register-validate-form -> register-validate-form*
- register-validate-page -> register-validate-page*
- demo-warning -> demo-warning*

Also remove unused old context-notification import in login.cljs.

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🔥 Remove unused onboarding-newsletter component

The newsletter opt-in is now handled directly in the registration
form via the newsletter-options* component, making the standalone
onboarding-newsletter modal obsolete.

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 🐛 Fix register test for UTM params to use prepare-register step

UTM params are now extracted and stored in token props during the
prepare-register step, not at register-profile time. Move utm_campaign
and mtm_campaign from the register-profile call to the
prepare-register-profile call in the test.

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-04-08 17:00:52 +02:00
Andrey Antukh
81b1b253f1
Add unique email domains to telemetry report (#8819)
Extend the telemetry payload with a sorted list of unique email domains
extracted from all registered profile email addresses. The new
:email-domains field is populated via a single SQL query using
split_part and DISTINCT, and is included in the stats sent when
telemetry is enabled.

Also update the tasks-telemetry-test to assert the new field is present
and contains the expected domain values.
2026-04-01 11:49:50 +02:00
Andrey Antukh
0dfa62a5b6
🐛 Improve error reporting on request parsing failures (#8805)
Include request URI and status in frontend handle-response error data,
and add request path/context to backend IOException handler logs and
response body. Previously these errors had no identifying information
about which endpoint or request caused the failure.
2026-03-26 15:42:49 +01:00
Andrey Antukh
cc03f3f884 📚 Add minor improvements to ai agents documentation 2026-03-24 18:00:39 +01:00
Andrey Antukh
2d616cf9c0
📚 Add better organization for AGENTS.md file (#8675) 2026-03-18 14:59:38 +01:00
Andrey Antukh
7ec9261475
Add improvements to AGENTS.md (#8586) 2026-03-11 15:24:40 +01:00
Andrey Antukh
ab90500ec8 🐛 Fix download-image to properly handle network errors and non-2xx responses (#8554)
The download-image function in app.media silently succeeded when the
remote image URL was unreachable or returned an error status code,
causing create-file-media-object-from-url to report success with no
actual image stored.

Add exception handling for connection refused, timeouts, and I/O errors
around the HTTP request, and validate the HTTP status code in
parse-and-validate before processing the response body.

Fixes #8499

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-03-10 15:01:23 +01:00
Andrey Antukh
0f47c30349 Merge branch 'main' into staging 2026-03-10 14:39:16 +01:00
Andrey Antukh
68fbacf8b3 Merge tag '2.14.0-RC2' 2026-03-10 14:38:58 +01:00
Andrey Antukh
32cf95265a 📚 Add GitHub Copilot instructions (#8548) 2026-03-10 13:12:15 +01:00
Andrey Antukh
77955d7f91 Add several redundant checks for library-id on file rpc methods 2026-03-09 10:01:29 +01:00
Andrey Antukh
151238e518 💄 Add cosmetic change to link-file-to-library rpc method impl 2026-03-09 10:01:29 +01:00
Andrey Antukh
86e851f408
🐛 Fix incorrect version visibility on workspace (#8463)
* 🐛 Add missing order by clause to snapshot query

This fixes the incorrect snapshot visibility when file
has a lot of versions.

*  Reduce allocation on milestone-group* component

* 🐛 Fix milestone group timestamp formatting

* 📎 Update changelog

* 🐛 Fix scroll on history panel

---------

Co-authored-by: Eva Marco <evamarcod@gmail.com>
2026-03-04 09:27:51 +01:00
Andrey Antukh
a4351d133b
Add minor improvements to error reporting (#8402) 2026-03-04 09:12:19 +01:00
Andrey Antukh
1b8afccba2 Remove usage of multipart body size config on backend 2026-02-23 14:44:44 +01:00
Andrey Antukh
3909bc0fc1 Merge remote-tracking branch 'origin/main' into staging 2026-02-16 14:17:46 +01:00
Andrey Antukh
b6427ecaac 🐛 Revert yetti upgrade
Because of regression introduced on undertow-core 2.3.19
2026-02-16 14:16:29 +01:00
Andrey Antukh
4a7b89a1da
Merge pull request #8327 from penpot/niwinz-develop-rlimit-notifications
 Add proper mattermost notifications for rlimit rejects
2026-02-13 17:11:54 +01:00
Andrey Antukh
12bc3ac9ed Update default cors headers 2026-02-13 13:17:08 +01:00
Sagar
cfcebf59d5
🐛 Make S3Client and S3Presigner use identical credential resolution (#8316) 2026-02-13 12:21:05 +01:00
Andrey Antukh
920fbe34ad 🐛 Fix invalid deps passed to http management routes service 2026-02-10 13:46:29 +01:00
Andrey Antukh
f08700945a Merge remote-tracking branch 'origin/staging' into develop 2026-02-10 11:58:09 +01:00
Andrey Antukh
06e5825c8a 🐛 Add proper input checking to font related RCP method 2026-02-10 10:36:57 +01:00
Pablo Alba
5d3ccbc8b4
Add managed profiles endpoint to nitrate api (#8292) 2026-02-09 15:52:18 +01:00
Andrey Antukh
ac1c3ff184 Merge branch 'staging-render' into develop 2026-02-09 14:14:02 +01:00
Andrey Antukh
43cd92c76d Merge remote-tracking branch 'origin/staging' into staging-render 2026-02-09 14:12:55 +01:00
Alejandro Alonso
3d20fc508d
🐛 Fix image magick info call (#8300) 2026-02-09 12:26:42 +01:00
Andrey Antukh
2ccb33ba89 📎 Add missing for-update for the migration 145 2026-02-05 18:12:11 +01:00
Andrey Antukh
ee88ee63a2 Add data migration for fix plugins data on profiles 2026-02-05 18:08:28 +01:00
Andrey Antukh
1656fefdc9 Merge remote-tracking branch 'origin/staging-render' into develop 2026-02-04 16:23:46 +01:00
Andrey Antukh
7f318bb110 Merge remote-tracking branch 'origin/staging' into staging-render 2026-02-04 16:22:13 +01:00
Andrey Antukh
06afd94a74 ⬆️ Update backend dependencies (mainly bugfixes) 2026-02-04 16:21:19 +01:00
Andrey Antukh
ca4d00df69 🐛 Fix latest error report related migration 2026-02-04 15:36:07 +01:00
Andrey Antukh
9667477d6b 🐛 Add missing dep for rpc routes on backend 2026-02-04 15:26:02 +01:00
Andrey Antukh
d80ba1856a
Add several improvements to frontend error reporting
*  Add major improvement on error handling

*  Add the ability to store frontend reports

* 📎 Add PR feedback changes
2026-02-04 12:45:38 +01:00
Andrey Antukh
32454f5959 Merge remote-tracking branch 'origin/staging-render' into develop 2026-01-29 10:23:46 +01:00
Andrey Antukh
b40e775a70
Add minor improvements to performance events (#8217)
*  Move devtools perf logging helpers to util.perf ns

* 💄 Move flag check to the entry point instead of initialize event

* ♻️ Make performance events consistent with other events
2026-01-28 20:47:14 +01:00
Pablo Alba
4ca82821c1
🐛 Fix shared keys init should be by keywords (2) (#8230) 2026-01-28 13:41:37 +01:00
Andrey Antukh
a9e2fc8d94 Backport linter fixes and config from develop 2026-01-28 12:58:54 +01:00
Pablo Alba
f76598f638
🐛 Fix shared keys init should be by keywords (#8228) 2026-01-28 12:56:04 +01:00
Andrey Antukh
17ffd9a5d0 Backport linter fixes and config from develop 2026-01-28 12:54:18 +01:00
Andrey Antukh
89935e2174 Make nitrate module loading conditional to flag
This removes the flag checking on each rpc method
2026-01-27 15:16:36 +01:00
Andrey Antukh
7f27e0326d Reuse basic team and profile schemas on nitrate 2026-01-27 15:14:32 +01:00
Andrey Antukh
9c539dfb2f 🔥 Remove subscriptions related management module 2026-01-27 15:14:32 +01:00
Andrey Antukh
50a4cf8b99 📎 Adapt nitrate module to auth changes 2026-01-27 15:14:32 +01:00
Andrey Antukh
f5996a7235 ♻️ Make several improvements to management API authentication 2026-01-27 15:14:32 +01:00
Andrey Antukh
0ab126748f
💄 Add format rule for code comments (#8211)
* 💄 Add format rule for code comments

* ⬆️ Update linter and formatter on devenv
2026-01-27 15:07:18 +01:00
Pablo Alba
d5abc52dac
🎉 Add first integration with nitrate (#7803)
* 🐛 Display missing selected tokens set info (#8098)

* 🐛 Display missing selected tokens set info

*  Add integration tests to verify current active set

* 🎉 Integration with nitrate platform

* 🐛 Fix nitrate get-teams returns deleted teams

*  Add nitrate to tmux devenv

*  Add retry and validation to nitrate module

*  Add photoUrl to profile on nitrate authenticate

*  Move nitrate url to an env variable

* ♻️ Change Nitrate organization-id schema to text

* ♻️ Cleanup unused imports

* 🔧 Add control-center to nginx

*  Add create org link

* 🔧 Fix nginx entrypoint

* 🐛 Fix control-center proxy pass

* 🎉 Add nitrate licence check

* Revert " Add nitrate to tmux devenv"

This reverts commit dc6f6c458995dac55cab7be365ced0972760a058.

*  Add feature flag check

* 🐛 Rename licences for licenses

*  MR changes

*  MR changes 2

* 📎 Add the ability to have local config on start backend

* 📎 Add FIXME comment

---------

Co-authored-by: Xaviju <xavier.julian@kaleidos.net>
Co-authored-by: Juanfran <juanfran.ag@gmail.com>
Co-authored-by: Yamila Moreno <yamila.moreno@kaleidos.net>
Co-authored-by: Marina López <marina.lopez.yap@gmail.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-01-27 10:04:53 +01:00