Andrey Antukh 7d4be33d4f 🎉 Add telemetry anonymous event collection (#9483)
* 🎉 Add telemetry anonymous event collection

Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:

Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
  telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
  telemetry flag active, audit-log flag inactive

Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
  prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
  webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
  Keeps UUID/boolean/number values; for login/identify events preserves
  lang, auth-backend, email-domain; for navigate events preserves route,
  file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
  Backend: version, initiator, client-version, client-user-agent.
  Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
  storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
  access-token fields removed from context

Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option

Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
  extraction from email addresses

Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)

RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation

RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API

SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys

Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
  payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
  per iteration, encodes and sends each page, deletes on success,
  stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
  blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
  oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id

Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
  safe base64 for JSON-safe string transport

Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
  efficient telemetry batch collection queries

Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support

Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
  happy-path batch send and delete, failure retention, payload anonymity,
  context stripping, timestamp day precision, batch encoding round-trip,
  multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📎 Add pr feedback changes

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-05-11 12:42:01 +02:00
2026-04-21 15:40:51 +02:00
2023-11-15 17:46:51 +01:00
2026-05-11 12:39:03 +02:00
2026-05-04 09:34:19 +02:00
2021-11-15 09:54:36 -05:00
2026-04-20 19:37:02 +02:00
2020-08-11 10:11:23 +02:00
2026-05-10 10:49:53 +02:00
2025-01-22 15:46:23 +01:00
2017-03-06 18:42:58 +01:00
2026-02-04 16:21:19 +01:00
2026-05-11 11:26:54 +02:00
2025-06-02 12:09:20 +02:00
2023-11-15 10:55:39 +01:00
2026-02-09 11:25:24 +01:00
2026-05-10 10:49:53 +02:00
2026-05-10 10:49:53 +02:00
2025-09-25 08:46:02 +02:00
2026-02-11 08:11:04 +01:00
2024-06-11 11:03:54 +02:00
2020-08-05 11:19:34 +02:00

penpot header image

License: MPL-2.0 Penpot Community Managed with Taiga.io Gitpod ready-to-code

WebsiteUser GuideLearning CenterCommunity

YoutubePeertubeLinkedinInstagramMastodonBlueskyX


Penpot video


Penpot is the first open-source design tool for design and code collaboration. Designers can create stunning designs, interactive prototypes, design systems at scale, while developers enjoy ready-to-use code and make their workflow easy and fast. And all of this with no handoff drama.

Available on browser or self-hosted, Penpot works with open standards like SVG, CSS, HTML and JSON, and its free!

The latest updates take Penpot even further. Its the first design tool to integrate native design tokens—a single source of truth to improve efficiency and collaboration between product design and development. With the huge 2.0 release, Penpot took the platform to a whole new level. This update introduces the ground-breaking CSS Grid Layout feature, a complete UI redesign, a new Components system, and much more. For organizations that need extra service for its teams, get in touch

🎇 Design, code, and Open Source meet at Penpot Fest! Be part of the 2025 edition in Madrid, Spain, on October 9-10.

Table of contents

Why Penpot

Penpot expresses designs as code. Designers can do their best work and see it will be beautifully implemented by developers in a two-way collaboration.

Plugin system

Penpot plugins let you expand the platform's capabilities, give you the flexibility to integrate it with other apps, and design custom solutions.

Designed for developers

Penpot was built to serve both designers and developers and create a fluid design-code process. You have the choice to enjoy real-time collaboration or play "solo".

Inspect mode

Work with ready-to-use code and make your workflow easy and fast. The inspect tab gives instant access to SVG, CSS and HTML code.

Self host your own instance

Provide your team or organization with a completely owned collaborative design tool. Use Penpot's cloud service or deploy your own Penpot server.

Integrations

Penpot offers integration into the development toolchain, thanks to its support for webhooks and an API accessible through access tokens.

Building Design Systems: design tokens, components and variants

Penpot brings design systems to code-minded teams: a single source of truth with native Design Tokens, Components, and Variants for scalable, reusable, and consistent UI across projects and platforms.



Getting started

Penpot is the only design & prototype platform that is deployment agnostic. You can use it in our SAAS or deploy it anywhere.

Learn how to install it with Docker, Kubernetes, Elestio or other options on our website.

Open Source


Community

We love the Open Source software community. Contributing is our passion and if its yours too, participate and improve Penpot. All your designs, code and ideas are welcome!

If you need help or have any questions; if youd like to share your experience using Penpot or get inspired; if youd rather meet our community of developers and designers, join our Community!

You will find the following categories:


Community


Code of Conduct

Anyone who contributes to Penpot, whether through code, in the community, or at an event, must adhere to the code of conduct and foster a positive and safe environment.

Contributing

Any contribution will make a difference to improve Penpot. How can you get involved?

Choose your way:

To find (almost) everything you need to know on how to contribute to Penpot, refer to the contributing guide.


Libraries and templates


Resources

You can ask and answer questions, have open-ended conversations, and follow along on decisions affecting the project.

💾 Documentation

🚀 Getting Started

✏️ Tutorials

🏘️ Architecture

📚 Dev Diaries

License

This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.

Copyright (c) KALEIDOS INC

Penpot is a Kaleidos open source project

Description
Penpot: The open-source design tool for design and code collaboration
Readme MPL-2.0 437 MiB
Languages
Clojure 69.9%
JavaScript 14.2%
Rust 5.6%
SCSS 4.8%
TypeScript 2.2%
Other 3.2%