Andrey Antukh e0194ce60a
✨ Deliver telemetry audit events independently (#13230)
* ✨ Ship telemetry shadows with a cursor consumer

Send anonymized audit events through a dedicated telemetry consumer,
including when Nexus or admin-console also persist full events. Keep the
legacy payload and cadence of 10,000 events per chunk and three hours
between passes. Nexus skips shadows to avoid archiving duplicates.

Move shadow deletion to the audit GC. Delete full rows and shadows along
independent cursor paths, and expire shadows after seven days even when
they have not been sent. Run the deletion paths in parallel with bounded
batches, separate time budgets and independent failure handling.

Replace the shared consumer catalog with transport definitions and one
generic pass runner. Keep the legacy telemetry report and newsletter in
the scheduled task, and update the tests, memories and rollout runbooks.

Addresses F1 of the audit-log consumer review.

AI-assisted-by: deepseek-v4-flash
AI-assisted-by: gpt-6.1-terra

* 🐛 Address audit log consumer review findings

Enable audit log GC by default and document how to pause it during rollout
or before registering a consumer that needs a complete replay.

Run GC paths in order on the runner thread. Count each committed batch and
preserve partial totals on ordinary failures while propagating interruption.

Skip full-row payloads in telemetry queries and decoding. Keep the age
limit for local retention only, and document the shared pass contract.

AI-assisted-by: gpt-6.2-sol

* 🐛 Close audit log consumer review findings

Verify telemetry counts processed and skipped events in a mixed batch.
Clarify how to pause GC, the scope of the replay race, and what the
retention warning counts. Remove private Nexus setup from the public
upgrade runbook.

AI-assisted-by: gpt-6.2-sol

* 📚 Simplify the audit log guide

Consolidate the guidance on event trust, privacy and delivery. Keep examples
that explain why the design works and link to code and runbooks for details.
Clarify replay limits and distinguish age-based deletion from missed delivery.

AI-assisted-by: gpt-6.2-sol
2026-11-20 16:28:42 +01:00
..
2022-11-08 13:02:14 +01:00