Add :uri and :scheme/:host keys to exceptions raised by
`validate-uri` for better error diagnostics. Also fix a bug
where (str url) was used instead of (str uri) in the
host-missing exception path.
Update the existing blocked-target test to verify the new :uri
key, and add three new tests covering scheme rejection, missing
host, and DNS failure error paths. All 27 tests pass with 60
assertions and 0 failures.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The /api/main/doc endpoint was returning HTML content with a
text/plain content-type header instead of text/html. This caused
browsers to render the response as plain text.
Added content-type: text/html; charset=utf-8 header to the
response in the doc handler and added a regression test to
verify the fix.
Closes#9680
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
- Add ::setup/props and ::db/pool to :app.http.assets/routes config
so session renewal works correctly for asset requests.
- Add actoken/authz middleware to the assets middleware chain so
access tokens are properly recognized.
- Add authenticated? helper that checks both ::session/profile-id
and ::actoken/profile-id, fixing 401 errors when accessing
protected assets with a valid access token.
- Add comprehensive test suite for assets auth scenarios.
Closes#9677
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Add a shared `schema:font-family` whitelist validator in
app.common.types.font that only allows letters, digits, spaces,
hyphens, underscores, and dots in font family names. Apply the schema
to create-font-variant and update-font RPC endpoints on the
backend, and add client-side validation in the dashboard fonts UI.
Include unit tests for the schema and integration tests for the RPC
handlers.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix library updates reappear after file is reloaded
Summary
Migrate synced_at timestamps to a standalone file_library_sync table to ensure sync state is tracked for both direct and transitive libraries.
Problem
Transitive libraries (libraries imported by other libraries) are not stored as direct rows in file_library_rel. Because the system previously coupled synced_at directly to the file_library_rel schema, transitive libraries lacked a persistent location for their sync timestamps. This caused sync states to be lost or incorrectly reported for nested dependencies.
Changes
Schema Migration: Created file_library_sync and migrated existing synced_at values from file_library_rel.
Decoupling: Removed tight Foreign Key coupling to allow sync rows to exist independently of specific relationship records.
Persistent Writes: Added upsert-file-library-sync! helper. Updated all import, duplication, and RPC write paths (v1/v2/v3 importers, link-file-library) to ensure every write persists a sync row.
Unified Reads: Updated both direct and recursive/transitive library queries to fetch synced_at from the new table.
Testing: Added regression tests to verify that sync rows are correctly created/updated even when a transitive relation is absent in file_library_rel.
Impact
This fix ensures that the system accurately records and retrieves sync states for the entire library dependency tree, resolving the bug where nested libraries appeared out of sync.
* ✨ MR review
* ✨ Add additional logging and validation for image upload
* 🎉 Add chunked upload support for font variants
Extend the font variant upload flow across frontend, backend, and common
to support the standardized chunked upload protocol.
**Backend:**
- Add \`:font-max-file-size\` config default (30 MiB) and schema entry
- Add \`validate-font-size!\` in \`media.clj\` (mirrors
\`validate-media-size!\`, raises \`:font-max-file-size-reached\`)
- Extend \`schema:create-font-variant\` to accept either \`:data\`
(legacy bytes or chunk-vector) or \`:uploads\` (new chunked session
map), with a validator requiring exactly one
- Add \`prepare-font-data-from-uploads\`: assembles each chunked
session via \`cmedia/assemble-chunks\`, validates type+size
- Add \`prepare-font-data-from-legacy\`: normalises legacy byte/chunk
entries, writing to a tempfile (joining via SequenceInputStream),
validates type+size
- Add structured logging ("init"/"end") with \`:size\`, \`:mtypes\`,
and \`:elapsed\` in \`create-font-variant\`
**Frontend:**
- \`upload-blob-chunked\` accepts a per-caller \`:chunk-size\` option
- Add \`font-upload-chunk-size\` (10 MiB) and \`upload-font-variant\`
fn that uploads each mtype as a separate chunked session
- \`on-upload*\` in dashboard fonts now calls \`upload-font-variant\`
instead of issuing \`create-font-variant\` RPC directly
- \`process-upload\` stores raw ArrayBuffer instead of chunking
client-side
**Common:**
- Replace \`"font/opentype"\` with \`"font/woff2"\` in \`font-types\`
**Tests:**
- 25 tests / 224 assertions covering all three upload paths (direct
bytes, legacy chunk-vector, new chunked sessions), size validation,
and media type validation
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add a script for check the commit format locally
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🎉 Add telemetry anonymous event collection
Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:
Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
telemetry flag active, audit-log flag inactive
Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
Keeps UUID/boolean/number values; for login/identify events preserves
lang, auth-backend, email-domain; for navigate events preserves route,
file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
Backend: version, initiator, client-version, client-user-agent.
Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
access-token fields removed from context
Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option
Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
extraction from email addresses
Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)
RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation
RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API
SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys
Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
per iteration, encodes and sends each page, deletes on success,
stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id
Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
safe base64 for JSON-safe string transport
Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
efficient telemetry batch collection queries
Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support
Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
happy-path batch send and delete, failure retention, payload anonymity,
context stripping, timestamp day precision, batch encoding round-trip,
multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add pr feedback changes
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🎉 Add telemetry anonymous event collection
Rewrite the audit logging subsystem to support three operating modes and
add anonymous telemetry event collection:
Modes:
- A (audit-log only): events persisted with full context
- B (audit-log + telemetry): same as A, plus events are collected for
telemetry shipping
- C (telemetry-only): events stored anonymously with PII stripped,
telemetry flag active, audit-log flag inactive
Audit system refactoring (app.loggers.audit):
- Replace qualified map keys (::audit/name etc.) with plain keywords
- Rename submit! -> submit, insert! -> insert, prepare-event ->
prepare-rpc-event
- Add submit* as a lower-level public API
- Add process-event dispatch function that handles all three modes and
webhooks in a single tx-run!
- Add :id to event schema (auto-generated if omitted)
- Add filter-telemetry-props: anonymises event props per event type.
Keeps UUID/boolean/number values; for login/identify events preserves
lang, auth-backend, email-domain; for navigate events preserves route,
file-id, team-id, page-id; instance-start trigger passes through.
- Add filter-telemetry-context: retains only safe context keys.
Backend: version, initiator, client-version, client-user-agent.
Frontend: browser, os, locale, screen metrics, event-origin.
- Timestamps truncated to day precision via ct/truncate for telemetry
storage
- PII stripped: props emptied, ip-addr zeroed, session-linking and
access-token fields removed from context
Config (app.config):
- Derive :enable-telemetry flag from telemetry-enabled config option
Email utilities (app.email):
- Add email/clean and email/get-domain helper functions for domain
extraction from email addresses
Setup (app.setup):
- Emit instance-start trigger event at system startup
- Simplify handle-instance-id (remove read-only check)
RPC layer (app.rpc):
- wrap-audit now activates when :telemetry flag is set
- Add :request-id to RPC params context for event correlation
RPC commands (management, teams_invitations, verify_token, OIDC auth,
webhooks): migrate all audit call sites to use the new plain-key API
SREPL (app.srepl.main):
- Migrate all audit/insert! calls to audit/insert with plain keys
Telemetry task (app.tasks.telemetry):
- Restructure legacy report into make-legacy-request; distinguish
payload type as :telemetry-legacy-report
- Add collect-and-send-audit-events: loop fetching up to 10,000 rows
per iteration, encodes and sends each page, deletes on success,
stops immediately on failure for retry
- Add send-event-batch: POSTs fressian+zstd batch (base64 via
blob/encode-str) to the telemetry endpoint with instance-id per event
- Add gc-telemetry-events: enforces 100,000-row safety cap by dropping
oldest rows first
- Add delete-sent-events: deletes successfully shipped rows by id
Blob utilities (app.util.blob):
- Add encode-str/decode-str: combine fressian+zstd encoding with URL-
safe base64 for JSON-safe string transport
Database:
- Add migration 0145: index on audit_log (source, created_at ASC) for
efficient telemetry batch collection queries
Frontend:
- Always initialize event system regardless of :audit-log flag
- Defer auth events (signin identify) to after profile is set
- Refactor event subsystem for telemetry support
Tests (21 test vars, 94 assertions in tasks-telemetry-test):
- Cover all code paths: disabled/enabled telemetry, no-events no-op,
happy-path batch send and delete, failure retention, payload anonymity,
context stripping, timestamp day precision, batch encoding round-trip,
multi-page iteration, GC cap enforcement, partial failure handling
- blob encode-str/decode-str round-trip tests (14 test vars)
- RPC audit integration tests (5 test vars)
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add pr feedback changes
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The recursive `read-items` function in `app.util.sse/read-stream`
caused a synchronous stack overflow when reading buffered stream
data. Each `rx/mapcat` call chained another recursive invocation
on the same call stack without yielding to the event loop.
Replace the recursive pattern with an `rx/create`-based async pump
that uses Promise `.then()` chaining, keeping the call stack depth
constant regardless of stream size.
Also add progress reporting with names and IDs during binfile
export and import, and bump `eventsource-parser` dependency.
Closes#9470
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The recursive `read-items` function in `app.util.sse/read-stream`
caused a synchronous stack overflow when reading buffered stream
data. Each `rx/mapcat` call chained another recursive invocation
on the same call stack without yielding to the event loop.
Replace the recursive pattern with an `rx/create`-based async pump
that uses Promise `.then()` chaining, keeping the call stack depth
constant regardless of stream size.
Also add progress reporting with names and IDs during binfile
export and import, and bump `eventsource-parser` dependency.
Closes#9470
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Only fall back to anonymous on :not-found in get-profile
::get-profile caught Throwable and silently returned the anonymous
user payload for every error - contradicting the in-code comment that
states in all other cases we need to reraise the exception. Under
transient DB conditions (pool checkout timeout, replica lag, statement
timeout, network blip) this masked real DB outages as ordinary
anonymous responses, returning HTTP 200 instead of 5xx and leaving
logged-in users on the login screen with a valid session cookie.
Narrow the catch so only :type :not-found falls through; everything
else propagates and reaches the standard error pipeline.
Closes#9235
Signed-off-by: jack-stormentswe <crazycoder131@gmail.com>
* 🐛 Only fall back to anonymous on :not-found in get-profile
::get-profile caught Throwable and silently returned the anonymous
user payload for every error - contradicting the in-code comment that
states in all other cases we need to reraise the exception. Under
transient DB conditions (pool checkout timeout, replica lag, statement
timeout, network blip) this masked real DB outages as ordinary
anonymous responses, returning HTTP 200 instead of 5xx and leaving
logged-in users on the login screen with a valid session cookie.
Narrow the catch so only :type :not-found falls through; everything
else propagates and reaches the standard error pipeline.
Closes#9253
Signed-off-by: jack-stormentswe <crazycoder131@gmail.com>
---------
Signed-off-by: jack-stormentswe <crazycoder131@gmail.com>
Signed-off-by: Jack Storment <88656337+jack-stormentswe@users.noreply.github.com>
