* ⚡ Improve performance and fix orphan detection in validate-file
- Add `*ref-shape-cache*` dynamic var to memoize `find-ref-shape`
lookups per page, avoiding repeated O(depth) ancestor walks.
- Add `*children-sets*` pre-computed maps for O(1) parent-child
containment checks, replacing linear `some` scans.
- Short-circuit `inside-component-main?` when the shape context
already implies a main component.
- Use single-pass reduce with early exit for duplicate detection
(children, swap slots) instead of count/distinct or frequencies.
- Guard `check-missing-slot` to skip expensive `find-near-match`
when the shape already has a swap slot.
- Refactor variant-set validation to use `run!` with direct `get`.
- Refactor `check-ref-cycles` to use a single `reduce-kv` pass.
- Fix `get-orphan-shapes`: the original `map` pipeline produced
nils so orphan shapes were never validated; rewrite with
`reduce-kv` for correct results.
- Add `validate-file-affected!` for change-scoped validation,
replacing full file validation in `process-changes-and-validate`
to only validate pages and components touched by the changes.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Improved validation
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: alonso.torres <alonso.torres@kaleidos.net>
Memories use a system of progressive disclosure:
Starting from a root memory, memories reference other memories using explicit
references.
The new system of hierarchical memories replaces AGENTS.md files.
GitHub #9215
Co-authored-by: Michael Panchenko <michael.panchenko@oraios-ai.de>
Co-authored-by: Codex <codex@openai.com>
* ✨ Add dedicated concurrency limit for restore-file-snapshot
This adds a dedicated climit configuration for the restore-file-snapshot
RPC method with :permits 1 per profile (plus queue of 2 and 60s timeout)
and a global limit of 3. Previously the method only used the generic
root/by-profile and root/global limits, allowing up to 7 concurrent
restore operations per profile which caused database row lock contention
on FOR UPDATE and connection pool exhaustion.
* ✨ Skip locking on restore! to avoid blocking other operations
Changes the row lock acquisition in restore! from a blocking FOR UPDATE
to FOR UPDATE SKIP LOCKED. If the file row is already locked by another
concurrent operation (e.g., another restore or an update-file), the query
returns no rows and the caller fails fast with a clear conflict error
instead of blocking indefinitely holding a database connection.
* ✨ Add queue and timeout limits to root/by-profile concurrency limit
Previously root/by-profile had no queue limit (unbounded Integer/MAX_VALUE)
and no timeout, allowing requests to pile up indefinitely behind a profile
whose permits were exhausted by long-running operations. This could lead
to memory pressure and cascading failures. Now limited to 30 queued
requests with a 30-second timeout so excess requests fail fast.
* ✨ Move backup snapshot creation outside restore transaction
The backup snapshot (fsnap/create!) is now created in its own short-lived
connection before the actual restore transaction begins. This ensures the
backup is persisted independently of the restore outcome and reduces the
restore transaction window.
The restore itself runs inside a db/tx-run! block with an optimistic
locking check: it reads the file with FOR UPDATE and compares its revn
against the value captured at backup time. If the file was edited
concurrently, the restore aborts with a conflict error to prevent data
loss.
Co-dependent with the SKIP LOCKED change in restore! — the FOR UPDATE
acquired here is in the same transaction as restore!, so the SKIP LOCKED
inside restore! correctly sees the row as unlocked (same transaction).
* ♻️ Remove unused private function get-minimal-file
The local get-minimal-file function in file_snapshots.clj is no longer
used since restore! switched to direct exec-one! with FOR UPDATE SKIP
LOCKED. The sql:get-minimal-file SQL constant is still used directly.
* ✨ Add minor improvements on db connection management
* ♻️ Refactor create-file-snapshot to use explicit transaction management
Remove automatic transaction wrapping (`::db/transaction true`) and
pass `cfg` through the call chain instead of destructured `conn`.
Wrap `fsnap/create!` in an explicit `db/tx-run!` for clearer
transaction boundaries.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add dedicated concurrency limit for create-file-snapshot
This adds a dedicated climit configuration for the create-file-snapshot
RPC method with :permits 1 per profile (plus queue of 2 and 60s timeout)
and a global limit of 3. Previously the method only used the generic
root/by-profile and root/global limits, allowing up to 10 concurrent
snapshot creation operations per profile which could cause database
contention and connection pool exhaustion.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Add :uri and :scheme/:host keys to exceptions raised by
`validate-uri` for better error diagnostics. Also fix a bug
where (str url) was used instead of (str uri) in the
host-missing exception path.
Update the existing blocked-target test to verify the new :uri
key, and add three new tests covering scheme rejection, missing
host, and DNS failure error paths. All 27 tests pass with 60
assertions and 0 failures.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The /api/main/doc endpoint was returning HTML content with a
text/plain content-type header instead of text/html. This caused
browsers to render the response as plain text.
Added content-type: text/html; charset=utf-8 header to the
response in the doc handler and added a regression test to
verify the fix.
Closes#9680
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
- Add ::setup/props and ::db/pool to :app.http.assets/routes config
so session renewal works correctly for asset requests.
- Add actoken/authz middleware to the assets middleware chain so
access tokens are properly recognized.
- Add authenticated? helper that checks both ::session/profile-id
and ::actoken/profile-id, fixing 401 errors when accessing
protected assets with a valid access token.
- Add comprehensive test suite for assets auth scenarios.
Closes#9677
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Add a shared `schema:font-family` whitelist validator in
app.common.types.font that only allows letters, digits, spaces,
hyphens, underscores, and dots in font family names. Apply the schema
to create-font-variant and update-font RPC endpoints on the
backend, and add client-side validation in the dashboard fonts UI.
Include unit tests for the schema and integration tests for the RPC
handlers.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 🐛 Fix library updates reappear after file is reloaded
Summary
Migrate synced_at timestamps to a standalone file_library_sync table to ensure sync state is tracked for both direct and transitive libraries.
Problem
Transitive libraries (libraries imported by other libraries) are not stored as direct rows in file_library_rel. Because the system previously coupled synced_at directly to the file_library_rel schema, transitive libraries lacked a persistent location for their sync timestamps. This caused sync states to be lost or incorrectly reported for nested dependencies.
Changes
Schema Migration: Created file_library_sync and migrated existing synced_at values from file_library_rel.
Decoupling: Removed tight Foreign Key coupling to allow sync rows to exist independently of specific relationship records.
Persistent Writes: Added upsert-file-library-sync! helper. Updated all import, duplication, and RPC write paths (v1/v2/v3 importers, link-file-library) to ensure every write persists a sync row.
Unified Reads: Updated both direct and recursive/transitive library queries to fetch synced_at from the new table.
Testing: Added regression tests to verify that sync rows are correctly created/updated even when a transitive relation is absent in file_library_rel.
Impact
This fix ensures that the system accurately records and retrieves sync states for the entire library dependency tree, resolving the bug where nested libraries appeared out of sync.
* ✨ MR review
* ✨ Add additional logging and validation for image upload
* 🎉 Add chunked upload support for font variants
Extend the font variant upload flow across frontend, backend, and common
to support the standardized chunked upload protocol.
**Backend:**
- Add \`:font-max-file-size\` config default (30 MiB) and schema entry
- Add \`validate-font-size!\` in \`media.clj\` (mirrors
\`validate-media-size!\`, raises \`:font-max-file-size-reached\`)
- Extend \`schema:create-font-variant\` to accept either \`:data\`
(legacy bytes or chunk-vector) or \`:uploads\` (new chunked session
map), with a validator requiring exactly one
- Add \`prepare-font-data-from-uploads\`: assembles each chunked
session via \`cmedia/assemble-chunks\`, validates type+size
- Add \`prepare-font-data-from-legacy\`: normalises legacy byte/chunk
entries, writing to a tempfile (joining via SequenceInputStream),
validates type+size
- Add structured logging ("init"/"end") with \`:size\`, \`:mtypes\`,
and \`:elapsed\` in \`create-font-variant\`
**Frontend:**
- \`upload-blob-chunked\` accepts a per-caller \`:chunk-size\` option
- Add \`font-upload-chunk-size\` (10 MiB) and \`upload-font-variant\`
fn that uploads each mtype as a separate chunked session
- \`on-upload*\` in dashboard fonts now calls \`upload-font-variant\`
instead of issuing \`create-font-variant\` RPC directly
- \`process-upload\` stores raw ArrayBuffer instead of chunking
client-side
**Common:**
- Replace \`"font/opentype"\` with \`"font/woff2"\` in \`font-types\`
**Tests:**
- 25 tests / 224 assertions covering all three upload paths (direct
bytes, legacy chunk-vector, new chunked sessions), size validation,
and media type validation
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Add a script for check the commit format locally
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
