Merge remote-tracking branch 'origin/develop' into staging

.agents/skills/create-commit/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: create-commit
 description: Stage, review, and commit files following Penpot commit conventions.
+slash: true
 ---
 
 # Skill: create-commit

.agents/skills/create-issue/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: create-issue
 description: Create or update GitHub issues (from PR, from draft body, retitle existing). Routes to the canonical flow in `mem:workflow/creating-issues`.
+slash: true
 ---
 
 # Skill: create-issue

.agents/skills/create-pr/SKILL.md

@@ -93,8 +93,8 @@ gh pr create --repo penpot/penpot --base "<BASE>" --title "<TITLE>" \
 repository default, which is wrong for a branch cut from `staging`. `--project
 "Main"` is required by `mem:workflow/creating-prs`.
 
-If an issue is present, run the explicit assignment and verification command
-from `mem:workflow/creating-prs` before reporting success:
+If an issue is present, run the explicit assignment command from
+`mem:workflow/creating-prs` before reporting success:
 
 ```bash
 python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
@@ -117,8 +117,7 @@ gh pr view <NUMBER> --repo penpot/penpot --json title,body
 ```
 
 If the updated body contains `Closes #NNNN`, run the explicit assignment
-command from `mem:workflow/creating-prs` and require its verification to
-succeed:
+command from `mem:workflow/creating-prs`:
 
 ```bash
 python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>

.agents/skills/implement-plan/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: implement-plan
-description: Implementation flow — execute a ready plan from the session context: read the plan, detect the flow, then present the full picture (issue and branch to create or the branch to continue on, execution style, task checklist) and wait for confirmation. Default is every task with one final commit; on request ("step by step"), one task and one commit at a time with a pause after each; on request ("direct"), no issue and no branch — the commit lands on the current branch. Use it when the user asks to implement or execute a plan, in any phrasing.
+description: Implementation flow — execute a ready plan from the session context
+slash: true
 ---
 
 # Implement Plan
@@ -13,6 +14,14 @@ By default it ends with exactly one commit. When the user asks for it
 ("step by step"), it commits once per task instead and waits for the
 user's confirmation after each one (see *Execution modes*).
 
+## Plan file handling
+
+Plan files are local workflow artifacts. They must never be staged or
+committed, even when `.gitignore` already excludes them. Never use
+`git add -f`, change ignore rules, or otherwise force a plan file into a
+commit. Close and update the plan when this flow requires it, but leave it
+in the local working tree.
+
 ## When to use
 
 - The user asks to implement or execute a plan, in any phrasing:
@@ -107,12 +116,13 @@ When the implementation is complete, close the plan file first: flip its
 with the issue URL when one exists (standalone mode, e.g.
 `https://github.com/penpot/penpot/issues/NNNN`); in continue/direct
 mode with no issue, just `done` with no invented identifier. Never
-record commit hashes. Then load the **`create-commit`** skill
-and follow its workflow to commit the changes together with the
-closed plan file, so plan and code land in the same commit. Provide a brief summary
-of what was implemented and why, the issue reference (`issue-NNNN`) when
-there is one, and the model name you are running as so the
-`AI-assisted-by` trailer is set correctly.
+record commit hashes. Leave the closed plan file local and uncommitted.
+Then load the **`create-commit`** skill and follow its workflow to commit
+only the implementation, tests, memory, and documentation changes. Before
+committing, verify that the staged file list does not contain the plan
+file. Provide a brief summary of what was implemented and why, the issue
+reference (`issue-NNNN`) when there is one, and the model name you are
+running as so the `AI-assisted-by` trailer is set correctly.
 
 ### Step-by-step mode (on request)
 
@@ -123,9 +133,9 @@ per task" — loop one task at a time:
 - Commit it now: load the **`create-commit`** skill and follow it —
   one commit per task, never two tasks in one commit. Same inputs as
   always: what and why, the issue reference, your model name.
-- After the final task, close the plan file (`Status: done`, one
-  `Review Log` line with the issue URL when one exists) and include
-  it in that last commit.
+- After the final task commit, close the plan file (`Status: done`, one
+  `Review Log` line with the issue URL when one exists). Leave it local
+  and uncommitted; never amend the task commit to include it.
 - Show the user the result (what changed, files touched, how it was
   verified).
 - WAIT for the user's confirmation before starting the next task.
@@ -143,11 +153,6 @@ instruction from me overrides them):
   `/make-a-plan` by itself if the findings need one.
 - `/create-pr` — when the task is done and the branch is ready to merge.
 
-## User context
+## User input, overrides and additional context
 
-Extra context in the user's invocation (the message that triggered this
-skill) plays the role command arguments play elsewhere: `standalone`,
-`continue`, `direct` (`no branch` / `direct commit`), `no issue` /
-`without issue`, an explicit base such as `from origin/develop`, or
-`step by step` / `one commit per task` for the step-by-step execution
-mode. Modes combine freely, for example "standalone step by step".
+$ARGUMENTS

.agents/skills/make-a-plan/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: make-a-plan
-description: Planning flow — research the subject of this session, produce an implementation plan with the planner skill, resolve open questions with the user in plain language, and save the final plan to .agents/plans/. Use it when the user asks to plan, design, or break down a task, in any phrasing.
+description: Planning flow — research the subject of this session, produce an implementation plan with the planner skill, resolve open questions with the user in plain language, and save the final plan to .agents/plans/.
+slash: true
 ---
 
 # Make a Plan
@@ -106,9 +107,6 @@ the final response by suggesting the next steps, in this order:
 These are suggestions, not a required pipeline — any instruction from me
 overrides them (for example, asking you to implement the plan directly).
 
-## User context
+## User input, overrides and additional context
 
-Extra context in the user's invocation (the message that triggered this skill)
-plays the role command arguments play elsewhere: for example, `delegated` to
-hand the research and drafting to the `general` subagent, or corrections and
-feedback about a previous plan.
+$ARGUMENTS

.agents/skills/review-code/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: review-code
 description: Code review flow — review a diff, PR, or code change, delegating the review to a subagent that follows the code-review-criteria skill. Use it when the user asks to review code or a PR, in any phrasing.
+slash: true
 ---
 
 # Review Code
@@ -66,8 +67,6 @@ agent again.
 6. Skip generated files, lockfile-only changes, and unrelated modifications
    unless they introduce security risks.
 
-## User context
+## User input, overrides and additional context
 
-Extra context in the user's invocation (the message that triggered this skill)
-plays the role command arguments play elsewhere: for example, a PR number or
-URL, a commit range, specific files, or a different agent to run the review.
+$ARGUMENTS

.agents/skills/review-plan/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: review-plan
 description: Plan review flow — evaluate an implementation plan before it is executed, delegating the review to a subagent that follows the plan-review-criteria skill. Use it when the user asks to review a plan, in any phrasing.
+slash: true
 ---
 
 # Review Plan
@@ -64,8 +65,6 @@ agent again.
 5. Judge the plan as the implementer would: every task executable without
    guessing, ordering follows the dependency graph, risks named.
 
-## User context
+## User input, overrides and additional context
 
-Extra context in the user's invocation (the message that triggered this skill)
-plays the role command arguments play elsewhere: for example, a plan file path
-to review, or a different agent to run the review.
+$ARGUMENTS

.clj-kondo/config.edn

@@ -22,6 +22,8 @@
    rumext.v2/defc hooks.export/rumext-defc
    rumext.v2/lazy-component hooks.export/rumext-lazycomponent
    shadow.lazy/loadable hooks.export/rumext-lazycomponent
+   app.util.i18n/tr hooks.i18n/tr-dynamic
+   app.common.i18n/tr hooks.i18n/tr-dynamic
    }}
 
  :output
@@ -45,6 +47,9 @@
   :potok/reify-type
   {:level :error}
 
+  :penpot/tr-dynamic
+  {:level :warning}
+
   :redundant-primitive-coercion
   {:level :off}
 

.clj-kondo/hooks/i18n.clj

@@ -0,0 +1,17 @@
+(ns hooks.i18n
+  (:require [clj-kondo.hooks-api :as api]))
+
+(defn tr-dynamic
+  [{:keys [:node]}]
+  (let [[_ code & _] (:children node)]
+    (when (and (some? code)
+               (not (api/string-node? code)))
+      (let [{:keys [:row :col :end-row :end-col]} (meta code)]
+        (api/reg-finding! {:message "dynamic key in (tr ...) is invisible to rehash; use a string literal or declare it with ;; (tr \"key\")"
+                           :type :penpot/tr-dynamic
+                           :row row
+                           :col col
+                           ;; end positions are required for #_:clj-kondo/ignore to match
+                           :end-row end-row
+                           :end-col end-col}))))
+  {:node node})

.github/workflows/build-tmp-tokens.yml

@@ -2,8 +2,8 @@ name: _TMP TOKENS
 
 on:
   workflow_dispatch:
-  schedule:
-    - cron: '46 5-20 * * 1-5'
+  # schedule:
+  #   - cron: '46 5-20 * * 1-5'
 
 concurrency:
   group: ${{ github.workflow }}

.github/workflows/tests-exporter.yml

@@ -9,6 +9,7 @@ on:
     paths:
       - 'exporter/**'
       - 'common/**'
+      - 'render-wasm/**'
 
     types:
       - opened
@@ -23,6 +24,7 @@ on:
     paths:
       - 'exporter/**'
       - 'common/**'
+      - 'render-wasm/**'
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}

.gitignore

@@ -24,7 +24,6 @@ opencode.json
 !AGENTS.md
 !CODE_OF_CONDUCT.md
 !SECURITY.md
-!HIGHLIGHTS.md
 /*.png
 /*.svg
 /*.sql
@@ -59,6 +58,8 @@ opencode.json
 /docker/images/bundle*
 /exporter/target
 /exporter/.shadow-cljs
+/exporter/resources/wasm/
+/exporter/src/app/wasm/shared.js
 /frontend/.storybook/preview-body.html
 /frontend/.storybook/preview-head.html
 /frontend/playwright-report/
@@ -90,6 +91,7 @@ opencode.json
 /playwright/.cache/
 /render-wasm/target/
 /media-processor/dist/
+/media-processor/target/
 /**/node_modules
 /**/.yarn/*
 /.pnpm-store

.serena/memories/backend/auth-permissions-product-domains.md

@@ -7,9 +7,13 @@
 - Basic Penpot registration is token staged: prepare/register creates or verifies temporary tokens, then profile creation/session setup is reused by other auth backends. The frontend `/auth/verify-token` flow is a hub for registration confirmation, email change, and invitation tokens.
 - OIDC-compatible providers share a generic flow: redirect to provider, validate callback/request token, fetch identity data, then login an existing profile or register a new one. Known providers may have hardcoded endpoints; generic OIDC can use discovery/configured endpoints.
 - LDAP login validates credentials against the external directory, fetches identity data, then logs in or registers a matching Penpot profile. LDAP registration is not a separate Penpot signup flow.
+- LDAP session identity MUST come from the directory-returned email (`info.email`): the profile matching the typed email can differ (aliases, UPNs, multi-valued `mail` attributes) and is used only for lockout checks, never to bind the session.
+- Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in.
+- Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`.
 - Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
 - Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
-- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/http-storage-filedata-subtleties`.
+- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile.
+- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/subtleties`.
 
 ## Permission model
 
@@ -37,4 +41,4 @@
 - Enable LDAP login locally with frontend flag `enable-login-with-ldap`; the devenv includes a configured test LDAP service.
 - OIDC testing requires external provider app credentials plus matching backend/frontend config.
 - Backend domain tests usually live under `backend/test/backend_tests/rpc/commands/*_test.clj` or nearby backend test namespaces. Use focused `clojure -M:dev:test --focus ...` from `backend/` when possible.
-- For auth/session or HTTP behavior, combine backend tests with the HTTP/session notes in `mem:backend/http-storage-filedata-subtleties` because RPC-level tests may not exercise cookie/header transforms.
+- For auth/session or HTTP behavior, combine backend tests with the HTTP/session notes in `mem:backend/subtleties` because RPC-level tests may not exercise cookie/header transforms.

.serena/memories/backend/core.md

@@ -4,9 +4,9 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
 
 ## Focused memories
 
-- RPC, DB helpers, workers, cron: `mem:backend/rpc-db-worker-subtleties`
+- Cross-cutting backend subtleties (RPC, DB, workers, cron, HTTP/sessions, storage, file data): `mem:backend/subtleties`
 - Storage abstraction, logical buckets, object lifecycle, deduplication, access, and garbage collection: `mem:backend/storage`.
-- HTTP sessions, config, media processing, and file data persistence: `mem:backend/http-storage-filedata-subtleties`.
+- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
 - Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
 - Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:backend/audit-log`
 - Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
@@ -44,13 +44,13 @@ Database migrations live in `backend/src/app/migrations/`; pure SQL migrations a
 For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump
 the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
 
-For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/rpc-db-worker-subtleties`.
+For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/subtleties`.
 
 ## Background tasks
 
 A task handler is an Integrant component with `ig/assert-key`, `ig/expand-key`, and `ig/init-key`, returning the function run by the worker. New tasks also need wiring in `app.main`: handler config, worker registry entry, and cron entry if scheduled.
 
-For worker dispatch, cron, retry semantics, deduplication, and queue internals: `mem:backend/rpc-db-worker-subtleties`.
+For worker dispatch, cron, retry semantics, deduplication, and queue internals: `mem:backend/subtleties`.
 
 ## REPL
 

.serena/memories/backend/graph-experiment.md

@@ -0,0 +1,631 @@
+# Graph Experiment
+
+## Scope
+
+- Purpose: project Penpot file data into an embedded Ladybug graph database.
+- Purpose: keep the graph current with Penpot file changes.
+- Purpose: expose a read-only graph console for backend debugging.
+- This is an experiment, not a replacement for PostgreSQL file storage.
+- The graph subsystem is off unless `:graph` is in the backend flags.
+- The main Penpot frontend has no graph feature code for this subsystem.
+- The graph console is a backend-served HTML template with JavaScript.
+
+## Memory Links
+
+- Read `mem:backend/core` for backend architecture, HTTP routes, DB rules, and test commands.
+- Read `mem:backend/subtleties` for RPC and message bus behavior, and for file data loading and realization.
+- Read `mem:common/changes-architecture` for the change record vocabulary.
+- Read `mem:frontend/routing-app-shell-subtleties` for the existing notification WebSocket.
+- Read `mem:prod-infra/core` for Redis or Valkey message bus topology.
+
+## Branch Surface
+
+- The graph experiment adds about 6,336 lines and changes about 27 files.
+- The graph implementation lives under `backend/src/app/graph/`.
+- The graph console lives at `backend/resources/app/templates/graph-console.tmpl`.
+- The existing debug page gains graph links in `backend/resources/app/templates/debug.tmpl`.
+- The existing debug HTTP routes gain graph handlers in `backend/src/app/http/debug.clj`.
+- The backend system passes the message bus to the debug route component in `backend/src/app/main.clj`.
+- The backend adds Ladybug and Arrow dependencies in `backend/deps.edn`.
+- The backend adds JVM options for Ladybug and Arrow native access.
+- The common flag registry adds `:graph` in `common/src/app/common/flags.cljc`.
+- The graph experiment adds `graph_sync_parity_test.clj` and `graph_binder_gate_test.clj`.
+
+## System Model
+
+### Storage layers
+
+- PostgreSQL remains the source of truth for Penpot files.
+- The graph database stores a projection of one file.
+- A persistent graph uses a `.lbug` path under `PENPOT_GRAPH_DIR`.
+- The default graph directory is `/tmp/penpot-graph`.
+- A debug session uses a Ladybug `:memory:` database.
+- A debug session database lives inside the backend JVM process.
+- A debug session does not survive a backend restart.
+- A debug session does not store file data back to PostgreSQL.
+
+### Two graph update paths
+
+- Cold projection reads the complete file and rebuilds the graph.
+- Incremental sync reads file change records and updates the open graph.
+- Both paths must produce the same graph for the same file state.
+- The parity test treats cold projection as the reference path.
+- A reload discards the session graph and uses cold projection again.
+
+## Main Namespaces
+
+### `app.graph.ladybug`
+
+- Opens and closes Ladybug `Database` and `Connection` objects.
+- Installs and loads the Ladybug JSON extension.
+- Executes Cypher statements.
+- Executes prepared statements.
+- Binds scalar parameters.
+- Formats UUID, string, integer, number, JSON, and timestamp values.
+- Formats compound values such as arrays, maps, and structs.
+- Converts Ladybug values back to Clojure values.
+- Limits normal query results to 200 rows by default.
+- Detects result truncation with `:truncated?`.
+- Uses query timeout `0` by default.
+- Query timeout `0` disables the timeout.
+- Provides `validate-on-connection!` for parse, bind, and read-only checks.
+- `exec-prepared-on-connection!` prepares every statement before the first execution.
+- A prepare failure stops the batch before a mutation runs.
+
+### `app.graph.schema`
+
+- Provides the public schema facade.
+- Exposes schema version `penpot-graph-slice-4`.
+- Delegates node and relationship definitions to `app.graph.schema.nodes`.
+
+### `app.graph.schema.nodes`
+
+- Holds the single registry for graph node tables.
+- Generates node DDL.
+- Generates relationship DDL.
+- Maps Penpot shape types to graph tables.
+- Projects source attributes into graph attributes.
+- Formats graph column values.
+- Quotes reserved graph labels such as `Group` and `Boolean`.
+- Defines container tables and shape tables.
+- Defines `IsChildOf`, `IsInstanceOf`, `RefersTo`, and `FillsSwapSlot`.
+
+### `app.graph.schema.contract`
+
+- Records deliberate graph contract decisions.
+- Renames graph columns such as `:revn` to `revision`.
+- Drops attributes that do not belong in this graph slice.
+- Records attributes that the graph does not project.
+- Applies per-table dropped attributes.
+- Defines type overrides for vectors, transforms, colors, maps, and JSON arrays.
+- Maps selected map keys to the frontend JSON naming convention.
+- `:background-blur` remains a declared unprojected attribute.
+
+### `app.graph.schema.projection`
+
+- Derives projected schemas from canonical Malli schemas.
+- Builds the projected document schema.
+- Builds projected shape schemas.
+- Selects the schema for each shape type.
+
+### `app.graph.schema.types`
+
+- Maps Malli types to Ladybug types.
+- Maps matrices to `DOUBLE[6]`.
+- Maps points to `DOUBLE[2]`.
+- Maps rectangles to `DOUBLE[4]`.
+- Maps colors to `UINT32`.
+- Maps collections to Ladybug arrays.
+- Maps `:map-of` schemas to `MAP`.
+- Maps closed scalar maps to `STRUCT`.
+- Maps other complex values to `JSON`.
+
+### `app.graph.schema.values`
+
+- Coerces source values to graph column values.
+- Writes fixed vectors with deterministic order.
+- Packs colors into the graph color representation.
+- Sorts set values when deterministic output is needed.
+
+### `app.graph.arrow`
+
+- Loads projection rows with Apache Arrow.
+- Creates temporary staged node and relationship tables.
+- Uses `COPY ... FROM (MATCH ...)` for bulk loading.
+- Groups relationship loads by source and target table pair.
+- Resolves relationship endpoints with joins.
+- Does not use `createArrowRelTable` for UUID relationship endpoints.
+- Keeps the Arrow `RootAllocator` alive until Ladybug releases staged buffers.
+- Closes the allocator after the connection and database close sequence.
+
+### `app.graph.ingest`
+
+- Fetches a complete file with `bfc/get-file` and `:realize? true`.
+- Rejects missing files.
+- Rejects files without file data.
+- Can run file data validation before projection.
+- Creates the DDL.
+- Loads nodes and edges through Arrow.
+- Executes post-load transforms.
+- Writes graph metadata last.
+- Treats the final metadata write as the complete-build marker.
+- Supports a persistent database path and an open connection.
+
+### `app.graph.projection.document`
+
+- Projects `Document`, `Page`, `Component`, and supported shape nodes.
+- Skips the page root frame.
+- Creates `IsChildOf` edges from shapes to parents.
+- Creates page edges to the document.
+- Creates component edges to the document.
+- Stores page order in `Page.index` and edge `position`.
+- Reverses the stored `:shapes` list for Penpot z-order.
+- Adds `page-id` to every projected shape.
+- Propagates an instance head `component-id` to descendants.
+- Stops component inheritance at a non-Frame shape with its own component ID.
+- Skips deleted components during cold projection.
+- Logs unsupported shape types and missing shape records.
+
+### `app.graph.projection.transforms`
+
+- Runs after the base nodes and edges load.
+- `link-component-instances` creates `IsInstanceOf` edges.
+- A Frame needs `component-file` to qualify as an instance head.
+- `link-shape-refs` creates `RefersTo` edges from `shape-ref`.
+- Ladybug limits multi-label relationship `MERGE` statements.
+- The transform emits one statement for each shape-table pair.
+- `link-swap-slots` creates `FillsSwapSlot` edges.
+- Swap slot IDs come from `swap-slot-<uuid>` entries in `touched`.
+- The transform removes swap slot entries from `touched` after edge creation.
+- The transform order matters because it reads and then changes `touched`.
+
+### `app.graph.meta`
+
+- Stores graph provenance in `GraphMeta`.
+- Stores schema version, source revision, producer, and build time.
+- The source revision identifies the file revision used for cold projection.
+
+### `app.graph.stats` and `app.graph.report`
+
+- `app.graph.stats` counts graph nodes and relationships from the live catalog.
+- `app.graph.report` prints ingest information for REPL use.
+
+## Cold Projection Flow
+
+1. Get the file row and realized file data from PostgreSQL.
+2. Read the file revision from the file row.
+3. Build the node and edge projection.
+4. Create all graph tables from the graph schema.
+5. Load node rows with Arrow.
+6. Load relationship rows with Arrow.
+7. Run `CHECKPOINT;`.
+8. Run the registered derived transforms.
+9. Write `GraphMeta` as the final build step.
+10. Return file ID, file revision, database path, projection stats, and transform stats.
+
+### Projection node groups
+
+- `Document` contains file-level attributes without the file data blob.
+- `Document.options` receives file-level options from the data blob.
+- `Page` contains page attributes without the page object map.
+- `Component` contains component attributes without component object maps.
+- Shape tables contain the supported shape attributes.
+- The graph stores selected derived attributes such as `page-id`.
+
+### Projection relationship groups
+
+- Structural edges use `IsChildOf`.
+- Page and component edges point to `Document`.
+- Derived edges come from the post-load transform registry.
+
+## Incremental Sync
+
+### Change source
+
+- `app.rpc.commands.files-update` persists the file update first.
+- The same command publishes a `:file-change` message to the file topic.
+- The topic key is the file UUID.
+- The message contains the file ID, profile ID, session ID, revision, version, and changes.
+- Library changes also publish a team-topic message.
+- The graph session only consumes the file-topic `:file-change` messages.
+
+### Session subscription
+
+- `app.graph.debug/start-sync-loop!` creates a channel with a dropping buffer of 64.
+- The session subscribes the channel to the file UUID topic.
+- The loop reads one message at a time.
+- The loop ignores message types other than `:file-change`.
+- The loop stops when the channel closes.
+- `destroy-session!` closes the channel and purges its message bus subscription.
+
+### Session state
+
+- Sessions are stored in a global `defonce` atom.
+- The map key is the string form of `profile-id`.
+- One profile has one graph session.
+- Loading another file first destroys the old session.
+- A session stores the Ladybug database and connection.
+- A session stores a shared lock for graph access.
+- A session stores file metadata.
+- A session stores the incremental sync index.
+- A session stores the message bus channel.
+- A session stores load time and profile ID.
+- The session keeps projection statistics but drops full projection rows after index creation.
+
+### Sync index
+
+- `build-index` starts from the complete cold projection.
+- The index stores the graph file ID and document ID.
+- The index stores the current graph revision.
+- The index stores page IDs, names, and positions.
+- The index stores component IDs, names, and deleted state.
+- The index stores shape table, parent, position, frame, page, and component context.
+- The index stores child IDs by parent ID.
+- The index supports later change application without another PostgreSQL file read.
+
+### Change application
+
+- `apply-changes!` processes the change list in source order.
+- Each supported change returns a new index and a list of Cypher statements.
+- Unsupported changes enter the `:skipped` result.
+- Supported changes enter the `:applied` result.
+- The function collects all statements before it executes them.
+- The function appends a document revision statement when at least one change applies.
+- The index revision advances only when at least one change applies.
+- A larger incoming revision than the index revision creates a warning.
+- A revision gap does not trigger catch-up.
+
+### Shape change rules
+
+- `:add-obj` reuses `projection.document/denormalized-shape`.
+- `:add-obj` creates the shape node and its parent edge.
+- `:mod-obj` applies supported `:set` operations to graph columns.
+- `:mod-obj` keeps false and zero values as values.
+- `:del-obj` deletes shapes in deep post-order.
+- `:mov-objects` detaches shapes from the old parent.
+- `:mov-objects` closes the old sibling position gap.
+- `:mov-objects` inserts shapes at the new position.
+- `:mov-objects` updates `parent_id` and `frame_id`.
+- `:mov-objects` rewrites container `shapes` values.
+- The parent columns and child lists must match a cold projection.
+
+### Page and component change rules
+
+- Page add creates a projected page node and a document edge.
+- Page delete removes the page subtree.
+- Page modification updates supported page attributes.
+- Component add creates a component node and document edge.
+- Component modification updates supported component attributes.
+- Component delete uses a soft-delete state.
+- Component restore removes the soft-delete state.
+- Component purge removes the component node and document edge.
+- Component sync paths need more parity coverage than the current tests provide.
+
+## Session Locking
+
+- The sync loop and HTTP handlers share one lock per session.
+- The lock protects one Ladybug connection from concurrent access.
+- Queries acquire the lock before binder validation and execution.
+- Graph data export acquires the lock before catalog reads.
+- Session export acquires the lock before `EXPORT DATABASE`.
+- A long query blocks sync for the same session.
+- A sync batch blocks queries for the same session.
+- Ladybug connection thread safety is not assumed.
+
+## Graph Query Rules
+
+- The console accepts Cypher text.
+- Blank query text raises a validation error.
+- The query first passes Ladybug prepare and bind checks.
+- The query must pass the engine read-only analysis.
+- A mutating query is rejected.
+- The graph console does not provide a write path.
+- A session graph is rebuilt from the file by Reload.
+- Normal query results have a 200-row limit.
+- Query results use string values for the HTML console representation.
+- JSON requests receive a Transit JSON response with the query and result.
+- HTML requests receive the rendered console with the result.
+
+## Graph Data Export
+
+### G6 data
+
+- `/dbg/actions/graph-data` reads the live Ladybug database.
+- It does not read the sync index for nodes and edges.
+- It therefore shows database drift if a batch fails after index update.
+- Node export covers all registered node tables.
+- Relationship export reads the Ladybug relationship catalog.
+- Relationship export includes source, target, relationship name, and position.
+- Node and relationship export uses a 100,000-row limit.
+- The response reports `truncated` when a limit cuts the result.
+- The response reports buffer-manager memory usage.
+
+### `.lbug` export
+
+- `source=file` rebuilds the persistent graph from PostgreSQL file data.
+- `source=file` runs a synchronous full ingest for each request.
+- `source=session` exports the caller profile's live in-memory graph.
+- Session export uses Ladybug `EXPORT DATABASE` to Parquet files.
+- Session export creates a new `.lbug` database with `IMPORT DATABASE`.
+- The temporary Parquet staging directory is deleted after import.
+- The final session `.lbug` file remains in the system temporary directory.
+- The HTTP response streams the database file to the caller.
+
+## HTTP Routes and Access
+
+- The graph routes live in `backend/src/app/http/debug.clj`.
+- The graph route list is added only when `:graph` is enabled.
+- `/dbg/graph` serves the graph console page.
+- `/dbg/actions/graph-files` returns the profile file tree.
+- `/dbg/actions/graph-load` loads a file into the profile session.
+- `/dbg/actions/graph-unload` closes the profile session.
+- `/dbg/actions/graph-reload` rebuilds the loaded file graph.
+- `/dbg/actions/graph-query` runs a read-only Cypher query.
+- `/dbg/actions/graph-sync-status` returns the sync state.
+- `/dbg/actions/graph-data` returns nodes and edges for G6.
+- `/dbg/actions/graph-export` streams a `.lbug` database.
+- The `/dbg` session middleware remains active.
+- The `/dbg` admin middleware remains active.
+- A devenv host with a profile ID passes the debug authorization rule.
+- Other hosts need a profile email in the configured admin set.
+- `/dbg/actions/graph-files` lists reachable teams, projects, and files.
+- The file tree query has a 500-file limit.
+- The graph handlers resolve graph namespaces at call time.
+- The backend requires `app.graph.debug` and `app.graph.ingest` when the flag is on.
+- Ladybug native loading then fails during route initialization instead of first use.
+
+## Console Frontend
+
+### Page type
+
+- `graph-console.tmpl` is a backend resource template.
+- It is not a Rumext component.
+- It is not part of the main frontend route table.
+- The page uses browser `fetch` calls and a browser WebSocket.
+- The page loads G6 version `5.1.1` from jsDelivr.
+
+### File tree
+
+- The page fetches `/dbg/actions/graph-files`.
+- The response contains team, project, and file groups.
+- The page creates the tree with DOM APIs.
+- A file click submits the graph load form.
+- The page shows a message when no file exists.
+
+### Graph rendering
+
+- The page fetches `/dbg/actions/graph-data`.
+- The page converts graph nodes and edges to G6 data.
+- The page skips repaint when the node and edge signature does not change.
+- The page marks added, removed, and changed graph entities.
+- The page supports tree, dagre, circular, force, and combo layouts.
+- The page supports collapsed container combos.
+- The page has render guards at 4,000 nodes and 8,000 edges.
+- The `?safe` query option bypasses the render guard.
+- The page shows graph size by node count and relationship count.
+- The page shows buffer-manager memory in MiB.
+- The page reports a CDN failure when G6 is undefined.
+
+### Query result filtering
+
+- A query can return `filter_*` columns with node IDs.
+- The HTML result table hides columns with the `filter_` prefix.
+- The JSON result keeps the full result.
+- The graph view uses the hidden IDs to select matching nodes.
+- The graph view re-runs the query after graph refresh.
+- This keeps the query filter aligned with the current graph.
+- A user column named `filter_*` follows the same hiding rule.
+
+### Node inspector
+
+- A node click creates a query for that node.
+- The inspector calls `/dbg/actions/graph-query` with JSON negotiation.
+- The inspector displays the full projected row.
+- The inspector uses table and ID values from the graph data.
+
+## WebSocket Data Flow
+
+1. The page opens `/ws/notifications` with a random `session-id` query value.
+2. The page sends `:subscribe-file` with a Transit UUID value.
+3. The server makes sure that the file exists and that the profile has read permission.
+4. The server subscribes the connection to the file topic.
+5. `files_update` publishes `:file-change` to the same topic.
+6. The graph session consumes the message from its message bus subscription.
+7. The WebSocket server sends the message to the browser connection.
+8. The browser adds the change to the changelog.
+9. The browser fetches sync status after 150 milliseconds.
+10. The browser fetches graph data after a 400-millisecond debounce.
+11. The browser repaints the G6 graph when the graph data changes.
+
+### WebSocket reconnect behavior
+
+- The page reconnects after three seconds when the socket closes.
+- The page resubscribes to the file after the socket opens.
+- The page refreshes sync status after reconnect.
+- The page refreshes graph data after reconnect.
+- Reconnect does not recover dropped message-bus changes.
+- The page shows the sync error or skipped-change state when the status reports it.
+
+## Feature Flag and Runtime Dependencies
+
+- `:graph` is defined in `common/src/app/common/flags.cljc`.
+- The flag is off by default.
+- `com.ladybugdb/lbug` version `0.19.1` is a backend dependency.
+- `org.apache.arrow/arrow-memory-netty` version `18.2.0` supports Arrow `RootAllocator`.
+- The JVM uses `--enable-native-access=ALL-UNNAMED`.
+- The JVM uses `--add-opens=java.base/java.nio=ALL-UNNAMED`.
+- The JVM uses `--sun-misc-unsafe-memory-access=allow`.
+- The JVM options appear in the development alias and backend launch scripts.
+- A Ladybug version change needs new binder and parity tests.
+- A JDK version change needs a startup test with the graph flag enabled.
+
+## Tests
+
+### `backend-tests.graph-sync-parity-test`
+
+- Uses two Ladybug `:memory:` databases.
+- Does not use PostgreSQL or a live graph session.
+- Projects initial file data into database A.
+- Applies changes to database A through incremental sync.
+- Applies the same changes to file data.
+- Projects the changed file data into database B.
+- Compares every node row and relationship row.
+- Reports differences by table, row key, and column.
+- Covers shape add, shape modification, shape deletion, movement, and page changes.
+- Contains a test that injects a sync defect and expects a graph difference.
+- Does not cover all component change variants.
+- Does not cover every movement insertion mode.
+
+### `backend-tests.graph-binder-gate-test`
+
+- Creates the live graph DDL in a Ladybug `:memory:` database.
+- Prepares each sync statement template without executing it.
+- Detects parse errors and missing tables.
+- Detects missing columns and bad label quoting.
+- Reports the expected read-only classification.
+- Covers reserved node labels across the node registry.
+- Reports an error result for an invalid statement.
+
+### Test gaps
+
+- No automated HTTP handler tests cover graph routes.
+- No automated session lifecycle tests cover load and unload.
+- No automated WebSocket tests cover graph subscription.
+- No automated export tests cover persistent and session sources.
+- Component add, modify, delete, restore, and purge need parity tests.
+- Page delete needs parity coverage.
+- Movement with `:after-shape` needs parity coverage.
+- Buffer overflow and revision gap behavior need tests.
+- Partial batch failure and recovery need tests.
+- Query timeout and long-query behavior need tests.
+
+## Known Risks and Limits
+
+### Dropped changes
+
+- The sync channel uses a dropping buffer of 64.
+- A burst can discard file-change messages.
+- The sync loop logs a revision gap when it sees a larger revision.
+- The sync loop does not fetch missing rows from `file_change`.
+- Reload is the only built-in recovery path.
+
+### Partial batch state
+
+- `apply-changes!` does not provide Ladybug transaction atomicity.
+- A statement failure can leave a partly changed graph.
+- The in-memory index can advance before the database state is complete.
+- `/dbg/actions/graph-data` reads the database and exposes this drift.
+- Reload rebuilds the graph from PostgreSQL file data.
+
+### Query resource use
+
+- The default session query timeout is zero.
+- A costly query can hold the session lock for a long time.
+- The same lock blocks incremental sync.
+- The graph export also holds the same lock during catalog reads.
+- The graph schema has a high memory floor.
+- The console reports about 115 MiB for the wide slice before file data.
+
+### Session lifecycle
+
+- Sessions have no TTL.
+- Sessions remain until unload, replacement, or process shutdown.
+- Each session owns native Ladybug memory.
+- Many profiles can create many native databases.
+- A profile load replaces its previous session.
+- Two browser tabs for one profile share one graph session.
+
+### Temporary files
+
+- Session export leaves the final `.lbug` file in the system temporary directory.
+- Long-lived servers can accumulate exported session databases.
+- The staging directory is deleted after import.
+
+### Browser dependency
+
+- The graph view depends on a runtime CDN request.
+- A network restriction can remove the G6 view.
+- Queries and session status still use backend endpoints without G6.
+
+### Data exposure
+
+- The graph console can list many files available to the profile.
+- The console can load complete projected file data.
+- The console can export a graph database.
+- The console can inspect all projected node attributes.
+- The console is safe only when the `/dbg` access boundary is correct.
+- The graph flag must remain off for deployments that do not need this tool.
+
+### Contract drift
+
+- The graph schema is a deliberate slice of the Penpot file model.
+- New source attributes do not enter the graph automatically in all cases.
+- Dropped and unprojected attributes need an explicit contract decision.
+- `applied_tokens` key mapping depends on the JSON naming function.
+- `filter_*` is a frontend convention, not a graph schema guarantee.
+
+### Ladybug dialect coupling
+
+- Cypher strings contain Ladybug-specific syntax.
+- Label quoting handles reserved labels explicitly.
+- Relationship transforms depend on Ladybug relationship limits.
+- Arrow loading depends on Ladybug `COPY FROM (MATCH ...)` behavior.
+- A dependency upgrade needs schema, binder, Arrow, and parity checks.
+
+## REPL Helpers
+
+- `app.srepl.main` resolves graph functions only when a helper runs.
+- `graph-smoke-test!` runs a basic Ladybug operation.
+- `graph-query-test!` runs a graph query test.
+- `ingest-file-to-graph!` projects a file into a graph database.
+- These helpers use `requiring-resolve` to keep the graph dependency lazy.
+
+## Operational Invariants
+
+- PostgreSQL file data remains authoritative.
+- Cold projection and incremental sync must produce equal graph state.
+- The graph revision must identify the last applied file revision.
+- The document revision must update when a sync batch applies.
+- A missing or skipped change must remain visible in sync status.
+- A graph query from the console must be read-only.
+- A graph session must serialize connection access.
+- Graph routes must remain behind the `:graph` flag and `/dbg` access control.
+- The Arrow allocator must outlive all Ladybug operations that use its buffers.
+- `GraphMeta` must be written after the full ingest and transforms finish.
+
+## Key Files
+
+- `backend/src/app/graph/ladybug.clj`: Ladybug API and query gates.
+- `backend/src/app/graph/arrow.clj`: Arrow bulk load.
+- `backend/src/app/graph/ingest.clj`: Complete file ingest.
+- `backend/src/app/graph/debug.clj`: Session lifecycle, sync loop, query, and export.
+- `backend/src/app/graph/sync.clj`: Incremental change application.
+- `backend/src/app/graph/meta.clj`: Graph provenance.
+- `backend/src/app/graph/stats.clj`: Graph counts.
+- `backend/src/app/graph/report.clj`: REPL ingest report.
+- `backend/src/app/graph/projection/document.clj`: Base document projection.
+- `backend/src/app/graph/projection/transforms.clj`: Derived relationship transforms.
+- `backend/src/app/graph/schema/nodes.clj`: Node and relationship registry.
+- `backend/src/app/graph/schema/contract.clj`: Projection contract decisions.
+- `backend/src/app/graph/schema/projection.clj`: Malli projection schemas.
+- `backend/src/app/graph/schema/types.clj`: Malli-to-Ladybug type mapping.
+- `backend/src/app/graph/schema/values.clj`: Value coercion.
+- `backend/src/app/http/debug.clj`: Graph route registration and handlers.
+- `backend/src/app/http/websocket.clj`: File WebSocket subscription handlers.
+- `backend/src/app/rpc/commands/files_update.clj`: File-change publication.
+- `backend/src/app/main.clj`: Integrant message bus wiring.
+- `backend/resources/app/templates/graph-console.tmpl`: Graph console browser code.
+- `backend/resources/app/templates/debug.tmpl`: Debug page graph links.
+- `common/src/app/common/flags.cljc`: `:graph` feature flag.
+- `backend/test/backend_tests/graph_sync_parity_test.clj`: Cold versus sync parity.
+- `backend/test/backend_tests/graph_binder_gate_test.clj`: Cypher binder gate.
+
+## Development Commands
+
+- Run backend commands from the `backend/` directory.
+- Run focused parity tests with `clojure -M:dev:test --focus backend-tests.graph-sync-parity-test`.
+- Run focused binder tests with `clojure -M:dev:test --focus backend-tests.graph-binder-gate-test`.
+- Run the backend test suite with `clojure -M:dev:test`.
+- Examine Clojure formatting with `pnpm run check-fmt:clj`.
+- Run backend Clojure lint with `pnpm run lint:clj`.
+- Write test output to a file before reading or filtering it.

.serena/memories/backend/http-storage-filedata-subtleties.md

@@ -1,28 +0,0 @@
-# Backend HTTP, Storage, Media, and File Data Subtleties
-
-## Config and HTTP/session middleware
-
-- `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them.
-- `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
-- The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
-- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`.
-- Response formatting negotiates with `Accept` or `_fmt=json`. Transit is the default for collection/boolean bodies; JSON encoding has special pointer-map handling.
-- Auth prefers the session cookie token before the `Authorization` header. Headers may be `Token` or `Bearer`; JWTs with `kid=1` and `ver=1` are decoded as v1 session tokens, otherwise they are treated as legacy tokens.
-- Shared-key auth requires `x-shared-key` as `<key-id> <key>` and stores the lowercased key id on the request. If no shared keys are configured it always rejects.
-- Session management uses DB storage unless the DB pool is read-only, then falls back to the in-memory manager. DB sessions support both legacy string ids and v2 UUID session ids.
-- Session cookies are renewed when using a legacy string id or when `modified-at` is older than the renewal interval. SameSite is `none` for CORS, otherwise strict/lax based on config.
-
-## Storage and media
-
-- Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
-- SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
-- Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
-- Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error.
-- Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts.
-
-## File data persistence
-
-- File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data.
-- `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob.
-- Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written.
-- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.

.serena/memories/backend/rpc-db-worker-subtleties.md

@@ -1,26 +0,0 @@
-# Backend RPC/DB/Worker Subtleties
-
-## RPC exposure and wrappers
-
-- RPC commands are discovered from vars created by `app.util.services/defmethod`; adding a command namespace is not enough unless `backend/src/app/rpc.clj` includes it in `resolve-methods`.
-- `GET`/`HEAD` RPC calls are only allowed for method names starting with `get-`. Other methods are method-not-allowed even if they are read-only internally.
-- RPC auth defaults to enabled. Public endpoints must set `::auth false` metadata explicitly.
-- The wrapper stack does auth before params validation, then auditing/rate/concurrency/metrics/retry/condition handling, with DB transaction handling inside that stack. `::db/transaction` metadata controls transaction wrapping.
-- Params with `::sm/params` are decoded/conformed through the JSON transformer and successful IObj results get `:encode/json` metadata. Legacy spec conforming only applies when no Malli params schema exists.
-- Nil RPC bodies become HTTP 204 unless explicit status metadata is present. Stream bodies default to `application/octet-stream` when no content type is set.
-
-## DB helpers
-
-- Most `app.db` helpers accept a pool, connection, or map containing `::db/pool` / `::db/conn`; preserve that convention in shared code.
-- `db/tx-run!` uses `next.jdbc.transaction/*nested-tx* :ignore`: nested transaction calls reuse the outer transaction, not a savepoint. Use explicit savepoints when nested rollback semantics matter.
-- `db/run!` opens/reuses one connection but does not create a transaction.
-- `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Worker task props use Transit and are decoded with `decode-transit-pgobject`.
-- Advisory transaction locks accept UUIDs or ints. UUID locks are hashed using a zero-UUID seeded siphash.
-
-## Workers and cron
-
-- Task queues are tenant-prefixed. Submit dedupe only removes not-yet-due `new` tasks with the same name/queue/label; it does not dedupe due, scheduled, retry, running, or completed work.
-- The dispatcher selects `new`/`retry` tasks with `FOR UPDATE SKIP LOCKED`, marks them `scheduled`, and publishes Redis payload `[id scheduled-at]`. The runner skips Redis messages whose scheduled timestamp no longer matches DB state.
-- Lost `scheduled` tasks are rescheduled after about 5 minutes; `running` tasks older than about 24 hours are marked failed as orphans.
-- A task handler that is missing or returns an invalid result currently defaults to completed after warning. Throwing with `ex-data :type ::retry` controls retry behavior; `:strategy ::noop` retries without incrementing retry count.
-- Cron jobs lock their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only.

.serena/memories/backend/storage.md

@@ -24,16 +24,52 @@
 - `get-object` excludes rows with `deleted_at`.
 - Existing object values can remain readable until physical deletion.
 - `:expired-at` blocks reads after the expiration time.
-- `del-object!` sets `deleted_at`. It does not remove backend content.
+- `del-object!` sets `deleted_at` on live rows only (`deleted_at IS NULL`): a repeated call returns `false`. It does not remove backend content.
 - `storage-gc-deleted` removes the database row and backend content after the deletion delay.
 - `storage-gc-touched` finds references before it sets `deleted_at`.
 - `objects-gc` removes deleted domain rows and touches their storage object IDs.
 - Use `::db/reuse-conn true` with `sto/resolve` inside a database transaction.
 
+## Connection Reuse Details
+
+### `app.storage/resolve` patterns:
+
+**1. Pool mode (default)** - `(sto/resolve cfg)`
+- Returns storage abstraction from config
+- Uses whatever database pool is available
+- **Safe to call outside transaction context**
+- Used in: `rpc/commands/media.clj:363`, `rpc/commands/auth.clj:327`, `rpc/commands/profile.clj:362`
+
+**2. Connection reuse mode** - `(sto/resolve cfg ::db/reuse-conn true)`
+- Internally calls `db/get-connection cfg` to obtain connectable
+- Configures storage with the specific connection from config
+- **Must be paired with transaction that owns this connection**
+- Used in: `features/fdata.clj:100`, `rpc/commands/media.clj:425`, `rpc/commands/files_thumbnails.clj:307,319`, `binfile/v3.clj:722`
+
+**3. Explicit configuration** - `(sto/configure storage conn)`
+- Sets `::db/conn` on storage map directly
+- Asserts `db/conn? connection` (storage.clj:349)
+- Used inside `db/tx-run!` blocks where `conn` is already available
+- Used in: `tasks/file_gc.clj:256`, `rpc/commands/files_thumbnails.clj:347,371`
+
+### Key Warning (from function notes):
+
+The improved note in `import-storage-objects` and `handle-persistence` warns:
+**Do not reuse the main database connection for storage operations within a transaction.** The storage upload process can fail mid-operation, leaving orphaned objects on the backend. If the outer transaction aborts, pending storage objects become unreconciliable because the storage subsystem registers its pending state in separate transactions.
+
+### Rule of Thumb for `sto/put-object!`:
+
+Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `impl/put-object`) and does not directly use `::db/conn` or `::db/pool`, **all usage of `put-object!` will never run inside a common transaction** (if configured at all). The storage backend operations are independent of the database transaction boundary.
+
 ## Deduplication
 
 - Deduplication requires `::sto/deduplicate?`, a content hash, and bucket metadata.
 - The lookup matches hash, bucket, backend, and `deleted_at IS NULL`.
+- The lookup only considers rows with `status='valid'`; pending rows are invisible.
+- A hit whose blob is missing is repaired in place: the same row/id is kept,
+  and `put-object!` rewrites the blob under that id. This heals all existing
+  references to the object. If the rewrite fails, the row is left live and
+  valid for a later retry.
 - The lookup does not include file ID, profile ID, team ID, or organization ID.
 - Objects can therefore share content across users and files within one bucket.
 - Deleted objects are not reused.
@@ -50,7 +86,8 @@
 | `file-thumbnail` | File grid thumbnails in `file_thumbnail.media_id`. | Yes | Authentication required | Reference scan. |
 | `profile` | User and team profile photos. References: `profile.photo_id` and `team.photo_id`. | Yes | Authentication required | Reference scan. |
 | `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
-| `tempfile` | Export files, chunked-upload chunks, and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
+| `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
+| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | No reference scan. A touched object is deleted after the delay; `gc-deleted` removes mappings before rows. |
 | `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
 | `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
 | `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
@@ -59,7 +96,7 @@
 - `file-media-object` is the default bucket for old rows without bucket metadata.
 - Do not assign a new bucket without adding its access and cleanup behavior.
 - The touched-object collector raises an internal error for an unknown bucket.
-- It supports `file-media-object`, `team-font-variant`, `file-object-thumbnail`, `file-thumbnail`, `profile`, `file-data`, `tempfile`, and `organization`.
+- It supports `file-media-object`, `team-font-variant`, `file-object-thumbnail`, `file-thumbnail`, `profile`, `file-data`, `tempfile`, `upload-session`, and `organization`.
 - It does not support `file-data-fragment` or `file-change`.
 
 ## Access Rules
@@ -81,3 +118,20 @@
 - The `file_data.metadata.storage-ref-id` value points to the storage object.
 - `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row.
 - File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
+
+## Metrics
+
+- `bucket` is always the Penpot logical bucket (object metadata), never an S3 bucket. Unknown/absent buckets are labeled `"unknown"`.
+- `target` is the physical S3 destination id. Today it is always `"default"` (hardcoded in `app.storage.s3/build-s3-client`; the `::target-id` config key was removed as unused until the per-bucket routing plan lands).
+- Physical S3 API calls (AWS SDK `MetricPublisher`, `app.storage.s3.metrics`):
+  - `penpot_storage_s3_requests_total{operation,target,result}` — one count per logical SDK call (the published `ApiCall` collection, not per attempt); retries are counted apart in `retries_total`, so total attempts = `requests + retries`. `result` is `"ok"` only when the SDK reports success as exactly `true`; a missing success flag counts as `error`.
+  - `penpot_storage_s3_retries_total{operation,target}` — SDK retry count.
+  - `penpot_storage_s3_timing{operation,target}` — call latency histogram (ms); explicit buckets up to 60000 ms (S3 slow calls exceed the default 7500 ms cap).
+- Logical storage operations (`app.storage`, `::mtx/metrics` required by the schema):
+  - `penpot_storage_operations_total{op,bucket,backend}` — `put`, `repair`, `get-data`, `get-bytes`, `del`, `touch`, `exists`. All ops are success-only: `put`/`repair` emit after the backend write, `get-*` after the backend fetch opens, `touch`/`del` only when a row actually changed. `touch-object!`/`del-object!` take the object id (UUID) only — no object overload. Labels come from the updated row itself via `UPDATE ... RETURNING id, backend, metadata` (no extra `SELECT`); with no row matched they emit nothing. `del-object!` only matches live rows (`deleted_at IS NULL`): a repeated del returns `false` and emits nothing. Post-open stream read errors stay counted as attempts. `del` only marks `deleted_at`; physical deletion is a GC concern. `exists` is emitted per deduplication-hit probe, always paired with a `hit`/`repair` outcome (never on probe failure), not per user-facing existence check.
+  - `penpot_storage_dedup_total{result,bucket}` — `hit`, `miss`, `repair`, `skip`.
+- Asset serving (`app.http.assets`, `::mtx/metrics` required in the handler cfg):
+  - `penpot_storage_asset_requests_total{route,backend,bucket,result}` — `route` is `by-id`, `by-file-media-id`, or `thumbnail`; `result` is `served` (<400), `not-found` (404), `unauthorized` (401/403), or `error` (everything else, including a nil/non-number status: every serve path must set `::yres/status`). Serve-path exceptions are counted by `serve-object-measured` and then rethrown. Permission-denied file-media requests and tempfile ownership mismatches both answer HTTP 404 (to avoid leaking existence) but are counted as `unauthorized`. Malformed UUIDs raise before any emission point and are never counted. Counts backend requests that trigger a browser GET to the object store (one per cache miss), so it is a proxy for object GETs, not an exact count.
+- The physical and logical counters intentionally overlap in coverage but differ in meaning; do not sum them.
+- Metrics is not optional: `::mtx/metrics` is required by the storage and s3-backend schemas, and the assets handler cfg always carries it. Wiring a component without metrics is a bug, not a supported mode.
+- Recording never fails: `app.metrics/run!` is safe by default at every emit point (`emit-op!`, `emit-dedup!`, `emit-asset!`, and the three S3 publisher emissions). The first recording failure per metric id logs at `warn`, later ones at `debug` (no log flood). The `instance` precondition is a plain assert and the collector lookup is outside the recording guard, so a missing instance fails hard (see `mem:backend/subtleties`). The `publish` outer try/catch stays: it is an SDK `MetricPublisher` contract boundary, not a metrics guard.

.serena/memories/backend/subtleties.md

@@ -45,6 +45,10 @@
 - The xnio worker MXBean can return transient `-1` (e.g. busy-thread count); negative samples are discarded (gauge keeps its previous value). Undertow exposes absolute request/error totals, so the sampler keeps a watermark atom and publishes deltas; a counter reset (decreasing totals) skips the negative delta and moves the watermark forward.
 - The `process_*` families (`process_open_fds`, `process_max_fds`, `process_cpu_seconds_total`, …) come from the prometheus client `StandardExports`, registered by `app.metrics/create-registry`. They read the OS MXBean reflectively and need the `jdk.management` module: on a pruned `jlink` JRE the MXBean is `sun.management.BaseOperatingSystemImpl`, the getters throw `NoSuchMethodException` and `StandardExports#collect` swallows it, so those families silently vanish from `/metrics`. `docker/images/Dockerfile.backend` keeps `jdk.management` in the `--add-modules` list, and `backend-tests.metrics-test` pins the contract.
 
+## Metrics recording
+
+- `app.metrics/run!` is safe by default: a recording failure never throws (a metrics bug must not change the behavior of the measured operation). The first failure per metric id logs at `warn`, later ones at `debug`. The `instance` precondition is a plain assert (the backend enables `:backend-asserts`), and the collector lookup sits outside the recording guard, so a missing instance fails hard even when asserts are disabled. `::mtx/metrics` is required by the storage, s3-backend, and db-pool schemas; `app.db` wires the prometheus `MetricsTrackerFactory` unconditionally. Storage-specific metric contracts: `mem:backend/storage`.
+
 ## Storage and media
 
 - Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.

.serena/memories/backend/testing.md

@@ -8,4 +8,5 @@ JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
 - Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
 - Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var.
 - Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas.
-- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
+- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
+- RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request.

.serena/memories/clojure/idioms.md

@@ -1,10 +1,10 @@
 # Clojure Idioms (verified)
 
-Behaviors confirmed against the language/stdlib — do not re-derive from
-assumption; a wrong assumption here already cost a review round.
+Behaviors confirmed against the language/stdlib — do not re-derive from assumption; a wrong assumption here already cost a review round.
 
-- `int?` is NOT 32-bit-only: true for `Long`, `Integer`, `Short`,
-  `Byte` (fixed-precision integers). Clojure integer literals are
-  `Long`, so `(int? 5000)` is true.
-- `integer?` is the general integer predicate; prefer it when any
-  integer kind must match, `int?` only when fixed precision is meant.
+- `int?` is NOT 32-bit-only: true for `Long`, `Integer`, `Short`, `Byte` (fixed-precision integers). Clojure integer literals are `Long`, so `(int? 5000)` is true.
+- `integer?` is the general integer predicate; prefer it when any integer kind must match, `int?` only when fixed precision is meant.
+- `await` is a `cljs.core` macro asserting `(:async &env)`: it fails at compile time outside an `^:async` context, never silently.
+- The analyzer reads `:async` only from the fn name meta and the `fn` operator meta; list-level meta is ignored (would make a MetaFn). `(fn ^:async [] …)` puts the meta on argv (pre/post only, NOT async).
+- Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself).
+- `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO).

.serena/memories/frontend/core.md

@@ -37,6 +37,7 @@ See `mem:scripts/paren-repair`.
 UI and packages:
 - App UI components, SCSS modules, style-system boundaries, accessibility, i18n, and render performance: `mem:frontend/ui-conventions-and-style-system`.
 - JS/TS packages, shared UI package, text editor, Storybook, and package builds: `mem:frontend/ui-packages-text-editor-workflow`.
+- PO translation workflow and per-locale conventions: `mem:frontend/translations`.
 
 Workspace behavior:
 - Workspace state, commits, persistence, undo, repo calls, and refs: `mem:frontend/workspace-state-persistence-subtleties`.

.serena/memories/frontend/dashboard-viewer-subtleties.md

@@ -13,4 +13,4 @@
 - Viewer bundle fetch sends the full supported feature set because anonymous shared viewers may not know team-enabled features.
 - View-only bundles can contain pointer values in `:pages-index` and file data. Viewer resolves those fragments with `:get-file-fragment` before storing the bundle.
 - `bundle-fetched` indexes pages and precomputes viewer frames/all-frames, stores libraries/users/thumbnails/permissions under `:viewer`, then navigates to frame id, query index, or auto-selected frame.
-- Viewer zoom and interaction mode changes update both `:viewer-local` and the `:viewer` route query params.
+- Viewer zoom and interaction mode changes update both `:viewer-local` and the `:viewer` route query params. `update-zoom-querystring` guards with a query-param comparison (`not= current expected`) and uses `::rt/replace`; without that guard the load sequence (`bundle-fetched` → `zoom-to-fill` → `update-zoom-querystring` → `rt/nav`) re-runs on every navigation and crashes with React "maximum update depth exceeded" (2.18.0-RC5 regression, fixed in `31b73460c3`; regression test: `bundle-fetched-with-zoom-fill-url-does-not-navigate`).

.serena/memories/frontend/routing-app-shell-subtleties.md

@@ -2,12 +2,17 @@
 
 ## Router, app shell, and errors
 
-- Routing uses browser-history hash tokens, but `on-navigate` rejects navigation if the current origin/path does not match `cf/public-uri`.
-- Route params are split into `:path` and `:query`; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required.
+- Routing uses browser-history query tokens (`?screen=<route-name>&params`, single `/` path), but `on-navigate` rejects navigation if the current origin/path does not match `cf/public-uri`.
+- Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required.
+- Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete).
 - Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions.
 - The root app renders an exception page from `:exception` state before the normal error boundary. `rt/navigated` clears `:exception`.
 - Frontend error handling treats stale cross-build JS chunk failures specially: messages containing `$cljs$cst$` or `$cljs$core$I` plus undefined/null/not-a-function signatures trigger throttled reload.
 - Plugin-originated uncaught errors are identified through the plugin runtime hook and logged rather than turning into the global exception page.
+- `app.main.errors/submit-report` is governed by a dedup governor: each report carries a fingerprint (`report-name|type|code|hint|first stack frame`; the report name is part of it so a handled report never coalesces with an unhandled/exception-page one), the first occurrence is always emitted, repeats within 2 minutes are counted and included in the next emitted report as `:occurrences`, and the fingerprint cache is bounded (first-inserted entry evicted, FIFO, via `:order` queue) so memory stays fixed. It applies to `handled-exception`, `unhandled-exception` and `exception-page`; a report without an exception cause is ignored and does not consume a reservation.
+- Errors caused by the environment (`environment-error-types`: `:network`, `:offline`, `:bad-gateway`, `:service-unavailable`, `:nitrate-unavailable`, `:nitrate-not-configured`) are not application defects: they are reported as audit-only `handled-exception` (never `unhandled-exception`/`exception-page`, so they skip internal reports and alerts), with a compact report and a fingerprint that drops the stack frame. `:offline` has its own handler and no longer falls through to `:default`; `:network`/`:offline` show the `errors.connection-error` toast.
+- `generate-report` accepts an explicit `{:format :compact|:full}` (default `:full`) chosen by its caller (`flash`, `exception-section*`): `:compact` keeps the context header plus type/code/uri, and skips the stack, the `ex-data` dump and the last-events list. It is total: if formatting fails it returns a minimal fallback string instead of nil, so an already reserved emission is never dropped.
+- `flash` runs its whole body (report pipeline first, toast after) inside a single `ts/schedule` callback: nothing executes synchronously on the error handler's stack. The report is reserved before being generated, so suppressed occurrences do not pay the `generate-report` cost; a reporting or notification failure is logged to the console and never propagates, and the toast is still attempted. `flash` returns a total promise (never rejects) resolving with the generated report, or nil when nothing is emitted, once the callback completes; production callers ignore it, tests await it. `flash` derives only the payload format from the cause (`environment-error?` → `:compact`); the audit event name is the canonical one requested by `:type` (`handled-exception`/`unhandled-exception`) and is never reclassified, because external tools filter on those names. `exception-section*` picks `handled-exception`/`exception-page` explicitly per cause; `flash-persistence` only adds the toast hint.
 
 ## Store and websocket
 

.serena/memories/frontend/testing.md

@@ -8,6 +8,38 @@ READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee
 
 Frontend unit tests live under `frontend/test/frontend_tests/` and use `cljs.test`. They should be deterministic, avoid DOM/UI integration where possible, and mock side effects such as RPC, storage, timers, or network access.
 
+### Async-first stance
+
+Frontend testing is async-first: everything essentially asynchronous is modeled with a test reproducing the asynchrony, even when the test could be written "synchronously". Sync-passing tests prove nothing about async behavior and rot as soon as an async boundary appears downstream. Consequences: mock through `frontend-tests.helpers.mock`, never `with-redefs`, except unit tests of purely synchronous functions; transport doubles deliver asynchronously (`observe-on :async`) while the test keeps scenario timing (explicit pushes); assertions always follow quiescence (`wait-for` on presence, bare `settle` tick for absence-only blocks), never a trigger.
+
+### Primitives
+
+- `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests.
+- `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`.
+- `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses).
+- Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for` (immediate check + bounded poll, fails instead of hanging), `observe` (stream → termination promise; asserts provided, timeout rejects).
+- Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left).
+- Valid `^:async` placements: `mem:clojure/idioms`.
+
+### Observing event streams
+
+To assert over emitted event sequences, observe termination: subscribe through `observe` (async delivery forced even for sync sources), `await` its promise, then assert the collected values. Never branch on nil (`when-let` skipping observation lets setup bugs pass as "empty"): producers answer refusals with empty streams, never nil, so every path subscribes uniformly. Observed termination is exact quiescence — no manual `settle` after it. Errors reject unless `:on-error` handles them.
+
+### Runner and library facts (verified: CLJS 1.12.145, beicon2 `df7058a`)
+
+- `cljs.test` keeps its env in a `set!` var: assertions inside deferred ticks count. `run-block`: double `done` only warns; missing `done` stalls.
+- `t/async` discards the body promise — completion signals ONLY via `done`. `t/deftest ^:async` adds auto async-context + auto `done`, but awaiting stays the author's job; without it the test passes empty.
+- `take 1` is per-subscription on a hot subject: subscribe-before-push or hang. `end!`/`.error` with pending takes only forwards valueless completion.
+
+### Traps that bit
+
+- `^:async` tests require map-style fixtures (`(t/use-fixtures :each {:before f})`): function-style fixtures abort the whole run ("Async tests require fixtures to be specified as maps").
+- `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`.
+- Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline.
+- Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis.
+- Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows).
+- A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop.
+
 From `frontend/`:
 - Full unit test run (always builds, suppressed output): `pnpm run test:quiet`.
 - Full unit test run (always builds, build output visible): `pnpm run test`.

.serena/memories/frontend/translations.md

@@ -0,0 +1,57 @@
+# Frontend Translations
+
+PO-based UI i18n. Files: `frontend/translations/*.po`. `en.po` is the
+source of truth (`msgid` = key, `msgstr` = English); `es.po` is a
+high-coverage support reference, never the base.
+
+## Workflow
+
+- Canonicalize with `node ./scripts/translations.js sync -l <locale>` from `frontend/`: sorts entries, syncs `#:` comments from `en`, deletes keys missing in `en`.
+- `sync` copies ALL comment flags from `en`, including `#, fuzzy`. Translated entries must NOT stay fuzzy: strip the flag after translating (mirror `es.po`, which keeps fuzzy only on genuinely untranslated entries). Fuzzy entries are excluded from `msgfmt --statistics` translated counts.
+- Canonical files carry NO `#~` obsolete blocks (`en`/`es` have zero). Drop them; `sync` does not resurrect them.
+- Definition of done: `msgfmt --check <locale>.po` exit 0 and `msgfmt --statistics` shows 0 fuzzy, 0 untranslated.
+- `rehash` scans `frontend/src` AND `common/src` for `(tr "key"` occurrences (any line counts, including `;;` comments) and refreshes `#:` refs in `en`, marking unreferenced keys `#, unused`.
+- Dynamic keys are invisible to `rehash`: eliminate them instead of
+  declaring them. Preference order: literal `(tr "key")` args only;
+  never put a branch inside `tr`, hoist it out (`(if cond (tr "a") (tr "b"))`);
+  resolve code→message maps with `case` returning literal calls;
+  key-forwarding components take pre-translated strings (callers translate
+  with literals). Only when no simple fix exists (open key sets sent by the
+  backend, large data-driven matrices like `shortcuts.*`) declare each key
+  with a `;; (tr "the.key")` comment next to the call site (grouped under
+  `;; Execution time translation strings:`), otherwise the next `rehash`
+  flags them `unused`. The declaration convention is legacy fallback, not
+  the default: never introduce new declarations where a hoist or
+  pre-translation works, and remove existing ones when fixing the call
+  site. Same for `:error/code "key"` data: prefer `:error/fn #(tr "key")`. Any non-literal first arg to `tr` is reported by the `:penpot/tr-dynamic` clj-kondo warning (hook in `.clj-kondo/hooks/i18n.clj`, registered for `app.util.i18n/tr` and `app.common.i18n/tr`) and forbidden by the `tr` docstring: only `(tr "literal" ...)` is statically resolvable. Translation keys must not contain spaces (use `-` or `.`).
+- Shortcut command/section/subsection labels (`shortcuts.*`, including `shortcuts.section.*` and `shortcuts.subsection.*`) resolve through `:label` fns holding static `(tr "literal")` calls, executed at render time: commands carry `:label` on their definitions in the `app.main.data.*.shortcuts` namespaces, sections/subsections in static registries in `app.main.ui.shortcuts` next to `translation-keyname` (id lookup with raw-key fallback for stale custom ids). When adding, renaming, or removing commands/sections/subsections, add/update/remove the `:label`/registry entry; `rehash` sees the literals with no comment needed, and the exhaustiveness test fails otherwise. Exempt, untranslated by design: debug `:preview-frame` and colorpicker `:delete-stop` (no PO keys).
+
+## Entry rules
+
+- Keys resolved dynamically from server-provided data carry a `#, backend`
+  flag (set it in `en`, `sync` copies it to the locales): do not remove or
+  rename these entries, and keep their `%s` placeholders (the backend only
+  sends the key). The code side keeps a `;; (tr "key")` marker for `rehash`
+  plus a `#_{:clj-kondo/ignore [:penpot/tr-dynamic]}` on the call site.
+- New entries take `#:` refs from `en`; copy `#, unused`, never `#, fuzzy`.
+- `en` keys with empty `msgstr` (or `#, fuzzy` + empty): translate from `es`/source context, never leave empty.
+- Entries whose `en` uses `msgid_plural` need `msgstr[0]`/`msgstr[1]` (header: `nplurals=2; plural=n != 1`). Keep the `msgid_plural` line: a singular `msgstr` on a plural key silently breaks count selection at runtime (the app build reads 1-elem `msgstr` as singular). Exception: single-form locales (`nplurals=1`, e.g. `jpn_JP`, `ko`) keep ONE `msgstr[0]` that includes the count `%s` (build emits a plain string, runtime formats it with the count for every n); `check` compares it against the `en` plural form.
+- `sync` re-adds `#, fuzzy` on EVERY run for entries fuzzy in `en`; re-strip after the last sync, never before it.
+- Preserve verbatim: `%s`/`%d`, `{var}`/`{{...}}`, markdown `[text](%s)`, HTML tags, `\n` positions, brand names (Penpot), key names (Ctrl/Shift/Alt), technical terms (SVG, CSS, HSV, RGB).
+
+## Catalan (ca) conventions
+
+- Normative IEC/Termcat Catalan. Address the user in VOSALTRES (2nd person plural): "Deseu", "Creeu", "Ja teniu un compte?". Buttons/menus use short imperatives ("Crea", "Mou", "Restaura").
+- Established glossary (reuse exactly, do not re-coin): layer=capa, board=tauler, stroke=traç, fill=Emplenat, blur=Difuminat, shadow=Ombra, clipboard=porta-retalls, delete=Elimina, rename=Canvia el nom, shortcut=drecera, grid=graella (keep "grid" where the file already does, e.g. grid-layout editing), plugins/extensions UI=extensions, layout=Disposició, gradient=Degradat, wireframing kept as loanword.
+- Ela geminada uses the middle dot: Cancel·la, paral·lel, al·lega.
+- ALL-CAPS source stays ALL-CAPS in Catalan; keep `$175`-style amounts in IEC format (`175 $/mes`) only where `es` already adapts.
+- Shortcut/action names (`shortcuts.*`) are noun/infinitive labels, not sentences. Error strings are direct, no hedging.
+- Same English source in different contexts may legitimately differ (verb "Copia" vs noun "Còpia"; "Desactivat" vs "Deshabilitada" agreeing with "drecera"). Normalize only true duplicates.
+
+## QA before commit
+
+- Run `node ./scripts/translations.js check -l <locale>` from `frontend/` (no default locale: pass `-l` explicitly): 0 errors required; review warnings by hand. Word lists live in `frontend/scripts/check-translations/words.<locale>.txt` (`[elision]` `[function]` `[common]` `[ok]` `[brands]`); new valid words that trip the gate go to `[ok]`; `check --self-test` covers the detector rules. Without a catalog only the universal checks run. `#, fuzzy` entries are skipped (known-pending, owned elsewhere).
+- Placeholder parity per entry (singular AND each plural form, also enforced by the script); verify `%s` against the `tr` call site when `en`/`es`/code disagree (a `%s` the code never passes renders literally; a dropped one swallows the argument). On `#, unused` keys the script only warns: never "fix" them by deleting placeholders or links, a reactivation may need them.
+- Glued words (AI batches drop spaces at wrap boundaries): the script flags function-word splits (`del'equip`, `lapolítica`, `sinecessiteu`), `,/.`/`:` without following space, lowercase+Uppercase joins (`delPenpot`, `oCapitalize`) and `%s` glued to a word.
+- Balanced `[]`/`()` in markdown links; no double spaces; no glued words around `·`; trailing spaces match the source.
+- `git diff --stat` must touch only `frontend/translations/<locale>.po`.

.serena/memories/frontend/ui-conventions-and-style-system.md

@@ -33,6 +33,7 @@
 ## Performance
 
 - Keep expensive derived data in refs, memoized selectors, or pure helpers. In hot render paths, prefer existing `app.common.data.macros` helpers where local code already uses them.
+- Derive index-aware or sorted/filtered sequences once: wrap the transformation in `mf/with-memo` keyed on the source collection instead of calling `d/enumerate` in the render body. Use the shared `d/xf:add-index`, which attaches `:app.common.data/index` to each item; the items must be associative (maps/records), so it does not work on keywords or plain ids.
 - Avoid creating new callback functions/objects inside hot renders when a named function, memoized callback, data attribute, or precomputed JS props object works.
 - Destructure props/state values used repeatedly. Avoid repeated deref/property access in render loops.
 

.serena/memories/frontend/workspace-state-persistence-subtleties.md

@@ -22,6 +22,10 @@
 - Persistence buffers local commits: status becomes pending after about 200ms, commits are flushed after about 3s or `::force-persist`, and buffered commits are merged per file before `:update-file`.
 - Persistence sends revn as the max of the commit revn and locally tracked latest revn; remote commits update that revn tracker.
 - Persistence is skipped in version preview/read-only mode or without edit permission.
+- Save failures split transient vs terminal (`transient-error?`: the repo retryable types `:network`/`:offline`/`:bad-gateway`/`:service-unavailable` plus `:invalid-save-response`; everything else is terminal). Terminal keeps the `:error` halt + `flash-persistence` path; transient enters a `:retrying` episode: the head commit stays queued and resends with backoff 2s/8s/20s (3 retries, then today's terminal path). Resends rotate the `::request-id` stamp only when the old request left `active-requests`, carry the same `:commit-id`, and never double-send while one request is in flight (`:in-flight` stays silent). Retry timers carry the episode token; a superseded token stays silent. Status stays `:retrying` through re-entries (`next-status` refuses `:pending`/`:saving` from it); waiters (`wait-persisted-or-error`) wait through it and reject only on `:error`.
+- One reconnect notice per episode: sticky toast tagged `:persistence-reconnecting` (single-toast store, re-show replaces), hidden by tag on drain (`:saved`) and on terminal failure; recovery is silent. Header indicator has a `:retrying` state (`workspace.header.retrying`).
+- Resume triggers: backoff timer and the browser `online` event (guarded by `exists? js/window`; re-enters the runner only for a live `:retrying` episode). New local edits during `:retrying` only join the queue: the episode keeps its `:run-id`, so `append-commit` does not re-enter, and the live runner (still waiting on the head's `commit-persisted`) sends them after the head. Re-entering on edits would bypass the backoff and spend an attempt per batched commit.
+- Tests instant-trigger retries by stubbing `rx/timer` (recording delays to assert the schedule); dynamic bindings do not survive `await` continuations, so no dynamic var for delays.
 - Undo transactions can stay open only temporarily; timed-out pending transactions are force-committed after about 20s. Undo entries are capped at 50.
 - Undo/redo are ignored while a normal editor/drawing interaction is active, except grid-layout edition handles undo through this path.
 - After local commits and when render-wasm is active, text shapes get derived `:position-data` recomputed in a separate commit tagged `#{:position-data}`; that tag is excluded from the position-data watcher to avoid loops.

.serena/memories/mcp/core.md

@@ -94,3 +94,9 @@ In the normal Penpot devenv MCP path, the browser plugin does not discover or ro
 The live plugin connection registry is in-memory inside each MCP server process (`PluginBridge.connectedClients` / `clientsByToken`). The database only stores MCP access tokens and profile props such as `mcp-enabled`; it does not manage which plugin is connected to which MCP server.
 
 For parallel devenvs, prefer same-origin MCP routing: each Penpot instance should expose `/mcp/ws` through its own nginx/Caddy path to the MCP server running inside the same main container. Keep container-internal ports fixed (MCP defaults `4401/4402/4403`, backend/exporter/frontend defaults, etc.) and only offset host-side published ports per instance. If internal ports are offset, hardcoded local proxy config such as `docker/devenv/files/nginx.conf` will misroute unless templated too.
+
+## Plugin reconnect policy
+
+- The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS.
+- The MCP server emits `1008` for a duplicate connection on the same user token (`PluginBridge`) and for a missing `userToken` in multi-user mode.
+- A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via "Connect here".

.serena/memories/prod-infra/core.md

@@ -31,3 +31,4 @@ Penpot in production lives with both: horizontal-scale deployments accept "exact
 
 - Devenv composition and the ws0-only worker placement: `mem:devenv/core`.
 - Storage backend resolution, dedup, bucket behavior, object lifecycle, and file-data lifecycle: `mem:backend/storage`.
+- Storage operation metrics (S3 API calls, logical ops, dedup, asset requests) and the logical-bucket vs physical-target distinction: `mem:backend/storage` (Metrics section).

.serena/memories/render-wasm/core.md

@@ -20,11 +20,18 @@
 
 `./build` sources `_build_env`, which sets the Emscripten paths and `EMCC_CFLAGS`. The WASM heap starts at 256 MB and uses geometric growth.
 
+- Linux builds require `flock` (util-linux).
+- `build`, target cleanup, and each watch rebuild share `render-wasm/.render-wasm-build.lock` per checkout. The dispatcher for both targets does not hold the lock in its parent process.
+- The lock covers setup, Cargo build, and artifact copy. The watch process releases it while waiting for changes. The operating system releases it on success, error, or signal; the file remains.
+- The lock only coordinates repository scripts. Manual Cargo commands do not take it.
+- Set `RENDER_WASM_LOCK_FILE` to the same path only when separate checkouts intentionally share a `CARGO_TARGET_DIR`.
+
 ## Commands
 
 From `render-wasm/`:
-- Build/copy frontend artifacts: `./build`.
-- Watch rebuild: `./watch`.
+- Build/copy frontend artifacts: `./build [frontend|export]`; no target builds frontend, then export.
+- Clean one target: `./clean [frontend|export]`; never run root `cargo clean` on the shared `target/`.
+- Watch rebuild: `./watch [frontend|export]`; its initial and change-triggered builds use `./build`.
 - Rust tests: `./test` or `cargo test <name>`.
 - Cross-cutting testing principles and anti-patterns: `mem:testing`.
 - Lint: `./lint`.

.serena/memories/render-wasm/ffi-rendering-subtleties.md

@@ -17,9 +17,32 @@
 
 ## Tile/render behavior
 
+- Raster `Fill::Image`: skip `save_layer` unless the shape has an image filter; plain
+  Rect/Frame (no corners) also skip the container clip (`draw_image_fill` in fills.rs).
+- `can_render_directly` paints onto Current (no Fills/Strokes blit) for plain geometry and
+  for stroke-free text (SrcOver, no blur/shadows). Multi-style text is fine: span styles
+  live in Paragraph `TextStyle`s. Text skips the `nested_fills` guard (fills are on spans).
+  `draw_text` only `save_layer`s when stroke-group opacity is set; plain fill paint is direct.
+- Plain text fill paint reuses `TextContent.layout` paragraphs when
+  `has_usable_paint_layout` (paragraphs present + version match; during
+  interactive transforms rotation/move skips width check via
+  `modifier_changes_text_layout`, resize falls back to `layout_width` vs
+  `get_width(selrect.width())`), via `text::try_paint_from_layout_cache`.
+  The walker computes `text_layout_cache_rotation_only` from `tree` and
+  passes it into `render_shape`; stroke/shadow paths pass `false`.
+- `TextContentLayout` paragraphs are `Rc`-shared on `Clone` so modifier clones
+  (rotate/pan) keep the paint cache; `needs_update` is paragraphs-empty only.
+  Decorations are skipped when no span requests underline/strike.
+- Zoom settle: visible tiles present via `FrameType::ViewportReady` before interest-ring
+  work; crop-cache rebuild is deferred to the later `Full` so the soft→sharp snap is
+  compose+present only.
 - Interactive transforms are distinct from viewport fast mode. `set_modifiers_start` enables fast mode and interactive transform; interactive transform still flushes each animation frame.
 - During interactive transform, modifier tile invalidation is deferred to `render()` once per rAF. Outside interactive transform, `set_modifiers` rebuilds modifier tiles immediately.
 - `set_modifiers_end` disables fast/interactive state and cancels pending async render; the caller must request the final full-quality render.
 - Plain viewport fast mode (`options.is_viewport_interaction()`) renders from cache and does not flush target output inside `process_animation_frame`; interactive transforms do flush.
-- Zoom changes rebuild the tile index while preserving cached tile textures. Avoid replacing that path with shallow rebuilds if blur/shadow cache preservation matters.
-- Pending tile priority is intentionally reversed by pop order; check the queue construction before changing tile scheduling.
+- Zoom settle wipes the tile texture cache in `set_view_end`. Mid-zoom overlays
+  key tiles by scale; shape edits must `invalidate_cached_tiles_intersecting`
+  the old∪new extrect so those overlays do not keep pre-edit pixels.
+- Pending tile priority is intentionally reversed by pop order; check the queue construction before changing tile scheduling.
+- Frames with a fill may use `render_frame_container_drop_shadow` (direct rrect +
+  blur saveLayer on `DropShadows`) when `uses_direct_container_drop_shadow` is true.

.serena/memories/scripts/gh.md

@@ -9,7 +9,7 @@ repository via GraphQL and REST APIs through the authenticated `gh` CLI.
 - Finding issues with no milestone.
 - Fetching PR details by number or by milestone.
 - Comparing milestone issues against CHANGES.md to find missing entries.
-- Explicitly linking a GitHub issue to a pull request and verifying both sides.
+- Explicitly linking a GitHub issue to a pull request.
 - Listing or inspecting GitHub Security Advisories (GHSA).
 
 ## Prerequisites
@@ -76,14 +76,14 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
 
 ### `link-issue`
 
-Explicitly assign a GitHub issue to a pull request and verify the relationship from both sides:
+Explicitly assign a GitHub issue to a pull request:
 
 ```bash
 python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
 # Short alias: python3 scripts/gh.py link <ISSUE_NUMBER> <PR_NUMBER>
 ```
 
-The command resolves both node IDs, calls `addCloseIssueReferences`, and checks the issue's manually linked PRs and the PR's closing issue references. It is safe to rerun, works for merged PRs, and does not close an issue retroactively. JSON goes to stdout; progress and errors go to stderr; a missing link exits non-zero.
+The command resolves both node IDs and calls `addCloseIssueReferences`. It trusts the successful mutation instead of re-querying, because GitHub does not reliably report mutation-created links through `closedByPullRequestsReferences(userLinkedOnly: true)`. It is safe to rerun, works for merged PRs, and does not close an issue retroactively. JSON goes to stdout; progress and errors go to stderr; a missing issue/PR or a failed mutation exits non-zero.
 
 ### `advisories`
 

.serena/memories/workflow/creating-issues.md

@@ -277,7 +277,7 @@ Add `Closes #<ISSUE_NUMBER>` to the PR body for readable context, then run the e
 python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
 ```
 
-The command creates the GitHub Development link and verifies it from both the issue and PR. It is safe to rerun and does not close an issue retroactively when the PR is already merged. Do not rely on the body keyword as the assignment operation.
+The command creates the GitHub Development link by calling `addCloseIssueReferences` and trusts the successful mutation (GitHub does not reliably report mutation-created links back through the API). It is safe to rerun and does not close an issue retroactively when the PR is already merged. Do not rely on the body keyword as the assignment operation.
 
 ### Clean up
 

.serena/memories/workflow/creating-prs.md

@@ -80,7 +80,7 @@ The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
 ## Explicit Issue Assignment
 
 - For each GitHub issue that a PR resolves, run `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` after creating or editing the PR. Do not rely on `Closes #NNNN` in the body; it is only human-readable context.
-- The command calls `addCloseIssueReferences`, verifies the relationship from both the issue and PR, and exits non-zero if either side is missing. It is safe to rerun and also works for an already merged PR; it does not close an issue retroactively.
+- The command calls `addCloseIssueReferences` and trusts the successful mutation: GitHub does not reliably report mutation-created links back through `closedByPullRequestsReferences(userLinkedOnly: true)`, so the command exits non-zero only when a link target is missing or the mutation fails. It is safe to rerun and also works for an already merged PR; it does not close an issue retroactively.
 - Skip this process for `Relates to #NNNN` and Taiga references, which do not represent a closing relationship.
 
 ## Before Opening

CHANGES.md

@@ -1,5 +1,52 @@
 # CHANGELOG
 
+## 2.19.0 (Unreleased)
+
+### :rocket: Epics and highlights
+
+- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237))
+- Improve path operations and edition in the path editor [#10889](https://github.com/penpot/penpot/issues/10889) (PR: [#10807](https://github.com/penpot/penpot/pull/10807))
+- Add auto-linking of libraries during import based on slugified name [#9263](https://github.com/penpot/penpot/issues/9263) (PR: [#9958](https://github.com/penpot/penpot/pull/9958))
+
+### :bug: Bugs fixed
+
+- Fix copying text from Penpot to the clipboard not working on MS Windows [#11303](https://github.com/penpot/penpot/issues/11303) (PR: [#11305](https://github.com/penpot/penpot/pull/11305))
+- Fix performance issue with WebGL render [#11240](https://github.com/penpot/penpot/issues/11240) (PR: [#11259](https://github.com/penpot/penpot/pull/11259))
+- Fix comment bubbles rendering on top of workspace dropdown menus [#10283](https://github.com/penpot/penpot/issues/10283) (PR: [#11201](https://github.com/penpot/penpot/pull/11201))
+- Fix inconsistent Mixed label in blur options and numeric inputs across 24 locales (by @filipsajdak) [#11148](https://github.com/penpot/penpot/issues/11148) (PR: [#11151](https://github.com/penpot/penpot/pull/11151))
+- Fix overlay shifting left when shown with top-center alignment in viewer prototype (by @filipsajdak) [#9048](https://github.com/penpot/penpot/issues/9048) (PR: [#10454](https://github.com/penpot/penpot/pull/10454))
+- Fix internal error when clicking the Copy button on the Access Token page (by @0xTHAC0) [#8496](https://github.com/penpot/penpot/issues/8496) (PR: [#11156](https://github.com/penpot/penpot/pull/11156))
+- Fix `disable-registration` flag not preventing non-users from creating accounts in the share prototypes page (by @0xTHAC0) [#5164](https://github.com/penpot/penpot/issues/5164) (PR: [#11199](https://github.com/penpot/penpot/pull/11199))
+- Fix "Cannot assign to read only property 'toString'" error during text resize (by @makesomethingshit) [#10168](https://github.com/penpot/penpot/issues/10168) (PR: [#11521](https://github.com/penpot/penpot/pull/11521))
+- Fix plugin postMessage channel broadcasting messages to all plugins without origin validation [#10968](https://github.com/penpot/penpot/issues/10968) (PR: [#10970](https://github.com/penpot/penpot/pull/10970))
+- Fix MCP plugin page navigation while connected crashing the workspace (by @makesomethingshit) [#11001](https://github.com/penpot/penpot/issues/11001) (PR: [#11521](https://github.com/penpot/penpot/pull/11521))
+- Fix shortcut search never matching on key combination, only on action label [#11003](https://github.com/penpot/penpot/issues/11003) (PR: [#11081](https://github.com/penpot/penpot/pull/11081))
+- Fix Shift + special character key shortcut capturing the shifted character instead of the physical key [#11004](https://github.com/penpot/penpot/issues/11004) (PR: [#11081](https://github.com/penpot/penpot/pull/11081))
+- Fix reassigning the "Paste" shortcut not updating the UI or taking effect in the workspace [#11005](https://github.com/penpot/penpot/issues/11005) (PR: [#11081](https://github.com/penpot/penpot/pull/11081))
+- Fix font-size dropdown clipping multi-digit values in Firefox (by @0xTHAC0) [#11008](https://github.com/penpot/penpot/issues/11008) (PR: [#11162](https://github.com/penpot/penpot/pull/11162), [#11500](https://github.com/penpot/penpot/pull/11500))
+- Fix exporting shortcuts producing an invalid "toggle-fullscreen" entry that breaks re-import [#11032](https://github.com/penpot/penpot/issues/11032) (PR: [#11081](https://github.com/penpot/penpot/pull/11081))
+- Fix plugin API missing permission checks in tokens, shapes, variants, flows, layouts, and user identity [#11137](https://github.com/penpot/penpot/issues/11137) (PR: [#11139](https://github.com/penpot/penpot/pull/11139))
+- Fix library summary Redis cache keys omitting the tenant [#11407](https://github.com/penpot/penpot/issues/11407) (PR: [#11408](https://github.com/penpot/penpot/pull/11408))
+- Fix active theme name in the inspect tab displaying an id instead of the name [#11437](https://github.com/penpot/penpot/issues/11437) (PR: [#11439](https://github.com/penpot/penpot/pull/11439))
+- Fix triple-click not selecting the full line in text editor v3 [#11483](https://github.com/penpot/penpot/issues/11483) (PR: [#11493](https://github.com/penpot/penpot/pull/11493))
+- Fix pasted text losing formatting on last lines after resizing and adding new lines from the top [#11501](https://github.com/penpot/penpot/issues/11501) (PR: [#11503](https://github.com/penpot/penpot/pull/11503))
+- Fix variant property dropdown appearing empty and throwing an internal error when the component has no sibling variants [#11524](https://github.com/penpot/penpot/issues/11524) (PR: [#11499](https://github.com/penpot/penpot/pull/11499))
+
+### :sparkles: New features & Enhancements
+
+- Make backend storage resilient to interrupted writes, missing files and stalled cleanup [#11344](https://github.com/penpot/penpot/issues/11344) (PR: [#11345](https://github.com/penpot/penpot/pull/11345))
+- Implement RTL support in the text editor v3 [#11262](https://github.com/penpot/penpot/issues/11262)
+- Improve path operations and edition in the path editor [#10889](https://github.com/penpot/penpot/issues/10889) (PR: [#10807](https://github.com/penpot/penpot/pull/10807))
+- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237))
+- Add auto-linking of libraries during import based on slugified name [#9263](https://github.com/penpot/penpot/issues/9263) (PR: [#9958](https://github.com/penpot/penpot/pull/9958))
+- Add support for internal libraries and file sync for Design Tokens [#9334](https://github.com/penpot/penpot/issues/9334)
+- Warn self-hosted users when their Penpot version is outdated and surface what they're missing [#10497](https://github.com/penpot/penpot/issues/10497) (PR: [#11411](https://github.com/penpot/penpot/pull/11411))
+- Add dedicated RPC methods for plugin registry operations with permission validation [#10952](https://github.com/penpot/penpot/issues/10952) (PR: [#10957](https://github.com/penpot/penpot/pull/10957))
+- Document MCP and internal resolver environment variables (by @ShreyashAgare26) [#11318](https://github.com/penpot/penpot/issues/11318) (PR: [#11572](https://github.com/penpot/penpot/pull/11572))
+- Add tokens source indicator to assets tab [#11365](https://github.com/penpot/penpot/issues/11365) (PR: [#11439](https://github.com/penpot/penpot/pull/11439))
+- Export multiple fills to SVG [#11466](https://github.com/penpot/penpot/issues/11466) (PR: [#11467](https://github.com/penpot/penpot/pull/11467))
+- Add Penpot-specific board size presets (file thumbnail, template cover, plugin icon/cover) [#11561](https://github.com/penpot/penpot/issues/11561) (PR: [#11565](https://github.com/penpot/penpot/pull/11565))
+
 ## 2.18.0
 
 ### :rocket: Epics and highlights
@@ -199,6 +246,10 @@
 ### :rocket: Epics and highlights
 
 - Render prototype viewer with WASM (Skia) engine instead of SVG [#10037](https://github.com/penpot/penpot/issues/10037) (PR: [#10038](https://github.com/penpot/penpot/pull/10038))
+- Add layer blur effect for visual depth and styling [#9844](https://github.com/penpot/penpot/issues/9844) (PR: [#10034](https://github.com/penpot/penpot/pull/10034))
+- Render guides in WebGL for consistent viewer performance [#10068](https://github.com/penpot/penpot/issues/10068) (PR: [#10014](https://github.com/penpot/penpot/pull/10014))
+- Add concurrency limiter and status indicators for MCP server communications [#9493](https://github.com/penpot/penpot/issues/9493) (PR: [#9748](https://github.com/penpot/penpot/pull/9748))
+- Add typography token row to multiselected texts for better token visibility [#9336](https://github.com/penpot/penpot/issues/9336) (PR: [#9128](https://github.com/penpot/penpot/pull/9128))
 
 ### :sparkles: New features & Enhancements
 
@@ -374,7 +425,7 @@
 
 ### :rocket: Epics and highlights
 
-- WebGL rendering (beta) user preference [#9683](https://github.com/penpot/penpot/issues/9683) (PR:[9113](https://github.com/penpot/penpot/pull/9113))
+- WebGL rendering (beta) user preference [#9683](https://github.com/penpot/penpot/issues/9683) (PR: [#9113](https://github.com/penpot/penpot/pull/9113))
 - Design Tokens at the design tab: numeric fields with token selection in place [#9358](https://github.com/penpot/penpot/issues/9358)
 
 ### :sparkles: New features & Enhancements
@@ -553,6 +604,10 @@
 
 ## 2.15.0
 
+### :rocket: Epics and highlights
+
+- Add MCP server integration for AI-assisted design workflows [#9174](https://github.com/penpot/penpot/issues/9174) (PR: [#9032](https://github.com/penpot/penpot/pull/9032), [#9321](https://github.com/penpot/penpot/pull/9321))
+
 ### :sparkles: New features & Enhancements
 
 - Add MCP server integration [GH #9174](https://github.com/penpot/penpot/issues/9174)

HIGHLIGHTS.md

@@ -1,26 +0,0 @@
-# HIGHLIGHTS
-
-## 2.17.0
-
-- Background blur is here
-- WebGL rendering gets stronger
-- MCP connection status and more
-- Design tokens: more visible, more user-friendly
-
-
-## 2.16.0
-
-- Design tokens in the design panel
-- Major community contributions
-- WebGL rendering (beta)
-
-
-## 2.15.0
-
-- AI connected to real design context
-- Multi-directional workflow
-- Your stack, your model, your decision
-
-
-
-

README.md

@@ -167,6 +167,6 @@ This Source Code Form is subject to the terms of the Mozilla Public
 License, v. 2.0. If a copy of the MPL was not distributed with this
 file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-Copyright (c) KALEIDOS INC Sucursal en España SL
+Copyright (c) KALEIDOS SUBSIDIARY SL
 ```
 Penpot is a Kaleidos’ [open source project](https://kaleidos.net/)

THANKYOU.md

@@ -72,7 +72,7 @@ list.
 * [NampoinaRal](https://hosted.weblate.org/user/NampoinaRal)
 * [nautilusx](https://hosted.weblate.org/user/nautilusx)
 * [niwinz](https://hosted.weblate.org/user/niwinz)
-* [pablo.alba](pablo.https://hosted.weblate.org/user/alba)
+* [pablo.alba](https://hosted.weblate.org/user/pablo.alba)
 * [PhilippeAccorsi](https://hosted.weblate.org/user/PhilippeAccorsi)
 * [rnarius](https://hosted.weblate.org/user/rnarius)
 * [rnd](https://hosted.weblate.org/user/rnd)
@@ -82,7 +82,7 @@ list.
 * [shahab](https://hosted.weblate.org/user/shahab)
 * [shuaib85](https://hosted.weblate.org/user/shuaib85)
 * [SiderealArt](https://hosted.weblate.org/user/SiderealArt)
-* [swapnil.cx](swapnil.https://hosted.weblate.org/user/cx)
+* [swapnil.cx](https://hosted.weblate.org/user/swapnil.cx)
 * [syuza](https://hosted.weblate.org/user/syuza)
 * [th3ph4nt0m](https://hosted.weblate.org/user/th3ph4nt0m)
 * [tiwb](https://hosted.weblate.org/user/tiwb)

backend/deps.edn

@@ -65,13 +65,19 @@
   ;; Pretty Print specs
   pretty-spec/pretty-spec {:mvn/version "0.1.4"}
   software.amazon.awssdk/s3 {:mvn/version "2.54.5"}
-  software.amazon.awssdk/sts {:mvn/version "2.54.5"}}
+  software.amazon.awssdk/sts {:mvn/version "2.54.5"}
+
+  com.ladybugdb/lbug {:mvn/version "0.19.1"}
+  ;; Required by Arrow RootAllocator (lbug only pulls arrow-memory-core).
+  org.apache.arrow/arrow-memory-netty {:mvn/version "18.2.0"}}
 
  :paths ["src" "resources" "target/classes"]
  :aliases
  {:dev
   {:jvm-opts ["--sun-misc-unsafe-memory-access=allow"
-              "--enable-native-access=ALL-UNNAMED"]
+              "--enable-native-access=ALL-UNNAMED"
+              ;; Arrow jars are on the classpath (unnamed module), not module-path.
+              "--add-opens=java.base/java.nio=ALL-UNNAMED"]
    :extra-deps
    {com.bhauman/rebel-readline {:mvn/version "0.1.11"}
     clojure-humanize/clojure-humanize {:mvn/version "0.2.2"}

backend/dev/script-fix-sobjects.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 ;; This is an example on how it can be executed:
 ;; clojure -Scp $(cat classpath) -M dev/script-fix-sobjects.clj

backend/dev/user.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns user
   (:require

backend/package.json

@@ -20,7 +20,7 @@
     "ws": "^8.21.1"
   },
   "scripts": {
-    "lint:clj": "clj-kondo --config-dir ../.clj-kondo --lint ../common/src src/",
+    "lint:clj": "clj-kondo --fail-level error --config-dir ../.clj-kondo --lint ../common/src src/",
     "check-fmt:clj": "cljfmt check --parallel=true src/ test/",
     "fmt:clj": "cljfmt fix --parallel=true src/ test/",
     "test:e2e": "node --test --test-concurrency=1 test/e2e/*.test.mjs"

backend/resources/app/email/change-email/en.html

@@ -205,7 +205,7 @@
                           <td align="center" bgcolor="#31EFB8" role="presentation"
                             style="border:none;border-radius:3px;cursor:auto;mso-padding-alt:10px 25px;background:#31EFB8;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/auth/verify-token?token={{token}}"
+                            <a href="{{ public-uri }}/?screen=auth-verify-token&token={{token}}"
                               style="display:inline-block;background:#31EFB8;color:#1F1F1F;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:3px;"
                               target="_blank"> Confirm email change </a>
                           </td>

backend/resources/app/email/change-email/en.txt

@@ -4,7 +4,7 @@ We received a request to change your current email to {{ pending-email }}.
 
 Click the link below to confirm the change.
 
-{{ public-uri }}/#/auth/verify-token?token={{token}}
+{{ public-uri }}/?screen=auth-verify-token&token={{token}}
 
 If you did not request this change, consider changing your password for security reasons.
 

backend/resources/app/email/invite-to-organization/en.html

@@ -243,7 +243,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/auth/verify-token?token={{token}}"
+                            <a href="{{ public-uri }}/?screen=auth-verify-token&token={{token}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> ACCEPT INVITE </a>
                           </td>

backend/resources/app/email/invite-to-organization/en.txt

@@ -11,7 +11,7 @@ If you can't get in, your account probably isn't in the directory yet. To get ac
 
 Accept invitation using this link:
 
-{{ public-uri }}/#/auth/verify-token?token={{token}}
+{{ public-uri }}/?screen=auth-verify-token&token={{token}}
 
 Enjoy!
 The Penpot team.

backend/resources/app/email/invite-to-team/en.html

@@ -220,7 +220,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/auth/verify-token?token={{token}}"
+                            <a href="{{ public-uri }}/?screen=auth-verify-token&token={{token}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> ACCEPT INVITE </a>
                           </td>

backend/resources/app/email/invite-to-team/en.txt

@@ -11,7 +11,7 @@ If you can't get in, your account probably isn't in the directory yet. To get ac
 
 Accept invitation using this link:
 
-{{ public-uri }}/#/auth/verify-token?token={{token}}
+{{ public-uri }}/?screen=auth-verify-token&token={{token}}
 
 Enjoy!
 The Penpot team.

backend/resources/app/email/join-team/en.html

@@ -199,7 +199,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/dashboard/team/{{team-id}}/projects"
+                            <a href="{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> GO TO THE TEAM </a>
                           </td>

backend/resources/app/email/join-team/en.txt

@@ -4,7 +4,7 @@ As you requested, {{invited-by|abbreviate:25}} has added you to the team “{{ t
 
 Go to the team with this link:
 
-{{ public-uri }}/#/dashboard/team/{{team-id}}
+{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}
 
 Enjoy!
 The Penpot team.

backend/resources/app/email/password-changed/en.html

@@ -0,0 +1,233 @@
+<!doctype html>
+<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml"
+  xmlns:o="urn:schemas-microsoft-com:office:office">
+
+<head>
+  <title>
+  </title>
+  <!--[if !mso]><!-- -->
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <!--<![endif]-->
+  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <style type="text/css">
+    #outlook a {
+      padding: 0;
+    }
+
+    body {
+      margin: 0;
+      padding: 0;
+      -webkit-text-size-adjust: 100%;
+      -ms-text-size-adjust: 100%;
+    }
+
+    table,
+    td {
+      border-collapse: collapse;
+      mso-table-lspace: 0pt;
+      mso-table-rspace: 0pt;
+    }
+
+    img {
+      border: 0;
+      height: auto;
+      line-height: 100%;
+      outline: none;
+      text-decoration: none;
+      -ms-interpolation-mode: bicubic;
+    }
+
+    p {
+      display: block;
+      margin: 13px 0;
+    }
+  </style>
+  <!--[if mso]>
+        <xml>
+        <o:OfficeDocumentSettings>
+          <o:AllowPNG/>
+          <o:PixelsPerInch>96</o:PixelsPerInch>
+        </o:OfficeDocumentSettings>
+        </xml>
+        <![endif]-->
+  <!--[if lte mso 11]>
+        <style type="text/css">
+          .mj-outlook-group-fix { width:100% !important; }
+        </style>
+        <![endif]-->
+  <!--[if !mso]><!-->
+  <link href="https://fonts.googleapis.com/css?family=Source%20Sans%20Pro" rel="stylesheet" type="text/css">
+  <style type="text/css">
+    @import url(https://fonts.googleapis.com/css?family=Source%20Sans%20Pro);
+  </style>
+  <!--<![endif]-->
+  <style type="text/css">
+    @media only screen and (min-width:480px) {
+      .mj-column-per-100 {
+        width: 100% !important;
+        max-width: 100%;
+      }
+
+      .mj-column-px-425 {
+        width: 425px !important;
+        max-width: 425px;
+      }
+    }
+  </style>
+  <style type="text/css">
+    @media only screen and (max-width:480px) {
+      table.mj-full-width-mobile {
+        width: 100% !important;
+      }
+
+      td.mj-full-width-mobile {
+        width: auto !important;
+      }
+    }
+  </style>
+</head>
+
+<body style="background-color:#E5E5E5;">
+  <div style="background-color:#E5E5E5;">
+    <!--[if mso | IE]>
+      <table
+         align="center" border="0" cellpadding="0" cellspacing="0" class="" style="width:600px;" width="600"
+      >
+        <tr>
+          <td style="line-height:0px;font-size:0px;mso-line-height-rule:exactly;">
+      <![endif]-->
+    <div style="margin:0px auto;max-width:600px;">
+      <table align="center" border="0" cellpadding="0" cellspacing="0" role="presentation" style="width:100%;">
+        <tbody>
+          <tr>
+            <td style="direction:ltr;font-size:0px;padding:0;text-align:center;">
+              <!--[if mso | IE]>
+                  <table role="presentation" border="0" cellpadding="0" cellspacing="0">
+
+        <tr>
+
+            <td
+               class="" style="vertical-align:top;width:600px;"
+            >
+          <![endif]-->
+              <div class="mj-column-per-100 mj-outlook-group-fix"
+                style="font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;">
+                <table border="0" cellpadding="0" cellspacing="0" role="presentation" style="vertical-align:top;"
+                  width="100%">
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:16px;word-break:break-word;">
+                      <table border="0" cellpadding="0" cellspacing="0" role="presentation"
+                        style="border-collapse:collapse;border-spacing:0px;">
+                        <tbody>
+                          <tr>
+                            <td style="width:97px;">
+                              <img height="32" src="{{ public-uri }}/images/email/logo-penpot.svg"
+                                style="border:0;display:block;outline:none;text-decoration:none;height:32px;width:100%;font-size:13px;"
+                                width="97" />
+                            </td>
+                          </tr>
+                        </tbody>
+                      </table>
+                    </td>
+                  </tr>
+                </table>
+              </div>
+              <!--[if mso | IE]>
+            </td>
+
+        </tr>
+
+                  </table>
+                </td>
+            </tr>
+        </tbody>
+      </table>
+    </div>
+    <!--[if mso | IE]>
+          </td>
+        </tr>
+      </table>
+
+      <table
+         align="center" border="0" cellpadding="0" cellspacing="0" class="" style="width:600px;" width="600"
+      >
+        <tr>
+          <td style="line-height:0px;font-size:0px;mso-line-height-rule:exactly;">
+      <![endif]-->
+    <div style="background:#FFFFFF;background-color:#FFFFFF;margin:0px auto;max-width:600px;">
+      <table align="center" border="0" cellpadding="0" cellspacing="0" role="presentation"
+        style="background:#FFFFFF;background-color:#FFFFFF;width:100%;">
+        <tbody>
+          <tr>
+            <td style="direction:ltr;font-size:0px;padding:20px 0;text-align:center;">
+              <!--[if mso | IE]>
+                  <table role="presentation" border="0" cellpadding="0" cellspacing="0">
+
+        <tr>
+
+            <td
+               class="" style="vertical-align:top;width:600px;"
+            >
+          <![endif]-->
+              <div class="mj-column-per-100 mj-outlook-group-fix"
+                style="font-size:0px;text-align:left;direction:ltr;display:inline-block;vertical-align:top;width:100%;">
+                <table border="0" cellpadding="0" cellspacing="0" role="presentation" style="vertical-align:top;"
+                  width="100%">
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
+                      <div
+                        style="font-family:Source Sans Pro, sans-serif;font-size:24px;font-weight:600;line-height:150%;text-align:left;color:#000000;">
+                        Hello {{name|abbreviate:25}}!</div>
+                    </td>
+                  </tr>
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
+                      <div
+                        style="font-family:Source Sans Pro, sans-serif;font-size:16px;line-height:150%;text-align:left;color:#000000;">
+                        Your password has been changed.
+                      </div>
+                    </td>
+                  </tr>
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
+                      <div
+                        style="font-family:Source Sans Pro, sans-serif;font-size:16px;line-height:150%;text-align:left;color:#000000;">
+                        If you did not make this change, please contact support immediately or reset your password.
+                      </div>
+                    </td>
+                  </tr>
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
+                      <div
+                        style="font-family:Source Sans Pro, sans-serif;font-size:16px;line-height:150%;text-align:left;color:#000000;">
+                        Enjoy!</div>
+                    </td>
+                  </tr>
+                  <tr>
+                    <td align="left" style="font-size:0px;padding:10px 25px;word-break:break-word;">
+                      <div
+                        style="font-family:Source Sans Pro, sans-serif;font-size:16px;line-height:150%;text-align:left;color:#000000;">
+                        The Penpot team.</div>
+                    </td>
+                  </tr>
+                </table>
+              </div>
+              <!--[if mso | IE]>
+            </td>
+
+        </tr>
+
+                  </table>
+                </td>
+            </tr>
+        </tbody>
+      </table>
+    </div>
+
+    {% include "app/email/includes/footer.html" %}
+
+  </div>
+</body>
+
+</html>

backend/resources/app/email/password-changed/en.subj

@@ -0,0 +1 @@
+Password changed

backend/resources/app/email/password-changed/en.txt

@@ -0,0 +1,8 @@
+Hello {{name|abbreviate:25}}!
+
+Your password has been changed.
+
+If you did not make this change, please contact support immediately or reset your password.
+
+Enjoy!
+The Penpot team.

backend/resources/app/email/password-recovery/en.html

@@ -199,7 +199,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/auth/recovery?token={{token}}"
+                            <a href="{{ public-uri }}/?screen=auth-recovery&token={{token}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> RESET PASSWORD </a>
                           </td>

backend/resources/app/email/password-recovery/en.txt

@@ -3,7 +3,7 @@ Hello {{name|abbreviate:25}}!
 We received a request to reset your password. Click the link below to choose a
 new one:
 
-{{ public-uri }}/#/auth/recovery?token={{token}}
+{{ public-uri }}/?screen=auth-recovery&token={{token}}
 
 If you received this email by mistake, you can safely ignore it. Your password
 won't be changed.

backend/resources/app/email/register/en.html

@@ -205,7 +205,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/auth/verify-token?token={{token}}"
+                            <a href="{{ public-uri }}/?screen=auth-verify-token&token={{token}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> VERIFY EMAIL </a>
                           </td>

backend/resources/app/email/register/en.txt

@@ -4,7 +4,7 @@ Welcome to Penpot!
 
 Please verify your email to get started with your first design and collaboration.
 
-{{ public-uri }}/#/auth/verify-token?token={{token}}
+{{ public-uri }}/?screen=auth-verify-token&token={{token}}
 
 Enjoy!
 

backend/resources/app/email/renewal-notice/en.txt

@@ -1,4 +1,4 @@
-Hi {% if user-name %}{{ user-name }}{% endif %},
+Hi{% if user-name %} {{ user-name }}{% endif %},
 
 Your Enterprise subscription is coming up for renewal. Here's a summary of what's included.
 

backend/resources/app/email/request-file-access-yourpenpot-view/en.html

@@ -207,7 +207,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
+                            <a href="{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> SEND A VIEW-ONLY LINK </a>
                           </td>

backend/resources/app/email/request-file-access-yourpenpot-view/en.txt

@@ -6,7 +6,7 @@ Since this file is in your Personal Projects, you can provide access by sending
 
 To proceed, please click the link below to generate and send the view-only link:
 
-{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
+{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
 
 
 

backend/resources/app/email/request-file-access-yourpenpot/en.html

@@ -230,7 +230,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
+                            <a href="{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
                                style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                                target="_blank"> SEND A VIEW-ONLY LINK </a>
                           </td>

backend/resources/app/email/request-file-access-yourpenpot/en.txt

@@ -19,7 +19,7 @@ Alternatively, you can create and share a view-only link to the file. This will
 
 Click the link below to generate and send the link:
 
-{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
+{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
 
 
 

backend/resources/app/email/request-file-access/en.html

@@ -214,7 +214,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/dashboard/members?team-id={{team-id}}&invite-email={{requested-by-email|urlescape }}"
+                            <a href="{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}&invite-email={{requested-by-email|urlescape }}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> GIVE ACCESS TO “{{team-name|abbreviate:25}}” TEAM </a>
                           </td>
@@ -247,7 +247,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
+                            <a href="{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true"
                                style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                                target="_blank"> SEND A VIEW-ONLY LINK </a>
                           </td>

backend/resources/app/email/request-file-access/en.txt

@@ -13,7 +13,7 @@ This will automatically include {{requested-by|abbreviate:25}} in the team, so t
 
 Click the link below to provide team access:
 
-{{ public-uri }}/#/dashboard/members?team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}
+{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}
 
 
 
@@ -23,7 +23,7 @@ Alternatively, you can create and share a view-only link to the file. This will
 
 Click the link below to generate and send the link:
 
-{{ public-uri }}/#/view?file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
+{{ public-uri }}/?screen=viewer&file-id={{file-id}}&page-id={{page-id}}&section=interactions&index=0&share=true
 
 
 If you do not wish to grant access at this time, you can simply disregard this email.

backend/resources/app/email/request-team-access/en.html

@@ -205,7 +205,7 @@
                           <td align="center" bgcolor="#6911d4" role="presentation"
                             style="border:none;border-radius:8px;cursor:auto;mso-padding-alt:10px 25px;background:#6911d4;"
                             valign="middle">
-                            <a href="{{ public-uri }}/#/dashboard/members?team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}"
+                            <a href="{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}"
                               style="display:inline-block;background:#6911d4;color:#FFFFFF;font-family:Source Sans Pro, sans-serif;font-size:16px;font-weight:normal;line-height:120%;margin:0;text-decoration:none;text-transform:none;padding:10px 25px;mso-padding-alt:0px;border-radius:8px;"
                               target="_blank"> GIVE ACCESS TO “{{team-name|abbreviate:25}}” TEAM </a>
                           </td>

backend/resources/app/email/request-team-access/en.txt

@@ -4,7 +4,7 @@ Hello!
 
 To provide access, please click the link below:
 
-{{ public-uri }}/#/dashboard/members?team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}
+{{ public-uri }}/?screen=dashboard-members&team-id={{team-id}}&invite-email={{requested-by-email|urlescape}}
 
 
 If you do not wish to grant access at this time, you can simply disregard this email.

backend/resources/app/templates/debug.tmpl

@@ -236,7 +236,23 @@ Debug Main Page
         </div>
       </form>
     </fieldset>
+    {% if graph-enabled %}
+    <fieldset>
+      <legend>Export graph (Ladybug):</legend>
+      <desc>Given a FILE-ID, builds the graph projection and downloads
+      the `.lbug` database file.</desc>
 
+      <form method="get" action="/dbg/actions/graph-export">
+        <div class="row">
+          <input type="text" style="width:300px" name="file-id" placeholder="file-id" />
+        </div>
+        <div class="row">
+          <input type="submit" value="Download .lbug" />
+          <a href="/dbg/graph">Open graph console</a>
+        </div>
+      </form>
+    </fieldset>
+    {% endif %}
     <fieldset>
       <legend>Import binfile:</legend>
       <desc>Import penpot file in binary format.</desc>
@@ -280,5 +296,60 @@ Debug Main Page
       </form>
     </fieldset>
   </section>
+
+</main>
+
+<main class="dashboard wide">
+  <section class="widget wide">
+    <fieldset>
+      <legend>Export jobs:</legend>
+      <desc>
+        Export jobs as the exporter left them in redis. Records expire an hour
+        after the export settles, so this is a live view, not a history.
+      </desc>
+
+      <form method="get" action="/dbg">
+        <div class="row">
+          <input type="text" style="width:300px" name="job-id"
+                 placeholder="filter by job id" value="{{export-job-filter}}" />
+          <input type="submit" value="Filter" />
+          <a href="/dbg">clear</a>
+        </div>
+      </form>
+
+      <div class="scroll-box">
+        <table>
+          <thead>
+            <tr>
+              <th>JOB ID</th>
+              <th>STATE</th>
+              <th>PROGRESS</th>
+              <th>CMD</th>
+              <th>BACKEND</th>
+              <th>NAME</th>
+              <th>CREATED</th>
+              <th>ENDED</th>
+            </tr>
+          </thead>
+          <tbody>
+            {% for job in export-jobs %}
+            <tr>
+              <td><tt>{{job.id}}</tt></td>
+              <td>{{job.state}}{% if job.interrupted %} (interrupted){% endif %}</td>
+              <td>{{job.done}} / {{job.total}}</td>
+              <td>{{job.cmd}}</td>
+              <td>{{job.backend}}</td>
+              <td>{{job.name}}</td>
+              <td>{{job.created-at}}</td>
+              <td>{{job.ended-at}}</td>
+            </tr>
+            {% empty %}
+            <tr><td colspan="8">No export jobs.</td></tr>
+            {% endfor %}
+          </tbody>
+        </table>
+      </div>
+    </fieldset>
+  </section>
 </main>
 {% endblock %}

backend/resources/app/templates/link-preview.tmpl

@@ -0,0 +1,22 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <title>{{title}}</title>
+    <meta name="robots" content="noindex" />
+    <meta name="description" content="{{description}}" />
+    <meta property="og:site_name" content="Penpot" />
+    <meta property="og:type" content="website" />
+    <meta property="og:locale" content="en_US" />
+    <meta property="og:title" content="{{title}}" />
+    <meta property="og:description" content="{{description}}" />
+    <meta property="og:image" content="{{image}}" />
+    <meta name="twitter:title" content="{{title}}" />
+    <meta name="twitter:card" content="summary_large_image" />
+    <meta name="twitter:description" content="{{description}}" />
+    <meta name="twitter:image" content="{{image}}" />
+  </head>
+  <body>
+    <script>location.replace((location.pathname.replace(/link-preview\/?$/, "") || "/") + location.search);</script>
+  </body>
+</html>

backend/resources/app/templates/styles.css

@@ -143,6 +143,35 @@ nav > div:not(:last-child) {
   height: fit-content;
 }
 
+/* A widget that holds a table rather than a form: full width, and tall
+   enough to be worth scrolling inside. */
+.dashboard.wide {
+  margin-top: 0px;
+}
+
+.widget.wide {
+  max-width: none;
+  width: 100%;
+}
+
+.widget.wide .scroll-box {
+  max-height: 320px;
+  overflow-y: auto;
+  margin-top: 10px;
+}
+
+.widget.wide table {
+  width: 100%;
+  border-collapse: collapse;
+}
+
+.widget.wide th {
+  text-align: left;
+  position: sticky;
+  top: 0;
+  background: white;
+}
+
 .widget input[type=submit] {
   outline: none;
   border: 1px solid gray;

backend/resources/rlimit.edn

@@ -98,6 +98,9 @@
    :main/get-subscription-usage}
  [[:get-access-tokens :bucket "150/75/30s"]]
 
+ #{:main/get-environment-data}
+ [[:get-environment-data :bucket "250/125/30s"]]
+
  #{:main/get-enabled-flags}
  [[:get-enabled-flags :bucket "250/125/30s"]]
 

backend/scripts/_env

@@ -13,6 +13,10 @@ export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
 # PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
 # docker/devenv/defaults.env and injected via the main service's env block.
 
+if [ -f /home/selfsigned.crt ]; then
+    export NODE_EXTRA_CA_CERTS=/home/selfsigned.crt;
+fi
+
 # Background worker flag is per-instance. Defaults to enabled (ws0); ws1+
 # overlays set PENPOT_BACKEND_WORKER=false so scheduled and async tasks only
 # run on ws0, keeping notification Pub/Sub bound to a single Valkey. See
@@ -44,6 +48,7 @@ export PENPOT_FLAGS="\
        enable-user-feedback \
        disable-secure-session-cookies \
        enable-smtp \
+       enable-account-lockout \
        enable-prepl-server \
        enable-urepl-server \
        enable-nrepl-server \
@@ -58,6 +63,7 @@ export PENPOT_FLAGS="\
        enable-file-validation \
        enable-file-schema-validation \
        enable-redis-cache \
+       enable-link-preview \
        enable-subscriptions";
 
 # Uncomment for nexus integration testing
@@ -89,7 +95,8 @@ export JAVA_OPTS="\
        -XX:-OmitStackTraceInFastThrow  \
        --sun-misc-unsafe-memory-access=allow \
        --enable-preview \
-       --enable-native-access=ALL-UNNAMED";
+       --enable-native-access=ALL-UNNAMED \
+       --add-opens=java.base/java.nio=ALL-UNNAMED";
 
 function setup_s3_bucket() {
     if [ "${PENPOT_OBJECTS_STORAGE_BACKEND}" != "s3" ]; then
@@ -119,5 +126,3 @@ function setup_s3_bucket() {
         sleep 1
     done
 }
-
-

backend/scripts/manage.py

@@ -4,7 +4,7 @@
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 #
-# Copyright (c) KALEIDOS INC Sucursal en España SL
+# Copyright (c) KALEIDOS SUBSIDIARY SL
 
 import argparse
 import json

backend/scripts/run.template.sh

@@ -18,7 +18,7 @@ if [ -f ./environ ]; then
     source ./environ
 fi
 
-export JAVA_OPTS="-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dlog4j2.configurationFile=log4j2.xml -XX:-OmitStackTraceInFastThrow --sun-misc-unsafe-memory-access=allow --enable-native-access=ALL-UNNAMED --enable-preview $JVM_OPTS $JAVA_OPTS"
+export JAVA_OPTS="-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dlog4j2.configurationFile=log4j2.xml -XX:-OmitStackTraceInFastThrow --sun-misc-unsafe-memory-access=allow --enable-native-access=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --enable-preview $JVM_OPTS $JAVA_OPTS"
 
 ENTRYPOINT=${1:-app.main};
 

backend/src/app/auth.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.auth
   (:require

backend/src/app/auth/ldap.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.auth.ldap
   (:require

backend/src/app/auth/login_lockout.clj

@@ -0,0 +1,132 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.auth.login-lockout
+  "Brute-force protection: per-account failed login counter backed by Redis.
+   Uses an atomic Lua script for the increment operation to prevent
+   concurrent login bypass. Stores count:window_start in the value;
+   time is passed from Clojure (ct/now) via ARGV, keeping it injectable
+   for tests. When the counter reaches the configured threshold within
+   the time window, the account is locked out until the window expires."
+  (:require
+   [app.common.generic-pool :as gpool]
+   [app.common.logging :as l]
+   [app.common.time :as ct]
+   [app.config :as cf]
+   [app.redis :as rds]
+   [app.redis.script :as-alias rscript]
+   [clojure.string :as str])
+  (:import
+   java.lang.AutoCloseable))
+
+(def ^:private key-prefix "penpot.login-lockout.")
+
+(def ^:private lockout-script
+  {::rscript/name ::login-lockout
+   ::rscript/path "app/auth/login_lockout.lua"})
+
+(defn- get-pool
+  [cfg]
+  (:app.redis/pool cfg))
+
+(defn- with-conn
+  [cfg f]
+  (let [pool (get-pool cfg)
+        conn (gpool/get pool)]
+    (try
+      (f @conn)
+      (finally
+        (.close ^AutoCloseable conn)))))
+
+(defn- parse-value
+  "Parse stored \"count:window_start\" string. Returns nil if missing
+   or expired. Throws NumberFormatException for malformed values
+   (caught by caller → fail-open)."
+  [value window-ms now-ms]
+  (when value
+    (let [sep (str/index-of value ":")]
+      (when sep
+        (let [count (Long/parseLong (subs value 0 sep))
+              start (Long/parseLong (subs value (inc sep)))]
+          (when (> (+ start window-ms) now-ms)
+            {:count count :window-start start}))))))
+
+(defn record-failed-attempt!
+  "Increment the failed-login counter for a profile-id (atomic via Lua).
+   Returns nil when the flag is disabled or on Redis error (fail-open).
+   Otherwise returns a map with :count (int), :ttl (int, seconds),
+   and :locked? (boolean)."
+  [cfg profile-id]
+  (when (contains? cf/flags :account-lockout)
+    (try
+      (let [threshold (cf/get :login-lockout-max-attempts)
+            window    (cf/get :login-lockout-window)
+            window-ms (if (integer? window) window (.toMillis window))
+            _         (assert (>= threshold 1) "login-lockout-max-attempts must be >= 1")
+            _         (assert (>= window-ms 60000) "login-lockout-window must be >= 60000ms (1 minute)")
+            key       (str key-prefix profile-id)
+            now-ms    (inst-ms (ct/now))
+            result    (with-conn cfg
+                        (fn [conn]
+                          (rds/eval conn
+                                    (assoc lockout-script
+                                           ::rscript/keys [key]
+                                           ::rscript/vals [threshold window-ms now-ms]))))
+            [count locked ttl] result]
+        {:count count
+         :locked? (= 1 locked)
+         :ttl (max 0 ttl)})
+      (catch Exception cause
+        (l/warn :hint "redis unavailable, failing open on login lockout"
+                :profile-id (str profile-id)
+                :cause cause)
+        nil))))
+
+(defn clear-attempts!
+  "Clear the failed-login counter (on successful login or password reset).
+   No-op when the flag is disabled."
+  [cfg profile-id]
+  (when (contains? cf/flags :account-lockout)
+    (try
+      (with-conn cfg
+        (fn [conn]
+          (rds/del conn (str key-prefix profile-id))))
+      (catch Exception cause
+        (l/warn :hint "redis unavailable, failed to clear login lockout"
+                :profile-id (str profile-id)
+                :cause cause)))))
+
+(defn locked?
+  "Check whether the account is currently locked out. Does not increment
+   the counter. Returns {:locked? false} when the flag is disabled or on
+   Redis error (fail-open). When locked, includes :ttl (seconds remaining)."
+  [cfg profile-id]
+  (if (contains? cf/flags :account-lockout)
+    (try
+      (let [threshold  (cf/get :login-lockout-max-attempts)
+            window    (cf/get :login-lockout-window)
+            window-ms (if (integer? window) window (.toMillis window))
+            _         (assert (>= threshold 1) "login-lockout-max-attempts must be >= 1")
+            _         (assert (>= window-ms 60000) "login-lockout-window must be >= 60000ms (1 minute)")
+            key       (str key-prefix profile-id)
+            now-ms    (inst-ms (ct/now))
+            result    (with-conn cfg
+                        (fn [conn]
+                          (if-let [current (parse-value (rds/get conn key) window-ms now-ms)]
+                            (let [elapsed (- now-ms (:window-start current))
+                                  ttl-ms  (- window-ms elapsed)
+                                  locked? (>= (:count current) threshold)]
+                              (cond-> {:locked? locked?}
+                                locked?
+                                (assoc :ttl (int (Math/ceil (/ ttl-ms 1000.0))))))
+                            {:locked? false})))]
+        result)
+      (catch Exception cause
+        (l/warn :hint "redis unavailable, failing open on lockout check"
+                :profile-id (str profile-id)
+                :cause cause)
+        {:locked? false}))
+    {:locked? false}))

backend/src/app/auth/login_lockout.lua

@@ -0,0 +1,46 @@
+-- This Source Code Form is subject to the terms of the Mozilla Public
+-- License, v. 2.0. If a copy of the MPL was not distributed with this
+-- file, You can obtain one at http://mozilla.org/MPL/2.0/.
+--
+-- Copyright (c) KALEIDOS SUBSIDIARY SL
+
+local key = KEYS[1]
+local threshold = tonumber(ARGV[1])
+local window_ms = tonumber(ARGV[2])
+local now_ms = tonumber(ARGV[3])
+local window_sec = math.ceil(window_ms / 1000)
+
+local val = redis.call('GET', key)
+local count
+local window_start
+
+if val then
+  local colon = string.find(val, ':')
+  if colon then
+    count = tonumber(string.sub(val, 1, colon - 1))
+    window_start = tonumber(string.sub(val, colon + 1))
+  else
+    count = 0
+    window_start = now_ms
+  end
+else
+  count = 0
+  window_start = now_ms
+end
+
+if (now_ms - window_start) > window_ms then
+  count = 0
+  window_start = now_ms
+end
+
+count = count + 1
+redis.call('SET', key, count .. ':' .. window_start, 'EX', window_sec)
+
+local locked = 0
+local ttl = 0
+if count >= threshold then
+  locked = 1
+  ttl = math.ceil((window_ms - (now_ms - window_start)) / 1000)
+end
+
+return {count, locked, ttl}

backend/src/app/auth/oidc.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.auth.oidc
   "OIDC client implementation."
@@ -685,10 +685,8 @@
 (defn- redirect-with-error
   ([error] (redirect-with-error error nil))
   ([error hint]
-   (let [params {:error error :hint hint}
-         params (d/without-nils params)
+   (let [params {:screen "auth-login" :error error :hint hint}
          uri    (-> (u/uri (cf/get :public-uri))
-                    (assoc :path "/#/auth/login")
                     (assoc :query (u/map->query-string params)))]
      (redirect-response uri))))
 
@@ -707,21 +705,19 @@
                       :iss :prepared-register
                       :exp (ct/in-future {:hours 48}))
 
-        params {:token (tokens/generate cfg info)
+        params {:screen "auth-register-validate"
+                :token (tokens/generate cfg info)
                 :provider (:provider (:id provider))
-                :fullname (:fullname info)}
-        params (d/without-nils params)]
+                :fullname (:fullname info)}]
 
     (redirect-response
      (-> (u/uri (cf/get :public-uri))
-         (assoc :path "/#/auth/register/validate")
          (assoc :query (u/map->query-string params))))))
 
 (defn- redirect-to-verify-token
   [token]
-  (let [params {:token token}
+  (let [params {:screen "auth-verify-token" :token token}
         uri    (-> (u/uri (cf/get :public-uri))
-                   (assoc :path "/#/auth/verify-token")
                    (assoc :query (u/map->query-string params)))]
 
     (redirect-response uri)))

backend/src/app/auth/passwords.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.auth.passwords
   "Password strength validation using Passay library."

backend/src/app/binfile/cleaner.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.binfile.cleaner
   "A collection of helpers for perform cleaning of artifacts; mainly

backend/src/app/binfile/common.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.binfile.common
   "A binfile related file processing common code, used for different
@@ -27,7 +27,6 @@
    [app.features.file-migrations :as fmigr]
    [app.loggers.audit :as-alias audit]
    [app.loggers.webhooks :as-alias webhooks]
-   [app.storage :as sto]
    [app.util.blob :as blob]
    [app.util.pointer-map :as pmap]
    [app.worker :as-alias wrk]
@@ -51,10 +50,34 @@
 (def temp-file-threshold
   (* 1024 1024 2))
 
-;; A maximum (storage) object size allowed: 100MiB
-(def ^:const max-object-size
+;; Maximum size allowed for a single binary entry during binfile
+;; import: 100MiB. Covers the storage blobs (`objects/` entries in v3,
+;; streams in v1), whose declared size and hash are verified against the
+;; imported bytes. Legitimate media objects fit comfortably below this;
+;; anything larger is rejected instead of being buffered into memory.
+(def ^:const default-max-binary-entry-size
   (* 1024 1024 100))
 
+;; Maximum decompressed size allowed for a single JSON/text zip entry
+;; (manifest, files, pages, shapes, colors, components, typographies,
+;; tokens, plugin-data) during binfile import: 20MiB. Legitimate entries
+;; are KB-sized, so this is deliberately much lower than
+;; default-max-binary-entry-size and bounds the DEFLATE amplification of any
+;; single entry.
+(def ^:const default-max-text-entry-size
+  (* 1024 1024 20))
+
+;; Maximum total decompressed size allowed for all JSON/text zip entries
+;; combined within a single import job: 200MiB. Bounds the case where many
+;; entries, each individually under default-max-text-entry-size, still sum
+;; to an unreasonable total.
+(def ^:const default-max-text-total-size
+  (* 1024 1024 200))
+
+;; Maximum number of entries allowed in the import zip: 500,000.
+(def ^:const default-max-zip-entries
+  (* 500 1000))
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 
 (declare get-resolved-file-libraries)
@@ -648,33 +671,18 @@
             data
             library-ids)))
 
-(defn disable-database-timeouts!
+(def ^:const import-transaction-timeout-ms
+  "Ceiling for binfile import transactions (20 minutes). Interpolated
+  directly into SQL: compile-time constant, never user input."
+  (* 20 60 1000))
+
+(defn configure-database-timeouts!
   [cfg]
   (let [conn (db/get-connection cfg)]
-    (db/exec-one! conn ["SET LOCAL idle_in_transaction_session_timeout = 0"])
+    (db/exec-one! conn [(str "SET LOCAL idle_in_transaction_session_timeout = "
+                             import-transaction-timeout-ms)])
     (db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])))
 
-(defn invalidate-thumbnails
-  [cfg file-id]
-  (let [storage (sto/resolve cfg)
-
-        sql-1
-        (str "update file_tagged_object_thumbnail "
-             "   set deleted_at = now() "
-             " where file_id=? returning media_id")
-
-        sql-2
-        (str "update file_thumbnail "
-             "   set deleted_at = now() "
-             " where file_id=? returning media_id")]
-
-    (run! #(sto/touch-object! storage %)
-          (sequence
-           (keep :media-id)
-           (concat
-            (db/exec! cfg [sql-1 file-id])
-            (db/exec! cfg [sql-2 file-id]))))))
-
 (defn process-file
   [cfg {:keys [id] :as file}]
   (let [libs (delay (get-resolved-file-libraries cfg file))]
@@ -875,8 +883,8 @@
 (defn get-resolved-file-libraries
   "Get all file libraries including itself. Returns an instance of
   LoadableWeakValueMap that allows do not have strong references to
-  the loaded libraries and reduce possible memory pressure on having
-  all this libraries loaded at same time on processing file validation
+  the loaded libraries and reduce memory pressure on having
+  all this libraries at the same time on processing file validation
   or file migration.
 
   This still requires at least one library at time to be loaded while
@@ -888,3 +896,47 @@
                          (cons (:id file)))
         load-fn     #(get-file cfg % :migrate? false)]
     (weak/loadable-weak-value-map library-ids load-fn {id file})))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; EXTERNAL LIBRARY RESOLUTION HELPERS
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn slugify-name
+  "Slugify a library name for cross-environment matching.
+  Lowercases, replaces non-alphanumeric runs with '-', strips
+  leading/trailing '-'."
+  [name]
+  (str/slug name))
+
+(def ^:private sql:get-files-names
+  "SELECT id, name FROM file WHERE id = ANY(?)")
+
+(defn get-files-names
+  "Return [{:id uuid :name string}] for the given file ids."
+  [cfg ids]
+  (db/run! cfg
+           (fn [{:keys [::db/conn]}]
+             (let [ids-arr (db/create-array conn "uuid" ids)]
+               (db/exec! conn [sql:get-files-names ids-arr])))))
+
+(def ^:private sql:get-shared-files-for-team
+  "SELECT f.id, f.name, f.project_id
+     FROM file AS f
+     JOIN project AS p ON (p.id = f.project_id)
+    WHERE p.team_id = ?
+      AND f.is_shared = true
+      AND f.deleted_at IS NULL
+      AND p.deleted_at IS NULL")
+
+(defn get-shared-files-for-team
+  "Return [{:id uuid :name string}] for all shared files in a team."
+  [cfg team-id]
+  (db/run! cfg
+           (fn [{:keys [::db/conn]}]
+             (db/exec! conn [sql:get-shared-files-for-team team-id]))))
+
+(defn find-shared-files-by-slug
+  "Return all shared files in `team-id` whose slugified name equals `slug`."
+  [cfg team-id slug]
+  (->> (get-shared-files-for-team cfg team-id)
+       (filter #(= slug (slugify-name (:name %))))))

backend/src/app/binfile/migrations.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.binfile.migrations
   "A binfile related migrations handling"

backend/src/app/binfile/v1.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.binfile.v1
   "A custom, perfromance and efficiency focused binfile format impl"
@@ -174,7 +174,7 @@
     (assert-mark m :obj)
     (let [size (read-long! input)]
       (assert (pos? size) "incorrect header size found on reading header")
-      (when (> size bfc/max-object-size)
+      (when (> size bfc/default-max-binary-entry-size)
         (ex/raise :type :validation
                   :code :max-file-size-reached
                   :hint (dm/str "unable to import object with size " size " bytes")))
@@ -249,7 +249,7 @@
         p (tmp/tempfile :prefix "penpot.binfile.")]
     (assert-mark m :stream)
 
-    (when (> s bfc/max-object-size)
+    (when (> s bfc/default-max-binary-entry-size)
       (ex/raise :type :validation
                 :code :max-file-size-reached
                 :hint (str/ffmt "unable to import storage object with size % bytes" s)))
@@ -455,7 +455,7 @@
 (defn- read-import-v1
   [{:keys [::db/conn ::bfc/project-id ::bfc/profile-id ::bfc/input] :as cfg}]
 
-  (bfc/disable-database-timeouts! cfg)
+  (bfc/configure-database-timeouts! cfg)
 
   (pu/with-open [input (zstd-input-stream input)
                  input (io/data-input-stream input)]

backend/src/app/binfile/v2.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.binfile.v2
   "A sqlite3 based binary file exportation with support for exportation

backend/src/app/config.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.config
   (:refer-clojure :exclude [get])
@@ -49,6 +49,9 @@
 
    :host "localhost"
    :tenant "default"
+   ;; The SaaS host also sets penpotIsSaas in the browser. Keep this server
+   ;; value in sync so Admin Console can query the same deployment type.
+   :is-saas false
 
    :redis-uri "redis://redis/0"
 
@@ -58,6 +61,7 @@
    :objects-storage-fs-directory "assets"
 
    :auth-token-cookie-name "auth-token"
+   :auth-token-cookie-max-age-absolute (ct/duration {:days 30})
 
    :assets-path "/internal/assets/"
    :smtp-default-reply-to "Penpot <no-reply@example.com>"
@@ -69,6 +73,9 @@
    :profile-bounce-max-age (ct/duration {:days 7})
    :profile-bounce-threshold 10
 
+   :login-lockout-max-attempts 5
+   :login-lockout-window (ct/duration "15m")
+
    :telemetry-uri "https://telemetry.penpot.app/"
 
    :media-max-file-size (* 1024 1024 30) ; 30MiB
@@ -95,11 +102,7 @@
 
    ;; SSRF protection
    :ssrf-allowed-hosts #{}
-   :ssrf-extra-blocked-cidrs #{}
-
-   ;; Binfile import limits
-   :binfile-import-max-object-size (* 1024 1024 100) ;; 100 MiB
-   :binfile-import-max-zip-entries (* 500 1000)})    ;; 500,000
+   :ssrf-extra-blocked-cidrs #{}})
 
 (def schema:config
   (do #_sm/optional-keys
@@ -109,6 +112,7 @@
     [:secret-key {:optional true} :string]
 
     [:tenant {:optional false} :string]
+    [:is-saas ::sm/boolean]
     [:public-uri {:optional false} ::sm/uri]
     [:host {:optional false} :string]
 
@@ -157,9 +161,14 @@
     [:media-processing-service-timeout {:optional true} ::sm/int]
 
     ;; Binfile import limits (PENPOT_BINFILE_IMPORT_*)
-    [:binfile-import-max-object-size {:optional true} ::sm/int]
+    [:binfile-import-max-binary-entry-size {:optional true} ::sm/int]
+    [:binfile-import-max-text-entry-size {:optional true} ::sm/int]
+    [:binfile-import-max-text-total-size {:optional true} ::sm/int]
     [:binfile-import-max-zip-entries {:optional true} ::sm/int]
 
+    [:login-lockout-max-attempts {:optional true} ::sm/int]
+    [:login-lockout-window {:optional true} ::ct/duration]
+
     [:deletion-delay {:optional true} ::ct/duration]
     [:file-clean-delay {:optional true} ::ct/duration]
     [:telemetry-enabled {:optional true} ::sm/boolean]
@@ -208,6 +217,7 @@
 
     [:auth-token-cookie-name {:optional true} :string]
     [:auth-token-cookie-max-age {:optional true} ::ct/duration]
+    [:auth-token-cookie-max-age-absolute {:optional true} ::ct/duration]
 
     [:registration-domain-whitelist {:optional true} [::sm/set :string]]
     [:email-verify-threshold {:optional true} ::ct/duration]
@@ -391,6 +401,22 @@
   (or (c/get config :file-clean-delay)
       (ct/duration {:days 2})))
 
+(defn join-uri
+  "Join path segments onto a base URI, preserving a potential subpath
+  (same semantics as the frontend config). The base is normalized with
+  a trailing slash; segments must not start with `/` (a leading slash
+  would resolve against the host root and drop the subpath)."
+  [base & segments]
+  (assert (not (some #(str/starts-with? % "/") segments))
+          "URI segments must be relative (no leading slash)")
+  (str (apply u/join (u/ensure-path-slash base) segments)))
+
+(defn get-public-uri
+  "Canonical public URI builder: `join-uri` over the configured
+  :public-uri. With no segments, returns the normalized base."
+  [& segments]
+  (apply join-uri (c/get config :public-uri) segments))
+
 (defn get
   "A configuration getter. Helps code be more testable."
   ([key]

backend/src/app/db.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.db
   (:refer-clojure :exclude [get run!])
@@ -65,7 +65,8 @@
    [::password {:optional true} :string]
    [::username {:optional true} :string]
    [::validation-timeout {:optional true} ::sm/int]
-   [::read-only {:optional true} ::sm/boolean]])
+   [::read-only {:optional true} ::sm/boolean]
+   [::mtx/metrics ::mtx/metrics]])
 
 (def defaults
   {::name :main
@@ -130,11 +131,9 @@
       (.setConnectionInitSql initsql)
       (.setInitializationFailTimeout -1))
 
-    ;; When metrics namespace is provided
-    (when-let [instance (::mtx/metrics cfg)]
-      (->> (mtx/get-registry instance)
-           (PrometheusMetricsTrackerFactory.)
-           (.setMetricsTrackerFactory config)))
+    (->> (mtx/get-registry (::mtx/metrics cfg))
+         (PrometheusMetricsTrackerFactory.)
+         (.setMetricsTrackerFactory config))
 
     (some->> ^String (::username cfg) (.setUsername config))
     (some->> ^String (::password cfg) (.setPassword config))

backend/src/app/db/sql.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.db.sql
   (:refer-clojure :exclude [update])

backend/src/app/email.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.email
   "Main api for send emails."
@@ -407,6 +407,16 @@
    :id ::password-recovery
    :schema schema:password-recovery))
 
+(def ^:private schema:password-changed
+  [:map
+   [:name ::sm/text]])
+
+(def password-changed
+  "A password changed notification email."
+  (template-factory
+   :id ::password-changed
+   :schema schema:password-changed))
+
 (def ^:private schema:change-email
   [:map
    [:name ::sm/text]
@@ -465,7 +475,7 @@
 
 (def ^:private schema:renewal-notice
   [:map
-   [:user-name [:maybe ::sm/text]]
+   [:user-name [:maybe :string]]
    [:renewal-date ::sm/text]
    [:estimated-amount ::sm/text]
    [:organizations [:vector schema:organization-data]]])

backend/src/app/email/blacklist.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.email.blacklist
   "Email blacklist provider"

backend/src/app/email/whitelist.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.email.whitelist
   "Email whitelist provider"

backend/src/app/features/fdata.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.features.fdata
   "A `fdata/*` related feature migration helpers"
@@ -151,6 +151,13 @@
 
   (cond
     (= backend "storage")
+    ;; IMPORTANT: we strongly do not reuse the main connection that can
+    ;; run inside a transaction because the storage upload process can
+    ;; fail in the middle of uploading and leave garbage on the underlying
+    ;; backend, if we participate in the main transaction and it aborts
+    ;; we will lose all registry of the pending to reconcile blobs
+    ;; what the storage subsystem registers in other parallel
+    ;; transaction
     (let [storage  (sto/resolve cfg)
           content  (sto/content data)
           sobject  (sto/put-object! storage

backend/src/app/features/file_migrations.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.features.file-migrations
   "Backend specific code for file migrations. Implemented as permanent feature of files."

backend/src/app/features/file_snapshots.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.features.file-snapshots
   (:require
@@ -326,8 +326,11 @@
     (let [file (d/update-when row :metadata fdata/decode-metadata)
           vern (rand-int Integer/MAX_VALUE)
 
+          ;; We reuse the main connection here for storage operations
+          ;; becaue the main operations are touching and we need them
+          ;; to be atomic with the current transaction
           storage
-          (sto/resolve cfg {::db/reuse-conn true})
+          (sto/resolve cfg ::db/reuse-conn true)
 
           snapshot
           (get-snapshot cfg file-id snapshot-id)]

backend/src/app/features/logical_deletion.clj

@@ -2,7 +2,7 @@
 ;; License, v. 2.0. If a copy of the MPL was not distributed with this
 ;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ;;
-;; Copyright (c) KALEIDOS INC Sucursal en España SL
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
 
 (ns app.features.logical-deletion
   "A code related to handle logical deletion mechanism"

backend/src/app/graph/arrow.clj

@@ -0,0 +1,370 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.graph.arrow
+  "Bulk Ladybug ingest through in-memory Arrow.
+
+  Rows are built as Arrow `VectorSchemaRoot`s in the JVM's off-heap memory,
+  handed to Ladybug as a virtual table, and `COPY`d into the real one. No file
+  is written and no value is rendered as text for the engine to re-parse, so
+  nothing in this path needs escaping. Arrow carries MAP, STRUCT, fixed-size
+  arrays and multi-line strings natively.
+
+  The type language is Ladybug's, read recursively by `app.graph.schema.values`;
+  this namespace adds the matching Arrow `Field` and a writer for each shape.
+  `values/coerce` shapes a value first — a matrix into six doubles, a colour
+  into a packed integer — exactly as it does for the Cypher path, so the two
+  writers cannot disagree.
+
+  Engine facts this file depends on, each verified against lbug 0.19.1:
+
+  - An Arrow table is **not** a `COPY` source identifier, but it *is* a
+    MATCH-able node label: `COPY T FROM (MATCH (n:stg) RETURN n.a AS a, …)`.
+  - A MAP vector's `entries` child struct must be non-nullable, and
+    `MapVector/getWriter` silently promotes it to a sparse union — so map
+    vectors are built from an explicit `Field` and filled child-first.
+  - Ladybug quotes the column and table names it interpolates into the staged
+    table's DDL, and does not quote a STRUCT member name. So a top-level field
+    arrives plain and a struct member whose name is a reserved word (`column`)
+    arrives backticked.
+  - `createArrowRelTable` resolves a UUID-keyed endpoint only from a
+    `FixedSizeBinary(16)` column carrying the `arrow.uuid` extension, so edges
+    are staged as a node table and joined by the `COPY` subquery instead."
+  (:require
+   [app.common.json :as json]
+   [app.graph.ladybug :as ladybug]
+   [app.graph.schema.nodes :as nodes]
+   [app.graph.schema.values :as values]
+   [clojure.string :as str])
+  (:import
+   com.ladybugdb.Connection
+   com.ladybugdb.QueryResult
+   java.nio.charset.StandardCharsets
+   java.util.ArrayList
+   java.util.List
+   org.apache.arrow.memory.BufferAllocator
+   org.apache.arrow.memory.RootAllocator
+   org.apache.arrow.vector.BigIntVector
+   org.apache.arrow.vector.BitVector
+   org.apache.arrow.vector.complex.ListVector
+   org.apache.arrow.vector.complex.MapVector
+   org.apache.arrow.vector.complex.StructVector
+   org.apache.arrow.vector.FieldVector
+   org.apache.arrow.vector.Float8Vector
+   org.apache.arrow.vector.TimeStampMicroVector
+   org.apache.arrow.vector.types.FloatingPointPrecision
+   org.apache.arrow.vector.types.pojo.ArrowType$Bool
+   org.apache.arrow.vector.types.pojo.ArrowType$FloatingPoint
+   org.apache.arrow.vector.types.pojo.ArrowType$Int
+   org.apache.arrow.vector.types.pojo.ArrowType$List
+   org.apache.arrow.vector.types.pojo.ArrowType$Map
+   org.apache.arrow.vector.types.pojo.ArrowType$Struct
+   org.apache.arrow.vector.types.pojo.ArrowType$Timestamp
+   org.apache.arrow.vector.types.pojo.ArrowType$Utf8
+   org.apache.arrow.vector.types.pojo.Field
+   org.apache.arrow.vector.types.pojo.FieldType
+   org.apache.arrow.vector.types.pojo.Schema
+   org.apache.arrow.vector.types.TimeUnit
+   org.apache.arrow.vector.UInt4Vector
+   org.apache.arrow.vector.VarCharVector
+   org.apache.arrow.vector.VectorSchemaRoot))
+
+(set! *warn-on-reflection* true)
+
+;; --------------------------------------------------------------- allocator
+
+(defn with-allocator!
+  "Invoke `(f allocator)` with a fresh Arrow `RootAllocator`.
+
+  The allocator must outlive the Ladybug connection, because Ladybug releases
+  its references to the staged buffers only when the Arrow tables are dropped —
+  which happens on connection close at the latest. Closing it first surfaces as
+  `IllegalStateException: Memory was leaked`, *thrown while unwinding*, which
+  hides whatever actually failed. Any diagnostic here must catch inside this
+  scope."
+  [f]
+  (with-open [allocator (RootAllocator.)]
+    (f allocator)))
+
+;; ------------------------------------------------------ Ladybug type → Field
+
+(def ^:private scalar-arrow-type
+  "Ladybug scalar → Arrow type. `UUID` and `JSON` ride as UTF-8: Ladybug
+  accepts a string into either column and does the conversion itself, which is
+  cheaper than teaching this side two more binary layouts."
+  {"STRING"    #(ArrowType$Utf8.)
+   "UUID"      #(ArrowType$Utf8.)
+   "JSON"      #(ArrowType$Utf8.)
+   "INT64"     #(ArrowType$Int. 64 true)
+   "UINT32"    #(ArrowType$Int. 32 false)
+   "DOUBLE"    #(ArrowType$FloatingPoint. FloatingPointPrecision/DOUBLE)
+   "BOOLEAN"   #(ArrowType$Bool.)
+   "TIMESTAMP" #(ArrowType$Timestamp. TimeUnit/MICROSECOND nil)})
+
+(defn column-field
+  "Arrow `Field` for a column of `ladybug-type`, recursively.
+
+  `nullable?` is false only where Arrow's own invariants demand it — a MAP's
+  `entries` struct and its key."
+  (^Field [^String field-name ladybug-type]
+   (column-field field-name ladybug-type true))
+  (^Field [^String field-name ladybug-type nullable?]
+   (cond
+     ;; A list first: `STRUCT(…)[]` starts with `STRUCT(` but is a list of them.
+     (ladybug/list-type? ladybug-type)
+     (Field. field-name (FieldType. nullable? (ArrowType$List.) nil)
+             [(column-field "item" (values/list-element ladybug-type))])
+
+     (ladybug/map-type? ladybug-type)
+     (let [[key-type value-type] (values/map-types ladybug-type)]
+       (Field. field-name (FieldType. nullable? (ArrowType$Map. false) nil)
+               [(Field. "entries" (FieldType. false (ArrowType$Struct.) nil)
+                        [(column-field "key" key-type false)
+                         (column-field "value" value-type)])]))
+
+     (ladybug/struct-type? ladybug-type)
+     (Field. field-name (FieldType. nullable? (ArrowType$Struct.) nil)
+             ;; Backticks kept: Ladybug quotes none of these when it names the
+             ;; staged struct's fields, so `column` has to arrive quoted.
+             (mapv (fn [[field field-type]] (column-field field field-type))
+                   (values/struct-fields-quoted ladybug-type)))
+
+     :else
+     (if-let [mk (get scalar-arrow-type ladybug-type)]
+       (Field. field-name (FieldType. nullable? (mk) nil) nil)
+       (throw (ex-info (str "no Arrow mapping for Ladybug type: " ladybug-type)
+                       {:ladybug-type ladybug-type}))))))
+
+;; ------------------------------------------------------------------- writer
+
+(defn- utf8
+  ^bytes [v]
+  (.getBytes (if (keyword? v) (name v) (str v)) StandardCharsets/UTF_8))
+
+(defn- epoch-micros
+  ^long [v]
+  (let [^java.time.Instant inst
+        (cond
+          (instance? java.time.Instant v) v
+          (instance? java.util.Date v)    (.toInstant ^java.util.Date v)
+          :else                           (java.time.Instant/parse (str v)))]
+    (+ (* (.getEpochSecond inst) 1000000) (long (quot (.getNano inst) 1000)))))
+
+(defn- write-scalar!
+  [^FieldVector fv ladybug-type ^long idx v]
+  (case ladybug-type
+    ("STRING" "UUID")        (.setSafe ^VarCharVector fv idx (utf8 v))
+    ;; A JSON column holds JSON, not a Clojure value's print form: `str` on a
+    ;; map yields `{:fill-color "#000000"}`, which is EDN and which every
+    ;; consumer of `fills`, `content` or `position_data` would fail to parse.
+    ;; Same encoder the Cypher path uses (`app.graph.ladybug/format-json`).
+    "JSON"                   (.setSafe ^VarCharVector fv idx
+                                       (.getBytes ^String (json/encode v)
+                                                  StandardCharsets/UTF_8))
+    "INT64"                  (.setSafe ^BigIntVector fv idx (long v))
+    "UINT32"                 (.setSafe ^UInt4Vector fv idx (unchecked-int (long v)))
+    "DOUBLE"                 (.setSafe ^Float8Vector fv idx (double v))
+    "BOOLEAN"                (.setSafe ^BitVector fv idx (if v 1 0))
+    "TIMESTAMP"              (.setSafe ^TimeStampMicroVector fv idx (epoch-micros v))
+    (throw (ex-info (str "no Arrow writer for Ladybug type: " ladybug-type)
+                    {:ladybug-type ladybug-type}))))
+
+(defn write-value!
+  "Write already-coerced `v` into `fv` at `idx`, per `ladybug-type`.
+
+  `map-key-fn` renders the keys of a `MAP(STRING, …)`, for the same reason
+  `app.graph.ladybug/format-typed-value` takes one: the right spelling is a
+  property of the column, not of the writer."
+  ;; `idx` is deliberately unhinted: Clojure only accepts primitive args on fns
+  ;; of four or fewer, and the map-key renderer has to travel with the value.
+  [^FieldVector fv ladybug-type idx v map-key-fn]
+  (if (nil? v)
+    (.setNull fv (int idx))
+    (cond
+      (ladybug/list-type? ladybug-type)
+      (let [^ListVector lv fv
+            child        (.getDataVector lv)
+            element-type (values/list-element ladybug-type)
+            elements     (vec (if (or (sequential? v) (set? v)) v [v]))
+            start        (.startNewValue lv (int idx))]
+        (dotimes [i (count elements)]
+          (write-value! child element-type (+ start i) (nth elements i) map-key-fn))
+        (.endValue lv (int idx) (count elements)))
+
+      (ladybug/map-type? ladybug-type)
+      (let [^MapVector mv fv
+            ^StructVector entries (.getDataVector mv)
+            [key-type value-type] (values/map-types ladybug-type)
+            key-vec   (.getChild entries "key")
+            value-vec (.getChild entries "value")
+            render-key (if (and map-key-fn (= "STRING" key-type)) map-key-fn identity)
+            pairs      (vec (seq v))
+            start      (.startNewValue mv (int idx))]
+        (dotimes [i (count pairs)]
+          (let [[k mv'] (nth pairs i)
+                at      (+ start i)]
+            ;; The entries struct is non-nullable: every slot must be defined.
+            (.setIndexDefined entries (int at))
+            (write-value! key-vec key-type at (render-key k) nil)
+            (write-value! value-vec value-type at mv' map-key-fn)))
+        (.endValue mv (int idx) (count pairs)))
+
+      (ladybug/struct-type? ladybug-type)
+      (let [^StructVector sv fv]
+        (.setIndexDefined sv (int idx))
+        (doseq [[quoted-field field-type] (values/struct-fields-quoted ladybug-type)]
+          ;; The child is named with its backticks; the coerced value is keyed
+          ;; without them.
+          (write-value! (.getChild sv quoted-field) field-type idx
+                        (get v (str/replace quoted-field "`" "")) map-key-fn)))
+
+      :else
+      (write-scalar! fv ladybug-type (long idx) v))))
+
+;; ------------------------------------------------------------------ batches
+
+(defn- fill-vector!
+  [^VectorSchemaRoot root ^String field-name ladybug-type rows value-fn map-key-fn]
+  (let [^FieldVector fv (.getVector root field-name)]
+    (.allocateNew fv)
+    (dotimes [i (count rows)]
+      (write-value! fv ladybug-type i
+                    (values/coerce ladybug-type (value-fn (nth rows i)))
+                    map-key-fn))
+    (.setValueCount fv (count rows))))
+
+(defn- node-batch
+  "One `VectorSchemaRoot` holding every projected row of `table`.
+
+  Fields carry the plain column name. Ladybug quotes every identifier it
+  interpolates into the staged table's DDL, so a name that is a reserved word
+  (`Page.index`, `Document.options`) arrives unquoted and a name arriving
+  pre-quoted comes out doubly backticked and fails to parse. The `COPY`
+  projection below is Cypher, not DDL, so it quotes the same names itself."
+  ^VectorSchemaRoot [^BufferAllocator allocator table rows]
+  (let [columns (nodes/column-keys table)
+        fields  (mapv (fn [k] (column-field (nodes/column-name table k)
+                                            (nodes/column-ladybug-type table k)))
+                      columns)
+        root    (VectorSchemaRoot/create (Schema. ^List fields) allocator)]
+    (doseq [k columns]
+      (fill-vector! root (nodes/column-name table k)
+                    (nodes/column-ladybug-type table k)
+                    rows #(get % k) (nodes/column-map-key-fn table k)))
+    (.setRowCount root (count rows))
+    root))
+
+(def ^:private edge-fields
+  "Edge staging columns. `id` is the staging table's own key — Ladybug wants a
+  first column to key the virtual table on — and `from`/`to` land as STRING,
+  hence the cast in the join."
+  [(Field. "id"       (FieldType. true (ArrowType$Utf8.) nil) nil)
+   (Field. "from"     (FieldType. true (ArrowType$Utf8.) nil) nil)
+   (Field. "to"       (FieldType. true (ArrowType$Utf8.) nil) nil)
+   (Field. "position" (FieldType. true (ArrowType$Int. 64 true) nil) nil)])
+
+(defn- edge-batch
+  ^VectorSchemaRoot [^BufferAllocator allocator edges]
+  (let [root (VectorSchemaRoot/create (Schema. ^List edge-fields) allocator)
+        ^VarCharVector iv (.getVector root "id")
+        ^VarCharVector fv (.getVector root "from")
+        ^VarCharVector tv (.getVector root "to")
+        ^BigIntVector  pv (.getVector root "position")
+        n (count edges)]
+    (doseq [^FieldVector v [iv fv tv pv]] (.allocateNew v))
+    (dotimes [i n]
+      (let [{:keys [from-id to-id position]} (nth edges i)]
+        (.setSafe iv i (utf8 i))
+        (.setSafe fv i (utf8 from-id))
+        (.setSafe tv i (utf8 to-id))
+        (if (nil? position) (.setNull pv i) (.setSafe pv i (long position)))))
+    (doseq [^FieldVector v [iv fv tv pv]] (.setValueCount v n))
+    (.setRowCount root n)
+    root))
+
+;; ------------------------------------------------------------------ staging
+
+(defn- batches
+  ^List [^VectorSchemaRoot root]
+  (doto (ArrayList.) (.add root)))
+
+(defn- check!
+  [^QueryResult result hint data]
+  (when-not (.isSuccess result)
+    (throw (ex-info (str hint ": " (.getErrorMessage result))
+                    (assoc data :err (.getErrorMessage result))))))
+
+(defn- with-staged-table!
+  "Create Arrow table `staging-name` from `root`, run `(f)`, always drop it."
+  [^Connection conn ^BufferAllocator allocator ^String staging-name
+   ^VectorSchemaRoot root data f]
+  (try
+    (with-open [^QueryResult r (.createArrowTable conn staging-name (batches root) allocator)]
+      (check! r "createArrowTable failed" data))
+    (f)
+    (finally
+      ;; Dropped even on failure: the staged buffers stay referenced by Ladybug
+      ;; until it is, and the allocator's leak check fires on close otherwise.
+      (try (.close ^QueryResult (.dropArrowTable conn staging-name))
+           (catch Throwable _ nil)))))
+
+(defn- copy-node-table!
+  [^Connection conn table ^String staging-name]
+  (let [projection (str/join ", " (for [k (nodes/column-keys table)
+                                        :let [c (nodes/cypher-property-key table k)]]
+                                    (str "n." c " AS " c)))
+        statement  (str "COPY `" table "` FROM (MATCH (n:" staging-name ") "
+                        "RETURN " projection ");")]
+    (with-open [^QueryResult r (.query conn statement)]
+      (check! r (str "COPY node table failed: " table)
+              {:table table :statement statement}))))
+
+(defn- copy-edge-group!
+  "Load one FROM/TO pair of `IsChildOf`.
+
+  `createArrowRelTable` is unusable here — it cannot resolve endpoints against a
+  UUID-keyed node table — so the edge list is staged as a node table and the
+  endpoints are resolved by the subquery. The `WHERE` is clause-level because
+  this dialect prohibits an inline pattern `WHERE`, and both sides are pinned by
+  label so the join cannot reach outside the pair."
+  [^Connection conn from-table to-table ^String staging-name]
+  (let [statement (str "COPY `IsChildOf` FROM ("
+                       "MATCH (e:" staging-name "), "
+                       "(a:" (nodes/match-label from-table) "), "
+                       "(b:" (nodes/match-label to-table) ") "
+                       "WHERE a.id = cast(e.from AS UUID) "
+                       "AND b.id = cast(e.to AS UUID) "
+                       "RETURN a.id, b.id, e.position) "
+                       "(from='" from-table "', to='" to-table "');")]
+    (with-open [^QueryResult r (.query conn statement)]
+      (check! r (str "COPY edge group failed: " from-table " -> " to-table)
+              {:from-table from-table :to-table to-table :statement statement}))))
+
+(defn- staging-name
+  [prefix & parts]
+  (str/replace (str/join "_" (cons (str "stg_" prefix) parts)) #"[^A-Za-z0-9_]" "_"))
+
+;; --------------------------------------------------------------------- load
+
+(defn load-projection!
+  "Load projected nodes and edges into an open Ladybug connection.
+
+  `allocator` must outlive `conn` — see `with-allocator!`."
+  [^Connection conn {:keys [nodes edges]} ^BufferAllocator allocator]
+  (doseq [[table rows] (sort-by key nodes)
+          :when (seq rows)]
+    (let [name (staging-name "node" table)]
+      (with-open [root (node-batch allocator table rows)]
+        (with-staged-table! conn allocator name root {:table table}
+          #(copy-node-table! conn table name)))))
+  (doseq [[[from-table to-table] group]
+          (sort-by key (group-by (juxt :from-table :to-table) edges))
+          :when (seq group)]
+    (let [name (staging-name "edge" from-table to-table)]
+      (with-open [root (edge-batch allocator group)]
+        (with-staged-table! conn allocator name root
+          {:from-table from-table :to-table to-table}
+          #(copy-edge-group! conn from-table to-table name))))))

backend/src/app/graph/debug.clj

@@ -0,0 +1,383 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.graph.debug
+  "In-memory Ladybug sessions for the debug graph console."
+  (:require
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.common.time :as ct]
+   [app.graph.ingest :as graph.ingest]
+   [app.graph.ladybug :as ladybug]
+   [app.graph.schema.nodes :as nodes]
+   [app.graph.sync :as graph.sync]
+   [app.msgbus :as mbus]
+   [clojure.java.io :as io]
+   [clojure.string :as str]
+   [promesa.exec.csp :as sp])
+  (:import
+   com.ladybugdb.Connection
+   com.ladybugdb.Database))
+
+(set! *warn-on-reflection* true)
+
+(def default-query
+  "Default console query, written to be self-explanatory in the textarea.
+  The `filter_*` columns carry node ids for the graph-view result filter;
+  the results table hides them (see `hide-filter-columns` and the
+  template's `renderQueryOutput`)."
+  (str "MATCH (s)-[r]->(t)\n"
+       "// WHERE some condition\n"
+       "RETURN label(s) AS src, s.name,\n"
+       "       label(r) AS rel,\n"
+       "       t.name, label(t) AS tgt,\n"
+       "\n"
+       "// filter_* columns omitted from table; these needed for graph view\n"
+       "s.id AS filter_src_id, t.id AS filter_tgt_id;"))
+
+(defonce ^:private sessions
+  (atom {}))
+
+(defn- session-key
+  [profile-id]
+  (str profile-id))
+
+(defn- destroy-session!
+  [{:keys [conn db sync-ch msgbus]}]
+  (when sync-ch
+    (sp/close! sync-ch)
+    (when msgbus
+      (mbus/purge! msgbus [sync-ch])))
+  (when conn
+    (ex/ignoring (.close ^Connection conn)))
+  (when db
+    (ex/ignoring (.close ^Database db))))
+
+(defn- slim-ingest-meta
+  "Drop full projection rows from session meta.
+
+  `build-index` needs `:nodes`/`:edges` once; keeping them in the session
+  duplicates the entire graph on the JVM heap for every Load."
+  [meta]
+  (update meta :projection #(select-keys % [:stats])))
+
+(defn- format-cell
+  [value]
+  (cond
+    (nil? value) "NULL"
+    (string? value) value
+    :else (str value)))
+
+(defn- format-query-result
+  [{:keys [columns rows truncated?]}]
+  {:columns (mapv str columns)
+   :rows    (mapv (fn [row]
+                    (mapv format-cell row))
+                  rows)
+   :truncated? truncated?
+   :row-count (count rows)})
+
+(defn- apply-file-change!
+  [conn profile-id {:keys [changes revn file-id]}]
+  (try
+    (some-> (get @sessions (session-key profile-id))
+            (as-> current
+                  (when (= file-id (:file-id current))
+                    (let [lock   (:lock current)
+                          result (locking lock
+                                   (graph.sync/apply-changes!
+                                    conn (:index current) changes revn))
+                          sync-at (ct/now)]
+                      (swap! sessions assoc-in [(session-key profile-id) :index]
+                             (:index result))
+                      (swap! sessions update-in [(session-key profile-id) :meta]
+                             (fn [meta]
+                               (cond-> (-> meta
+                                           (update :sync dissoc :error)
+                                           (assoc-in [:sync :last-at] sync-at)
+                                           (assoc-in [:sync :last-applied] (:applied result))
+                                           (assoc-in [:sync :last-skipped] (:skipped result)))
+                                 (seq (:applied result))
+                                 (assoc :revn (:revn result)))))
+                      (when (seq (:skipped result))
+                        (l/dbg :hint "graph sync skipped changes"
+                               :file-id (str file-id)
+                               :revn revn
+                               :skipped (:skipped result)))))))
+    (catch Throwable cause
+      (l/wrn :hint "graph sync failed"
+             :file-id (str file-id)
+             :cause cause)
+      (swap! sessions assoc-in [(session-key profile-id) :meta :sync :error]
+             (ex-message cause)))))
+
+(defn- start-sync-loop!
+  [{:keys [conn profile-id file-id] :as session}]
+  (if-let [msgbus (:msgbus session)]
+    (let [sync-ch (sp/chan :buf (sp/dropping-buffer 64))]
+      (mbus/sub! msgbus :topic file-id :chan sync-ch)
+      ;; Recur ONLY while the channel is open. A bare `(recur)` after
+      ;; `take!` returns nil would spin forever and pin this Connection
+      ;; (and its Ladybug Database native memory) across every Load.
+      (sp/go-loop []
+        (when-let [message (sp/take! sync-ch)]
+          (when (= :file-change (:type message))
+            (apply-file-change! conn profile-id message))
+          (recur)))
+      (assoc session :sync-ch sync-ch))
+    session))
+
+(defn session-info
+  "Return a public view of the current session for `profile-id`, if any."
+  [profile-id]
+  (when-let [{:keys [file-id meta loaded-at index]} (get @sessions (session-key profile-id))]
+    {:file-id        file-id
+     :name           (:name meta)
+     :revn           (:revn meta)
+     :graph-revn     (:revn index)
+     :schema-version (:schema-version meta)
+     :projection     (:projection meta)
+     :sync           (:sync meta)
+     :loaded-at      (ct/format-inst loaded-at :iso)}))
+
+(defn sync-status
+  "Return incremental sync status for the active session."
+  [profile-id]
+  (when-let [session (get @sessions (session-key profile-id))]
+    (let [{:keys [file-id meta index loaded-at]} session]
+      {:file-id    file-id
+       :revn       (:revn meta)
+       :graph-revn (:revn index)
+       :sync       (:sync meta)
+       :loaded-at  (ct/format-inst loaded-at :iso)})))
+
+(defn unload-session!
+  "Close and discard the in-memory graph for `profile-id`."
+  [profile-id]
+  (when-let [session (get @sessions (session-key profile-id))]
+    (destroy-session! session))
+  (swap! sessions dissoc (session-key profile-id)))
+
+(defn load-session!
+  "Ingest `file-id` into a new in-memory Ladybug database for `profile-id`."
+  [cfg profile-id file-id]
+  (unload-session! profile-id)
+  (let [^Database db (Database.)
+        ^Connection conn (Connection. db)
+        msgbus     (::mbus/msgbus cfg)]
+    (.setQueryTimeout conn 0)
+    (ladybug/ensure-extensions! conn)
+    (try
+      (let [meta  (graph.ingest/ingest-on-connection! cfg conn file-id
+                                                      :db-path ":memory:"
+                                                      :skip-stats? true
+                                                      :skip-validation? true)
+            index (graph.sync/build-index file-id (:revn meta) (:projection meta))
+            ;; Discard projection rows after indexing — they are only needed
+            ;; to seed the sync index and would otherwise leak heap on each Load.
+            meta  (slim-ingest-meta meta)
+            session
+            ;; :lock serializes access to the shared Connection between the
+            ;; msgbus sync loop (writes) and HTTP handlers (reads); the Java
+            ;; binding gives no thread-safety guarantee for one Connection.
+            (-> {:db db
+                 :conn conn
+                 :lock (Object.)
+                 :file-id file-id
+                 :meta meta
+                 :index index
+                 :msgbus msgbus
+                 :profile-id profile-id
+                 :loaded-at (ct/now)}
+                start-sync-loop!)]
+        (swap! sessions assoc (session-key profile-id) session)
+        meta)
+      (catch Throwable cause
+        (destroy-session! {:conn conn :db db :msgbus msgbus})
+        (throw cause)))))
+
+(defn query-session!
+  "Run a read-only `statement` against the in-memory graph for `profile-id`.
+
+  The statement is bound against the live schema before it runs, so a query
+  naming a table or a property that does not exist reports the binder's own
+  message and executes nothing. The engine's read/write analysis then decides
+  whether it may run at all: the console is an inspection surface, and a
+  session graph is rebuilt from the file by Reload, so a mutation from here
+  would produce a graph no rebuild reproduces."
+  [profile-id statement]
+  (when (str/blank? statement)
+    (ex/raise :type :validation
+              :code :missing-query
+              :hint "cypher query is required"))
+  (if-let [{:keys [conn lock]} (get @sessions (session-key profile-id))]
+    (locking lock
+      (let [{:keys [ok? error read-only?]} (ladybug/validate-on-connection! conn statement)]
+        (when-not ok?
+          (ex/raise :type :validation
+                    :code :graph-query-invalid
+                    :hint error))
+        (when-not read-only?
+          (ex/raise :type :validation
+                    :code :graph-query-not-read-only
+                    :hint "the graph console runs read-only queries"))
+        (-> (ladybug/query-on-connection! conn statement)
+            format-query-result)))
+    (ex/raise :type :not-found
+              :code :graph-session-not-loaded
+              :hint "load a file graph before running queries")))
+
+(def ^:private export-max-rows
+  "Row cap for graph-view export queries; far above expected per-file node
+  and edge counts. `:truncated` in the export signals when it was hit."
+  100000)
+
+(defn- export-nodes
+  [conn]
+  (reduce
+   (fn [acc {:keys [table]}]
+     (let [stmt (str "MATCH (n:" (nodes/match-label table)
+                     ") RETURN n.id AS id, n.name AS name;")
+           {:keys [rows truncated?]}
+           (ladybug/query-on-connection! conn stmt :max-rows export-max-rows)]
+       (-> acc
+           (update :nodes into
+                   (map (fn [[id label]]
+                          {:id (str id) :label (str label) :table table}))
+                   rows)
+           (update :truncated? #(or % truncated?)))))
+   {:nodes [] :truncated? false}
+   nodes/node-types))
+
+(defn rel-tables
+  "Every relationship table in the open database, with whether it carries a
+  `position` property.
+
+  Read from the catalog rather than listed here, so a newly ported transform's
+  rel table appears in the graph view without the console being told about it."
+  [conn]
+  (for [[table] (:rows (ladybug/query-on-connection!
+                        conn "CALL show_tables() WHERE type = 'REL' RETURN name;"
+                        :max-rows 1000))
+        :let [props (->> (ladybug/query-on-connection!
+                          conn (str "CALL table_info('" table "') RETURN *;")
+                          :max-rows 1000)
+                         :rows
+                         (into #{} (map (comp str second))))]]
+    {:table table :position? (contains? props "position")}))
+
+(defn- export-edges
+  [conn]
+  (reduce
+   (fn [acc {:keys [table position?]}]
+     (let [stmt (str "MATCH (a)-[r:`" table "`]->(b) "
+                     "RETURN a.id AS source, b.id AS target, "
+                     (if position? "r.position" "NULL") " AS position, "
+                     "'" table "' AS rel;")
+           {:keys [rows truncated?]}
+           (ladybug/query-on-connection! conn stmt :max-rows export-max-rows)]
+       (-> acc
+           (update :edges into
+                   (map (fn [[source target position rel]]
+                          (cond-> {:source (str source)
+                                   :target (str target)
+                                   :rel    (str rel)}
+                            (some? position) (assoc :position position))))
+                   rows)
+           (update :truncated? #(or % truncated?)))))
+   {:edges [] :truncated? false}
+   (rel-tables conn)))
+
+(defn- bm-usage-bytes
+  "Buffer-manager memory in use by this session's in-memory database
+  (`CALL bm_info()` → [mem_limit mem_usage]); nil if the call fails."
+  [conn]
+  (ex/ignoring
+   (-> (ladybug/query-on-connection! conn "CALL bm_info() RETURN *;" :max-rows 1)
+       :rows first second)))
+
+(defn export-graph-data!
+  "Export the node/edge inventory of the in-memory graph for `profile-id`
+  as plain data for the debug graph view. Returns nil when no session is
+  loaded. Queries the Ladybug database (not the sync index) so the view
+  reflects actual DB state, including drift."
+  [profile-id]
+  (when-let [{:keys [conn lock file-id index]} (get @sessions (session-key profile-id))]
+    (locking lock
+      (let [{:keys [nodes] nodes-truncated? :truncated?} (export-nodes conn)
+            {:keys [edges] edges-truncated? :truncated?} (export-edges conn)]
+        {:file-id   (str file-id)
+         :revn      (:revn index)
+         :truncated (boolean (or nodes-truncated? edges-truncated?))
+         :bm-bytes  (bm-usage-bytes conn)
+         :nodes     nodes
+         :edges     edges}))))
+
+(defn- delete-tree!
+  [^java.io.File file]
+  (when (.exists file)
+    (doseq [f (reverse (file-seq file))]
+      (.delete ^java.io.File f))))
+
+(defn export-session-database!
+  "Materialize the in-memory session graph of `profile-id` as a `.lbug` file.
+
+  The console's graph is in-memory and live-synced, so it can differ from a
+  fresh projection of the same file — which is exactly when someone wants to
+  take it away and query it elsewhere. There is no \"save this database\"
+  primitive, so the transfer goes through Ladybug's `EXPORT DATABASE` (Parquet
+  per table) into a fresh on-disk database via `IMPORT DATABASE`.
+
+  Note the round trip drops table comments. Nothing in the graph is addressed
+  by a table comment: every table is resolved by name, so the loss costs
+  nothing.
+
+  Returns the path of the written database, or nil when no session is loaded.
+  The caller owns the file and must delete it once streamed."
+  [profile-id]
+  (when-let [{:keys [conn lock file-id]} (get @sessions (session-key profile-id))]
+    (let [stamp       (System/nanoTime)
+          staging     (io/file (System/getProperty "java.io.tmpdir")
+                               (str "penpot-graph-session-" file-id "-" stamp))
+          db-path     (str (io/file (System/getProperty "java.io.tmpdir")
+                                    (str file-id "-session-" stamp ".lbug")))]
+      (try
+        (locking lock
+          (ladybug/exec-on-connection!
+           conn [(str "EXPORT DATABASE '" (.getAbsolutePath staging)
+                      "' (format='parquet');")]))
+        (ladybug/with-connection! db-path
+          (fn [target]
+            (ladybug/exec-on-connection!
+             target [(str "IMPORT DATABASE '" (.getAbsolutePath staging) "';")
+                     "CHECKPOINT;"])))
+        db-path
+        (finally
+          (delete-tree! staging))))))
+
+(defn- hide-filter-columns
+  "Drop `filter_*` columns from a query result before HTML table render;
+  they exist to feed node ids to the graph-view filter, not for reading.
+  The JSON response path keeps the full result."
+  [{:keys [columns rows] :as result}]
+  (let [idxs (vec (keep-indexed
+                   (fn [i c] (when-not (str/starts-with? (str c) "filter_") i))
+                   columns))]
+    (if (or (empty? idxs) (= (count idxs) (count columns)))
+      result
+      (assoc result
+             :columns (mapv (vec columns) idxs)
+             :rows    (mapv (fn [row] (mapv (vec row) idxs)) rows)))))
+
+(defn console-context
+  "Build template data for the graph debug console page."
+  [profile-id & {:keys [query query-result error message]}]
+  {:session       (session-info profile-id)
+   :query         (or query default-query)
+   :query-result  (some-> query-result hide-filter-columns)
+   :error         error
+   :message       message
+   :default-query default-query})

backend/src/app/graph/ingest.clj

@@ -0,0 +1,106 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.graph.ingest
+  "Penpot file -> Ladybug graph projection."
+  (:require
+   [app.binfile.common :as bfc]
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.common.types.file :as ctf]
+   [app.db :as db]
+   [app.graph.arrow :as graph.arrow]
+   [app.graph.ladybug :as ladybug]
+   [app.graph.meta :as graph.meta]
+   [app.graph.projection.document :as projection.document]
+   [app.graph.projection.transforms :as projection.transforms]
+   [app.graph.schema :as schema]
+   [app.graph.stats :as stats]
+   [app.srepl.helpers :as h])
+  (:import
+   com.ladybugdb.Connection
+   org.apache.arrow.memory.BufferAllocator))
+
+(defn- fetch-file!
+  [system file-id]
+  (let [file-id (h/parse-uuid file-id)
+        file    (db/run! system #(bfc/get-file % file-id :realize? true))]
+    (when-not file
+      (ex/raise :type :not-found
+                :code :file-not-found
+                :file-id (str file-id)))
+    (when-not (:data file)
+      (ex/raise :type :validation
+                :code :file-without-data
+                :hint "file has no data to project"
+                :file-id (str file-id)))
+    [file-id file]))
+
+(defn- ingest-on-connection*!
+  [system ^Connection conn file-id ^BufferAllocator allocator
+   {:keys [db-path skip-stats? skip-validation?] :or {skip-stats? true}}]
+  (let [[file-id file] (fetch-file! system file-id)
+        db-path        (or db-path (ladybug/db-path-for-file file-id))
+        data           (:data file)]
+    (when-not skip-validation?
+      (ctf/check-file-data data))
+    (l/inf :hint "graph ingest"
+           :file-id (str file-id)
+           :revn (:revn file)
+           :db-path db-path
+           :schema schema/schema-version)
+    (let [ddl (schema/ddl-statements)
+          {:keys [nodes edges stats]}
+          (projection.document/projection-data data file)]
+      (ladybug/exec-on-connection! conn ddl)
+      (graph.arrow/load-projection! conn {:nodes nodes :edges edges} allocator)
+      (ladybug/exec-on-connection! conn ["CHECKPOINT;"])
+      (let [transforms (projection.transforms/apply-transforms! system conn data file)]
+        ;; Written last: its presence doubles as the build-complete marker.
+        (graph.meta/write! conn {:file-id file-id
+                                 :revn    (:revn file)})
+        {:file-id        file-id
+         :revn           (:revn file)
+         :name           (or (:name data) (:name file))
+         :db-path        db-path
+         :schema-version schema/schema-version
+         :projection     {:stats stats
+                          :nodes nodes
+                          :edges edges}
+         :transforms     transforms
+         :stats          (when-not skip-stats?
+                           (stats/summarize-connection conn))}))))
+
+(defn ingest-on-connection!
+  "Project `file-id` into an already open Ladybug `conn`.
+
+  Takes an `:arrow-alloc` when the caller already owns one; otherwise it makes
+  a short-lived allocator around this call. A caller that opened the connection
+  itself should pass its own, because the allocator has to be closed *after*
+  the connection — see `app.graph.arrow/with-allocator!`."
+  [system ^Connection conn file-id & {:keys [arrow-alloc] :as opts}]
+  (if arrow-alloc
+    (ingest-on-connection*! system conn file-id arrow-alloc opts)
+    (graph.arrow/with-allocator!
+      (fn [allocator] (ingest-on-connection*! system conn file-id allocator opts)))))
+
+(defn ingest-file!
+  [system file-id & {:keys [db-path reset-db? skip-stats? skip-validation?]
+                     :or   {reset-db? true}}]
+  (let [db-path (or db-path (ladybug/db-path-for-file (h/parse-uuid file-id)))]
+    (when reset-db?
+      (ladybug/reset-db-path! db-path))
+    ;; Allocator outermost: Ladybug holds the staged Arrow buffers until its
+    ;; tables are dropped, which is no later than connection close, so the
+    ;; allocator must be closed after the connection and the database.
+    (graph.arrow/with-allocator!
+      (fn [allocator]
+        (ladybug/with-connection! db-path
+          (fn [conn]
+            (ingest-on-connection*! system conn file-id allocator
+                                    {:db-path db-path
+                                     :skip-stats? skip-stats?
+                                     :skip-validation? skip-validation?})))))))