Merge remote-tracking branch 'origin/develop' into iop-vc-preview

Merges develop at f6f9df9a1a. Twelve files conflicted; each keeps
both intents.

files_update.clj: develop's per-stage validation timings wrap the
branch save path. `update-file` keeps the branch's early timer and
passes it to `update-file*`, inside develop's try/catch timing log.
`process-changes-and-validate` times the changes application and
still returns the file, the fixed-up change vector and the media
index. The audit props carry develop's revn and modified-at plus the
branch props. The timing test destructures the new return shape.

v3.clj keeps the branch-file overwrite guard and takes develop's
`configure-import-transaction`. migrations.clj puts develop's 0155
to 0157 before the branch's 0800 to 0805, so the branch's stay the
tail of the vector.

errors.cljs checks the merged and archived branch refusals before
develop's retrying case. left_header.cljs keeps the branch wrapper,
the review badge and the branch bar on develop's markup: the status
dot without icons and the DS library icon on the shared badge.
palette.scss and assets.scss keep the branch clearance terms on
develop's logical properties. The test runners, requires and
deftests are unions.

AI-assisted-by: claude-opus-5-5
Signed-off-by: Álvaro Tejero Cantero <alvorithm@teje.ro>

.agents/README.md

@@ -410,6 +410,10 @@ append-only `Review Log` (UTC ISO 8601, one line per entry):
 
 A plan approved with no changes goes `draft` → `done` directly.
 
+A plan is never staged or committed: the `done` status and its
+`Review Log` are session bookkeeping, not part of the change. Their
+absence from the commit is expected.
+
 ## 7. Connecting `gh` CLI with a token
 
 The `create-issue` and `create-pr` flows need an authenticated `gh`

.agents/skills/create-commit/SKILL.md

@@ -23,10 +23,11 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
 
 ## Iron Rules (non-negotiable)
 
-1. **Wrap every body line at 76 characters or fewer.** Count characters, do
+1. **Wrap every body line at 80 characters or fewer.** Count characters, do
    not eyeball. Exceptions: `Signed-off-by:` / `AI-assisted-by:` trailers and
    lines carrying a URL. This is the rule agents skip most often.
-2. **Subject ≤70 chars**, imperative, capitalized, no trailing period.
+2. **Subject ≤80 chars**, imperative, capitalized, no trailing period.
+   `Merge`/`Revert`/`Reapply` commits skip the length check.
 3. **Blank line between subject and body.**
 4. **Run `./scripts/check-commit` and require exit code 0.** It mechanically
    checks rules 1–3. A non-zero exit is a hard blocker: fix the message and
@@ -41,7 +42,7 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
    that does not match the stated intent, **STOP** and tell the user before
    committing.
 3. Draft the message following the format in the memory doc, wrapping the body
-   at 76 characters per line, and run:
+   at 80 characters per line, and run:
    ```bash
    git commit -m "<subject>" -m "<body>"
    ```
@@ -62,5 +63,6 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
 - Do not pass `--author`. Author identity comes from the local git config.
 - Do not amend a commit you did not create in this session, unless explicitly asked.
 - Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
+- Never add a `Co-authored-by:` trailer. It is forbidden and fails `./scripts/check-commit`.
 - Do not add untracked files that were not created in this session.
 - Do not skip the `scripts/check-commit` verification step (Iron Rule 4).

.agents/skills/rewrite-commit-message/SKILL.md

@@ -0,0 +1,38 @@
+---
+name: rewrite-commit-message
+description: Rewrite HEAD's commit message (typically a squash that concatenated old messages) into one proper title plus description following Penpot commit conventions, preserving every distinct AI-assisted-by attribution. Amends in place, never pushes.
+slash: true
+---
+
+# Skill: rewrite-commit-message
+
+HEAD's message is a concatenation of older messages (typical after a
+squash): several subjects, bodies and repeated trailers stacked in a
+single message. Replace it with one proper message and amend HEAD.
+
+## When to Use
+
+- The user asks to rewrite, retitle or fix the last commit message
+- After a squash left HEAD with stacked messages and trailer repeats
+
+## Steps
+
+1. Read the full current message (`git log -1 --format=%B HEAD`) and
+   the branch diff (`git diff develop..HEAD --stat`, plus the code
+   when the summary needs grounding).
+2. Collect every DISTINCT `AI-assisted-by:` value in the current
+   message. All of them survive into the new message, one line each:
+   attributions are never dropped in a rewrite, even when most of
+   the old text is.
+3. Draft one message per `mem:workflow/creating-commits` (read it
+   end-to-end first): single subject (imperative, capitalized, no
+   trailing period, ≤80 chars), blank line, body wrapped at ≤80
+   chars covering the whole diff (not the old per-commit slices),
+   then the preserved trailers.
+4. Write it to a file (a heredoc keeps real newlines; inline `-m`
+   with `\n` does not) and `git commit --amend -F <file>`.
+5. Run `./scripts/check-commit` and require exit code 0. A non-zero
+   exit is a hard blocker: fix the message, amend again.
+6. Report the new hash. Never push: the user pushes from their own
+   shell. Warn when local history was rewritten over pushed commits
+   (the push will need force — their call, not yours).

.github/workflows/auto-label.yml

@@ -25,7 +25,7 @@ jobs:
     steps:
       - name: Generate GitHub App token
         id: triage-app-token
-        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
+        uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
         with:
           client-id: ${{ secrets.TRIAGE_APP_CLIENT_ID }}
           private-key: ${{ secrets.TRIAGE_APP_PRIVATE_KEY }}

.github/workflows/build-bundle.yml

@@ -61,6 +61,7 @@ jobs:
     outputs:
       gh_ref: ${{ steps.vars.outputs.gh_ref }}
       bundle_version: ${{ steps.vars.outputs.bundle_version }}
+      build_key: ${{ steps.vars.outputs.build_key }}
       short_sha: ${{ steps.vars.outputs.short_sha }}
       sha: ${{ steps.vars.outputs.sha }}
       commit_title: ${{ steps.vars.outputs.commit_title }}
@@ -79,16 +80,26 @@ jobs:
         env:
           GH_REF: ${{ inputs.gh_ref || github.ref_name }}
         run: |
+          # Same command the build uses to stamp version.txt.
+          BUNDLE_VERSION=$(./manage.sh version)
+          SHORT_SHA=$(git rev-parse --short=12 HEAD)
+          # The key takes only the closest tag (--abbrev=0): the sha already
+          # pins the commit, and the full describe output is not stable
+          # across checkouts (its -g<sha> suffix length depends on the
+          # local object count). Must match build-docker.yml.
+          VERSION_TAG=$(git describe --tags --match '*.*.*' --abbrev=0)
           echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
-          echo "bundle_version=$(git describe --tags --always)" >> $GITHUB_OUTPUT
-          echo "short_sha=$(git rev-parse --short=12 HEAD)" >> $GITHUB_OUTPUT
+          echo "bundle_version=$BUNDLE_VERSION" >> $GITHUB_OUTPUT
+          echo "build_key=sha-${SHORT_SHA}-${VERSION_TAG}" >> $GITHUB_OUTPUT
+          echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
           echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
           echo "commit_title=$(git log -1 --pretty=%s)" >> $GITHUB_OUTPUT
 
-      # Keyed by commit sha, same as build-docker.yml's marker check, so
-      # both pipelines agree on what "already built" means: any ref that
-      # points at an already-bundled commit skips the build, regardless of
-      # which ref built it first. `force` bypasses this check entirely.
+      # Keyed by commit sha AND version, same as build-docker.yml's marker
+      # check, so both pipelines agree on what "already built" means. The
+      # version is stamped into the bundle, so a new tag on an already
+      # bundled commit (2.18.1-RC1 -> 2.18.1) must produce a new bundle.
+      # `force` bypasses this check entirely.
       - name: Check if this bundle is already built
         id: check
         env:
@@ -97,8 +108,7 @@ jobs:
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
           FORCE: ${{ inputs.force }}
-          SHORT_SHA: ${{ steps.vars.outputs.short_sha }}
-          BUNDLE_VERSION: ${{ steps.vars.outputs.bundle_version }}
+          BUILD_KEY: ${{ steps.vars.outputs.build_key }}
         run: |
           if [ "$FORCE" = "true" ]; then
             echo "exists=false" >> $GITHUB_OUTPUT
@@ -112,7 +122,7 @@ jobs:
 
           BUNDLE_EXISTS=$(aws s3api head-object \
             --bucket "$S3_BUCKET" \
-            --key "penpot-sha-${SHORT_SHA}.zip" \
+            --key "penpot-${BUILD_KEY}.zip" \
             > /dev/null 2>&1 && echo "true" || echo "false")
 
           if [ "$BUNDLE_EXISTS" = "true" ]; then
@@ -120,7 +130,7 @@ jobs:
             {
               echo "### ⏭️ Bundle build skipped"
               echo ""
-              echo "The bundle in S3 was already built from \`sha-${SHORT_SHA}\` (\`${BUNDLE_VERSION}\`)."
+              echo "The bundle in S3 was already built for \`${BUILD_KEY}\`."
             } >> "$GITHUB_STEP_SUMMARY"
           else
             echo "exists=false" >> $GITHUB_OUTPUT
@@ -164,11 +174,11 @@ jobs:
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
-          SHORT_SHA: ${{ needs.check.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.check.outputs.build_key }}
           BUNDLE_VERSION: ${{ needs.check.outputs.bundle_version }}
         run: |
           aws s3 cp zips/penpot.zip \
-            "s3://${S3_BUCKET}/penpot-sha-${SHORT_SHA}.zip" \
+            "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" \
             --metadata "bundle-version=${BUNDLE_VERSION}"
 
       - name: Write step summary
@@ -180,7 +190,7 @@ jobs:
           {
             echo "### ✅ Bundle built"
             echo ""
-            echo "- Version: \`${BUNDLE_VERSION}\` (\`git describe --tags --always\`)"
+            echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
             echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
             echo "- Built at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
           } >> "$GITHUB_STEP_SUMMARY"

.github/workflows/build-docker.yml

@@ -85,6 +85,7 @@ jobs:
       bundle_version: ${{ steps.vars.outputs.bundle_version }}
       sha: ${{ steps.vars.outputs.sha }}
       short_sha: ${{ steps.vars.outputs.short_sha }}
+      build_key: ${{ steps.vars.outputs.build_key }}
       commit_title: ${{ steps.vars.outputs.commit_title }}
       exists: ${{ steps.check.outputs.exists }}
 
@@ -105,10 +106,15 @@ jobs:
           REF_SLUG=$(printf '%s' "$GH_REF" | tr -c 'A-Za-z0-9._-' '-' | cut -c1-100)
           echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
           echo "ref_slug=$REF_SLUG" >> $GITHUB_OUTPUT
+          # Must match the key computed by build-bundle.yml.
+          BUNDLE_VERSION=$(./manage.sh version)
+          SHORT_SHA=$(git rev-parse --short=12 HEAD)
+          VERSION_TAG=$(git describe --tags --match '*.*.*' --abbrev=0)
           echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
-          echo "short_sha=$(git rev-parse --short=12 HEAD)" >> $GITHUB_OUTPUT
+          echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
           echo "commit_title=$(git log -1 --pretty=%s)" >> $GITHUB_OUTPUT
-          echo "bundle_version=$(git describe --tags --always)" >> $GITHUB_OUTPUT
+          echo "bundle_version=$BUNDLE_VERSION" >> $GITHUB_OUTPUT
+          echo "build_key=sha-${SHORT_SHA}-${VERSION_TAG}" >> $GITHUB_OUTPUT
 
       # The image set is a single block, so a single set-level check is
       # enough: `promote` drops a marker object in S3 only after every
@@ -123,14 +129,14 @@ jobs:
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
           FORCE: ${{ inputs.force }}
-          SHORT_SHA: ${{ steps.vars.outputs.short_sha }}
+          BUILD_KEY: ${{ steps.vars.outputs.build_key }}
         run: |
           if [ "$FORCE" = "true" ]; then
             echo "exists=false" >> $GITHUB_OUTPUT
             mkdir -p "$BUNDLE_CACHE"
             find "$BUNDLE_CACHE" -type f -mtime +1 -delete || true
-            ZIP="$BUNDLE_CACHE/penpot-sha-${SHORT_SHA}.zip"
-            aws s3 cp "s3://${S3_BUCKET}/penpot-sha-${SHORT_SHA}.zip" "$ZIP.$$.tmp"
+            ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
+            aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
             mv "$ZIP.$$.tmp" "$ZIP"
             {
               echo "### 🔁 Image set build forced"
@@ -142,13 +148,13 @@ jobs:
 
           if aws s3api head-object \
                --bucket "$S3_BUCKET" \
-               --key "markers/images-sha-${SHORT_SHA}" \
+               --key "markers/images-${BUILD_KEY}" \
                > /dev/null 2>&1; then
             echo "exists=true" >> $GITHUB_OUTPUT
             {
               echo "### ⏭️ Image set build skipped"
               echo ""
-              echo "The whole set was already built and promoted for \`sha-${SHORT_SHA}\`."
+              echo "The whole set was already built and promoted for \`${BUILD_KEY}\`."
             } >> "$GITHUB_STEP_SUMMARY"
           else
             echo "exists=false" >> $GITHUB_OUTPUT
@@ -158,9 +164,9 @@ jobs:
             # prune stale bundles while at it.
             mkdir -p "$BUNDLE_CACHE"
             find "$BUNDLE_CACHE" -type f -mtime +1 -delete || true
-            ZIP="$BUNDLE_CACHE/penpot-sha-${SHORT_SHA}.zip"
+            ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
             if [ ! -f "$ZIP" ]; then
-              aws s3 cp "s3://${S3_BUCKET}/penpot-sha-${SHORT_SHA}.zip" "$ZIP.$$.tmp"
+              aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
               mv "$ZIP.$$.tmp" "$ZIP"
             fi
           fi
@@ -231,14 +237,14 @@ jobs:
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
-          SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
           IMAGE: ${{ matrix.image }}
         run: |
-          ZIP="$BUNDLE_CACHE/penpot-sha-${SHORT_SHA}.zip"
+          ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
           if [ ! -f "$ZIP" ]; then
             echo "Bundle not found in host cache; falling back to S3."
             mkdir -p "$BUNDLE_CACHE"
-            aws s3 cp "s3://${S3_BUCKET}/penpot-sha-${SHORT_SHA}.zip" "$ZIP.$$.tmp"
+            aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
             mv "$ZIP.$$.tmp" "$ZIP"
           fi
           # Extract only the bundle this job needs.
@@ -274,7 +280,7 @@ jobs:
           sbom: true
           # Immutable tag only; branch tags are moved atomically for the
           # whole image set by the `promote` job.
-          tags: ${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:sha-${{ needs.prepare.outputs.short_sha }}
+          tags: ${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:${{ needs.prepare.outputs.build_key }}
           labels: ${{ steps.meta.outputs.labels }}
           cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache
           cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache,mode=max
@@ -305,13 +311,13 @@ jobs:
         env:
           DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
           REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
-          SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
         run: |
           set -e
           for image in $ALL_IMAGES; do
             docker buildx imagetools create \
               -t "${DOCKER_REGISTRY}/${image}:${REF_SLUG}" \
-              "${DOCKER_REGISTRY}/${image}:sha-${SHORT_SHA}"
+              "${DOCKER_REGISTRY}/${image}:${BUILD_KEY}"
           done
 
       # The marker is written LAST: its presence certifies that all five
@@ -322,26 +328,27 @@ jobs:
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
-          SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
         run: |
           echo "$GITHUB_RUN_ID" | aws s3 cp - \
-            "s3://${S3_BUCKET}/markers/images-sha-${SHORT_SHA}"
+            "s3://${S3_BUCKET}/markers/images-${BUILD_KEY}"
 
       - name: Write step summary
         env:
           REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
           SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
           BUNDLE_VERSION: ${{ needs.prepare.outputs.bundle_version }}
           COMMIT_TITLE: ${{ needs.prepare.outputs.commit_title }}
         run: |
           {
             echo "### ✅ Image set promoted"
             echo ""
-            echo "- Version: \`${BUNDLE_VERSION}\` (\`git describe --tags --always\`)"
+            echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
             echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
             echo "- Built at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
             echo ""
-            echo "All \`:${REF_SLUG}\` tags now point to \`sha-${SHORT_SHA}\`."
+            echo "All \`:${REF_SLUG}\` tags now point to \`${BUILD_KEY}\`."
           } >> "$GITHUB_STEP_SUMMARY"
 
   # ── 3b. Skip path: make sure THIS ref's tags point to the existing
@@ -374,30 +381,31 @@ jobs:
         env:
           DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
           REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
-          SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
         run: |
           set -e
           for image in $ALL_IMAGES; do
             docker buildx imagetools create \
               -t "${DOCKER_REGISTRY}/${image}:${REF_SLUG}" \
-              "${DOCKER_REGISTRY}/${image}:sha-${SHORT_SHA}"
+              "${DOCKER_REGISTRY}/${image}:${BUILD_KEY}"
           done
 
       - name: Write step summary
         env:
           REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
           SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
           BUNDLE_VERSION: ${{ needs.prepare.outputs.bundle_version }}
           COMMIT_TITLE: ${{ needs.prepare.outputs.commit_title }}
         run: |
           {
             echo "### ✅ Image set already built (branch tags ensured)"
             echo ""
-            echo "- Version: \`${BUNDLE_VERSION}\` (\`git describe --tags --always\`)"
+            echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
             echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
             echo "- Checked at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
             echo ""
-            echo "All \`:${REF_SLUG}\` tags now point to \`sha-${SHORT_SHA}\`."
+            echo "All \`:${REF_SLUG}\` tags now point to \`${BUILD_KEY}\`."
           } >> "$GITHUB_STEP_SUMMARY"
 
   # ── 4. Single failure notification for the whole workflow ─────────────

.github/workflows/build-renovate.yml

@@ -27,7 +27,7 @@ jobs:
     permissions:
       contents: read
     outputs:
-      short_sha: ${{ steps.bundle.outputs.short_sha }}
+      bundle_key: ${{ steps.bundle.outputs.bundle_key }}
       images: ${{ steps.changes.outputs.images }}
       devenv: ${{ steps.changes.outputs.devenv }}
       imagemagick: ${{ steps.changes.outputs.imagemagick }}
@@ -60,7 +60,9 @@ jobs:
           } >> "$GITHUB_OUTPUT"
 
       # Renovate only changes Dockerfiles and package manager versions, so
-      # the most recent develop bundle is enough to build the images.
+      # the most recent develop bundle is enough to build the images. The
+      # version part of the key does not matter here: take any bundle of
+      # the commit.
       - name: Find the latest develop bundle
         id: bundle
         env:
@@ -72,10 +74,12 @@ jobs:
         run: |
           for SHA in $(git rev-list --first-parent --max-count=50 "$BASE_SHA"); do
             SHORT_SHA=$(git rev-parse --short=12 "$SHA")
-            if aws s3api head-object --bucket "$S3_BUCKET" \
-                 --key "penpot-sha-${SHORT_SHA}.zip" > /dev/null 2>&1; then
-              echo "short_sha=$SHORT_SHA" >> "$GITHUB_OUTPUT"
-              echo "Using bundle \`penpot-sha-${SHORT_SHA}.zip\`" >> "$GITHUB_STEP_SUMMARY"
+            BUNDLE_KEY=$(aws s3api list-objects-v2 --bucket "$S3_BUCKET" \
+                           --prefix "penpot-sha-${SHORT_SHA}-" \
+                           --query 'Contents[0].Key' --output text)
+            if [ -n "$BUNDLE_KEY" ] && [ "$BUNDLE_KEY" != "None" ]; then
+              echo "bundle_key=$BUNDLE_KEY" >> "$GITHUB_OUTPUT"
+              echo "Using bundle \`${BUNDLE_KEY}\`" >> "$GITHUB_STEP_SUMMARY"
               exit 0
             fi
           done
@@ -135,13 +139,13 @@ jobs:
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
           S3_BUCKET: ${{ secrets.S3_BUCKET }}
-          SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
+          BUNDLE_KEY: ${{ needs.prepare.outputs.bundle_key }}
           IMAGE: ${{ matrix.image }}
         run: |
-          ZIP="$BUNDLE_CACHE/penpot-sha-${SHORT_SHA}.zip"
+          ZIP="$BUNDLE_CACHE/${BUNDLE_KEY}"
           if [ ! -f "$ZIP" ]; then
             mkdir -p "$BUNDLE_CACHE"
-            aws s3 cp "s3://${S3_BUCKET}/penpot-sha-${SHORT_SHA}.zip" "$ZIP.$$.tmp"
+            aws s3 cp "s3://${S3_BUCKET}/${BUNDLE_KEY}" "$ZIP.$$.tmp"
             mv "$ZIP.$$.tmp" "$ZIP"
           fi
           pushd docker/images

.github/workflows/commit-checker.yml

@@ -47,4 +47,46 @@ jobs:
       #     pattern: '^.+(Resolves|Fixes): \#[0-9]+$'
       #     error: 'You need at least one "Resolves|Fixes: #<issue number>" line.'
 
+  check-commit-guidelines:
+    if: ${{ !github.event.pull_request.draft }}
+    name: Check Commit Guidelines
+    runs-on: ubuntu-latest
+    steps:
+      # On PR events check out the base ref, so the script below is
+      # always the trusted version: PR commits are only read, never
+      # executed. On push events this resolves to the pushed sha.
+      - name: Checkout trusted code
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+          fetch-depth: 0
+          ref: ${{ github.event.pull_request.base.sha || github.sha }}
+
+      - name: Fetch PR commits
+        if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_target' }}
+        run: git fetch --no-tags origin "pull/${{ github.event.pull_request.number }}/head:pr-head"
+
+      - name: Run scripts/check-commit on every new commit
+        run: |
+          set -euo pipefail
+          if [ "${{ github.event_name }}" = "push" ]; then
+            if [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ]; then
+              RANGE="${{ github.sha }}"
+            else
+              RANGE="${{ github.event.before }}..${{ github.sha }}"
+            fi
+          else
+            RANGE="${{ github.event.pull_request.base.sha }}..pr-head"
+          fi
+          COMMITS="$(git rev-list --reverse "$RANGE")"
+          if [ -z "$COMMITS" ]; then
+            echo "No new commits to check."
+            exit 0
+          fi
+          FAILED=0
+          for COMMIT in $COMMITS; do
+            ./scripts/check-commit -c "$COMMIT" || FAILED=1
+          done
+          exit "$FAILED"
+
 

.github/workflows/lint-workflows.yml

@@ -34,7 +34,7 @@ permissions: {}
 
 env:
   ACTIONLINT_VERSION: '1.7.12'
-  ZIZMOR_VERSION: '1.30.1'
+  ZIZMOR_VERSION: '1.31.1'
 
 jobs:
   lint:

.github/workflows/plugins-deploy-api-doc.yml

@@ -48,10 +48,10 @@ jobs:
       # No dependency cache here: a deploy must not restore a cache that
       # other workflows can write to.
       - name: Setup pnpm + Node.js
-        uses: pnpm/setup@fbda4c85fc2e1e08721cd8763afea8f48d60f024 # v3.0.0
+        uses: pnpm/setup@a0881977cd8ea73b1ed212236c99ca10dcc80cf4 # v3.1.1
         with:
           working-directory: plugins
-          runtime: node@24.21.0
+          runtime: node@24.22.1
           install: false
 
       - name: Install deps

.github/workflows/plugins-deploy-styles-doc.yml

@@ -46,10 +46,10 @@ jobs:
       # No dependency cache here: a deploy must not restore a cache that
       # other workflows can write to.
       - name: Setup pnpm + Node.js
-        uses: pnpm/setup@fbda4c85fc2e1e08721cd8763afea8f48d60f024 # v3.0.0
+        uses: pnpm/setup@a0881977cd8ea73b1ed212236c99ca10dcc80cf4 # v3.1.1
         with:
           working-directory: plugins
-          runtime: node@24.21.0
+          runtime: node@24.22.1
           install: false
 
       - name: Install deps

.github/workflows/tests-e2e.yml

@@ -206,7 +206,7 @@ jobs:
             --reporter=blob
 
       - name: Upload blob report
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         if: always()
         with:
           name: integration-blob-report-${{ matrix.shard }}
@@ -215,7 +215,7 @@ jobs:
           retention-days: 3
 
       - name: Upload test result
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         if: always()
         with:
           name: integration-tests-result-${{ matrix.shard }}
@@ -338,7 +338,7 @@ jobs:
           pnpm install --frozen-lockfile;
 
       - name: Download blob reports
-        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
         with:
           path: frontend/all-blob-reports
           pattern: integration-blob-report-*
@@ -367,7 +367,7 @@ jobs:
                >> "$GITHUB_STEP_SUMMARY"
 
       - name: Upload JSON report
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         if: always()
         with:
           name: integration-json-report
@@ -377,7 +377,7 @@ jobs:
           retention-days: 7
 
       - name: Upload HTML report
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         if: always()
         with:
           name: integration-html-report

.github/workflows/tests-exporter.yml

@@ -61,8 +61,3 @@ jobs:
         working-directory: ./exporter
         run: |
           ./scripts/test
-
-      - name: Check no invoke_* in exporter JS glue
-        working-directory: ./render-wasm
-        run: |
-          ./check-no-invoke-v export

.github/workflows/tests-wasm.yml

@@ -60,13 +60,3 @@ jobs:
         working-directory: ./render-wasm
         run: |
           ./test
-
-      - name: Build
-        working-directory: ./render-wasm
-        run: |
-          NODE_ENV=production ./build frontend
-
-      - name: Check no invoke_*V in JS glue
-        working-directory: ./render-wasm
-        run: |
-          ./check-no-invoke-v

.gitignore

@@ -16,7 +16,10 @@
 .repl
 opencode.json
 .opencode/package-lock.json
+/*.html
 /*.jpg
+/*.json
+!/package.json
 /*.md
 !CHANGES.md
 !CONTRIBUTING.md
@@ -92,6 +95,12 @@ opencode.json
 /playwright/.cache/
 /render-wasm/target/
 /render-wasm/.render-wasm-build.lock
+
+# render-wasm build artifacts copied into the exporter (autogenerated)
+/exporter/resources/wasm/render-wasm.js
+/exporter/resources/wasm/render-wasm.wasm
+/exporter/src/app/wasm/shared.js
+
 /media-processor/dist/
 /media-processor/target/
 /**/node_modules
@@ -114,5 +123,6 @@ opencode.json
 /.ci-logs
 /tmp/
 /.codex/
+/scripts/__pycache__/
 /tools/__pycache__
 /scripts/__pycache__

.nvmrc

@@ -1 +1 @@
-v24.21.0
+v24.22.1

.serena/memories/audit-log.md

@@ -12,14 +12,15 @@ The system has:
 
 - **Three producers** (they create events): the Penpot backend (RPC layer), the Penpot frontend (browser app), and the Nitrate admin-console, which sends both server-side and client-side events.
 - **One central buffer**: the Postgres `audit_log` table in Penpot. It is temporary: it holds events for minutes/hours until they are archived, then they are deleted.
-- **Four consumers** (they read events): webhooks, error reporters, telemetry, and the external archive in Nexus.
-- **One long-term archive**: **Nexus**, a separate service that stores events for the long term. Archiving to **Nitrate** is in development and **does not exist on the current branch** (see section 11).
+- **Four consumers** (they read events): webhooks, error reporters, telemetry, and the external archives in Nexus and in the admin console.
+- **Two long-term archives**: **Nexus**, a separate service that stores events for the long term, and the **admin console** (Nitrate), which ingests its own allowlisted events through a second archive job (see section 12).
 
 ```
 Penpot frontend ─ POST push-audit-events (public) ─────────▶┐
 Nitrate admin-console (browser) ─ POST push-audit-events ──▶│
-Nitrate admin-console (server) ─ POST push-audit-events  ──▶│ ─▶ audit_log table (buffer) ── cron every 5m ──▶ Nexus (archive)
-Penpot backend (RPC wrap-audit) ─ submit ───────────────────┘         │   ├──▶ webhooks
+Nitrate admin-console (server) ─ POST push-audit-events  ──▶│ ─▶ audit_log table (buffer) ──┬─ cron 0s every 5m ──▶ Nexus (archive)
+Penpot backend (RPC wrap-audit) ─ submit ───────────────────┘         │                      └─ cron 30s every 5m ─▶ admin console (archive)
+                                                                      │   ├──▶ webhooks
                                                                       │   ├──▶ error reporters
                                                                       └──▶ telemetry (shadow rows)
 ```
@@ -44,7 +45,7 @@ Rule of thumb: if you need to know for sure "did this user do X?", look at `sour
 
 ## 3. The `audit_log` table: a buffer, not a store
 
-The Penpot `audit_log` table **is not the final destination**. It is a waiting tray: events sit here until the archive cron sends them to Nexus and the cleanup cron (GC) deletes them.
+The Penpot `audit_log` table **is not the final destination**. It is a waiting tray: events sit here until every active consumer (Nexus, admin console) confirms them, and whoever finishes last deletes the row.
 
 Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table.sql` and later ones):
 
@@ -56,15 +57,16 @@ Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table
 - `source` text: `backend` / `frontend` for full rows, `telemetry:backend` / `telemetry:frontend` for anonymized copies (see section 10).
 - `ip_addr` inet: real IP (or `0.0.0.0` on telemetry rows).
 - `props` / `context` jsonb holding transit-encoded maps: action data and metadata (versions, browser, request-id...).
-- `archived_at` timestamptz: NULL until Nexus confirms receipt; only then the row can be deleted.
+- No `archived_at`: the `audit_log_pending` queue (migration `0157`, consumer `1` = Nexus, `2` = admin console, ids never reused) tracks what is left. Every record is born pending only for the consumers enabled at insert time (never a fixed both): cheaper on single-consumer hosts and true to the design doc. Consequence, accepted on purpose: rows born with no send flag on get no pendings, so enabling archiving later does not pick them up — the sweep deletes them unarchived.
 
 Indexes (each consumer has its own, so a slow consumer never blocks the others):
 
-- Partial `created_at WHERE archived_at IS NULL`: used by the archive scan.
-- Partial `archived_at WHERE archived_at IS NOT NULL`: used by the GC.
+- `audit_log_pending` PK `(consumer, audit_log_id)`: each consumer owns its queue at constant cost, even with millions pending.
+- `audit_log (created_at, id)` (migration `0157`): the chronological drain order (`ORDER BY created_at ASC, id ASC`; same total order for both consumers, so concurrent drains cannot deadlock).
+- `audit_log_pending (audit_log_id)`: serves the "is anything still pending?" check of the final delete.
 - `(source, created_at)`: used by the telemetry scan.
 
-Life of a row: `INSERT` → (every 5 min) archive sets `archived_at = now()` **only if Nexus answers 204**, in the same transaction → (every 5 min) GC runs `DELETE WHERE archived_at IS NOT NULL` **with no age filter**. So archive must run before GC, or data is deleted without being sent.
+Life of a row: `INSERT` (row plus one pending per enabled consumer, same transaction; a rollback takes both) → (every 5 min, Nexus at second 0 and admin console at second 30) each archive job sends its own 128-row batches oldest-first and deletes its pendings in the same transaction → whoever deletes the last pending deletes the row. A batch the destination refuses rolls back and stays for the next tick. Pendings of consumers that are off are reaped at the start of each job run (covers later deactivations), and orphan rows are swept at the end of each run, so there is no separate GC task. The FK is plain `DEFERRABLE` (never `INITIALLY DEFERRED`): every deleter removes pendings before rows, including the telemetry task's own deletes.
 
 ## 4. Anatomy of an event
 
@@ -118,6 +120,21 @@ Critical `profile-id` INVARIANT: the event belongs to **whoever made the request
 
 The `::audit/profile-id` override goes through `coerce-profile-id`: a string is parsed, anything that cannot become a uuid is dropped (with a warning) and the event falls back to the caller. Do not pass the value raw: `schema:event` needs a uuid and `submit*` swallows the validation error (it only logs, the RPC still succeeds), so a non-uuid override **loses the row silently**. Hand-built events with `event-from-rpc-params` + `submit` (clone template, accept team invitation, `management/nitrate` push-audit-events) do **not** go through that coercion: they must already carry a uuid.
 
+### Team id and entity ids (projection data)
+
+The event stream is meant to let a consumer (Nexus) rebuild what it needs **without joining production tables**, so events carry the ids a projection keys on:
+
+- **Team id enrichment.** After `clean-props`, `prepare-rpc-event` calls `enrich-team-id`: when props have no `:team-id` but carry a `:project-id` or `:file-id`, it resolves the team with a local query and adds `:team-id`. It never resolves from `:id` (read commands put the entity id there, and that would add a query to every read), and it skips read events entirely (`get-*` names plus `has-file-libraries`: their events carry no consumer state). Commands that already loaded the team add `:team-id` themselves in `::audit/props` (`delete-file`, comments, media upload, `delete/update-webhook`, `clone-template`, ...), so the resolver does not run.
+- **Membership.** `add-profile-to-team!` emits `add-team-member` (props `:team-id`, `:member-id`, `:role`, `:is-owner`, `:is-admin`, `:can-edit`) whenever it actually inserts a row, only with `:audit-log` on, inside the caller's transaction. It covers the flows that add a member with no event of their own (`create-team` owner, invitee added directly when `:email-verification` is off, approved join-request, duplicated team). With `on-conflict-do-nothing?` a conflict returns nil and emits nothing, so an existing membership is reported once. The event belongs to the actor: the two flows where the owner adds somebody else (`create-invitation`, `add-member-to-team`) pass `:actor-id` through a separate `event-opts` arg (never in `params`, which is inserted as-is); everywhere else actor and member coincide. `:role` is derived in the emitter from the flags (inverse of `permissions-for-role`: owner > admin > editor > viewer) because the table has no role column; request context travels in `event-opts` as `:rpc-params`.
+- **Created entity ids.** `create-project` and `create-file` add the new row's `:id` (and `:project-id`/`:team-id`) to `::audit/props`, so a projection can key it.
+- **Bulk operations.** `clone-template`, `import-binfile` and `duplicate-project` create entities through internal calls that emit nothing on their own; they emit a `create-file` (and `create-project`) per entity (only with `:audit-log`). The import paths go through `audit/submit-file-created-event`, which reads the light columns (`id`, `name`, `project-id`, timestamps, `revn`) back from the row — the file must really exist, a phantom id emits nothing. `duplicate-project` already holds the rows and uses `audit/submit-create-event` directly. Both binfile versions return the same shape from `import-files!` (`{:file-ids [...]}`).
+- **Deletes.** `delete-team` still returns nil but carries the deleted row (`:id`, `:name`, `:created-at`, `:modified-at`, `:deleted-at`) in `::audit/props`, like `delete-project`. `delete-profile` carries `:id`, `:email`, `:fullname`, `:created-at`, `:modified-at` and `:deleted-at` (without this its props are empty: every param is namespaced and `clean-props` drops them). `create-team` adds `:created-at` and `:modified-at`. `delete-team-member` needs nothing: the delete is physical and the params already carry `:team-id`/`:member-id`.
+- **Hard deletes.** `objects-gc` emits one `hard-delete-objects` event per purged chunk, carrying only that chunk's entity key (`:team-ids`, `:project-ids` or `:file-ids`; one chunk transaction purges a single entity, so one event can never span sections) with context `{:triggered-by "objects-gc"}`, only with `:audit-log` on. No other GC path emits (profiles, fonts, thumbnails); those purges stay silent.
+- **SREPL deletes.** `delete-profile!` carries the subject id in props (it used to emit nothing usable); the event profile stays zero because SREPL has no actor. `delete-team!` props already carried the id. Both travel as actions with a `triggered-by: srepl` context.
+- **SREPL restores.** `restore-profile!` carries the full row (with id) in props with a zero event profile; `restore-team!` and `restore-project!` already carried their ids. The consumer resolves the subject from props by a uniform rule, with no SREPL special case.
+- **Cascades.** The `:delete-object` worker task emits one `delete-object-with-cascade` event per execution, type `action`, with `:object`/`:id`/`:deleted-at` plus the affected ids nested under `:cascade` (`:team-ids`, `:project-ids`, `:file-ids`), in the same transaction. The handler submits unconditionally (`submit` fans out per flag, so no `:audit-log` re-check); the `audit_log` row itself is still only written with `:audit-log` on. Direct calls (objects-gc) and snapshot cascades stay silent on purpose.
+- **Ownership.** `create-team-with-invitations` carries the new team `:id` like `create-team`; `update-team-member-role` names the demoted member in `:prev-owner-id` on owner transfers.
+
 Entry points:
 
 - `submit`: the normal one (fills defaults, validates, runs inside `tx-run!`, never breaks the RPC).
@@ -223,6 +240,8 @@ Webhooks let an external URL get notified when something happens in a team (for
 
 One guarantee: sending webhooks never changes the audit event itself. If delivery fails, only the webhook (and its delivery log) is affected; the audit row stays untouched and still gets archived normally.
 
+Every match also writes a `webhook` / `trigger` row to `audit_log` (even with zero matches): its props are `{:name <origin event name>, :event-id <origin row id>, :total-affected <matched webhook count>}`. `process-event` assigns the origin row id up-front so the trigger can reference it; the telemetry shadow row always gets its own id (reusing it would collide on the PK).
+
 ## 10. Consumers II: error reporters
 
 Both `app.loggers.database` and `app.loggers.mattermost` listen to backend `:error` logs **and** frontend crash events (found by the `::audit/event` marker), through a sliding-buffer channel so a flood cannot stall the app. The database one stores them in `server-error-report` (source 3 = backend log, 4 = audit-log origin, 5 = rate-limit); the Mattermost one forwards a short note to `:error-report-webhook` when set.
@@ -252,17 +271,21 @@ Nexus dispatches on `:type`: `"telemetry-events"` batches are coerced and insert
 
 - **Anonymization happens in Penpot, before sending.** By the time events leave, times are day-truncated, IPs are already `0.0.0.0`, props/context are filtered (see above), and emails are gone (only derived `email-domain` survives). Nexus additionally forces the outcome: it prefixes any non-`telemetry:` source, sets `received-at = tracked-at` (already truncated), pins IP to `0.0.0.0`, and stamps the sending `instance-id` into every event context.
 - **No personal data is expected.** Profile ids are random UUIDs with no PII, so the telemetry consumer needs no login guard; grouping is by `instance`, not by person (see posthog-telemetry below).
-- **The endpoint always answers 200.** Failures are swallowed and only counted (`nexus_telemetry_errors_total`), so a bad batch never breaks the sending instance; successes are counted (`nexus_telemetry_events_total`, `nexus_telemetry_reports_total`). Old instances may send a fressian+zstd base64 blob instead of a plain vector; Nexus decodes both (50 MiB cap against decompression bombs).
+- **The endpoint always answers 200.** Failures are swallowed and only counted (`nexus_telemetry_errors_total`), so a bad batch never breaks the sending instance; successes are counted (`nexus_telemetry_events_total`, `nexus_telemetry_reports_total`). A body the parser cannot read never reaches the handler: it answers 400 and counts in `nexus_http_malformed_requests_total`. Old instances may send a fressian+zstd base64 blob instead of a plain vector; Nexus decodes both (50 MiB cap against decompression bombs).
 
-## 12. Archive to Nexus (today) and to Nitrate (future)
+## 12. Archive to Nexus and to the admin console
 
-### Nexus: how it works today
+### How it works today
 
-Nexus is the long-term store. Every 5 min the `:audit-log-archive` cron takes 128-row chunks of unarchived rows (`FOR UPDATE SKIP LOCKED`, ordered by `created_at`), POSTs them as transit `{:events [...]}` with `x-shared-key: "nexus <key>"` (`:nexus-shared-key`, or derived from the instance secret) and `origin` = public-uri, and marks `archived_at = now()` **only on HTTP 204**, in the same transaction. Anything else is retried next run; flag on but no URI raises `:task-not-configured`. It loops with 100 ms pauses until empty.
+Every 5 min each archive job drains its own queue (`:audit-log-consumer-nexus` on flag `:nexus`, cron `0 */5 * * * ?`; `:audit-log-consumer-admin-console` on flag `:admin-console`, cron `30 */5 * * * ?`, 30s offset so the two drains never start on the same second), in 128-row chunks taken oldest-first (`ORDER BY created_at ASC, id ASC`) with `FOR UPDATE OF audit_log` (`SET LOCAL lock_timeout = '30s'`) inside the same transaction that sends and completes them, with 100 ms pauses between chunks. Batches arrive at Nexus in chronological order (it numbers by arrival for order-sensitive consumers). Nexus receives transit `{:events [...]}` with `x-shared-key: "nexus <key>"` (`:nexus-shared-key`, or derived from the instance secret) and `origin` = public-uri; flag on but no URI raises `:task-not-configured`. The admin console receives only its allowlisted event names (enriched with team names, props emails, and optional top-level `:profile-email` / `:profile-name` from the actor `:profile-id`); anything filtered out counts as done for the admin-console queue. A chunk whose destination refuses it (Nexus non-204 raises `:consumer-chunk-refused`) or throws (transport failure, admin-console error) rolls back and stays for the next tick, and the job aborts as failed: it archives what it can, and the cron retries the rest. Committed chunks stay sent. The send fns return counts or throw carrying the attempted count as `:events`; `run-single-consumer` records the `failed` metric from it, outside every chunk transaction (a local DB failure carries no `:events` and stays uncounted — it is not a destination failure). No `warn` before the throw: the runner reports the aborted job. Each job touches only its own queue, so a fallen Nexus never starves the admin console (nor vice versa).
+
+A chunk is **one POST** and the task advances only on 204, so a single oversized event wedges the whole queue: a crash report whose message embeds a whole payload (a serialized file, for example) can push the request past the proxy body limit and the rejection blocks every later event until it is pruned by hand. Frontend crash reports are therefore size-bounded at the source (`mem:frontend/routing-app-shell-subtleties`): `app.common.exceptions/get-hint` and the CLJS `format-throwable` clamp exception messages (`max-message-length`).
 
 On the other side, Nexus checks `origin` against its allowlist (403 if not allowed), coerces the events, and inserts them into its own `audit_log` (partitioned by `received_at`, with an incrementing sequence number and `source`). Watch the rename: Penpot's `created-at` is stored as **`received-at`** in Nexus; Nexus's `created-at` is when it was created there. `tracked-at` keeps the origin time. Telemetry batches arriving at `telemetry.penpot.app` land in the same table through the telemetry endpoint above.
 
-Concurrent runs are safe: rows are locked with `FOR UPDATE SKIP LOCKED` inside the same transaction that sends and marks them, so two archivers running at once each take different rows. Only rows Nexus acknowledges (HTTP 204) get marked; anything else is retried on the next run.
+Concurrent jobs are safe: both lock rows in `created_at ASC, id ASC` order (one total order, deliberately blocking — no `SKIP LOCKED`, so archive arrival stays chronological), so the second waits for the first to commit and then sees whether it was last. Only acknowledged work deletes pendings; an admin-console failure never blocks the Nexus send of the same rows (each job owns its queue).
+
+Observability (`backend/src/app/loggers/audit/consumer_task.clj`, definitions in `backend/src/app/main.clj:default-metrics`): one `dbg` summary per job run (`:consumer` label, sent/skipped/chunks, reaped disabled pendings, swept orphans, `:elapsed` as `"1.5s"` via `ct/format-duration`, frozen in a strict binding first because log props evaluate lazily in the logger thread), one `dbg` per admin-console send call (`ready` with totals, `sent`/`failed` with counts), plus `dbg` when the queue is empty or the job's send flag is off. Queue depth is Grafana's business: it queries the database directly, so the run performs no `GROUP BY` for the logs. No `warn` before a throw: the runner reports the aborted job (refusal details travel in the error data). Metrics, all recorded with bounded labels only: `penpot_audit_consumer_events_total{ destination = nexus|admin-console, outcome = archived|failed|skipped }` (`skipped` = filtered by the admin-console allowlist) and `penpot_audit_consumer_discarded_total{ reason = reaped|swept }` (entries discarded without archiving: `reaped` counts pendings dropped for disabled consumers, `swept` counts rows deleted while nobody waited for them — the silent R3 loss, now dashboard-visible instead of log-only). Timing needs no custom histogram: the jobs substrate already records execution time per job (`jobs-execution-timing`, split by job name), so each archive job is timed as itself. No queue-depth gauge: pending depth is read off the replica database directly. The bulk frontend ingest counts `penpot_audit_ingest_events_total{ kind = full|shadow, outcome = inserted|failed }` (both faces share one transaction, so a failure drops the batch on both). Archive success counts are recorded by `run-single-consumer` on the drain totals: a chunk that returns normally committed its transaction (a rollback always throws instead), so they cannot inflate. Logs never carry event data (`props`, `context`, `profile-id`, `ip-addr`, `team-name`, emails): only counts, statuses and the uri host.
 
 ### How Nexus consumes the data (and forwards to PostHog)
 
@@ -272,20 +295,16 @@ Stored rows are not the end: a consumer pipeline reads them back in sequence ord
 2. **`posthog`** (runs after `local`): forwards our own SaaS archived events to PostHog. It sends an allowlist of backend track events (teams, members, projects, files, comments, Nitrate/SSO, ...) plus frontend events minus a noise exclusion list, grouped by **team**, with person identify data from the local profile mirror and snake_case props. Both paths include the `initiator` property when the stored context has one, so PostHog can split Penpot vs admin-console traffic.
 3. **`posthog-telemetry`**: forwards events with `source = telemetry:frontend|telemetry:backend` to a separate PostHog project. Because these rows are already anonymous (no IPs, truncated times, filtered props), grouping is by **instance** (the self-hosted server, from `context.instance-id`), with one `$groupidentify` event per instance per chunk so PostHog learns per-instance properties (version, URI, flags). Each consumer has its own PostHog credentials.
 
-So the full journey is: Penpot buffer → Nexus `audit_log` (via archive cron or telemetry endpoint) → consumers → PostHog (SaaS events grouped by team, telemetry grouped by instance) plus local analytics views and materialized KPIs.
+So the full journey is: Penpot buffer → Nexus `audit_log` (via either archive cron or the telemetry endpoint) → consumers → PostHog (SaaS events grouped by team, telemetry grouped by instance) plus local analytics views and materialized KPIs.
 
-### Nitrate: what exists and what is planned
+### Admin console: the second consumer
 
-Today **only Nexus archives**. Archiving Penpot → Nitrate is in development and not on this branch.
-
-What already exists (do not confuse with archiving):
+Penpot → admin-console archiving lives in its own job (`:audit-log-consumer-admin-console`, consumer `2`, flag `:admin-console`, params closed empty): it drains its own pending queue and ingests allowlisted events through `nitrate/call :ingest-audit-log`, enriched as described above. What also exists (do not confuse with archiving):
 
 - **Nitrate → Penpot**: `POST /api/management/methods/push-audit-events` (`backend/src/app/rpc/management/nitrate.clj`) lets the trusted Nitrate backend inject events into the Penpot audit log (no RPC auth, `::audit/skip`, request context + `submit`). Section 7a is the main user.
 - **Admin-console browser → collector**: usage events from the admin-console UI traveling through the normal public `push-audit-events` (section 7b).
 - **Penpot frontend Nitrate bits**: membership/subscription events (`frontend/src/app/main/data/nitrate.cljs`, `nitrate-audit/*`) through the same public endpoint.
 
-The future Penpot → Nitrate archive will follow the Nexus pattern (cron reading the buffer and POSTing batches), but there is no contract or flag on this branch yet. When it lands, document here: endpoint, auth, format, which `source` values travel, and how it shares the GC.
-
 ## 13. Flags and config
 
 Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=enable-<name>`):
@@ -293,29 +312,31 @@ Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=ena
 | Flag | What it does |
 |---|---|
 | `:audit-log` | Stores full rows; turns on `wrap-audit` and frontend ingest |
-| `:audit-log-archive` | Turns on the cron that sends to Nexus (needs `:audit-log-archive-uri`) |
-| `:audit-log-gc` | Turns on the cron that deletes archived rows |
+| `:nexus` | Send flag of the Nexus archive job `:audit-log-consumer-nexus` (cron `0 */5 * * * ?`; needs `:nexus-uri` (ingest path `api/audit` appended by the job)) |
+| `:admin-console` | Send flag of the admin-console archive job `:audit-log-consumer-admin-console` (cron `30 */5 * * * ?`, 30s after Nexus); also the module flag for the admin-console integration |
 | `:audit-log-logger` | Structured `app.audit` log per event |
 | `:error-reporting` | Starts the error reporters (off by default); database reporter on flag alone, Mattermost on flag plus webhook URL |
 | `:telemetry` | Stores shadow rows and turns on shipping (auto with `:telemetry-enabled`) |
 
-Config (`backend/src/app/config.clj`): `:audit-log-archive-uri`, `:telemetry-uri` (default `https://telemetry.penpot.app/`), `:nexus-shared-key` / instance secret, `:public-uri` (sent as `origin` to Nexus).
+Config (`backend/src/app/config.clj`): `:nexus-uri`, `:telemetry-uri` (default `https://telemetry.penpot.app/`), `:nexus-shared-key` / instance secret, `:public-uri` (sent as `origin` to Nexus).
 
 ## 14. Practical guide: add or change events
 
 - **New backend command that must (or must not) emit**: `wrap-audit` does it alone; to exclude it set `::audit/skip`. To customize props/name/type use result metadata. Add a test mocking `app.loggers.audit/submit` (nil; `helpers.clj` stubs it globally) and assert `:call-args-list` — see `backend_tests/rpc-audit-test.clj` (full-row, telemetry-only, dual, no-op without flags, `insert`, `prepare-rpc-event`) and `rpc-team-test/accept-organization-invitation-audit-event` (three cases, exact props; counts matter: asserting with `first` passes even when a command emits the same name twice).
 - **New Penpot frontend event**: create the data-event or implement `Event`/`PerformanceEvent` in `app.main.data.event`, with `::name`/`::type`/`::origin`. Honor `skip-audit?` if the action can resume. Remember: with no props schema, **your asserts are the contract**.
 - **New admin-console event**: server action → add the backend-log call next to the mutation, best-effort so it never breaks the action; browser action → add a new event helper plus its test.
+- **Command that creates an entity or already knows its team**: put the new row's `:id` in `::audit/props` (`create-project`/`create-file`) and set `:team-id` when the team is loaded, so the consumer never has to join production tables. For entities created inside an internal call (`clone-template`, `import-binfile`, `duplicate-project`), emit one event per entity: `audit/submit-file-created-event` when only the id is known (it reads the row back), `audit/submit-create-event` when the row is already in hand.
 - **Never** put secrets in `props`; `clean-props` is a last defense, not an excuse.
 - **Do not duplicate events**: if the backend already emits an event for an action, do not add a frontend (or admin-console browser) event for the same action — the backend one wins because it is reliable. A second event for the same action is only acceptable with a strong reason (for example the backend cannot see it at all), and then it must be clearly justified.
 - **Follow the naming policy**: use kebab-case names (`"create-organization"`, not `"createOrganization"`), and before inventing prop keys, look at similar existing events and reuse their key names (`organization-id`, `team-id`, `file-id`, `event-origin`...). Same concept, same key spelling — that is what keeps the data queryable later.
 
 ## 15. Code map
 
-- Backend producer: `backend/src/app/loggers/audit.clj` (`submit`, `submit*`, `insert`, `prepare-rpc-event`, `event-from-rpc-params`, telemetry filters), `backend/src/app/rpc.clj` (`wrap-audit`), `backend/src/app/rpc/commands/audit.clj` (`push-audit-events`, `get-environment-data`, deprecated `get-enabled-flags`), `backend/src/app/rpc/management/nitrate.clj` (`push-audit-events` Nitrate→Penpot), `backend/src/app/http/middleware.clj` + `backend/src/app/http/management.clj` (shared-key auth).
+- Backend producer: `backend/src/app/loggers/audit.clj` (`submit`, `submit*`, `insert`, `prepare-rpc-event`, `enrich-team-id`, `submit-create-event`, `event-from-rpc-params`, telemetry filters), `backend/src/app/rpc.clj` (`wrap-audit`), `backend/src/app/rpc/commands/audit.clj` (`push-audit-events`, `get-environment-data`, deprecated `get-enabled-flags`), `backend/src/app/rpc/management/nitrate.clj` (`push-audit-events` Nitrate→Penpot), `backend/src/app/http/middleware.clj` + `backend/src/app/http/management.clj` (shared-key auth).
+- Membership emitter: `backend/src/app/rpc/commands/teams.clj` (`add-profile-to-team!` → `add-team-member`).
 - Penpot frontend producer: `frontend/src/app/main/data/event.cljs` (collector, buffer, send), `frontend/src/app/main/repo.cljs` + `frontend/src/app/util/http.cljs` (headers), `frontend/src/app/main/data/nitrate.cljs` (Nitrate membership events).
 - Admin-console producer (private repo, no paths cited here): server-side organization/subscription events through the management `push-audit-events`, browser-side UI events through the public `push-audit-events`.
 - Consumers: `backend/src/app/loggers/webhooks.clj`, `backend/src/app/loggers/database.clj`, `backend/src/app/loggers/mattermost.clj`, `backend/src/app/tasks/telemetry.clj`.
-- Archive/GC: `backend/src/app/loggers/audit/archive_task.clj`, `backend/src/app/loggers/audit/gc_task.clj`, wiring and crons in `backend/src/app/main.clj`.
+- Archive (two jobs sharing one namespace + pending queue, no GC): `backend/src/app/loggers/audit/consumer_task.clj` (`:audit-log-consumer-nexus` / `:audit-log-consumer-admin-console` job-defs, shared drain/reap/sweep helpers), producer hook in `backend/src/app/loggers/audit.clj` (`append-audit-entry`, single `queue-pendings` helper) plus the bulk ingest in `backend/src/app/rpc/commands/audit.clj` (`handle-events`), metrics/cron wiring in `backend/src/app/main.clj` (one cron entry per job, Nexus at second 0 and admin console at second 30, each gated by its own send flag), single migration `0157` (queue, drop of `archived_at`, covering `(created_at, id)` index for the chronological scan; no backfill — pre-existing rows are swept unsent, an accepted loss).
 - Nexus (private repo, no paths cited here): archive ingest, ordered consumers (local profile mirror, PostHog SaaS events, PostHog telemetry), telemetry endpoint, analytics views.
 - Tests: `backend_tests/rpc-audit-test.clj`, `rpc-team-test/accept-organization-invitation-audit-event`.

.serena/memories/backend/auth-permissions-product-domains.md

@@ -13,7 +13,7 @@
 - Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
 - Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
 - When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile.
-- HTTP/session parsing details such as cookie/header precedence and SameSite behavior are in `mem:backend/subtleties`.
+- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/subtleties`.
 
 ## Permission model
 

.serena/memories/backend/core.md

@@ -4,7 +4,7 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
 
 ## Focused memories
 
-- Cross-cutting backend subtleties (RPC, DB, workers, cron, CLIMIT, HTTP/sessions, storage, file data): `mem:backend/subtleties`
+- Cross-cutting backend subtleties (RPC, DB helpers, workers, cron, CLIMIT, config and HTTP/session middleware, http self-metrics, media processing, and file data persistence): `mem:backend/subtleties`
 - Storage abstraction, logical buckets, object lifecycle, deduplication, access, and garbage collection: `mem:backend/storage`.
 - Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
 - Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`.
@@ -49,13 +49,15 @@ Backend RPC command areas without focused memories include access tokens, binfil
 
 Database migrations live in `backend/src/app/migrations/`; pure SQL migrations are under `backend/src/app/migrations/sql/`. SQL filenames conventionally start with a sequence and verb/table description, e.g. `0026-mod-profile-table-add-is-active-field`. Applied migrations are tracked in the `migrations` table.
 
+Migrations must stay within PostgreSQL 15 syntax: the official docker images (`docker/images/docker-compose.yaml`) pin `postgres:15`, so anything newer breaks those deployments (`SET STORAGE DEFAULT` is PostgreSQL 16, for instance). The documented floor is >=15.
+
 For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
 
 For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/subtleties`.
 
 ## Background tasks (unified jobs)
 
-Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` in its namespace + a malli params schema + an `ig/init-key` that returns the job-def map `{::jobs/name, ::jobs/schema, ::jobs/handler, ::jobs/decoder, ::jobs/validator}` (decoder/validator precompiled at init). The registry is the `::jobs/defs` wiring in `app.main`, which also populates a module-level registry used by `jobs/submit` as fallback — job-def components cannot ig/ref `::jobs/defs` (wiring cycle). Submit from RPC code passes its RPC cfg (it carries `::jobs/defs` via ig/ref).
+Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` in its namespace + a malli params schema + an `ig/init-key` that returns the job-def map `{::jobs/name, ::jobs/schema, ::jobs/handler, ::jobs/decoder, ::jobs/validator}` (decoder/validator precompiled at init), optionally with `::jobs/result-schema` when the result carries types plain JSON does not keep (a uuid, whether a value or a map key): the substrate compiles the decoder from that schema on registry init and checks a result against the schema before storing it. The registry is the `::jobs/defs` wiring in `app.main`, which also populates a module-level registry used by `jobs/submit` as fallback — job-def components cannot ig/ref `::jobs/defs` (wiring cycle). Submit from RPC code passes its RPC cfg (it carries `::jobs/defs` via ig/ref).
 
 - `::jobs/handler` is `(fn [context params] ...)`. It is NOT given the runner cfg: a job-def closes over its own dependencies, and handing it the runner cfg would let a handler reach the pool or any other component. A wrapper passes the context to its implementation only when the implementation takes it; most do not, and their implementation stays `[cfg params]`.
 - The context is built by `jobs/make-context` and has exactly five keys: `id`, `name`, `label`, `resource-id`, `profile-id`, behind a closed schema. `profile-id` is the owner of a user-facing job and nil for an internal one, and it is there so a handler can revalidate permissions and check who owns its resource without a second query. `queue` is absent on purpose (a routing detail of the dispatcher, not a property of the job) and so are `retry-num`/`max-retries`/`attempt` (retry policy belongs to the runner, and an attempt number cannot be durable because the `noop` strategy re-runs without incrementing the counter).
@@ -64,14 +66,14 @@ Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` i
 - A job-def may carry optional metadata keys read from the registry by its consumers, never by the substrate: `::jobs/family` (e.g. `:export`) groups the jobs of a family for quotes, `::jobs/resource-role` (`:input` or `:output`) says whether the job's `resource_id` is what it consumes or what it produces, and `::jobs/queue-name` names the queue the creation command routes the job to (`submit-job` falls back to `:binfile` when the def does not set it).
 - Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
 - The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
-- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it, because the job-def has no result schema of its own.
+- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
 - The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
 - Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
 - The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.
 - A user-facing job is created through `app.rpc.commands.jobs`. One command per job type: `create-export-binfile-job` and `create-import-binfile-job` name the job they create and their body carries only `:params`, whose schema the command imports from the job namespace (`::sm/params`), so the RPC validates the business params up front, before the command runs; the job-def is still the place that owns them and validates them again at run time. The commands check the permission and the quote of the family before the expensive part, submit with `max-retries 0`, an expiry from `:jobs-user-ttl` and the queue the job-def routes to through `::jobs/queue-name` (`:binfile` when the def says nothing; read by the command, never by the substrate), and answer with a summary of the row. The import command has its own creation schema (`import-binfile/schema:create-params`, version optional) because the request is not the stored params: the command fills the version (from the caller or the package header) and the manifest metadata kept for audit. It assembles the upload, reads the manifest of a version 3 package for the audit (never its content) and stores the package in `job-resource` as the resource of the job, releasing it when the job never comes to own it.
-- `get-job` is the read surface a profile has over its own jobs: it answers with the same summary the creation commands return plus the `result` of a finished job or its public error (through `decode-job-error`, so the keywords come back), and a job of another profile is not distinguishable from one that does not exist: both answer `not-found`. `decode-row` decodes `params`, `result` and `error`, and the result is the plain JSON the row holds, so a uuid comes back as the text it is on disk (a result has no schema of its own like the params do). The event log still has no read surface: a client that follows a job takes its progress from the websocket.
+- `get-job` is the read surface a profile has over its own jobs: it answers with the same summary the creation commands return plus the `result` of a finished job or its public error (through `decode-job-error`, so the keywords come back), and a job of another profile is not distinguishable from one that does not exist: both answer `not-found`. `decode-row` decodes `params`, `result` and `error`, and the result is then decoded with the decoder the substrate compiled from the result schema the job-def the row names declares (`decode-result`; a def without one, or a row whose job-def is gone, keeps the plain JSON the row holds: a uuid as the text it is on disk, a uuid used as a map key as a keyword). The event log still has no read surface: a client that follows a job takes its progress from the websocket.
 - `heartbeat` uses named options `[cfg & {:keys [job-id progress] :as options}]` and returns the number of durable writes, so 0 means there was nothing to write: no job context, or the throttle did not allow it. A write that finds the job no longer active (terminal or gone) raises an `:interrupt` (`code :job-interrupted`, carrying the job id and the status it found) through the shared `jobs/check-active` instead of counting zero, so a task in course stops at its next beat; the interrupt is internal and never reaches HTTP (the management progress report answers `:skip` for it). The job id falls back to `::jobs/job-id` on the cfg and then to the runner-bound `*job-id*`. The `modified_at` touch is throttled to ~1s and the progress event to ~200ms, so the check fires at most once per second with no extra query (the read of the row happens only when a write already found nothing). A progress report is a milestone (`schema:progress`): a required `:stage` keyword plus an optional `:counters` map of `scope -> {:current, :total?}`, and no other key. Counters are non-negative, a `total` is positive and never lower than its `current`, and stage and counter keys are names of at most 64 characters. A milestone without counters is valid (a point of the run with no units to count), the stage vocabulary belongs to the worker and not to the substrate, and a client must degrade for a stage it does not know. Read a stored payload back with `jobs/decode-progress`: the database holds the keywords as strings. The 1s beat (down from 60s, so a cancelled heavy job is seen as gone at its next beat) costs one `modified_at` touch per beating job per second: cheap while the only heavy beater runs alone on its queue, revisit if concurrent beating jobs ever grow.
-- `job.error` conforms to `app.jobs/schema:job-error` (`type` and `code` keywords, `hint` text, extra details allowed). Read it back with `jobs/decode-job-error`; a plain `db/decode-json-pgobject` leaves the keywords as strings. `complete` takes named options and an optional `resource-id`, which is only ever set, never replaced: it reads the row locked inside its own transaction and applies the rules in a fixed order, skipping a job that is no longer completable before refusing one for its resource, so the answer is a function of the row and not of the order two callers arrive in.
+- `job.error` conforms to `app.jobs/schema:job-error` (`type` and `code` keywords, `hint` text, extra details allowed). Read it back with `jobs/decode-job-error`; a plain `db/decode-json-pgobject` leaves the keywords as strings. `complete` takes named options and an optional `resource-id`, which is only ever set, never replaced: it reads the row locked inside its own transaction and applies the rules in a fixed order, skipping a job that is no longer completable before refusing one for its resource, so the answer is a function of the row and not of the order two callers arrive in. It checks the result against the schema the job-def of the row declares before it writes (a violation raises `:invalid-result` and nothing is stored), and only for a transition that is going to write: a terminal job answers `:skip` without validating. `complete` is the single door of a result, so the check covers the runner and the external worker that completes a job through the management RPC (`::complete-job`), which stores a result the substrate would otherwise take blind.
 
 For worker dispatch, cron, retry semantics (`ex/raise :type ::wrk/retry` with `:delay`/`:strategy`), deduplication, job events, and queue internals: `mem:backend/subtleties`.
 

.serena/memories/backend/storage.md

@@ -25,8 +25,9 @@
 - Existing object values can remain readable until physical deletion.
 - `:expired-at` blocks reads after the expiration time.
 - `del-object!` sets `deleted_at` on live rows only (`deleted_at IS NULL`): a repeated call returns `false`. It does not remove backend content.
-- `storage-gc-deleted` removes the database row and backend content after the deletion delay, streaming the pending rows with `db/plan` inside a read-only transaction and deleting them in chunks of 25, each chunk in its own transaction: a chunk that fails (for example, an object a foreign key still references) is logged and skipped instead of aborting the run. A physical-deletion failure inside a chunk defers its rows one day (`deletion_attempts + 1`) and gives up after 7 attempts, accepting the orphan blob; chunk mappings (`upload_session_chunk`) are removed before the `storage_object` rows (NO ACTION foreign keys).
+- `storage-gc-deleted` removes the database row and backend content after the deletion delay, processing the pending objects in chunks of 25 with keyset pagination (`(deleted_at, id)` cursor, matching the `storage_object(deleted_at, id)` index). Only `status = 'valid'` objects are read: pending ones (still being written, see the status lifecycle) stay untouched even when already marked as deleted. Each chunk is one short read-write transaction: `SELECT ... FOR UPDATE SKIP LOCKED` (so two concurrent runs never take the same chunk; no separate read transaction exists and the run does not block the table vacuum) followed by the database delete and the backend delete grouped by backend. The cursor advances to the `(deleted_at, id)` of the last row read regardless of the deletion result, so a chunk that fails is skipped and not visited again in the same run: the error aborts the transaction (nothing of the chunk is deleted, locks are released), the handler catches it inside the transaction function and rolls it back explicitly — the commit that `tx-run!` performs afterwards is a JDBC-driver no-op on the already-rolled-back connection — and the loop continues with the returned rows. Accepted quirks, both benign and self-healing: (1) with rows of several backends in one chunk, a failure after one group's physical delete leaves those rows pending with already-removed content — pending rows are excluded from reads and the next run finishes them, since the physical delete is idempotent (S3 ignores missing keys, FS uses `deleteIfExists`); (2) rows locked by a concurrent run are skipped via `SKIP LOCKED` and deferred to the next scheduled run, and the `deleted_at <= now` cutoff is sampled per chunk. It emits the `penpot_storage_gc_deleted_objects_total{backend}` counter (one increment per deleted backend group, reported only after the chunk commits, so rolled-back chunks are never counted twice); run duration is covered by the generic `penpot_tasks_timing{name="storage-gc-deleted"}` histogram and chunk failures by the log-derived `penpot_log_events_total`. Every 5000 deleted objects the run pauses 5 seconds (throttle between chunk transactions, never holding one open). It runs as the `storage-gc-deleted` job (`::job-def`) and beats `jobs/heartbeat` after each committed chunk, so a long sweep keeps its lease and never holds it during a chunk transaction.
 - `storage-gc-touched` finds references before it sets `deleted_at`.
+- Orphan watch: `SELECT COALESCE(metadata->>'bucket', metadata->>'~:bucket') AS bucket, count(*) FROM storage_object so WHERE touched_at IS NULL AND deleted_at IS NULL AND status = 'valid' AND NOT EXISTS (SELECT 1 FROM upload_session_chunk usc WHERE usc.object_id = so.id) GROUP BY 1;` Only the `upload-session` row of that output is a leak signal (its rows always have a mapping or a pending touch/deleted mark); other buckets use domain-table references, not this mapping.
 - `objects-gc` removes deleted domain rows and touches their storage object IDs. Each proc streams its pending keys with `db/plan` inside a read-only transaction and deletes them in chunks, each chunk in its own transaction with `FOR UPDATE SKIP LOCKED`: a chunk that fails is logged and skipped instead of aborting the run.
 - Use `::db/reuse-conn true` with `sto/resolve` inside a database transaction.
 
@@ -34,7 +35,7 @@
 
 - `sto/touch-object!` is one `UPDATE storage_object SET touched_at`: DB-only, never touches S3/FS backend content.
 - Physical FK cascades are avoided by convention; domain deletion is logical (`deleted_at`) everywhere. GC removes small objects first, then parents, then grandparents (see the call order in `app.tasks.objects-gc/execute-objects-gc`). Per-row delete cost is small and bounded.
-- `objects-gc` sweeps in chunks of 100 (`::chunk-size`, overridable per job params); `storage-gc-deleted` in chunks of 25. The cursor that streams the pending keys runs in a read-only transaction of its own (`db/plan`), and every chunk is deleted in a transaction of its own: the cursor fetch and the deletes are never the same transaction.
+- `objects-gc` sweeps in chunks of 100 (`::chunk-size`, overridable per job params); `storage-gc-deleted` in chunks of 25. `objects-gc` (and the other `db/plan` sweeps) stream the pending keys in a read-only transaction of their own and delete every chunk in a transaction of its own: the cursor fetch and the deletes are never the same transaction. `storage-gc-deleted` has no separate cursor transaction: it reads, locks and deletes each chunk in the same short transaction (see the keyset pagination above).
 - Consequence: per-chunk cost is DB-only and bounded, so these sweeps cannot outrun the 30-minute jobs lease in practice (would need millions of pending rows in one proc). Per-chunk `heartbeat` where present is belt-and-braces, not load-bearing.
 
 ## Connection Reuse Details
@@ -91,7 +92,7 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
 | `profile` | User and team profile photos. References: `profile.photo_id` and `team.photo_id`. | Yes | Authentication required | Reference scan. |
 | `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
 | `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
-| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | No reference scan. A touched object is deleted after the delay; `gc-deleted` removes mappings before rows. |
+| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. |
 | `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
 | `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
 | `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
@@ -124,6 +125,11 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
 - The `file_data.metadata.storage-ref-id` value points to the storage object.
 - `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row.
 - File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
+- Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer holds at least `FOR SHARE` on the `file` row. A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`.
+- Offload pacing: none by deliberate decision. Sleeps only make sense when batching multiple transactions; the per-file transaction is short by construction and the scheduler run takes minutes per production metrics, so neither loop sleeps. `jobs/heartbeat` (pool-based liveness, ~1 write/s) stays on both paths.
+- Offload metrics (bounded labels only, never ids or error text): `penpot_file_data_offload_rows_total{type}` and `penpot_file_data_offload_bytes_total{type}` (`type`: `main`/`snapshot`/`fragment`, successful runs only) plus `penpot_file_data_offload_timing_ms{result}` (`result`: `ok`/`skip`/`error`). Helpers live in `app.jobs.metrics`; the job-def requires `::mtx/metrics`.
+- Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`.
+- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/process-metadata`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
 
 ## Metrics
 

.serena/memories/backend/subtleties.md

@@ -29,7 +29,7 @@
 - Whether a call is nested is decided by the CONNECTION, not by an ambient flag: `db/transact!` binds `*after-commit-conn*` around the body, and a transactable that is not identical to that connection is an independent unit of work that owns its own after-commit context. A `tx-run!` that passes a pool while a transaction is open therefore commits on its own and drains its own callbacks, instead of leaving them in a context that may roll back.
 - `after-commit` callbacks run on commit only. A transaction marked `::db/rollback` (used by `srepl` and `app.tasks/offload-file-data`) commits nothing, so it runs none of them.
 - `db/run!` opens/reuses one connection but does not create a transaction.
-- `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Job params (`job.params`) are plain JSON, decoded with the job-def decoder (`app.jobs/decode-params`); only the legacy `task` props were Transit (`decode-transit-pgobject`).
+- `db/tjson` is Transit JSON for jsonb storage; `db/json` is plain JSON. Job params (`job.params`) are plain JSON, decoded with the job-def decoder (`app.jobs/decode-params`); only the legacy `task` props were Transit (`decode-transit-pgobject`). Job results (`job.result`) are plain JSON too: they are read with the result decoder of the job-def (`app.jobs/decode-result`), and a result map keyed by uuid must declare its schema (`[:map-of ::sm/uuid ...]`, the import resolution does) or every key comes back as a keyword and every uuid as the text it is on disk.
 - Advisory transaction locks accept UUIDs or ints. UUID locks are hashed using a zero-UUID seeded siphash.
 
 ## Workers and cron
@@ -52,7 +52,7 @@
 - Duration config values are parsed as ISO-8601 with the `PT` prefix, so a day is written in hours (`:jobs-retention` and `:jobs-user-ttl` default to a week, and an operator writes `168h`): `7d` does not parse, and the value fails validation at boot instead of being silently ignored.
 - `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
 - The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
-- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`.
+- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413.
 - Response formatting negotiates with `Accept` or `_fmt=json`. Transit is the default for collection/boolean bodies; JSON encoding has special pointer-map handling.
 - Auth prefers the session cookie token before the `Authorization` header. Headers may be `Token` or `Bearer`. Only `kid=1`/`ver=1` tokens are decoded as session tokens; anything else is left unauthenticated (legacy v1 tokens were removed).
 - Shared-key auth requires `x-shared-key` as `<key-id> <key>` and stores the lowercased key id on the request. If no shared keys are configured it always rejects.
@@ -72,9 +72,15 @@
 - `app.metrics/run!` is safe by default: a recording failure never throws (a metrics bug must not change the behavior of the measured operation). The first failure per metric id logs at `warn`, later ones at `debug`. The `instance` precondition is a plain assert (the backend enables `:backend-asserts`), and the collector lookup sits outside the recording guard, so a missing instance fails hard even when asserts are disabled. `::mtx/metrics` is required by the storage, s3-backend, db-pool and jobs APIs; jobs helpers never silently skip a missing metrics instance. The resolver is `app.metrics/instance` and it raises, unlike `run!`.
 - `app.msgbus` (`pub!`, `sub!`, `purge!`) keeps taking the **instance** on purpose: msgbus is what produces it, and `app.rpc.notifications` already shows the caller passing a cfg. `app.jobs` resolves `::mbus/msgbus` from the cfg internally, and `insert-event` keeps its explicit pre-insert check so a profile job never loses its notification.
 - `app.db/after-commit` is the boundary for SQL-backed metrics: callbacks are drained by the commit of the transaction that owns the connection they were registered on, and discarded when it rolls back. Because nesting follows the connection, that transaction is the outermost one only for a real nested call.
+- `penpot_http_malformed_requests_total` (counter, no labels) counts requests whose body the parser could not read. `app.http.middleware/wrap-parse-request` emits it where it classifies the error, so the instance travels as a middleware argument (`[mw/parse-request (::mtx/metrics cfg)]`) and each service declares the definition with its own prefix (nexus: `nexus_http_malformed_requests_total`). The per-endpoint request counters of nexus do not count these requests: they sit inside the parse middleware, so the failure happens before them.
+
+## Log-derived metrics
+
+- `app.loggers.metrics/reporter` counts the log records the application emits and publishes them as `penpot_log_events_total{logger, level}` (counter), so the error panels do not have to count text in Loki. It is a component that hooks `add-watch` on `app.common.logging/log-record`, like `app.loggers.database`, but with no thread and no channel: the increment is in memory, so it goes straight in the watcher. Only `warn`, `error` and `fatal` are counted, and only the records that go through `app.common.logging` (the application's own logs; a library writing straight to log4j2 is not counted, and today that is one line in a thousand). The body of the watcher is wrapped in a `try/catch` that swallows the error on purpose: the watcher fires from the `swap!` of the log record, inside the logging path. With `::l/sync?` (`app.binfile.v1`, `app.msgbus`) an exception there would reach whoever was logging; on the async path it escapes to the logging executor worker, whose trace goes to the thread's uncaught handler -- outside the application's logging -- and the pool replaces the dead worker, so logging continues and that line is lost. The catch also makes a publication that fails every time invisible: `penpot_log_events_total` stays flat and nothing is written about it. The published `level` values are lowercase (`warn`), unlike the `level` label of the Loki logs (`ERROR`), so a panel that moves from one source to the other has to change it. Cardinality is `logger` (fixed names taken from the code) times `level`; a logger built from a dynamic value would break that.
 
 ## Storage and media
 
+- The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation.
 - Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
 - SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
 - Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
@@ -86,4 +92,12 @@
 - File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data.
 - `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob.
 - Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written.
-- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
+- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
+- A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data.
+- Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision.
+- File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only written by the `legacy-db` backend, plus the rows written before the 2025-07 file storage refactor.
+- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label.
+- The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again.
+- Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter.
+- A deletion date is assigned once and never moved: `restore-files` (`app.rpc.commands.files`) unmarks only durable history, the `:file` method of `app.tasks.delete-object` dates only rows that have no date yet, and `objects-gc` collects only rows that carry one. Unmarking a revision leaves it invisible to `objects-gc` for ever, because nothing sets that date again.
+- Binfile v1/v3 import and the v3 export `get-file` step run inside one long transaction. They raise `idle_in_transaction_session_timeout` to 30 min (`app.binfile.common/transaction-idle-timeout-ms`) instead of disabling it, so a connection left idle while the ZIP is decoded or the file data is realized is bounded; `statement_timeout` stays at 300 s.

.serena/memories/clojure/design-rules.md

@@ -2,10 +2,10 @@
 
 How to shape a function in this codebase. Each rule came from a review round; do not re-derive from taste.
 
-- **Preconditions belong to the boundary, not to the core.** A function documents what it needs (hints, docstring) and assumes it. Absence checks live where the value enters the system: the caller that resolves optional objects, `ig/init-key`, the request handler. A `(when (and (some? a) (some? b)) ...)` inside a leaf function means the guard is in the wrong place — if you cannot get `a` or `b`, you should not be calling it.
-- **Optional-by-design data is guarded where the optionality is born.** Example: `ConnectorStatistics` is nil when the server option is off, so the guard is `when-let [cs (some-> server ...)]` at the entry point and the consumer assumes `cs`.
+- **Preconditions belong to the boundary, not to the core.** A function documents what it needs (hints, docstring) and assumes it. Absence checks live where the value enters the system: the caller that resolves optional objects, `ig/init-key`, the request handler — including `nil` that is optional-by-design (`when-let [cs (some-> server ...)]` at the entry point, as with `ConnectorStatistics` when the server option is off). A `(when (and (some? a) (some? b)) ...)` inside a leaf function means the guard is in the wrong place — if you cannot get `a` or `b`, you should not be calling it.
 - **A static set of operations is written statically.** With a fixed, small set (four metrics) write the four calls. Do not build a collection and iterate it (`doseq` over a literal vector): it allocates, adds indirection and hides each operation. Use a collection when the set is dynamic (config, registry, input).
 - **Default to no `!`; existing `!` is legacy being removed gradually.** Do not introduce new `!` names except the allowed cases documented in `mem:clojure/idioms` (add and justify the special case there first). Allowed: mutating public fns in `app.db` (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, ...) and fns whose contract is to work over transients (stdlib `conj!`/`assoc!`/`dissoc!`/`persistent!` and custom helpers over them). Existing `!` elsewhere (e.g. `emit!`, `send!`, `submit!`, `validate-file!`) stays until its refactor; keep a family consistent in the meantime. Reporting helpers (log, sample-and-publish) never take `!`: the mutation happens in the `!` API they call.
+- **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is.
 - **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one.
 - **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them.
 

.serena/memories/clojure/idioms.md

@@ -10,3 +10,4 @@ Behaviors confirmed against the language/stdlib — do not re-derive from assump
 - `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO).
 - `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
 - Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant).
+- `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`).

.serena/memories/critical-info.md

@@ -7,15 +7,16 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
 - Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
 - Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
 - Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
-- When designing a solution or an API, read `mem:clojure/design-rules` (function shape, boundary checks, naming).
+- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
 
 # Development workflow
 
 - Commit/PR/issue creation is **on explicit request only**. Before any of these actions, read the relevant memory — don't infer format from prior examples:
-  - Before `git commit` → `mem:workflow/creating-commits` (subject/body format, 76-char body wrapping enforced by `scripts/check-commit`, `AI-assisted-by: model-name` trailer)
+  - Before `git commit` → `mem:workflow/creating-commits` (subject/body format, 80-char body wrapping enforced by `scripts/check-commit`, `AI-assisted-by: model-name` trailer)
   - Before `gh issue create` → `mem:workflow/creating-issues` (title derivation, body template, labels, Issue Type)
   - Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, "Note:" line)
 - Before a repo-wide pnpm version update → `mem:workflow/updating-pnpm` (workspace layout, `scripts/sync-pnpm-version` flow, the stamp-missing-field and ignored-builds gotchas, verification steps)
+- Before a Node version update → `mem:workflow/updating-node` (the six pin sites in `.nvmrc`, the devenv and image Dockerfiles and the plugins-deploy workflows, the DHI tag and digest rules, what to verify)
 - **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked.
 - **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via the `update-changelog` skill flow or on explicit user request.
 - You have access to the GitHub CLI `gh` or corresponding MCP tools.
@@ -59,6 +60,7 @@ The memory is structured in a way that you can get the critical information abou
 - `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`.
 - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`.
 - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `<repo>/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`.
+- `scripts/setup-opencode` — Install the opencode CLI in-container as a global pnpm package in `$PNPM_HOME`. Not part of the devenv image; see `mem:devenv/core`.
 - `scripts/ci` — CI orchestration script: runs lint, tests, and format checks per module (`frontend backend common render-wasm exporter mcp plugins library`). Logs go to `.ci-logs/`; read the log file on failure. It is the preferred way to run the backend tests, and it exports `PENPOT_FLAGS` with `enable-backend-asserts`, which is what makes `clojure.core/assert` exist in the compiled test code. See `mem:scripts/ci`.
 
 # Dependency graph

.serena/memories/devenv/core.md

@@ -24,6 +24,8 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
 - Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead.
 - No cross-project `depends_on`. `manage.sh ensure-infra-up` uses Compose `--wait`; PostgreSQL, RustFS, and Mailpit expose healthchecks. PostgreSQL checks TCP so its temporary init server cannot report ready.
 - `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`.
+- opencode is not part of the devenv image. `scripts/setup-opencode` installs the `@opencode/cli` global pnpm package into the penpot user's `$PNPM_HOME` (`/home/penpot/.local/share/pnpm`, inside the user-data volume, so it survives a container re-creation); nothing pins the version in the repo, it tracks the `latest` dist-tag at install time. pnpm 12 puts the global packages in `$PNPM_HOME/global` and the bin dir in `$PNPM_HOME/bin`, and aborts with `ERR_PNPM_GLOBAL_BIN_DIR_NOT_IN_PATH` unless that bin dir is on `PATH`; `files/bashrc` and the script both add it. The script exports `PNPM_HOME` itself because non-interactive shells never read `~/.bashrc`.
+- `files/entrypoint.sh` chowns penpot's whole `~/.local` and `~/.cache` on every start, because `EXTERNAL_UID` remaps penpot's UID to the host user and build-time ownership then stops matching. That covers pnpm's global dir, store, `package-manager-store` and metadata cache, plus opencode's own state; without it, global installs and pnpm's `packageManager` auto-download fail on permissions. The dotfiles copied from `/root` (`.bashrc`, `.vimrc`, `.tmux.conf`) are chowned in the same pass. `~/.config` stays out on purpose (the opencode overlay bind-mounts a host directory inside it) and so does `/home/penpot` itself, which holds the repo bind mount.
 
 ## Worker policy
 
@@ -75,3 +77,42 @@ No `--delete` on the working-tree pass: gitignored caches in the workspace survi
 - `start-devenv` / `log-devenv` / `drop-devenv`: legacy paths around ws0 + shared infra. `drop-devenv` never removes volumes.
 
 `exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.
+
+## Image and dependency pins
+
+Two Dockerfile trees, both multi-arch (`linux/amd64,linux/arm64`):
+`docker/devenv` (dev) and `docker/images` + `docker/imagemagick` (prod).
+
+- Hand-pinned versions, one source each:
+  - `docker/devenv/Dockerfile`: `NODE_VERSION` (+ per-arch `NODE_SHA256`),
+    `PNPM_VERSION` (+ `PNPM_SHA256`), `CADDY_VERSION`, `CLOJURE_VERSION`,
+    Zulu JDK URL + `ESUM`, `RUSTUP_VERSION`/`RUST_VERSION`/
+    `EMSCRIPTEN_VERSION`, `CLJKONDO_VERSION`, `BABASHKA_VERSION`,
+    `CLJFMT_VERSION`, `PIXI_VERSION`, `GITHUB_CLI_VERSION`,
+    `AWS_CLI_VERSION` (+ `BINARY_SHA256`), `UV_VERSION`, `SERENA_VERSION`,
+    `postgresql-client-18`, and the `penpotapp/imagemagick:<tag>`
+    `COPY --from`.
+  - `docker/images/Dockerfile.backend`: Zulu JDK URL + `ESUM`. The `jlink`
+    `--add-modules` list must keep `jdk.management`
+    (`mem:backend/subtleties`).
+  - `docker/images/Dockerfile.{exporter,mcp,media-processor}`: `ARG
+    PNPM_VERSION` (+ arch SHAs, same values as devenv).
+- Node and pnpm must agree in five places: `.nvmrc`, the devenv tarball pin,
+  the two `plugins-deploy-*` workflows that set `runtime: node@<version>`, and
+  the four Dockerfile pins. pnpm procedure: `mem:workflow/updating-pnpm`.
+- `dhi.io` base images are pinned `tag@sha256:<index digest>`; Renovate moves
+  them (`penpot-renovate[bot]`, commits titled `:arrow_up: Update DHI base
+  images`). Tag and digest move together: a new tag with the old digest still
+  pulls the old image, and the digest must be the multi-arch index, not a
+  per-platform one.
+- Every hand-pinned checksum is per architecture; one wrong value breaks half
+  of the multi-arch build.
+- `SERENA_UPDATE_VERSION` in `docker-compose.main.yml` must never be lower than the Dockerfile's `SERENA_VERSION`. Any value makes `start-tmux.sh` install that Serena release on start, overriding the version baked into the image; a stale value silently downgrades Serena (it once replaced 1.7.0 with 1.6.1). If the image already has that version the install is an offline no-op; otherwise it downloads on every start (`run-devenv` recreates the container) and, offline, fails after ~25s and keeps the image version. Bump both together; empty means use the image version.
+- Infra images live in `docker-compose.infra.yml`: `rustfs/rustfs` (owns the
+  S3 data volume), `axllent/mailpit` (SQLite in a volume), `valkey/valkey`,
+  `postgres:18.6`, `rroemhild/test-openldap`. Keep the volume names stable
+  when bumping them; replacement services get new volumes and leave the old
+  data untouched.
+- Build locally: `docker build -f docker/devenv/Dockerfile docker/devenv/`
+  and `cd docker/images && ./build.sh <image> --local`. CI:
+  `build-docker-devenv.yml`, `build-docker.yml`.

.serena/memories/exporter/core.md

@@ -5,6 +5,7 @@
 ## Layout and commands
 
 - Source: `exporter/src/`; config: `deps.edn`, `shadow-cljs.edn`, `package.json`; runtime helpers/assets: `vendor/`, `scripts/`.
+- `exporter/resources/wasm/render-wasm.{js,wasm}` and `exporter/src/app/wasm/shared.js` are generated render-wasm build output, not source: they are gitignored. Never hand-edit or commit them; regenerate by rebuilding render-wasm.
 - From `exporter/`: setup `./scripts/setup`; watch `pnpm run watch` or `pnpm run watch:app`; production build `pnpm run build`; test bundle `pnpm run build:test`; tests `pnpm run test` or `pnpm run test:quiet`; lint `pnpm run lint:clj`; format check/fix `pnpm run check-fmt:clj` / `pnpm run fmt:clj`.
 - Because exporter consumes `common/`, shared file/shape/model changes may need exporter verification even when the immediate change is not under `exporter/`.
 - Cross-cutting testing principles and anti-patterns: `mem:testing`.

.serena/memories/frontend/composable-component-tests.md

@@ -150,5 +150,5 @@ The frontend bundle is built once per SHA and restored by this job (`mem:fronten
 Details: README, "Running in CI".
 
 ## Substrate
-`mem:common/test-setup`, `mem:common/component-data-model`, `mem:common/component-swap-pipeline`,
+`mem:common/testing`, `mem:common/component-data-model`, `mem:common/component-swap-pipeline`,
 `mem:frontend/testing`.

.serena/memories/frontend/dashboard-viewer-subtleties.md

@@ -5,7 +5,7 @@
 - Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id.
 - Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly.
 - Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications.
-- The import wizard runs the package as a job of the backend: it reads the manifest of the file in the main thread (`app.main.data.imports`, no web worker) and follows the job, so its rows paint the milestone of the job (the stage and its counter, with the file in course) instead of the messages the old SSE stream sent. Before the job exists, the rows paint a client-side `:upload` milestone built from the chunked upload (`uploads/upload-blob-chunked` `:on-progress`, chunks done/total), which never comes from the backend; the chunks go one at a time and the row writes them as a percentage of the file, not as a counter. Once created and still waiting for a worker, each entry reports `:queued` ("Queued"); the `:start`/`:retry` events move it to bare progress, and only `:progress` carries a milestone. The export modal follows the same one-job-per-file model, but every file is queued up front (even the ones whose job is created only after the previous one is over) and its queue ends only with a milestone: `:start`/`:retry` do not end it there. Neither modal paints a generic "…ing file…" line any more: each row carries its own status. The `:error` of an import entry is user-facing text (the backend hint, or a translated label), never an i18n key: the wizard renders it as-is, it does not translate it again. The legacy SSE import stays in the backend for other clients.
+- The import wizard runs the package as a job of the backend: it reads the manifest of the file in the main thread (`app.main.data.imports`, no web worker) and follows the job, so its rows paint the milestone of the job (the stage and its counter, with the file in course) instead of the messages the old SSE stream sent. Before the job exists, the rows paint a client-side `:upload` milestone built from the chunked upload (`uploads/upload-blob-chunked` `:on-progress`, chunks done/total), which never comes from the backend; the chunks go one at a time and the row writes them as a percentage of the file, not as a counter. Once created and still waiting for a worker, each entry reports `:queued` ("Queued"); the `:start`/`:retry` events move it to bare progress, and only `:progress` carries a milestone. The export modal follows the same one-job-per-file model, but every file is queued up front (even the ones whose job is created only after the previous one is over) and its queue ends only with a milestone: `:start`/`:retry` do not end it there. Neither modal paints a generic "…ing file…" line any more: each row carries its own status. The `:error` of an import entry is user-facing text (the backend hint, or a translated label), never an i18n key: the wizard renders it as-is, it does not translate it again. The link-later step takes the file id from the `:id` of its resolution entry, never from the key of the resolution map: the map comes from the job result, so what a key holds depends on the result schema of the job-def that wrote it, while the wizard indexes its selection by the id of the entry. The legacy SSE import stays in the backend for other clients.
 - File creation/duplication strips file `:data` before putting file summaries into dashboard state.
 
 ## Viewer

.serena/memories/frontend/routing-app-shell-subtleties.md

@@ -14,9 +14,14 @@
 - Frontend error handling treats stale cross-build JS chunk failures specially: messages containing `$cljs$cst$` or `$cljs$core$I` plus undefined/null/not-a-function signatures trigger throttled reload.
 - Plugin-originated uncaught errors are identified through the plugin runtime hook and logged rather than turning into the global exception page.
 - `app.main.errors/submit-report` is governed by a dedup governor: each report carries a fingerprint (`report-name|type|code|hint|first stack frame`; the report name is part of it so a handled report never coalesces with an unhandled/exception-page one), the first occurrence is always emitted, repeats within 2 minutes are counted and included in the next emitted report as `:occurrences`, and the fingerprint cache is bounded (first-inserted entry evicted, FIFO, via `:order` queue) so memory stays fixed. It applies to `handled-exception`, `unhandled-exception` and `exception-page`; a report without an exception cause is ignored and does not consume a reservation.
-- Errors caused by the environment (`environment-error-types`: `:network`, `:offline`, `:bad-gateway`, `:service-unavailable`, `:nitrate-unavailable`, `:nitrate-not-configured`) are not application defects: they are reported as audit-only `handled-exception` (never `unhandled-exception`/`exception-page`, so they skip internal reports and alerts), with a compact report and a fingerprint that drops the stack frame. `:offline` has its own handler and no longer falls through to `:default`; `:network`/`:offline` show the `errors.connection-error` toast.
+- Errors caused by the environment (`environment-error-types`: `:network`, `:offline`, `:bad-gateway`, `:service-unavailable`, `:gateway-error`, `:nitrate-unavailable`, `:nitrate-not-configured`) are not application defects: they are reported as audit-only `handled-exception` (never `unhandled-exception`/`exception-page`, so they skip internal reports and alerts), with a compact report and a fingerprint that drops the stack frame. `:offline` has its own handler and no longer falls through to `:default`; `:network`/`:offline`/`:gateway-error` show the `errors.connection-error` toast.
+- `repo/handle-response` classifies responses the backend never wrote (body is not a transit map) by status: 504 and Cloudflare 520–524 → `:gateway-error` (retryable, so GETs and saves retry); 429 → `:rate-limit`; any other ≥400 → `:unexpected-response` (both toast-only via `handle-transport-error`, never the exception page). `:internal :unable-to-process-repository-response` is left for a <400 response that does not decode. Uninterpretable bodies are capped at 500 chars in `:data`.
 - `generate-report` accepts an explicit `{:format :compact|:full}` (default `:full`) chosen by its caller (`flash`, `exception-section*`): `:compact` keeps the context header plus type/code/uri, and skips the stack, the `ex-data` dump and the last-events list. It is total: if formatting fails it returns a minimal fallback string instead of nil, so an already reserved emission is never dropped.
 - `flash` runs its whole body (report pipeline first, toast after) inside a single `ts/schedule` callback: nothing executes synchronously on the error handler's stack. The report is reserved before being generated, so suppressed occurrences do not pay the `generate-report` cost; a reporting or notification failure is logged to the console and never propagates, and the toast is still attempted. `flash` returns a total promise (never rejects) resolving with the generated report, or nil when nothing is emitted, once the callback completes; production callers ignore it, tests await it. `flash` derives only the payload format from the cause (`environment-error?` → `:compact`); the audit event name is the canonical one requested by `:type` (`handled-exception`/`unhandled-exception`) and is never reclassified, because external tools filter on those names. `exception-section*` picks `handled-exception`/`exception-page` explicitly per cause; `flash-persistence` only adds the toast hint.
+- `flash` accepts a `:links` vector of `{:label :callback}` actions, appended after the download link that `:report-link?` produces. `flash-persistence` passes a reload link, so a terminal save failure hands the user both the report and the only action that restarts persistence for the session. `labels.reload-page` plus `js/location.reload` is the established full-restart pattern; the link cannot be invoked under the test runner because `js/location` does not exist there, so it is covered by its label and by being callable.
+- `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler.
+- The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission.
+- Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason.
 
 ## Store and websocket
 
@@ -24,4 +29,4 @@ For general store mechanics such as `emit!`, `last-events`, persistence, and und
 
 - Websocket initialization uses `cf/public-uri` joined with `ws/notifications`, converting `http/https` to `ws/wss`, and includes the current `session-id` as query param.
 - Reinitializing or finalizing websocket stops the previous receive stream. Incoming websocket payloads become Potok data events under `app.main.data.websocket/message`.
-- A job the client created is followed with `app.main.data.jobs/watch-job`: it emits the `:job-event` messages of that job (filtered by `:job-id`) and one final outcome read from the row with `get-job`, which is read when the stream starts and again on every `opened` of the websocket, so a job that ended before the client started following it, or while the socket was down, still answers. The stream ends after that outcome, and a read that fails reaches the caller: the repo retries the transient transport errors of a GET, and anything else is a real failure the flow has to show. The `:stage` of a progress event is a stable key of the vocabulary of the worker, never a display string: `app.main.ui.jobs.progress` resolves it with a `case` of literal `tr` calls, falls back to a generic label for a stage it does not know, and the counter of the activity is named after its own stage (the counter of `:files` is the outer context).
+- A job the client created is followed with `app.main.data.jobs/watch-job`: it emits the `:job-event` messages of that job (filtered by `:job-id`) and one final outcome read from the row with `get-job`, which is read when the stream starts and again on every `opened` of the websocket, so a job that ended before the client started following it, or while the socket was down, still answers. The stream ends after that outcome, and a read that fails reaches the caller: the repo retries the transient transport errors of a GET, and anything else is a real failure the flow has to show. The `:stage` of a progress event is a stable key of the vocabulary of the worker, never a display string: `app.main.ui.jobs.progress` resolves it with a `case` of literal `tr` calls, falls back to a generic label for a stage it does not know, and the counter of the activity is named after its own stage (the counter of `:files` is the outer context).

.serena/memories/frontend/workspace-state-persistence-subtleties.md

@@ -11,21 +11,30 @@
 ## Repo calls
 
 - `app.main.repo/send!` uses GET only when the RPC name starts with `get-`, when all params are query params, or for configured special cases. Only GET requests are retried.
-- GET retry is limited to transient `:network`, `:bad-gateway`, `:service-unavailable`, and `:offline` errors with exponential backoff. Mutations are not retried.
+- `repo/retryable-types` is public and is the single source of truth for the transient transport classification. `persistence/transient-error?` and `errors/environment-error-types` both derive from it; extend that set, never copy it.
+- GET retry is limited to transient `:network`, `:bad-gateway`, `:service-unavailable`, `:gateway-error` (504, Cloudflare 52x), and `:offline` errors with exponential backoff. Mutations are not retried.
 - A server SSE response is only accepted when the command is configured `:stream?`; otherwise it raises an unexpected-response assertion.
 
 ## Commits, undo, persistence
 
-- `commit-changes` refuses to create commits unless `:permissions :can-edit` is true. It captures file revn/vern, selected-before, features, tags, undo group, and translation flag into a `::commit` event.
+- `commit-changes` refuses to create commits unless `:permissions :can-edit` is true. It captures file revn/vern, selected-before, features, tags, undo group, and translation flag into a `::commit` event. A refusal answers with an empty stream, never nil, so callers subscribe uniformly instead of branching on nil.
 - Applying a remote commit first rolls back pending local commits, applies the remote changes, then replays pending local redo changes. Index updates are emitted for undo, remote redo, and replayed redo paths.
 - Local commits are independently consumed by undo, persistence, WASM model updates, thumbnail/library watchers, and text position-data recalculation.
 - Persistence buffers local commits: status becomes pending after about 200ms, commits are flushed after about 3s or `::force-persist`, and buffered commits are merged per file before `:update-file`.
 - Persistence sends revn as the max of the commit revn and locally tracked latest revn; remote commits update that revn tracker.
 - Persistence is skipped in version preview/read-only mode or without edit permission.
-- Save failures split transient vs terminal (`transient-error?`: the repo retryable types `:network`/`:offline`/`:bad-gateway`/`:service-unavailable` plus `:invalid-save-response`; everything else is terminal). Terminal keeps the `:error` halt + `flash-persistence` path; transient enters a `:retrying` episode: the head commit stays queued and resends with backoff 2s/8s/20s (3 retries, then today's terminal path). Resends rotate the `::request-id` stamp only when the old request left `active-requests`, carry the same `:commit-id`, and never double-send while one request is in flight (`:in-flight` stays silent). Retry timers carry the episode token; a superseded token stays silent. Status stays `:retrying` through re-entries (`next-status` refuses `:pending`/`:saving` from it); waiters (`wait-persisted-or-error`) wait through it and reject only on `:error`.
+- A terminal failure halts the queue on purpose: the edits stay queued so `wait-persisted-or-error` keeps rejecting, and `workspace/versions.cljs` relies on that to avoid snapshotting a file whose edits were never saved. `recover-persistence` lifts the halt when a file is opened again or the role changes. A transport failure still inside its retry window (`slow-retry-pending?`) is resent once (`resume-persistence true`), never released; any other halt emits `release-failed-file` (drops every queued commit of the file that failed) then `resume-persistence`. Opening a file also replays the queued edits of its version onto the loaded data (`pending-file-changes` → `restore-pending-changes` in `bundle-fetched`); a replay that throws keeps the server data and the queue. That is what unblocks the editor without a reload; because the queue is shared, dropping the failed file is also what lets other files' pending edits drain. `release-failed-file` runs at recovery time, never at failure time — clearing the error eagerly would make waiters resolve.
+- The split between `initialize-persistence-state` and `recover-persistence` is deliberate: initialization deliberately does NOT lift a halt, so undeliverable edits are released when the user acts, not when the view mounts.
+- **Every non-transport terminal save failure is permanent.** There is no "stale but valid" category: permission lost, file gone, and server-rejected changes all fail again on a resend. Losing edit permission is not temporary — the backend raises `:not-found` for it (`check-edition-permissions!` → `perms/make-check-fn`), and `:not-found` routes to the exception page. So `:save-permission-denied` (a client-side `can-edit` check) must never be resumed: replaying the save to discover that routes the user to the exception page. Release it instead.
+- Save failures split transient vs terminal (`transient-error?`: the repo retryable types `:network`/`:offline`/`:bad-gateway`/`:service-unavailable`/`:gateway-error`, plus `:invalid-save-response`, plus save-only `busy-server-error?`: `:rate-limit` (429 from rlimit/climit — a save parked on another save's file lock holds the profile's single `:update-file/by-profile` permit, so the user's saves to other files 429 after 60 s) and backend-written 504 `:server-error` with `:statement-timeout`/`:idle-in-transaction-timeout` (Postgres cancels the lock wait at 300 s, before nginx; rolled back, so resend is safe). Kept out of `rp/retryable-types` on purpose so GETs do not retry them; everything else is terminal). Terminal keeps the `:error` halt + `flash-persistence` path; transient enters a `:retrying` episode: the head commit stays queued and resends with backoff 2s/8s/20s (3 retries, then the `:error` path). A transient failure that outlasts the backoff arms `slow-retry-cycle`: one silent resend (`:recovering`) every `slow-retry-delay-ms` (30s) until a save lands, `retry-give-up-ms` (24h since `:failing-since`) passes, or the head commit collects `max-slow-failures` (3) attempts that failed after `slow-failure-ms` (90s) or more (`:slow-failures`, counted in `persistence-transient-failure` from the attempt's elapsed time): a gateway timeout may leave the backend applying the save, so a save that keeps failing slowly must not be replayed for a day, while fast failures (offline) never count; the cycle stops on `::resume-persistence`, `::initialize-persistence` and `finalize-workspace`. Leaving the workspace also emits `suspend-retries` (`:suspended` in `:persistence`, drops `:retry-token` so a pending backoff timer goes stale): no backoff, slow cycle or `online` resend runs outside the workspace; only `workspace-initialized` (not a role change) lifts it via `recover-persistence` (`opened-file?`), which also resends a `:retrying` episode whose backoff was cut short. Its toast is `errors.save-still-retrying` without a reload link; a landing save hides the failure toast (`errors/persistence-failed-tag`). Replaying a save is safe because changes are idempotent, so there is no unknown-outcome guard: a commit whose earlier request left `active-requests` is simply sent again under the same `:commit-id`; only a request still in flight blocks a resend (`:in-flight` stays silent). Tests hold the slow cycle by stubbing `rx/timer` to never fire for `slow-retry-delay-ms`, or instant timers loop forever. Retry timers carry the episode token; a superseded token stays silent. Status stays `:retrying` through re-entries (`next-status` refuses `:pending`/`:saving` from it); waiters (`wait-persisted-or-error`) wait through it and reject only on `:error`.
 - One reconnect notice per episode: sticky toast tagged `:persistence-reconnecting` (single-toast store, re-show replaces), hidden by tag on drain (`:saved`) and on terminal failure; recovery is silent. Header indicator has a `:retrying` state (`workspace.header.retrying`).
 - Resume triggers: backoff timer and the browser `online` event (guarded by `exists? js/window`; re-enters the runner only for a live `:retrying` episode). New local edits during `:retrying` only join the queue: the episode keeps its `:run-id`, so `append-commit` does not re-enter, and the live runner (still waiting on the head's `commit-persisted`) sends them after the head. Re-entering on edits would bypass the backoff and spend an attempt per batched commit.
 - Tests instant-trigger retries by stubbing `rx/timer` (recording delays to assert the schedule); dynamic bindings do not survive `await` continuations, so no dynamic var for delays.
+- `save-request-timeout-ms` is 7 minutes and wraps the `update-file` request in `rx/timeout`, failing it as a transient `:network` error so the retry path resends instead of blocking the queue forever. It is deliberately longer than `saving-stall-timeout-ms` (5 min) so the stalled report fires before the retry.
+- `sync-unload-guard` keeps `:persistence` in `st/ongoing-tasks` while edits are buffered or queued, which is what holds the browser leave-page warning on. It syncs on `::append-commit`, `::discard-commit` and `::release-failed-file` (every queue change). It is the one effect that must stay in `store.cljs`'s `omitset`, and `store.cljs`'s `ongoing-tasks` watcher is guarded by `exists? js/window` because tests run without a window.
+- `leave-signals` (document `visibilitychange`→hidden, window `blur`) emits `::force-persist` when edits are buffered, so leaving the tab/window sends the 3 s batch at once; tests mock `dps/leave-signals` BEFORE `initialize-persistence` runs. pagehide/beforeunload are too late for an async save.
+- Exports and the `.penpot` download call `force-persist-files-and-wait file-ids timeout` (waits only for those files, ends on `:error` or timeout, never raises) and then `unsaved-changes? state file-ids` to warn; it reads `refs/persistence`, so tests mock it with `(l/derived :persistence store)`.
+- `persistence-failed` emits a `save-failed` audit event (`:cause-type :code :retrying :queued-commits`) whenever the user is notified (`notify-failure?`, same rule as the toast), including delegated auth/not-found failures.
 - Undo transactions can stay open only temporarily; timed-out pending transactions are force-committed after about 20s. Undo entries are capped at 50.
 - Undo/redo are ignored while a normal editor/drawing interaction is active, except grid-layout edition handles undo through this path.
 - After local commits and when render-wasm is active, text shapes get derived `:position-data` recomputed in a separate commit tagged `#{:position-data}`; that tag is excluded from the position-data watcher to avoid loops.

.serena/memories/prod-infra/core.md

@@ -5,7 +5,7 @@ Backend (`app.config`, `PENPOT_*` env vars) is parameterized; deployments choose
 ## Services
 
 - **PostgreSQL**: durable store. Profiles, teams, files, sessions, audit, `storage_object` metadata, the `job` work queue (plus the dormant legacy `task` table), `scheduled_task` cron registry, migrations. File-data also lives here when the file-data backend is `legacy-db`/`db`. One shared DB across all backends.
-- **Redis (Valkey-compatible)**: per-backend message bus and cache. Concrete uses: msgbus Pub/Sub for collaborative-editing broadcasts and team/profile-org notifications fired by RPC handlers (`app.rpc.notifications`, `files_update`, `teams`, `websocket`); file-summary cache gated by `enable-redis-cache`; rate-limit counters; and the dispatcher→runner work hand-off list `penpot.worker.queue:<tenant>:<queue>`. `PENPOT_REDIS_URI`.
+- **Redis (Valkey-compatible)**: per-backend message bus and cache. Concrete uses: msgbus Pub/Sub for collaborative-editing broadcasts and team/profile-org notifications fired by RPC handlers (`app.rpc.notifications`, `files_update`, `teams`, `websocket`); file-summary cache gated by `enable-redis-cache`; rate-limit counters; and the dispatcher→runner work hand-off list `penpot.worker.queue:<tenant>:<queue>`. `PENPOT_REDIS_URI`. A burst of client-side `RedisCommandTimeoutException` (2s on `DEL`/`SET`, 10s on `BLPOP`) with the exporter healthy is the JVM paused or starved, not Redis: the timeout fires because nothing reads the reply.
 - **Object storage**: backends `:s3` and `:fs`. S3 in prod; devenv uses RustFS. Holds uploaded media, file-data when the file-data backend is `storage`, exports. Backend-side details (resolve, dedup, bucket set, object lifecycle, and file-data backends): `mem:backend/storage`.
 - **SMTP mailer**: invitations, password resets, email verification (sent via the `:sendmail` worker job).
 - **LDAP** (optional auth provider): helpers in `app.auth.*`, gated by `enable-login-with-ldap`.
@@ -22,7 +22,7 @@ Postgres row locking is the only correctness primitive: `job` claims via `FOR UP
 
 Two known race patterns survive multi-backend operation:
 
-- **Cron dedup is best-effort.** The lock on `scheduled_task` is released when the task body finishes. If two backends' cron timers fire for the same scheduled instant with a gap larger than the task body's runtime, both execute it. Penpot's cron entries are idempotent (`session-gc`, `objects-gc`, `storage-gc-*`, `storage-pending-gc`, `jobs-gc`, `file-gc-scheduler`); the exceptions are `:telemetry` (would double-report) and `:audit-log-archive` (depends on archive target idempotency).
+- **Cron dedup is best-effort.** The lock on `scheduled_task` is released when the task body finishes. If two backends' cron timers fire for the same scheduled instant with a gap larger than the task body's runtime, both execute it. Penpot's cron entries are idempotent (`session-gc`, `objects-gc`, `storage-gc-*`, `storage-pending-gc`, `jobs-gc`, `file-gc-scheduler`); the exceptions are `:telemetry` (would double-report) and `:audit-log-consumer-nexus` / `:audit-log-consumer-admin-console` (depend on archive target idempotency).
 - **`jobs/submit ::dedupe true`** does a non-atomic `DELETE` then `INSERT`. Concurrent cross-backend submits can both bypass the `DELETE` (each sees the other's uncommitted insert as absent) and end up with duplicate `'new'` rows. Each row claims and runs once independently, so the underlying work is fine; the "at most one pending" guarantee weakens.
 
 Penpot in production lives with both: horizontal-scale deployments accept "exactly-once" as "essentially-once for idempotent operations." Devenv parallel instances handle it by running workers only on ws0 (see `mem:devenv/core`).
@@ -43,6 +43,7 @@ Production runs the deployed toolbox, not the devenv one:
 - **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups.
 - **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them.
 - The read-only replica cuts long queries: see `mem:prod-infra/grafana`.
+- A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table (file saves write to `file_change` on every save) build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block); the shipped migration uses `IF NOT EXISTS`, finds it and does nothing. A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_indexes` and `pg_index.indisvalid`).
 
 ## See also
 

.serena/memories/prod-infra/grafana.md

@@ -6,16 +6,23 @@ Keep this memory free of internal names: no host names, no deployed versions, no
 
 ## Datasources
 
-- Metrics — **Prometheus**, uid `xkRmyFBGk`. PromQL. Metric families: `penpot_*` (RPC, task timings, CLIMIT, HTTP, websocket, storage), `hikaricp_*`, `jvm_*` and `process_*` on the backend (`job="app"`); `pg_*` from the Postgres exporter (`job="postgres"`); `redis_*` from the Redis exporter (`job="redis"`); `node_*` and `container_*` from node-exporter and cAdvisor. Backend histograms are in **milliseconds**.
-- Logs — **Loki**, uid `h3CGKhEMk`. LogQL, and filter by the structured metadata `logger`/`level` (`{job="penpot"} | logger="..." | level="ERROR"`). Signals that only exist as logs: the profile behind a CLIMIT queue (`logger="app.rpc.climit"`, whose `id` carries the key). CLIMIT rejections have their own counters (`penpot_rpc_climit_rejected_total`, by `name` and `reason`), and their `hint="rejected"` line goes out at WARN only when something stays queued, so counting the lines is not counting the rejections.
-- SQL — **Postgres**, uid `y8ea9tfMk`. Panels need `rawQuery: true`, `editorMode: "code"` and `format: "table"`. The `task` table is the only truth about the queue depth. It is a **read-only replica**: it rejects any write, `CREATE TEMP TABLE` included, so queries through it must be plain `SELECT`s. It also cuts long-running queries (the MCP path has a short timeout), so bound them (e.g. a list of ids); an empty answer there is not proof that there is no data.
+- Metrics — **Prometheus**, uid `xkRmyFBGk`. PromQL. Metric families: `penpot_*` (RPC, task timings, CLIMIT, HTTP, websocket, storage, and the application log events of the backend), `hikaricp_*`, `jvm_*` and `process_*` on the backend (`job="app"`); `pg_*` from the Postgres exporter (`job="postgres"`); `redis_*` from the Redis exporter (`job="redis"`); `node_*` and `container_*` from node-exporter and cAdvisor. Backend histograms are in **milliseconds**. Tails (ms): requests and waits to 60000, background tasks to 300000, Redis eval to 10000; the sub-7500 buckets are identical everywhere, so old quantiles stay comparable.
+  - The Postgres exporter scrapes **the primary as well as the read-only replica**, so the primary's `pg_stat_activity`, `pg_settings_*` and `pg_stat_bgwriter_*` are in Prometheus even though the SQL datasource cannot reach them (the replica's views describe the replica, not the primary).
+  - Per-database data from the exporter is **aggregate only**: `pg_settings_*` (the whole server configuration, so autovacuum, checkpoint and memory parameters are readable from Prometheus), `pg_stat_database_*`, `pg_stat_bgwriter_*`, `pg_stat_activity_*`, and replication/lock counts. It publishes **no per-table series** (`pg_stat_user_tables_*` resolves empty), so dead tuples, per-table reloptions and relation sizes are SQL-only questions.
+  - It publishes **no `pg_stat_progress_*` collector** and `pg_stat_activity_count` carries no `backend_type` label, so a session is only identifiable by its remaining labels: autovacuum workers are the bucket with `usename` and `application_name` empty plus `state="active"`, and `pg_wait_events_connections{wait_event="AutoVacuumMain"}` is the autovacuum launcher alive (absent on the replica).
+  - A metric the exporter does not publish itself has to be added as a custom query block in its YAML queries file on the exporter host (that is where `pg_wait_events_connections` comes from), and in this build the file is read at startup: restart the exporter after editing. There is no per-metric flag to turn on.
+- Logs — **Loki**, uid `h3CGKhEMk`. LogQL, and filter by the structured metadata `logger`/`level` (`{job="penpot"} | logger="..." | level="ERROR"`). Two things to remember when counting from it: the CLIMIT rejections have their own counters (`penpot_rpc_climit_rejected_total`, by `name` and `reason`) and their `hint="rejected"` line goes out at WARN only when something is left queued behind the call (a lone caller that times out is written at TRACE), so counting lines is not counting rejections -- the same goes for `hint="enqueued"` and `hint="acquired"`; and the only signal that exists nowhere else is the profile behind a queue (`logger="app.rpc.climit"`, whose `id` carries the key), chased there on demand.
+  - The Postgres server also ships its own log (`job="postgresql"`, primary only). Those streams carry just `filename`, `instance`, `job` and `level` — no `environment`/`hostname` — so selectors need `instance=~"$host.$env.*"`. `log_autovacuum_min_duration` is low enough that every autovacuum leaves an `automatic vacuum of table "db.schema.table"` line with pages, tuples, index scans and `elapsed`: a per-table history of **finished** vacuums (retrospective, not a view of what runs now). A day of vacuums is only a couple of hundred lines, so pulling them raw over `query_range` and parsing the table, the pages scanned, `tuples: N removed` and the `removable cutoff` XID age in the client works well — less fighting than making LogQL's `pattern`/`regexp` chew a multi-line message.
+- SQL — **Postgres**, uid `y8ea9tfMk`. Panels need `rawQuery: true`, `editorMode: "code"` and `format: "table"`. The `task` table is the only truth about the queue depth. It is a **read-only replica**: it rejects any write, `CREATE TEMP TABLE` included, so queries through it must be plain `SELECT`s. Catalog data (`pg_class`, `pg_partition_tree`, `pg_stats`, and `reloptions`) describes the primary faithfully, because catalogs replicate; the statistics views (`pg_stat_user_tables`, `pg_stat_all_tables`) read as zero there, since statistics are per-instance and are not replicated. It also cuts long-running queries (the MCP path has a short timeout), so bound them (e.g. a list of ids); an empty answer there is not proof that there is no data.
 - **Postgres (telemetry)**, uid `9YAUuM_Gz`.
 - `$__rate_interval` and `$__interval` are substituted by Grafana; use them instead of fixed windows in graphs.
 
 ## Main dashboard
 
-- `Production status`, uid `penpot-production-status`. Sections: general state and deploy, RPC and CLIMIT, PostgreSQL and pool, Redis/msgbus, tasks, JVM/process. Variables `$env` and `$host` (custom), no auto refresh.
+- `Production status`, uid `penpot-production-status`. Sections: general state and deploy, `RPC y CLIMIT`, PostgreSQL and pool, Redis/msgbus, tasks, JVM/process. Variables `$env` and `$host` (custom), no auto refresh. Every section opens collapsed except `RPC y CLIMIT`, which is the one read on a normal day.
+  - `$host` is a custom multi-variable: its `options` list and its `allValue` regex must both enumerate every host. A host added to `query` but missing from `options`/`allValue` is silently dropped when `All` is selected, so panels that filter with `instance=~"$host..."` stop showing it (panel 104 is the visible symptom). Update all three places together when adding a host.
 - Per-host investigation dashboards are separate, and the alert rules deeplink into them.
+- The PostgreSQL section ends with a block of vacuum panels: two stats (autovacuum workers active on the primary, and the launcher alive), a Loki timeseries of finished vacuums per interval and table, and then the `Vacuum TXID age` table, which is the last panel of the section. The worker stat is an approximation — it counts the `pg_stat_activity` bucket described above, because the exporter publishes no `backend_type` — and the Loki panel is retrospective by construction. A per-table, in-flight view (table, phase, progress) needs a `pg_stat_progress_vacuum` collector added to the exporter itself.
 
 ## Conventions
 
@@ -26,6 +33,20 @@ Keep this memory free of internal names: no host names, no deployed versions, no
 - Cron "did it run" comes from the task metrics, not from the cron registry table (it keeps rows from old versions).
 - Dashboard thresholds must match the alert rule thresholds.
 
+## Editing dashboards through the API
+
+- Saving through the API normalizes the dashboard: submit the native form, rows with `panels: []` and their panels top-level with contiguous `y`. A row that has to start collapsed (or open) is sent holding its children in `panels` with `collapsed` set, and the top-level `y` are recomputed by hand: one unit per collapsed row, with the expanded block kept contiguous. Grafana accepts it and returns the same shape.
+- A `remove` operation on a panel leaves its `gridPos` hole behind: Grafana does not refill the layout (the UI behaves the same), so a neighbouring panel has to be moved by hand when the gap matters.
+- Check `list_dashboard_versions` before writing and prefer `operations` patches: the user edits dashboards by hand and a full-JSON overwrite silently reverts that work.
+
+## Reading the API by hand
+
+- The MCP authenticates with a service account token (`GRAFANA_SERVICE_ACCOUNT_TOKEN`, set in the environment of the MCP server), and a service account has no user behind it: a 403 is never the token being bad. On `/api/user` and `/api/user/orgs` it is normal; on a single dashboard (`Access denied to this dashboard`, which is not a 404) it means that identity cannot see it, a permission on that dashboard or its folder. Restarting the MCP or rotating the token changes nothing, and `X-Grafana-Org-Id` does not help when the identity belongs to another organization.
+- Listing is reliable, filtering is not: a dashboard search with a text query answers empty in this instance (through the tool and through `/api/search?query=...` alike), and `list_prometheus_metric_names` also lists names that survive only in the retained series of a metric that stopped being published. Use the listing without a query, and `count(<metric>)` to tell whether a metric is alive today.
+- The MCP Prometheus tool can answer empty for every query, `up` included, while the datasource itself is fine: run PromQL by hand with `grafana_api_request` against `/api/datasources/uid/<uid>/resources/api/v1/query?query=...`.
+- `curl` and `python` from the shell cannot reach the Grafana host of the deployment: Cloudflare blocks them (`error code: 1010`, which looks like a 403 and has nothing to do with Grafana). Probe the API through the MCP (`grafana_api_request`), which is what the deployment lets through.
+- A Loki query by hand goes through `GET /api/datasources/proxy/uid/<loki-uid>/loki/api/v1/query_range?query=<urlencoded>&start=<ns>&end=<ns>&step=<seconds>` (the range comes back as `resultType: matrix`, one series per group). For Loki the `/api/datasources/uid/<uid>/resources/...` route answers 500, and `/api/ds/query` is not reachable from `grafana_api_request` at all (404), even though that is the path the dashboards use.
+
 ## Gotchas
 
 - PromQL literal dots need `\\.`; simpler to use `[.]` or a plain dot.
@@ -34,9 +55,5 @@ Keep this memory free of internal names: no host names, no deployed versions, no
   - Prometheus: `label_replace(<expr>, "host", "$1", "instance", "([^.]+)[.][^.]+.*")` plus `legendFormat: "... {{host}}"`.
   - Loki metric query: `| label_format host="{{ regexReplaceAll "[.][^.]*[.][^.]*[.][^.]*$" .instance "" }}"` plus `legendFormat: "{{host}}"`. Quotes inside the template must be escaped as `\"`, or LogQL closes the string early (`not a valid duration string`).
 - Tables from a Prometheus raw selector carry a `__name__` column; hide it with `organize`.
-- Saving through the API normalizes the dashboard (rows may be flattened, `y` recomputed): submit the native form (rows with `panels: []`, panels top-level, contiguous `y`).
 - Panel images cannot be rendered (no image renderer installed) and there is no browser here: verify a dashboard by running its queries with the variables substituted.
 - Signals that exist only as logs are counted with Loki, and the count is then cross-checked against the nginx access log for the same window and the same status code (`job="frontend"`, e.g. a `429`): two independent sources must agree before a number is trusted.
-- The app writes its own logs at WARN only when something is left queued behind the call and at TRACE otherwise, so production Loki holds just the under-pressure lines: a count of `hint="enqueued"` or `hint="acquired"` is not the number of calls, and neither is a count of `hint="rejected"` lines, because a rejection that leaves the queue empty (a lone caller that times out) is written at TRACE. The rejection counters are the real number.
-- The MCP Prometheus tool can answer empty for every query, `up` included, while the datasource itself is fine: an empty result from it is not proof that a panel has no data, and blaming the datasource for it is wrong. Run PromQL by hand with `grafana_api_request` against `/api/datasources/uid/<uid>/resources/api/v1/query?query=...` (or `POST /api/ds/query`, the path dashboards use), and check when a series is expected but the tool says nothing.
-- **Check `list_dashboard_versions` before writing.** The user edits dashboards by hand and a full-JSON overwrite silently reverts that work; prefer `operations` patches per panel.

.serena/memories/render-wasm/core.md

@@ -25,6 +25,7 @@
 - The lock covers setup, Cargo build, and artifact copy. The watch process releases it while waiting for changes. The operating system releases it on success, error, or signal; the file remains.
 - The lock only coordinates repository scripts. Manual Cargo commands do not take it.
 - Set `RENDER_WASM_LOCK_FILE` to the same path only when separate checkouts intentionally share a `CARGO_TARGET_DIR`.
+- Build output is gitignored and stays untracked everywhere it lands: `frontend/resources/public/js/render-wasm.{js,wasm}` and `frontend/src/app/render_wasm/api/shared.js` from `./build`, plus `exporter/resources/wasm/render-wasm.{js,wasm}` and `exporter/src/app/wasm/shared.js`. Never commit generated JS/WASM; rebuild instead.
 
 ## Commands
 

.serena/memories/scripts/gh.md

@@ -119,6 +119,7 @@ python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
 
 ## Key principles
 
+- `issues --compare` collects every `#NNNN` token in the file regardless of its link target, so a same-numbered reference to another tracker (e.g. `[Taiga #12312](...)`) or an inline prose number falsely marks that GitHub issue as already listed; verify against `changelog.py cross-ref` when a number collides.
 - Changelog-specific checks (merge-status verification, milestone-vs-changelog cross-reference, anomaly report) live in `scripts/changelog.py`, which calls this script via subprocess — see `mem:scripts/changelog`.
 
 - All output is JSON — pipe into `jq` or other tools for further processing.

.serena/memories/workflow/creating-commits.md

@@ -2,6 +2,12 @@
 
 Commit only on explicit request. Before commit: `git status`; exclude unrelated user changes.
 
+**Never stage a plan file.** A plan is session bookkeeping, not part of the
+change: the commit carries the code and the tests. The `implement-plan` flow
+still closes the plan (`Status: done` plus one `Review Log` line with the issue
+URL); that is not a file to ship, and its absence from the commit is correct,
+not a missing piece.
+
 Do not guess or hallucinate git author information (Name or Email). Never include the
 `--author` flag in git commands unless specifically instructed by the user for a unique
 case; assume the local environment is already configured. Allow git commit to
@@ -11,12 +17,11 @@ automatically pull the identity from the local git config `user.name` and `user.
 ## Message Format
 
 ```
-:emoji: Subject line (imperative, capitalized, no period, <=70 chars)
+:emoji: Subject line (imperative, capitalized, no period, <=80 chars)
 
 Body explaining what changed and why.
-Wrap lines at 76 characters — git log adds a
-four-space indent, so 76 + 4 fits an 80-column
-terminal. Keep each line concise.
+Wrap lines at 80 characters.
+Keep each line concise.
 
 AI-assisted-by: model-name
 ```
@@ -26,19 +31,22 @@ AI-assisted-by: model-name
 These rules are not advisory. Do not commit until every one holds. A commit
 that breaks them is wrong, even if the code is right.
 
-- **Body lines MUST wrap at 76 characters or fewer.** Measure every line; do
-  not eyeball it. This is the rule most often skipped. Rationale: `git log`
-  indents the body four spaces, so 76 + 4 fits an 80-column terminal.
-- **Subject MUST be ≤70 chars**, imperative, capitalized, no trailing period.
+- **Body lines MUST wrap at 80 characters or fewer.** Measure every line; do
+  not eyeball it. This is the rule most often skipped.
+- **Subject MUST be ≤80 chars**, imperative, capitalized, no trailing period.
+  Merge/Revert/Reapply commits skip the length check: git prepends those
+  prefixes to an existing subject.
 - **MUST be a blank line** between subject and body.
 - **MUST run `scripts/check-commit` and get exit code 0 before finishing.**
   It mechanically validates the rules above; a failing run is a blocker.
   - It checks `HEAD` by default: `./scripts/check-commit`
   - For another commit: `./scripts/check-commit -c <ref>`
-- **NEVER** hand-wave the body as "one long line". If a line exceeds 76,
+- **NEVER** hand-wave the body as "one long line". If a line exceeds 80,
   break it at a space.
+- **NEVER add a `Co-authored-by:` trailer.** It is forbidden; `scripts/check-commit` fails the commit when any line matches it (case-insensitive).
+- CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build.
 - Exceptions inside the body (do not wrap these): `Signed-off-by:`,
-  `Co-authored-by:`, `AI-assisted-by:` trailers, and lines carrying a URL.
+  `AI-assisted-by:` trailers, and lines carrying a URL.
 
 **AI-assisted-by trailer rules:**
 - Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash`

.serena/memories/workflow/creating-prs.md

@@ -27,7 +27,7 @@ Always add the PR to the Main project (`--project "Main"`) unless the user expli
 PR titles follow commit title conventions:
 
 ```
-:emoji: Subject line (imperative, capitalized, no period, <=70 chars)
+:emoji: Subject line (imperative, capitalized, no period, <=80 chars)
 ```
 
 See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters.

.serena/memories/workflow/updating-node.md

@@ -0,0 +1,26 @@
+# Updating Node Across the Toolchain
+
+No single source of truth: nothing reads `.nvmrc` at build time, and the version is repeated in a different shape per file. Six sites move together, and the image ones carry a digest beside the tag.
+
+## Pin sites
+
+- `.nvmrc` → `v<X.Y.Z>`, with the leading `v`.
+- `docker/devenv/Dockerfile`, `setup-node` stage → `ENV NODE_VERSION=v<X.Y.Z>`; downloads `https://nodejs.org/dist/${NODE_VERSION}/node-${NODE_VERSION}-linux-{arm64,x64}.tar.gz`, one tarball per arch, and verifies each against a `NODE_SHA256` right below its `BINARY_URL`. Both hashes come from `SHASUMS256.txt` in that same dist directory, so a bump must recompute the two.
+- `docker/images/Dockerfile.mcp` → two stages: `FROM dhi.io/node:<X.Y.Z>-debian13-dev AS build` and `FROM dhi.io/node:<X.Y.Z>-debian13 AS image`. The release stage carries no `-dev`.
+- `docker/images/Dockerfile.media-processor` and `docker/images/Dockerfile.exporter` → single `FROM dhi.io/node:<X.Y.Z>-debian13-dev`.
+- `.github/workflows/plugins-deploy-api-doc.yml` and `.github/workflows/plugins-deploy-styles-doc.yml` → `runtime: node@<X.Y.Z>` in the `pnpm/setup` step. Its `working-directory: plugins` makes pnpm's own version come from the manifest, so this pin is the Node one; keep it equal to `.nvmrc` minus the `v`.
+- DHI tags take the bare version; only the devenv `NODE_VERSION` carries the `v`.
+- Every `dhi.io` base image in the repo (the node ones above, plus `debian-base` and `nginx`) is pinned by digest: `FROM dhi.io/<image>:<tag>@sha256:<digest>`. Tag and digest are a pair; Renovate's `Update DHI base images` commits rewrite both in one go, so a hand-made tag bump that leaves the digest behind keeps the old image.
+
+## Gotchas
+
+- DHI tags are exact-version and lag the upstream release, and dhi.io answers 401 to anonymous requests, so a tag that does not exist yet can only be caught by a real build. `build-docker.yml` and `build-docker-devenv.yml` log into dhi.io, so the failure surfaces in CI, not locally.
+- Node 24.x is the Krypton LTS line: take the newest patch of that major unless a major bump is the point.
+- Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites.
+- A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift.
+- Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import.
+
+## Verification
+
+- `rg '<old version>'` finds nothing outside `CHANGES.md` and untracked scratch files (a repo-wide grep without path filters hits multi-megabyte ClojureScript analyzer caches: always scope it). Grep `dhi\.io/node` as well: a line whose tag moved but whose digest did not still pulls the old image.
+- After a devenv rebuild, `node --version` matches `.nvmrc` and `pnpm --version` matches `PNPM_VERSION` in `docker/devenv/Dockerfile`.

.serena/memories/workflow/updating-pnpm.md

@@ -23,7 +23,7 @@ file (never pipe tool output through filters).
 - The store survives `node_modules` cleans. It is content-addressed and
   integrity-verified, so it cannot go stale; staleness lives in
   node_modules. Only `scripts/clean-node-modules --store` removes it.
-- Every first-party `package.json` (35 files: workspace roots plus
+- Every first-party `package.json` (34 files: workspace roots plus
   members) must carry a `packageManager` field with the identical
   `pnpm@<version>+sha512.<hash>` value. Do not let them drift.
   `external/` (vendored trees with their own lifecycles), `.opencode/`,
@@ -33,7 +33,9 @@ file (never pipe tool output through filters).
   images). Nothing may call corepack: it is gone from Node 25+. pnpm
   auto-downloads the `packageManager` version on mismatch
   (`pmOnFail: download`, the default), so drift self-heals; aligned pins
-  just skip the download.
+  just skip the download. Inside the devenv that download lands in the
+  penpot user's `package-manager-store`, which `files/entrypoint.sh` chowns
+  to penpot on every start (see `mem:devenv/core`).
 - The pnpm version pin lives in three places that move together:
   `PNPM_VERSION` (+ arch SHAs) in `docker/devenv/Dockerfile` and in
   `docker/images/Dockerfile.{media-processor,exporter,mcp}`, and the
@@ -49,14 +51,23 @@ file (never pipe tool output through filters).
    `pnpm view pnpm dist-tags --json` for `latest-12` (latest 12.x). The
    tag moves over time; always re-check.
 2. Bump `PNPM_VERSION` (+ arch SHAs) in `docker/devenv/Dockerfile` and in
-   `docker/images/Dockerfile.{media-processor,exporter,mcp}`.
+   `docker/images/Dockerfile.{media-processor,exporter,mcp}`. Those SHAs are
+   of the **GitHub release tarballs**, one per arch: download
+   `pnpm-linux-{arm64,x64}.tar.gz` from the `v<version>` release and
+   `sha256sum` each. The npm `dist.integrity` that the script stamps in step
+   3 is a different hash of a different artifact — never reuse it for the
+   Dockerfiles.
 3. From the repo root, on a host whose system pnpm is the target version
    (rebuilt devenv), run `scripts/sync-pnpm-version`. It resolves the
    integrity hash via `pnpm view` + node (no npm, no corepack, no
-   python3) and stamps the identical field into all 35 files, replacing
+   python3) and stamps the identical field into all 34 files, replacing
    only the value line (each file keeps its own indent) and inserting
    the key after `"name"`/`"version"` where missing. Explicit version
-   instead: `scripts/sync-pnpm-version <version>`.
+   instead: `scripts/sync-pnpm-version <version>`. The explicit form works
+   from a host whose system pnpm is older than the target (the registry
+   lookup resolves the hash, and the first later `pnpm` call auto-downloads
+   the target into `package-manager-store`), so a rebuild is only needed for
+   the in-container `pnpm --version` to match.
 4. Run `scripts/sync-pnpm-version --install` (plain `pnpm install` in
    each of the 11 workspace roots) to refresh lockfile metadata, then
    fix any failing workspace (see gotchas) and re-run that directory.
@@ -76,10 +87,15 @@ file (never pipe tool output through filters).
   gone on purpose: pnpm picks the nearest `pnpm-workspace.yaml` walking up,
   so a nested one silently forks install and lockfile behavior. Do not
   reintroduce it.
+- `media-processor/target/package.json` is a generated, gitignored copy that
+  the script also stamps, so stamping and `--check` report 35 files while
+  only 34 are tracked. A stale copy there makes `--check` fail with nothing
+  wrong in the repo: delete `media-processor/target` and re-run.
 - Expect metadata-only lockfile diffs when only the pnpm version moves:
   the pnpm self-reference entries, plus a new `packageManagerDependencies`
   section in lockfiles last written by older pnpm. Large diffs mean
   re-resolution; inspect them before accepting.
+- A backport merge of main into staging can bring a newer `PNPM_VERSION` in `docker/` while every `packageManager` field stays behind, since main receives partial `.github/`+`docker/` backports from develop. Resolve those conflicts with main's version and run `scripts/sync-pnpm-version <version>` in the same merge.
 
 ## Verification
 

.serena/project.yml

@@ -1,42 +1,54 @@
 # the name by which the project can be referenced within Serena/when chatting with the LLM.
 project_name: "penpot"
 
-# list of languages for which language servers are started (LSP backend only); choose from:
-#   ada                 al                  angular             ansible             bash
-#   bsl                 clojure             cpp                 cpp_ccls            crystal
-#   csharp              csharp_omnisharp    cue                 dart                elixir
-#   elm                 erlang              fortran             fsharp              gdscript
-#   go                  groovy              haskell             haxe                hlsl
-#   html                java                json                julia               kotlin
-#   latex               lean4               lua                 luau                markdown
-#   matlab              msl                 nix                 ocaml               pascal
-#   perl                php                 php_phpactor        php_phpantom        powershell
-#   python              python_jedi         python_pyrefly      python_ty           r
-#   rego                ruby                ruby_solargraph     rust                scala
-#   scss                solidity            svelte              swift               systemverilog
-#   terraform           toml                typescript          typescript_vts      vue
-#   yaml                zig
+# list of language servers to start when using the LSP backend; choose from:
+#   ada                    al                     angular                ansible                astro
+#   bash                   bsl                    clojure                cpp                    cpp_ccls
+#   crystal                csharp                 csharp_omnisharp       cue                    dart
+#   deno                   elixir                 elm                    erlang                 fortran
+#   fsharp                 gdscript               gleam                  go                     groovy
+#   haskell                haxe                   hlsl                   html                   java
+#   json                   julia                  julia_fatou            kotlin                 latex
+#   lean4                  lua                    lua_emmylua            luau                   markdown
+#   matlab                 msl                    nextflow               nix                    ocaml
+#   pascal                 perl                   php                    php_devsense           php_phpactor
+#   php_phpantom           powershell             python                 python_basedpyright    python_jedi
+#   python_pyrefly         python_ty              qml                    r                      rego
+#   ruby                   ruby_solargraph        rust                   scala                  scss
+#   solidity               svelte                 swift                  systemverilog          terraform
+#   toml                   typescript             typescript_vts         vue                    wolfram
+#   yaml                   zig
 #   (This list may be outdated; generated with scripts/print_language_list.py;
-#   For the current list, see values of Language enum here:
+#   For the current list, see values of the LanguageServerId enum here:
 #   https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
-# For some languages, there are alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
+# For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
 # Note:
 #   - For C, use cpp
 #   - For JavaScript, use typescript
 #   - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
+#   - For Astro projects, use astro (subsumes typescript/javascript for .astro projects; requires npm)
 #   - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
+#   - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
 #   - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
 #   - For Free Pascal/Lazarus, use pascal
+#   - External Python adapters may add further registered IDs; install the adapter package first
+#     and then use its ID here, for example: example
 # Special requirements:
-#   Some languages require additional setup/installations.
+#   Some language servers require additional setup/installations.
 #   See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
-# When using multiple languages, the first language server that supports a given file will be used for that file.
-# The first language is the default language and the respective language server will be used as a fallback.
+# When using multiple language servers, the first language server that supports a given file will be used for that file.
+# The first language server is the default language and the respective language server will be used as a fallback.
 # Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
+language_servers:
+  - clojure
+  - typescript
+  - rust
+
+# same list for Serena 1.x, which only reads this key; keep both lists equal
 languages:
-- clojure
-- typescript
-- rust
+  - clojure
+  - typescript
+  - rust
 
 # the encoding used by text files in the project
 # For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
@@ -177,3 +189,18 @@ activation_command:
 # maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
 # must be a positive number.
 activation_command_timeout: 180.0
+
+# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.get_diagnostics_for_symbol") to include in the REPL
+# that would otherwise be disabled (particularly optional methods, which are disabled by default).
+# This extends the existing inclusions (e.g. from the global configuration).
+included_apis: []
+
+# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.find_symbol") to exclude from the REPL.
+# This extends the existing exclusions (e.g. from the global configuration).
+excluded_apis: []
+
+# The interface through which the agent (LLM) accesses Serena's functionality (overrides the global setting).
+# Valid values: tools, REPL (see the global configuration for details); leave empty to use the global setting.
+# Note: the interface is fixed at startup. If a project is activated post-init, its setting is not applied.
+agent_interface: REPL
+

AGENTS.md

@@ -17,7 +17,8 @@
 - **Read `AGENTS.local.md` after this file whenever it exists.** It carries
   the developer's steering for this project and overrides this file on prose style and
   workflow, but not on project policy, where instructions here prevail.
-- **Commit message body lines MUST wrap at ≤76 chars** (subject ≤70 chars) and
+- **Commit message body lines MUST wrap at ≤80 chars** (subject ≤80 chars,
+  except `Merge`/`Revert`/`Reapply` which skip the subject length check) and
   the commit MUST pass `./scripts/check-commit` with exit code 0 before you
   consider it done. This is mechanically checked — do not eyeball it.
 - **Read the workflow memory BEFORE the corresponding action**:
@@ -31,7 +32,7 @@
 
 Do this **before planning, before coding, before touching any file**:
 
-1. Read `critical-info` (use `serena_read_memory critical-info` or read `.serena/memories/critical-info.md`).
+1. Read `critical-info` through Serena (see "Accessing memories" below) or read `.serena/memories/critical-info.md`.
    It describes the project structure and tells you which modules exist.
 2. From `critical-info`, identify which modules your task affects.
 3. Read each affected module's **core memory** — the name is `<module>/core`
@@ -102,9 +103,12 @@ When working on a task:
 
 ## Accessing memories
 
-- **If `serena_read_memory` / `serena_list_memories` tools are available**: use them.
-  `serena_read_memory` takes a memory name (e.g. `critical-info`, `frontend/core`).
-- **If tools are NOT available**: read the filesystem directly.
+- **If Serena's `serena_repl` tool is available** (Serena 2.x): call `s.mem.read_memory(name)`
+  and `s.mem.list_memories()` through it.
+- **If Serena's `read_memory` / `list_memories` tools are available** (Serena 1.x; some clients
+  show them as `serena_read_memory` / `serena_list_memories`): use them.
+- Either way, `read_memory` takes a memory name (e.g. `critical-info`, `frontend/core`).
+- **If Serena is NOT available**: read the filesystem directly.
   Memory name `mem:foo/bar` maps to file `.serena/memories/foo/bar.md`.
 
 ## Cross-reference convention

CHANGES.md

@@ -129,11 +129,15 @@
 - Fix file restore leaving its storage objects marked for deletion [#12278](https://github.com/penpot/penpot/issues/12278) (PR: [#12269](https://github.com/penpot/penpot/pull/12269))
 - Fix backend workers staying parked forever when nested calls take the same concurrency limits [#12279](https://github.com/penpot/penpot/issues/12279) (PR: [#12269](https://github.com/penpot/penpot/pull/12269))
 - Fix one profile's activity blocking the audit events of every other profile [#12280](https://github.com/penpot/penpot/issues/12280) (PR: [#12269](https://github.com/penpot/penpot/pull/12269))
+- Fix storage garbage collection blocking database vacuum during long deletions [#12396](https://github.com/penpot/penpot/issues/12396) (PR: [#12397](https://github.com/penpot/penpot/pull/12397))
+- Fix saving a version taking seconds and the version quota never counting saved versions [#12404](https://github.com/penpot/penpot/issues/12404) (PR: [#12405](https://github.com/penpot/penpot/pull/12405))
+- Fix file_change accumulating revisions that the objects GC never collects [#12418](https://github.com/penpot/penpot/issues/12418) (PR: [#12419](https://github.com/penpot/penpot/pull/12419))
 
 ### :sparkles: New features & Enhancements
 
 - Add a graceful instance drain through the /readyz endpoint [#12281](https://github.com/penpot/penpot/issues/12281) (PR: [#12269](https://github.com/penpot/penpot/pull/12269))
 - Add an accumulated queue gauge to the CLIMIT metrics [#12282](https://github.com/penpot/penpot/issues/12282) (PR: [#12269](https://github.com/penpot/penpot/pull/12269))
+- Expose the backend log events as a metric by logger and level [#12312](https://github.com/penpot/penpot/issues/12312) (PR: [#12313](https://github.com/penpot/penpot/pull/12313))
 
 ## 2.18.3
 

CONTRIBUTING.md

@@ -67,6 +67,11 @@ If AI-assisted content is used, it must be carefully reviewed and verified by a
 Report bugs via [GitHub Issues](https://github.com/penpot/penpot/issues).
 Before filing, search existing issues to avoid duplicates.
 
+> **Note:** Every new issue starts under the `needs triage` label until the
+> product team reviews it. Issues under `needs triage` or `internal` are not
+> open to contributions — do not start work on them. Wait for a maintainer to
+> confirm it in the issue itself.
+
 Include the following when possible:
 
 1. Steps to reproduce the error.
@@ -119,7 +124,7 @@ commit history — what matters is the [PR title](#title).
 Pull request titles **must** follow the same convention as commit subjects:
 
 ```
-:emoji: Subject line (imperative, capitalized, no period, <=70 chars)
+:emoji: Subject line (imperative, capitalized, no period, <=80 chars)
 ```
 
 Read [Creating Commits](./.serena/memories/workflow/creating-commits.md)
@@ -164,10 +169,15 @@ To save time on both sides, please avoid submitting PRs that:
 - Submit AI-generated code without human review.
 - Skip local syntax and formatting checks before submitting.
 - Skip the [discussion step](#workflow) for non-bug-fix changes.
+- Open a PR for an issue under `needs triage` or `internal`. We may close
+  these PRs without review, without an explanation and without a reply.
 
 ### Good first issues
 
 We use the `good first issue` label to mark issues appropriate for newcomers.
+An issue is open to contributions only after a maintainer says so in the issue
+itself. Issues under `needs triage` or `internal` stay closed, however simple
+they look.
 
 ## Commit Guidelines
 
@@ -210,11 +220,17 @@ Commit messages must follow this format:
 - Capitalize the first letter of the subject
 - Add clear and concise description on the body
 - Do not end the subject with a period
-- Keep the subject to **70 characters** or fewer
-- **Wrap body lines at 76 characters or fewer** (trailers and URLs excepted)
+- Keep the subject to **80 characters** or fewer
+  (`Merge`/`Revert`/`Reapply` commits skip this limit, since git prepends
+  those prefixes to an existing subject)
+- **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted)
 - Separate the subject from the body with a **blank line**
+- Never add a `Co-authored-by` trailer — it is forbidden and fails
+  `./scripts/check-commit`
 
 You can check a commit against these rules with `./scripts/check-commit`.
+CI runs the same script on every new commit in
+`.github/workflows/commit-checker.yml`, so violations fail the build.
 
 ### Examples
 

backend/package.json

@@ -4,7 +4,7 @@
   "license": "MPL-2.0",
   "author": "Kaleidos INC Sucursal en España SL",
   "private": true,
-  "packageManager": "pnpm@12.8.1+sha512.f64ba907507f5ceafe06c8d38e6052d0179444580ec1279ddd5bfc11cb48aa8a2644b66598e07e761da84872a9fc57d5f902b87fa49d024198d558612aabbe45",
+  "packageManager": "pnpm@12.15.0+sha512.b3951d6d6ff4a9f13dc7cf56cd28d2b8eb37dbcc9c68e52e1e1191873ea9f82a017287ab50baadc5ccc494f036c67c0e47eb13608c2fa36f47711b61ea128e61",
   "repository": {
     "type": "git",
     "url": "https://github.com/penpot/penpot"

backend/pnpm-lock.yaml

@@ -7,153 +7,153 @@ importers:
     configDependencies: {}
     packageManagerDependencies:
       pnpm:
-        specifier: 12.8.1
-        version: 12.8.1
+        specifier: 12.15.0
+        version: 12.15.0
 
 packages:
 
-  '@pnpm/exe.android-arm64@12.8.1':
-    resolution: {integrity: sha512-siDxcidfz5eM6L2tJfik+i2bbUEw8UC7MZ3Yl2HYdRmqbdhHGAOQACtOOu6QVW4Jul4/w5wVKm+wM8ACtgh1Pg==}
+  '@pnpm/exe.android-arm64@12.15.0':
+    resolution: {integrity: sha512-eVCqmscY70LGvgRzu3mi8IBWBFGix+a6mNZoWmXxESsBdeiKEF9xQSuaeel10yn+zI7TXy2jlI63vFLzETk+aQ==}
     cpu: [arm64]
     os: [android]
 
-  '@pnpm/exe.android-x64@12.8.1':
-    resolution: {integrity: sha512-0F/uqPRc3aN4vLHsa0f+6fTtmsigFQ9/yxEU43fionF//pNHEd5e97b3w/nKIgqYMCUQCJphmBRMA9Q8/wlo9Q==}
+  '@pnpm/exe.android-x64@12.15.0':
+    resolution: {integrity: sha512-Gx/WbkFcdUFUg8g8AShezcS9EiQxfYvdAm2hfKLO8sfJmJvKOCt5fzlgHWDzmGhoclibgW+tUGZ/KF4F4wOTCA==}
     cpu: [x64]
     os: [android]
 
-  '@pnpm/exe.darwin-arm64@12.8.1':
-    resolution: {integrity: sha512-/rwavvMQJsl2RIxOHBnDF+4Gk+fhQFAY4M3PQZoKskJOEnaJcygG9p1xby6wcQcbS4d9dIubv09CQhufTgfmpw==}
+  '@pnpm/exe.darwin-arm64@12.15.0':
+    resolution: {integrity: sha512-N0ZykiDWCvqSBJvt71j7cSoBR993FqvYZFtmpCtqvW3pguiI+NAgraLPA5OTwu5mV+/d8hYpVcPktJA3xFDLtA==}
     cpu: [arm64]
     os: [darwin]
 
-  '@pnpm/exe.darwin-x64@12.8.1':
-    resolution: {integrity: sha512-htYt2gN7zqJKLhC99Ft6EUiO7myjZAZXfmGCTJUL1v+o7vE9hZdE1+fAkk4Oj1ZHNRti6b2woOxEkRkAusi8OQ==}
+  '@pnpm/exe.darwin-x64@12.15.0':
+    resolution: {integrity: sha512-7Ntr2HlBUxzwWOg7NwDxCbRW/3vXuqKO+hcXm+w3SBi9rrHY53li1wLqWVT7Gjxd+lVEHM1QJZo6WJEGFU8C6A==}
     cpu: [x64]
     os: [darwin]
 
-  '@pnpm/exe.freebsd-x64@12.8.1':
-    resolution: {integrity: sha512-PB5BULNgytxvExoOXUUeseS/DcpPENs+cg3iA6xHHeaLT4ctEGKSGXEbuj7USeqvo8JBpAx039hivnbzGqDwnA==}
+  '@pnpm/exe.freebsd-x64@12.15.0':
+    resolution: {integrity: sha512-btOiyL7fL7W4uQGH0aqFai2TNC3IoKcZ7gNHBVbEFZt2dWrnrD0SufL3iGs3fLlmb4OtuU35uf4Zpr9V9Jh9mA==}
     cpu: [x64]
     os: [freebsd]
 
-  '@pnpm/exe.linux-arm64-musl@12.8.1':
-    resolution: {integrity: sha512-kKeyC/ArjrgdciQRBrOL0j9HtOI4gDRgoyP+eJ1CQKyadpKlf/DhDDyspXe07R3B9KagTVLUF2nlt6Dr5quFIg==}
+  '@pnpm/exe.linux-arm64-musl@12.15.0':
+    resolution: {integrity: sha512-pNC38k+O+90FOjbPNSnw0E7apq1PKTmz3fFe1amn+lys/U5kL++uzir8bRiho8ONcyA3d9EU7wdqcY0DKkj+JA==}
     cpu: [arm64]
     os: [linux]
     libc: [musl]
 
-  '@pnpm/exe.linux-arm64@12.8.1':
-    resolution: {integrity: sha512-q4s9a9X2O3N9aeUFooW24orNrxvu20+jyVUFR5RanPCqOKKPBrgs2iywMkBBx1aXkkzdWT54/mfz8Be8sJlWEw==}
+  '@pnpm/exe.linux-arm64@12.15.0':
+    resolution: {integrity: sha512-qML/8aCXslh8td/w4U+HJNqUcZ2+m6EfTbWScIzZRmyPsRFN+xB4LP1U/hymxc1iSJiKtG56HJx2fvUErWFIGw==}
     cpu: [arm64]
     os: [linux]
     libc: [glibc]
 
-  '@pnpm/exe.linux-ppc64@12.8.1':
-    resolution: {integrity: sha512-F7XSjKJthwCh/L6abZiMpAfo2Cm61SZIT3e4dfgiIjESndlWaXCnfLKngSgZ/SEMsdmHT6dud3+kWIB3tw0/xQ==}
+  '@pnpm/exe.linux-ppc64@12.15.0':
+    resolution: {integrity: sha512-tjRDUUl7RCJ6T/6eoNPxOqkwU4Mzi5RtTXmSAxujyz37OTwwsMU7Ps087Zk4gHcUvRSeyF40ZFsew3vtE+nmUA==}
     cpu: [ppc64]
     os: [linux]
     libc: [glibc]
 
-  '@pnpm/exe.linux-riscv64@12.8.1':
-    resolution: {integrity: sha512-z1ecpIK/ZY08Unk69KWWE9867oaKUytGUjrHnBoGFAAhsK+q+0cfvBH+f5fWobJ7FX97leMlLO+gkJolddM45Q==}
+  '@pnpm/exe.linux-riscv64@12.15.0':
+    resolution: {integrity: sha512-bF1uBWEfbwaeryz6lIiaJ8AWD082otZfvzGX5l+6LZRpRwUEpJHUsJrORaYP5/cNMv5ZR2p6TvvcJcQzwtDzmw==}
     cpu: [riscv64]
     os: [linux]
     libc: [glibc]
 
-  '@pnpm/exe.linux-s390x@12.8.1':
-    resolution: {integrity: sha512-KU6tnrBfu4uKXTq6N5qJwSm8FSrs9Wc8b5AUEsHK9bRXwNdsZ1/lRxG1++X29SKRW9OSNuDSjxASv5mAvGo1gA==}
+  '@pnpm/exe.linux-s390x@12.15.0':
+    resolution: {integrity: sha512-zKMhIzsexYMToeLmqV5jPsvV/f9vwsJb2kUH2NFYzalbxImeBrlA4kL7GwF8gD4pagV6jYYcJb8RjbZIdjHptg==}
     cpu: [s390x]
     os: [linux]
     libc: [glibc]
 
-  '@pnpm/exe.linux-x64-musl@12.8.1':
-    resolution: {integrity: sha512-eUE8BWJkDr2tJb/yrk0yVnszlmNqFarVxUlqiZ9BAkct28Nq/5g2H0tvMDmPA7fAQW3STNj+omJPPRVT+4s5sQ==}
+  '@pnpm/exe.linux-x64-musl@12.15.0':
+    resolution: {integrity: sha512-A9J0Jg0Rdx+qQU4qXtwyST5tto9IOu1zi4fDk35nbc6DWahggbcKtFvZPKFwL2qgUXGXOX//hXv6z/sp6emO8A==}
     cpu: [x64]
     os: [linux]
     libc: [musl]
 
-  '@pnpm/exe.linux-x64@12.8.1':
-    resolution: {integrity: sha512-8bDZ0lZlCdi2rvnFul7EYgcC7zKeWSe76y8JV2oXobaS8p9i9d6PSf6LgLtTM0fI7R9Oh4eLCbveXyNDMmvZ+g==}
+  '@pnpm/exe.linux-x64@12.15.0':
+    resolution: {integrity: sha512-yc+QlZR+9+nr29J4fRzhMoapgxTBDndq/+peyoq1XUufnQFnfbXUMFudDIKzHi545linGqmOmXfn0NoufNgcOg==}
     cpu: [x64]
     os: [linux]
     libc: [glibc]
 
-  '@pnpm/exe.win32-arm64@12.8.1':
-    resolution: {integrity: sha512-MsT0dlrRxnCRCHCaosNbhWpEiAFnFuMEu3KuDJDU41BXgTwiINxU8goloUSUDejWZUqKaUqNC/qUTihl9nFtqg==}
+  '@pnpm/exe.win32-arm64@12.15.0':
+    resolution: {integrity: sha512-fPXZxuY1VclkOga9bc53qNCXcegfeoBR8+Ue+AospDwKzuGGSHll0oDxkLfkmpjyRqpe7bH5DyQPYp/xSMaIVA==}
     cpu: [arm64]
     os: [win32]
 
-  '@pnpm/exe.win32-x64@12.8.1':
-    resolution: {integrity: sha512-SWtPe0PsbTTk//gJND10tMdOfCYjrkq5+qV49hzVhY7xz2iJ339Rc+xwASlcvbwTfrvH39YNqpKYibD+CRGLwA==}
+  '@pnpm/exe.win32-x64@12.15.0':
+    resolution: {integrity: sha512-nrd+Tp1We1DGj4DGG5Bea9lZNh9y1PjlLBdEwdInS4D0h0UIDtM7JaGGGDJhNzIwcHASyihWI7HLV+dtG+fhYg==}
     cpu: [x64]
     os: [win32]
 
-  pnpm@12.8.1:
-    resolution: {integrity: sha512-9kupB1B/XOr+BsjTjmBS0BeURFgOwSed3Vv8EctIqoomRLZlmOB+dh2oSHKp/FfV+QK4f6SdAkGY1VhhKqu+RQ==}
-    engines: {node: '>=18.*'}
+  pnpm@12.15.0:
+    resolution: {integrity: sha512-s5UdbW/0qfE9x89WzSjSuOs328ycaOUuHhGRhz6p+CoBcoerULqtxczElPA2xnwOR+sTYIwvo29HcRth6hKOYQ==}
+    engines: {node: '>=18'}
     hasBin: true
 
 snapshots:
 
-  '@pnpm/exe.android-arm64@12.8.1':
+  '@pnpm/exe.android-arm64@12.15.0':
     optional: true
 
-  '@pnpm/exe.android-x64@12.8.1':
+  '@pnpm/exe.android-x64@12.15.0':
     optional: true
 
-  '@pnpm/exe.darwin-arm64@12.8.1':
+  '@pnpm/exe.darwin-arm64@12.15.0':
     optional: true
 
-  '@pnpm/exe.darwin-x64@12.8.1':
+  '@pnpm/exe.darwin-x64@12.15.0':
     optional: true
 
-  '@pnpm/exe.freebsd-x64@12.8.1':
+  '@pnpm/exe.freebsd-x64@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-arm64-musl@12.8.1':
+  '@pnpm/exe.linux-arm64-musl@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-arm64@12.8.1':
+  '@pnpm/exe.linux-arm64@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-ppc64@12.8.1':
+  '@pnpm/exe.linux-ppc64@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-riscv64@12.8.1':
+  '@pnpm/exe.linux-riscv64@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-s390x@12.8.1':
+  '@pnpm/exe.linux-s390x@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-x64-musl@12.8.1':
+  '@pnpm/exe.linux-x64-musl@12.15.0':
     optional: true
 
-  '@pnpm/exe.linux-x64@12.8.1':
+  '@pnpm/exe.linux-x64@12.15.0':
     optional: true
 
-  '@pnpm/exe.win32-arm64@12.8.1':
+  '@pnpm/exe.win32-arm64@12.15.0':
     optional: true
 
-  '@pnpm/exe.win32-x64@12.8.1':
+  '@pnpm/exe.win32-x64@12.15.0':
     optional: true
 
-  pnpm@12.8.1:
+  pnpm@12.15.0:
     optionalDependencies:
-      '@pnpm/exe.android-arm64': 12.8.1
-      '@pnpm/exe.android-x64': 12.8.1
-      '@pnpm/exe.darwin-arm64': 12.8.1
-      '@pnpm/exe.darwin-x64': 12.8.1
-      '@pnpm/exe.freebsd-x64': 12.8.1
-      '@pnpm/exe.linux-arm64': 12.8.1
-      '@pnpm/exe.linux-arm64-musl': 12.8.1
-      '@pnpm/exe.linux-ppc64': 12.8.1
-      '@pnpm/exe.linux-riscv64': 12.8.1
-      '@pnpm/exe.linux-s390x': 12.8.1
-      '@pnpm/exe.linux-x64': 12.8.1
-      '@pnpm/exe.linux-x64-musl': 12.8.1
-      '@pnpm/exe.win32-arm64': 12.8.1
-      '@pnpm/exe.win32-x64': 12.8.1
+      '@pnpm/exe.android-arm64': 12.15.0
+      '@pnpm/exe.android-x64': 12.15.0
+      '@pnpm/exe.darwin-arm64': 12.15.0
+      '@pnpm/exe.darwin-x64': 12.15.0
+      '@pnpm/exe.freebsd-x64': 12.15.0
+      '@pnpm/exe.linux-arm64': 12.15.0
+      '@pnpm/exe.linux-arm64-musl': 12.15.0
+      '@pnpm/exe.linux-ppc64': 12.15.0
+      '@pnpm/exe.linux-riscv64': 12.15.0
+      '@pnpm/exe.linux-s390x': 12.15.0
+      '@pnpm/exe.linux-x64': 12.15.0
+      '@pnpm/exe.linux-x64-musl': 12.15.0
+      '@pnpm/exe.win32-arm64': 12.15.0
+      '@pnpm/exe.win32-x64': 12.15.0
 
 ---
 lockfileVersion: '9.0'

backend/resources/app/templates/debug.tmpl

@@ -326,6 +326,7 @@ Debug Main Page
               <th>PROGRESS</th>
               <th>CMD</th>
               <th>BACKEND</th>
+              <th>REPLICA</th>
               <th>NAME</th>
               <th>CREATED</th>
               <th>ENDED</th>
@@ -339,12 +340,13 @@ Debug Main Page
               <td>{{job.done}} / {{job.total}}</td>
               <td>{{job.cmd}}</td>
               <td>{{job.backend}}</td>
+              <td>{{job.replica}}</td>
               <td>{{job.name}}</td>
               <td>{{job.created-at}}</td>
               <td>{{job.ended-at}}</td>
             </tr>
             {% empty %}
-            <tr><td colspan="8">No export jobs.</td></tr>
+            <tr><td colspan="9">No export jobs.</td></tr>
             {% endfor %}
           </tbody>
         </table>

backend/scripts/_env

@@ -69,7 +69,7 @@ export PENPOT_FLAGS="\
 
 # Uncomment for nexus integration testing
 # export PENPOT_FLAGS="$PENPOT_FLAGS enable-nexus";
-# export PENPOT_AUDIT_LOG_ARCHIVE_URI="http://localhost:6070/api/audit";
+# export PENPOT_NEXUS_URI="http://localhost:6070/";
 
 # Default deletion delay for devenv
 export PENPOT_DELETION_DELAY="24h"

backend/scripts/decode-file-data

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# Decode a file_data blob (app.util.blob) to JSON for inspection with jq.
+#
+# Offline use (no DB):
+#   backend/scripts/decode-file-data --input /tmp/blob.bin --format summary | jq .
+#
+# DB use (backend running, via nREPL):
+#   ./scripts/nrepl-eval.mjs "(require '[app.scripts.decode-file-data :as dfd] :reload)"
+#   ./scripts/nrepl-eval.mjs '(dfd/get-file-json sys/system #uuid "..." :format :summary)'
+
+SCRIPT_DIR=$(dirname "$0")
+
+source "$SCRIPT_DIR/_env"
+
+if [ -f "$SCRIPT_DIR/_env.local" ]; then
+    source "$SCRIPT_DIR/_env.local"
+fi
+
+exec clojure -M:dev -m app.scripts.decode-file-data "$@"

backend/src/app/auth/oidc.clj

@@ -629,7 +629,7 @@
         (l/warn :hint "received incomplete user info object (please set correct scopes)" :info info)
         (ex/raise :type :internal
                   :code :incomplete-user-info
-                  :hint "inconmplete user info"
+                  :hint "incomplete user info"
                   :info info)))
 
     ;; If the provider is OIDC, we can proceed to check
@@ -1037,7 +1037,9 @@
         (when (and session organization-id)
           (let [props (-> (or (:props session) {})
                           (update :sso assoc organization-id exp))]
-            (session/update-session (::session/manager cfg) (assoc session :props props))))
+            (session/update-session (::session/manager cfg) (assoc session :props props))
+            (submit-organization-sso-auth-event
+             cfg request (:profile-id session) organization-id "organization-sso-token-issued")))
         (submit-organization-sso-auth-event
          cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded")
         (redirect-response dest-url))

backend/src/app/binfile/common.clj

@@ -14,6 +14,7 @@
    [app.common.files.changes :as cpc]
    [app.common.files.helpers :as cfh]
    [app.common.files.migrations :as fmg]
+   [app.common.files.tokens :as cfo]
    [app.common.files.validate :as fval]
    [app.common.logging :as l]
    [app.common.media :as cm]
@@ -957,26 +958,57 @@
              colors
              colors))
 
+(defn absorb-unlinked-tokens-source
+  "Make the file its own tokens source when its tokens source is not one
+  of the given library-ids, as happens when the library was deleted. The
+  tokens lib of the source is copied while the library still exists, even
+  if deleted; otherwise the tokens source and status are removed."
+  [cfg data library-ids]
+  (let [source-id (cfo/get-tokens-source data)]
+    (if (and (some? source-id)
+             (not (contains? library-ids source-id)))
+      (if-let [library (get-file cfg source-id
+                                 :include-deleted? true
+                                 :throw-if-not-exists? false)]
+        (cfo/absorb-tokens-lib data (:data library))
+        (dissoc data :tokens-source :tokens-status))
+      data)))
+
 (defn embed-assets
   [cfg data file-id]
-  (let [library-ids (get-libraries cfg [file-id])]
-    (reduce (fn [data library-id]
-              (if-let [library (get-file cfg library-id :include-deleted? true)]
-                (ctf/absorb-assets data (:data library))
-                data))
-            data
-            library-ids)))
+  (let [library-ids (get-libraries cfg [file-id])
+        data        (reduce (fn [data library-id]
+                              (if-let [library (get-file cfg library-id :include-deleted? true)]
+                                (cond-> (ctf/absorb-assets data (:data library))
+                                  (= library-id (cfo/get-tokens-source data))
+                                  (cfo/absorb-tokens-lib (:data library)))
+                                data))
+                            data
+                            library-ids)]
+    (absorb-unlinked-tokens-source cfg data library-ids)))
 
-(def ^:const import-transaction-timeout-ms
-  "Ceiling for binfile import transactions (20 minutes). Interpolated
-  directly into SQL: compile-time constant, never user input."
-  (* 20 60 1000))
+(def transaction-idle-timeout-ms
+  "Idle-in-transaction guard for the long binfile transactions: 30
+  minutes. The import and the export of a file run inside one
+  transaction that can go quiet while the ZIP is decoded and the file
+  data is realized. The longest operation observed in production is
+  under 8 minutes, so this leaves room for bigger files and still
+  aborts a connection that is left idle for half an hour."
+  1800000)
 
-(defn configure-database-timeouts!
+(defn set-transaction-idle-timeout
+  "Set `idle_in_transaction_session_timeout` for the current
+  transaction. Must be called inside a transaction: `SET LOCAL` is
+  scoped to it."
   [cfg]
   (let [conn (db/get-connection cfg)]
     (db/exec-one! conn [(str "SET LOCAL idle_in_transaction_session_timeout = "
-                             import-transaction-timeout-ms)])
+                             transaction-idle-timeout-ms)])))
+
+(defn configure-import-transaction
+  [cfg]
+  (set-transaction-idle-timeout cfg)
+  (let [conn (db/get-connection cfg)]
     (db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])))
 
 (defn process-file

backend/src/app/binfile/jobs.clj

@@ -161,7 +161,7 @@
                                  :input input})]
     (with-progress cfg context
       #(let [result (case (int version)
-                      1 {:file-ids (vec (bf.v1/import-files! cfg))}
+                      1 (bf.v1/import-files! cfg)
                       3 (bf.v3/import-files! cfg)
                       (ex/raise :type :validation
                                 :code :unsupported-version

backend/src/app/binfile/v1.clj

@@ -455,7 +455,7 @@
 (defn- read-import-v1
   [{:keys [::db/conn ::bfc/project-id ::bfc/profile-id ::bfc/input] :as cfg}]
 
-  (bfc/configure-database-timeouts! cfg)
+  (bfc/configure-import-transaction cfg)
 
   (pu/with-open [input (zstd-input-stream input)
                  input (io/data-input-stream input)]
@@ -482,9 +482,9 @@
         (bfm/apply-pending-migrations! cfg)
 
         ;; Knowing that the ids of the created files are in index,
-        ;; just lookup them and return it as a set
+        ;; just lookup them and return them in the same shape as v3.
         (let [files (-> bfc/*state* deref :files)]
-          (into #{} (keep #(get-in @bfc/*state* [:index %])) files))))))
+          {:file-ids (into [] (keep #(get-in @bfc/*state* [:index %])) files)})))))
 
 (defmethod read-import :v1
   [options]

backend/src/app/binfile/v3.clj

@@ -274,6 +274,7 @@
   (let [detach? (= export-type :detach-libraries)
         embed?  (= export-type :merge-libraries)]
     (db/tx-run! cfg (fn [cfg]
+                      (bfc/set-transaction-idle-timeout cfg)
                       (cond-> (bfc/get-file cfg file-id
                                             {:realize? true
                                              :include-deleted? true
@@ -285,6 +286,10 @@
                         embed?
                         (update :data #(bfc/embed-assets cfg % file-id))
 
+                        (not (or detach? embed?))
+                        (update :data #(bfc/absorb-unlinked-tokens-source
+                                        cfg % (bfc/get-libraries cfg [file-id])))
+
                         :always
                         (bfc/clean-file-features))))))
 
@@ -843,7 +848,7 @@
                  {})
          (not-empty))))
 
-(defn- read-file-tokens-lib
+(defn- read-file-tokens-lib*
   [{:keys [::bfc/input ::entries-index] :as cfg} file-id]
   (when-let [{:keys [entry]} (first (get-in entries-index [:tokens (str file-id)]))]
     (tap-progress cfg {:section :tokens-lib :file-id file-id})
@@ -851,6 +856,19 @@
          (decode-tokens-lib)
          (validate-tokens-lib))))
 
+(defn- read-file-tokens-lib
+  "Read the tokens lib of a file once per import. Decoding generates new
+  set ids, so the tokens lib of a library must be the same instance when
+  it is imported and when it resolves the tokens status of a file that
+  uses it as tokens source."
+  [cfg file-id]
+  (let [cache (:tokens-libs @bfc/*state*)]
+    (if (contains? cache file-id)
+      (get cache file-id)
+      (let [tokens-lib (read-file-tokens-lib* cfg file-id)]
+        (vswap! bfc/*state* assoc-in [:tokens-libs file-id] tokens-lib)
+        tokens-lib))))
+
 (defn- read-file-tokens-status
   [{:keys [::bfc/input ::entries-index] :as cfg} file-id tokens-lib]
   (when-let [{:keys [entry]} (first (get-in entries-index [:tokens-status (str file-id)]))]
@@ -1217,7 +1235,7 @@
 
 (defn- import-files*
   [{:keys [::manifest] :as cfg}]
-  (bfc/configure-database-timeouts! cfg)
+  (bfc/configure-import-transaction cfg)
 
   (vswap! bfc/*state* update :index bfc/update-index (:files manifest) :id)
 
@@ -1302,7 +1320,7 @@
               :hint "branch file data is derived from the branch base and the operation log and can't be overwritten by an import"
               :file-id file-id))
 
-  (bfc/configure-database-timeouts! cfg)
+  (bfc/configure-import-transaction cfg)
 
   (let [ref-file (bfc/get-minimal-file cfg file-id ::db/for-update true)
         file     (first (get manifest :files))
@@ -1383,7 +1401,8 @@
   format. There are some options available for customize the output:
 
   `::bfc/export-type`: determines how linked libraries are handled.
-  Valid values: `:include-libraries` (include linked libraries),
+  Valid values:
+  `:include-libraries` (include linked libraries),
   `:merge-libraries` (embed library assets in the file),
   `:detach-libraries` (treat assets as basic objects),
   `:link-later` (preserve component metadata for relinking on import)."

backend/src/app/config.clj

@@ -231,7 +231,7 @@
     [:rpc-rlimit-config {:optional true} ::fs/path]
     [:rpc-climit-config {:optional true} ::fs/path]
 
-    [:audit-log-archive-uri {:optional true} :string]
+    [:nexus-uri {:optional true} ::sm/uri]
     [:audit-log-http-handler-concurrency {:optional true} ::sm/int]
 
     [:default-executor-parallelism {:optional true} ::sm/int] ;; REVIEW

backend/src/app/http.clj

@@ -244,6 +244,7 @@
    [::mgmt/routes schema:routes]
    ::session/manager
    ::setup/props
+   ::mtx/metrics
    ::db/pool])
 
 (defmethod ig/assert-key ::router
@@ -260,7 +261,7 @@
                       [mw/auth {:bearer (partial session/decode-token cfg)
                                 :cookie (partial session/decode-token cfg)
                                 :token  (partial actoken/decode-token cfg)}]
-                      [mw/parse-request]
+                      [mw/parse-request (::mtx/metrics cfg)]
                       [mw/errors errors/handle]
                       [mw/restrict-methods]]}
 

backend/src/app/http/errors.clj

@@ -122,7 +122,11 @@
           {::yres/status 400 ::yres/body data}))
 
       :else
-      {::yres/status 400 ::yres/body data})))
+      (let [explain (ex/explain data)]
+        {::yres/status 400
+         ::yres/body   (-> data
+                           (dissoc ::sm/explain)
+                           (cond-> explain (assoc :explain explain)))}))))
 
 (defmethod handle-error :assertion
   [error request parent-cause]

backend/src/app/http/middleware.clj

@@ -14,7 +14,9 @@
    [app.config :as cf]
    [app.http :as-alias http]
    [app.http.errors :as errors]
+   [app.metrics :as mtx]
    [app.tokens :as tokens]
+   [app.util.inet :as inet]
    [app.util.pointer-map :as pmap]
    [cuerdas.core :as str]
    [yetti.adapter :as yt]
@@ -44,27 +46,81 @@
     (java.io.BufferedReader.
      (java.io.InputStreamReader. body))))
 
+(def ^:private not-a-body
+  "Marker returned by `read-body-params` when the `content-type` header names
+  neither transit nor json; the request is left untouched.
+
+  A fresh object and not a keyword: a keyword is a legal value for a decoded
+  body, so identity is the only safe comparison."
+  (Object.))
+
+(defn- raise-malformed-body
+  "Signals a body that the parser could not read.
+
+  A failure while reading the body means the body does not match what the
+  client declared, so it is a client error: the exception class says nothing
+  here, because `clojure.data.json` signals its syntax errors with a bare
+  `java.lang.Exception`."
+  [request metrics content-type cause]
+  ;; A single line, without the stack trace: the trace of a parse failure
+  ;; hides the fields (path, peer address, content-type) that identify the
+  ;; client sending the broken body.
+  (l/wrn :hint "malformed request body"
+         :path (yreq/path request)
+         :remote-addr (inet/parse-request request)
+         :content-type content-type
+         :error (ex/get-hint cause))
+
+  (mtx/run! metrics
+            :id :http-malformed-requests-total
+            :inc 1)
+
+  (ex/raise :type :validation
+            :code :malformed-json
+            :hint "invalid JSON in request body"
+            :cause cause))
+
+(defn- read-body-params
+  "Reads the request body according to the `content-type` header.
+
+  `RequestTooBigException` keeps its own code (413) and a structured error
+  raised from deeper (an `ex-info`) travels untouched; anything else is a
+  malformed body."
+  [request metrics content-type]
+  (try
+    (cond
+      (str/starts-with? content-type "application/transit+json")
+      (with-open [^InputStream is (yreq/body request)]
+        (t/read! (t/reader is)))
+
+      (str/starts-with? content-type "application/json")
+      (with-open [reader (get-reader request)]
+        (json/read reader :key-fn json/read-kebab-key))
+
+      :else
+      not-a-body)
+
+    (catch RequestTooBigException cause
+      (ex/raise :type :validation
+                :code :request-body-too-large
+                :hint "request body exceeds size limit"
+                :cause cause))
+
+    (catch Throwable cause
+      (if (ex/error? cause)
+        (throw cause)
+        (raise-malformed-body request metrics content-type cause)))))
+
 (defn wrap-parse-request
-  [handler]
+  [handler metrics]
   (letfn [(process-request [request]
-            (let [header (yreq/get-header request "content-type")]
-              (cond
-                (str/starts-with? header "application/transit+json")
-                (with-open [^InputStream is (yreq/body request)]
-                  (let [params (t/read! (t/reader is))]
-                    (-> request
-                        (assoc :body-params params)
-                        (update :params merge params))))
-
-                (str/starts-with? header "application/json")
-                (with-open [reader (get-reader request)]
-                  (let [params (json/read reader :key-fn json/read-kebab-key)]
-                    (-> request
-                        (assoc :body-params params)
-                        (update :params merge params))))
-
-                :else
-                request)))
+            (let [header (yreq/get-header request "content-type")
+                  params (read-body-params request metrics header)]
+              (if (identical? params not-a-body)
+                request
+                (-> request
+                    (assoc :body-params params)
+                    (update :params merge params)))))
 
           ;; The specific-exception branches below (IAE,
           ;; RequestTooBigException, EOFException) raise with
@@ -107,10 +163,14 @@
 
     (fn [request]
       (if (= (yreq/method request) :post)
-        (try
-          (-> request process-request handler)
-          (catch Throwable cause
-            (handle-error cause request)))
+        ;; The body is read outside the `try`: a failure there is always
+        ;; raised already classified by `read-body-params`, and goes out as a
+        ;; throw like the rest of the specific-exception branches above.
+        (let [request (process-request request)]
+          (try
+            (handler request)
+            (catch Throwable cause
+              (handle-error cause request))))
         (handler request)))))
 
 (def parse-request

backend/src/app/jobs.clj

@@ -9,11 +9,15 @@
 
   Every job is defined as a job-def map returned by the `ig/init-key` of its
   module: {::name, ::schema, ::handler, ::decoder, ::validator} — decoder and
-  validator are precompiled at init time. A job-def may also carry the
-  optional metadata {::family, ::resource-role}, which the consumers of the
-  job (quotes, storage helpers) read from the registry: the substrate never
-  reads them. The registry of job-defs is plain integrant wiring (::jobs/defs);
-  submit, dispatch and management consume it by reference.
+  validator are precompiled at init time. A def whose result needs its types
+  back declares the optional ::result-schema: the substrate compiles the
+  decoder from that schema and checks a result against it before storing it,
+  while a def without one answers with the JSON the row holds. A job-def may
+  also carry the optional metadata {::family, ::resource-role}, which the
+  consumers of the job (quotes, storage helpers) read from the registry: the
+  substrate never reads them. The registry of job-defs is plain integrant
+  wiring (::jobs/defs); submit, dispatch and management consume it by
+  reference.
 
   Two execution modes are provided:
   - `submit` (durable): validates + JSON-encodes params and inserts a row
@@ -24,6 +28,11 @@
 
   Params payloads are stored as plain JSON (not transit) in the `params` jsonb
   column and decoded back to typed Clojure values using the job-def decoder.
+  Results are plain JSON in the `result` jsonb column: they are decoded with the
+  decoder the substrate compiled from the schema the def declares, and a result
+  that does not conform to that schema is refused before it is stored. Without a
+  schema a uuid comes back as the text it is on disk and a uuid used as a map
+  key as a keyword, so a def that keys a result map by uuid must declare it.
 
   Any job that can run longer than `:jobs-lease` must call `heartbeat`
   on its loop, otherwise the dispatcher marks it `aborted` (a system-side
@@ -151,7 +160,14 @@
    ;; implementation stays [cfg params].
    [::handler ::sm/fn]
    [::decoder ::sm/fn]
-   [::validator ::sm/fn]])
+   [::validator ::sm/fn]
+   ;; A result crosses a plain JSON column on its way out of the job, so a
+   ;; uuid comes back as the text it is on disk and a uuid used as a map
+   ;; key comes back as a keyword. A def that declares the schema of its
+   ;; result gets its types back when the row is read (the substrate
+   ;; compiles the decoder from the schema), and its result is checked
+   ;; against that schema before it is stored.
+   [::result-schema {:optional true} any?]])
 
 (def ^:private schema:job-defs
   [:map-of :keyword schema:job-def])
@@ -189,11 +205,24 @@
                 :job (d/name name)
                 :expected (d/name (::name job-def))))))
 
+(defn- compile-result-schema
+  "Compile the decoder of a job-def from the result schema it declares, so the
+  pair cannot drift from the schema and a def without one keeps working
+  untouched. The write path checks the result against the schema itself."
+  [job-def]
+  (if-let [schema (::result-schema job-def)]
+    (assoc job-def ::result-decoder (sm/decoder schema sm/json-transformer))
+    job-def))
+
 (defmethod ig/init-key ::defs
   [_ defs]
-  (reset! defs-registry defs)
-  (l/inf :hint "job definitions initialized" :jobs (count defs))
-  defs)
+  (let [defs (reduce-kv (fn [acc name job-def]
+                          (assoc acc name (compile-result-schema job-def)))
+                        {}
+                        defs)]
+    (reset! defs-registry defs)
+    (l/inf :hint "job definitions initialized" :jobs (count defs))
+    defs))
 
 (defmethod ig/halt-key! ::defs
   [_ defs]
@@ -222,6 +251,32 @@
         db/decode-json-pgobject)
       ((::decoder job-def))))
 
+(defn decode-result
+  "Decode the raw JSON result (pgobject or decoded map) with the decoder the
+  substrate compiled from the result schema of the job-def. A def without one
+  answers with the JSON the row holds: every key keywordized and every uuid as
+  the text it is on disk."
+  [job-def result]
+  (when (some? result)
+    (let [value (cond-> result
+                  (db/pgobject? result)
+                  db/decode-json-pgobject)]
+      (if-let [decoder (::result-decoder job-def)]
+        (decoder value)
+        value))))
+
+(defn- check-result
+  "Validate the result of a job against the schema its job-def declares, with
+  the malli explanation on failure. A job-def without a result schema has
+  nothing to check, and so has a job whose definition is gone."
+  [job-def result]
+  (when-let [schema (::result-schema job-def)]
+    (sm/check schema result
+              :hint "invalid result of a job"
+              :type :validation
+              :code :invalid-result))
+  result)
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; SUBMIT API
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@@ -432,11 +487,21 @@
       (decode-json-col :result)
       (decode-json-col :error)))
 
+(defn- decode-result-of
+  "Decode the result of a row with the decoder of the job-def that owns it.
+  A kind the registry no longer knows (a retired job-def) answers with the
+  row untouched instead of raising."
+  [cfg row]
+  (if-let [job-def (get (get-defs cfg) (keyword (:name row)))]
+    (update row :result #(decode-result job-def %))
+    row))
+
 (defn get-job
-  "Retrieve the job row (with the raw JSON params decoded to a plain map)."
+  "Retrieve the job row, with params and result decoded to plain maps (the
+  result only when its job-def declares a result decoder)."
   [cfg job-id]
-  (some-> (db/get* cfg :job {:id job-id})
-          (decode-row)))
+  (when-let [row (db/get* cfg :job {:id job-id})]
+    (decode-result-of cfg (decode-row row))))
 
 (defn get-user-status
   "Map the internal job status to the user-facing status."
@@ -1058,7 +1123,8 @@
 
   - `:job-id`      the job to complete.
   - `:result`      the JSON-encodable result, nil when the job returns
-                   nothing.
+                   nothing; a result that does not conform to the schema
+                   its job-def declares is refused.
   - `:resource-id` optional storage object to associate with the job. A
                    resource is only ever set, never replaced: passing one
                    for a job that already has a resource is a validation
@@ -1073,11 +1139,13 @@
   aborted by the dispatcher — is never overwritten) and writes no event. Returns
   the number of affected rows.
 
-  The row is read locked inside that same transaction, so the two rules
-  are applied in a fixed order against the row and not against the
-  clock: a job that is no longer completable is skipped, and only a job
-  that is still completable can be refused for its resource. A
-  concurrent completion waits for the lock and then sees the outcome."
+  The row is read locked inside that same transaction, so the rules are
+  applied in a fixed order against the row and not against the clock: a
+  job that is no longer completable is skipped, only a job that is still
+  completable can be refused for its resource, and the result is checked
+  against the schema its job-def declares before it is stored (a def
+  without one has nothing to check). A concurrent completion waits for
+  the lock and then sees the outcome."
   [cfg & {:keys [job-id result resource-id]}]
   (assert (mtx/instance cfg) "missing ::mtx/metrics on the jobs cfg")
   (check-resource-id resource-id)
@@ -1097,6 +1165,9 @@
                                           :hint "the job already has a resource, it cannot be replaced"
                                           :job-id job-id
                                           :resource-id (:resource-id job)))
+                              (check-result (get (get-defs tx-cfg)
+                                                 (keyword (:name job)))
+                                            result)
                               (if (db/exec-one! conn [sql:complete-job (ct/now) (ct/now)
                                                       encoded resource-id job-id])
                                 (do

backend/src/app/jobs/metrics.clj

@@ -176,6 +176,46 @@
     (record-value metrics :jobs-request-timing [outcome]
                   (max 0 (long millis)))))
 
+(defn- offload-result-label
+  "Coerce an offload outcome to its label: lowercase, with `failed`
+  only when there is no outcome at all. Values come from the fixed
+  call sites of the offload handler (`ok`, `skip`, `error`), never
+  from params, IDs or exception text."
+  [result]
+  (str/lower (mtx/label result "failed")))
+
+(defn- offload-type-label
+  "Coerce a file data type to its label: lowercase, with `other` only
+  when there is no type at all. Values are the fixed `file_data.type`
+  values (`main`, `snapshot`, `fragment`)."
+  [type]
+  (str/lower (mtx/label type "other")))
+
+(defn record-offload-rows
+  "Count file data rows moved by the offload job, broken down by row
+  type. Only successful runs emit rows, so there is no result label."
+  [cfg type amount]
+  (record-count (mtx/instance cfg) :file-data-offload-rows
+                [(offload-type-label type)]
+                amount))
+
+(defn record-offload-bytes
+  "Count file data bytes moved by the offload job, broken down by row
+  type. Only successful runs emit bytes, so there is no result label."
+  [cfg type amount]
+  (record-count (mtx/instance cfg) :file-data-offload-bytes
+                [(offload-type-label type)]
+                amount))
+
+(defn record-offload-duration
+  "Time one offload run from lock to summary (milliseconds). Recorded
+  for every outcome (`ok`, `skip`, `error`), so its count is the run
+  total per outcome."
+  [cfg result millis]
+  (record-value (mtx/instance cfg) :file-data-offload-timing
+                [(offload-result-label result)]
+                (max 0 (long millis))))
+
 (def ^:private backlog-statuses
   ["new" "scheduled" "running" "retry" "completed" "failed" "cancelled" "aborted"])
 

backend/src/app/loggers/audit.clj

@@ -21,6 +21,7 @@
    [app.http :as-alias http]
    [app.http.access-token :as-alias actoken]
    [app.jobs :as jobs]
+   [app.loggers.audit.consumer-task :as consumer]
    [app.loggers.audit.tasks :as-alias tasks]
    [app.loggers.webhooks :as-alias webhooks]
    [app.rpc :as-alias rpc]
@@ -235,12 +236,19 @@
 (defn- append-audit-entry
   [cfg params]
   (let [params (-> params
-                   (assoc :id (uuid/next))
+                   ;; Keep a pre-assigned id (process-event assigns one so
+                   ;; the webhook task can reference the stored row).
+                   (update :id #(or % (uuid/next)))
                    (update :props db/tjson)
                    (update :context db/tjson)
                    (update :ip-addr db/inet))
-        params (select-keys params event-keys)]
-    (db/insert! cfg :audit-log params)))
+        params (select-keys params event-keys)
+        row    (db/insert! cfg :audit-log params)]
+    ;; The consumer queue follows the row in the same transaction: one
+    ;; pending per enabled consumer. A rollback takes both on its way
+    ;; out, so a row never outlives its queue.
+    (consumer/queue-pendings cfg [(:id row)])
+    row))
 
 (def ^:private xf:filter-telemetry-props
   "Transducer that keeps only map entries whose values are UUIDs,
@@ -254,67 +262,75 @@
 
 (defn- process-event
   [cfg event]
-  (when (contains? cf/flags :audit-log-logger)
-    (l/log! ::l/logger "app.audit"
-            ::l/level :info
-            :profile-id (str (:profile-id event))
-            :ip-addr (str (:ip-addr event))
-            :type (:type event)
-            :name (:name event)
-            :props (json/encode (:props event) :key-fn json/write-camel-key)
-            :context (json/encode (:context event) :key-fn json/write-camel-key)))
+  ;; Assign the row id up-front so every sink below shares it: the
+  ;; stored row and the `:process-webhook-event` task params. The
+  ;; webhook trigger row links back to its origin event through props
+  ;; `:event-id`, which used to land nil because the id was generated
+  ;; inside `append-audit-entry` without reaching the worker params.
+  (let [event (assoc event :id (or (:id event) (uuid/next)))]
+    (when (contains? cf/flags :audit-log-logger)
+      (l/log! ::l/logger "app.audit"
+              ::l/level :info
+              :profile-id (str (:profile-id event))
+              :ip-addr (str (:ip-addr event))
+              :type (:type event)
+              :name (:name event)
+              :props (json/encode (:props event) :key-fn json/write-camel-key)
+              :context (json/encode (:context event) :key-fn json/write-camel-key)))
 
-  (when (contains? cf/flags :audit-log)
-    (append-audit-entry cfg event))
+    (when (contains? cf/flags :audit-log)
+      (append-audit-entry cfg event))
 
-  (when (contains? cf/flags :telemetry)
-    ;; NOTE: when both audit-log and telemetry are enabled, events are stored
-    ;; twice: once with full details (above) and once stripped of props and
-    ;; ip-addr, tagged with source="telemetry" so the telemetry task can
-    ;; collect and ship them.  The profile-id is preserved (UUIDs are already
-    ;; anonymous random identifiers).  Only a safe subset of context fields
-    ;; is kept: initiator, version, client-version and client-user-agent.
-    ;; Timestamps are truncated to day precision to avoid leaking exact event
-    ;; timing.
-    (let [event (-> event
-                    (filter-telemetry-props)
-                    (filter-telemetry-context)
-                    (update :created-at ct/truncate :days)
-                    (update :tracked-at ct/truncate :days)
-                    (assoc :source "telemetry:backend")
-                    (assoc :ip-addr "0.0.0.0"))]
-      (append-audit-entry cfg event)))
+    (when (contains? cf/flags :telemetry)
+      ;; NOTE: when both audit-log and telemetry are enabled, events are stored
+      ;; twice: once with full details (above) and once stripped of props and
+      ;; ip-addr, tagged with source="telemetry" so the telemetry task can
+      ;; collect and ship them.  The profile-id is preserved (UUIDs are already
+      ;; anonymous random identifiers).  Only a safe subset of context fields
+      ;; is kept: initiator, version, client-version and client-user-agent.
+      ;; Timestamps are truncated to day precision to avoid leaking exact event
+      ;; timing. The shadow row gets its own id: reusing the full row id
+      ;; would collide on the primary key.
+      (let [event (-> event
+                      (assoc :id (uuid/next))
+                      (filter-telemetry-props)
+                      (filter-telemetry-context)
+                      (update :created-at ct/truncate :days)
+                      (update :tracked-at ct/truncate :days)
+                      (assoc :source "telemetry:backend")
+                      (assoc :ip-addr "0.0.0.0"))]
+        (append-audit-entry cfg event)))
 
-  (when (and (contains? cf/flags :webhooks)
-             (::webhooks/event? event))
-    (let [batch-key     (::webhooks/batch-key event)
-          batch-timeout (::webhooks/batch-timeout event)
-          label         (dm/str "rpc:" (:name event))
-          label         (cond
-                          (ifn? batch-key)    (dm/str label ":" (batch-key (::rpc/params event)))
-                          (string? batch-key) (dm/str label ":" batch-key)
-                          :else               label)
-          dedupe?       (boolean (and batch-key batch-timeout))]
+    (when (and (contains? cf/flags :webhooks)
+               (::webhooks/event? event))
+      (let [batch-key     (::webhooks/batch-key event)
+            batch-timeout (::webhooks/batch-timeout event)
+            label         (dm/str "rpc:" (:name event))
+            label         (cond
+                            (ifn? batch-key)    (dm/str label ":" (batch-key (::rpc/params event)))
+                            (string? batch-key) (dm/str label ":" batch-key)
+                            :else               label)
+            dedupe?       (boolean (and batch-key batch-timeout))]
 
-      (jobs/submit cfg
-                   {::jobs/name :process-webhook-event
-                    ::jobs/queue :webhooks
-                    ::jobs/max-retries 0
-                    ::jobs/delay (or batch-timeout 0)
-                    ::jobs/dedupe dedupe?
-                    ::jobs/label label
-                    ;; The event travels as an opaque transit blob:
-                    ;; transit preserves the instant/UUID/set types
-                    ;; that plain JSON props cannot carry.
-                    ::jobs/params {:event-blob
-                                   (t/encode-str
-                                    (-> event
-                                        (d/without-qualified)
-                                        (dissoc :source)
-                                        (dissoc :context)
-                                        (dissoc :ip-addr)
-                                        (dissoc :type)))}})))
-  event)
+        (jobs/submit cfg
+                     {::jobs/name :process-webhook-event
+                      ::jobs/queue :webhooks
+                      ::jobs/max-retries 0
+                      ::jobs/delay (or batch-timeout 0)
+                      ::jobs/dedupe dedupe?
+                      ::jobs/label label
+                      ;; The event travels as an opaque transit blob:
+                      ;; transit preserves the instant/UUID/set types
+                      ;; that plain JSON props cannot carry.
+                      ::jobs/params {:event-blob
+                                     (t/encode-str
+                                      (-> event
+                                          (d/without-qualified)
+                                          (dissoc :source)
+                                          (dissoc :context)
+                                          (dissoc :ip-addr)
+                                          (dissoc :type)))}})))
+    event))
 
 (defn submit*
   "A public API, lower-level than submit, assumes all required fields are filled"
@@ -402,6 +418,47 @@
 
     coerced))
 
+(def ^:private sql:team-id-by-project
+  "SELECT id FROM team
+    WHERE id = (SELECT team_id FROM project WHERE id = ?)")
+
+(def ^:private sql:team-id-by-file
+  "SELECT t.id FROM team AS t
+     JOIN project AS p ON (p.team_id = t.id)
+     JOIN file AS f ON (f.project_id = p.id)
+    WHERE f.id = ?")
+
+(def ^:private read-event-names
+  "Read commands excluded from team-id resolution on top of the `get-*`
+  prefix: their events carry no consumer state, so the lookup would only
+  tax the read path."
+  #{"has-file-libraries"})
+
+(defn- read-event?
+  [event-name]
+  (or (str/starts-with? event-name "get-")
+      (contains? read-event-names event-name)))
+
+(defn- enrich-team-id
+  "Adds the team id to the event props when it is missing and can be resolved
+  locally, so consumers do not have to join production tables. Only events that
+  already carry a project or file id are considered: read commands put the
+  entity id under `:id`, and resolving those would add a query to every read."
+  [cfg event-name props]
+  (if (or (:team-id props)
+          (read-event? event-name)
+          (and (nil? (:project-id props))
+               (nil? (:file-id props))))
+    props
+    (let [project-id (:project-id props)
+          file-id    (:file-id props)
+          team-id    (or (when (some? project-id)
+                           (:id (db/exec-one! cfg [sql:team-id-by-project project-id])))
+                         (when (some? file-id)
+                           (:id (db/exec-one! cfg [sql:team-id-by-file file-id]))))]
+      (cond-> props
+        (some? team-id) (assoc :team-id team-id)))))
+
 (defn prepare-rpc-event
   [cfg mdata params result]
   (let [resultm      (meta result)
@@ -416,25 +473,32 @@
                          (::rpc/profile-id params)
                          uuid/zero)
 
+        module       (get cfg ::rpc/module)
+        event-name   (or (::name resultm)
+                         (let [sname (::sv/name mdata)]
+                           (if (not= module "main")
+                             (str module "-" sname)
+                             sname)))
+
         props        (-> (or (::replace-props resultm)
                              (merge params (::props resultm)))
                          (clean-props))
 
+        ;; Resolve the team locally when the event carries a project or file
+        ;; id, so the consumers do not need to join production tables.
+        ;; Reads are excluded: their events carry no consumer state.
+        props        (enrich-team-id cfg event-name props)
+
         context      (-> (::context resultm)
                          (merge (prepare-context-from-request request))
                          (assoc :request-id (::rpc/request-id params))
                          (d/without-nils))
 
-        ip-addr      (inet/parse-request request)
-        module       (get cfg ::rpc/module)]
+        ip-addr      (inet/parse-request request)]
 
     {:type (or (::type resultm)
                (::rpc/type cfg))
-     :name (or (::name resultm)
-               (let [sname (::sv/name mdata)]
-                 (if (not= module "main")
-                   (str module "-" sname)
-                   sname)))
+     :name event-name
 
      :profile-id profile-id
      :ip-addr ip-addr
@@ -491,12 +555,41 @@
                   (d/without-nils))]
     (submit* cfg event)))
 
+(defn submit-create-event
+  "Emits a `create-*` audit event for an entity created as a side effect of a
+  bulk operation (import, duplicate, clone). Those entities get no event from
+  their own command, so a local projection cannot key them otherwise. The
+  event is built by hand and does not pass through `prepare-rpc-event`, so its
+  props are passed as-is minus nils."
+  [cfg params event-name props]
+  (when (contains? cf/flags :audit-log)
+    (submit cfg
+            (-> (event-from-rpc-params params)
+                (assoc :name event-name)
+                (assoc :props (d/without-nils props))))))
+
+(def ^:private sql:get-file-event-columns
+  "SELECT id, name, project_id, created_at, modified_at, revn
+     FROM file WHERE id = ?;")
+
+(defn submit-file-created-event
+  "Emit a `create-file` event for a file born inside a bulk operation,
+  with the same prop shape as the `create-file` command event. Only
+  the file id is needed: the light columns are read back (no data
+  blob), so projections that key on name, revn or timestamps work.
+  `extra` carries what the row lacks (the team id)."
+  [cfg params file-id extra]
+  (when-let [row (db/exec-one! cfg [sql:get-file-event-columns file-id])]
+    (submit-create-event cfg params "create-file" (merge row extra))))
+
 (defn insert
   "Submit audit event to the collector, intended to be used only from
   command line helpers because this skips all webhooks and telemetry
   logic."
   [cfg event]
   (when (contains? cf/flags :audit-log)
+    ;; Transactional (not a bare connection): the row and its
+    ;; pendings commit or vanish together (see append-audit-entry).
     (let [tnow  (ct/now)
           event (-> event
                     (assoc :created-at tnow)
@@ -507,4 +600,4 @@
                     (update :source d/nilv "backend")
                     (select-keys event-keys)
                     (check-event))]
-      (db/run! cfg append-audit-entry event))))
+      (db/tx-run! cfg append-audit-entry event))))

backend/src/app/loggers/audit/archive_task.clj

@@ -1,308 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS SUBSIDIARY SL
-
-(ns app.loggers.audit.archive-task
-  (:require
-   [app.common.exceptions :as ex]
-   [app.common.logging :as l]
-   [app.common.schema :as sm]
-   [app.common.transit :as t]
-   [app.common.uri :as u]
-   [app.common.uuid :as uuid]
-   [app.config :as cf]
-   [app.db :as db]
-   [app.http.client :as http]
-   [app.jobs :as jobs]
-   [app.nitrate :as nitrate]
-   [app.setup :as-alias setup]
-   [integrant.core :as ig]
-   [promesa.exec :as px]))
-
-;; This is a task responsible to send the accumulated events to
-;; external service for archival.
-
-(defn- decode-row
-  [{:keys [props ip-addr context] :as row}]
-  (cond-> row
-    (db/pgobject? props)
-    (assoc :props (db/decode-transit-pgobject props))
-
-    (db/pgobject? context)
-    (assoc :context (db/decode-transit-pgobject context))
-
-    (db/pgobject? ip-addr "inet")
-    (assoc :ip-addr (db/decode-inet ip-addr))))
-
-(def ^:private event-keys
-  [:type
-   :name
-   :source
-   :created-at
-   :tracked-at
-   :profile-id
-   :ip-addr
-   :props
-   :context])
-
-(defn- row->event
-  [row]
-  (select-keys row event-keys))
-
-(defn- send!
-  [{:keys [::uri] :as cfg} events]
-  (let [skey    (-> cfg ::setup/shared-keys :nexus)
-        body    (t/encode {:events events})
-        headers {"content-type" "application/transit+json"
-                 "origin" (str (cf/get :public-uri))
-                 "x-shared-key" (str "nexus " skey)}
-        params  {:uri uri
-                 :timeout 12000
-                 :method :post
-                 :headers headers
-                 :body body}
-        resp    (http/req cfg params {:skip-ssrf-check? true})]
-
-    (if (= (:status resp) 204)
-      true
-      (do
-        (l/error :hint "unable to archive events"
-                 :resp-status (:status resp)
-                 :resp-body (:body resp))
-        false))))
-
-(def ^:private event-names-for-nitrate
-  #{"accept-organization-invitation"
-    "accept-team-invitation"
-    "accept-team-invitation-from"
-    "add-member-to-organization"
-    "add-team-to-organization"
-    "cancel-organization-invitation"
-    "create-export-binfile-job"
-    "create-file"
-    "create-organization"
-    "create-organization-attribute"
-    "create-organization-invitation"
-    "create-project"
-    "create-team"
-    "create-team-access-request"
-    "create-team-invitation"
-    "create-team-invitations"
-    "create-webhook"
-    "delete-file"
-    "delete-font"
-    "delete-organization"
-    "delete-organization-member"
-    "delete-project"
-    "delete-team"
-    "delete-team-invitation"
-    "delete-team-member"
-    "leave-team"
-    "move-project"
-    "move-team-to-organization"
-    "organization-sso-auth-failed"
-    "organization-sso-auth-started"
-    "organization-sso-auth-succeeded"
-    "permanently-delete-team-files"
-    "remove-organization-team"
-    "remove-team-from-organization"
-    "rename-organization"
-    "rename-project"
-    "restore-deleted-team-files"
-    "success-activate-organization-sso"
-    "success-apply-organization-sso-changes"
-    "update-organization-invitation"
-    "update-organization-permissions"
-    "update-team-invitation"
-    "update-team-invitation-role"
-    "update-team-member-role"
-    "update-team-photo"
-    "verify-token"})
-
-;; Props id key -> email prop filled from profile.email (nitrate path only).
-;; Some producers store ids as UUID strings (e.g. :user-who-send-invitation).
-(def ^:private id->email-prop
-  {:profile-id                 :profile-email
-   :user-id                    :user-email
-   :member-id                  :member-email
-   :user-who-send-invitation   :user-who-send-invitation-email})
-
-(defn- collect-team-ids-needing-name
-  [events]
-  (into #{}
-        (comp (map :props)
-              (keep (fn [props]
-                      (when-let [team-id (and (not (contains? props :team-name))
-                                              (uuid/coerce (:team-id props)))]
-                        team-id))))
-        events))
-
-(defn- collect-profile-ids-needing-email
-  [events]
-  (into #{}
-        (mapcat (fn [{:keys [props]}]
-                  (keep (fn [[id-key email-key]]
-                          (when-not (contains? props email-key)
-                            (uuid/coerce (get props id-key))))
-                        id->email-prop)))
-        events))
-
-(defn- load-team-names
-  [conn ids]
-  (if (seq ids)
-    (let [arr  (db/create-array conn "uuid" ids)
-          rows (db/exec! conn ["SELECT id, name FROM team WHERE id = ANY(?)" arr])]
-      (into {} (map (juxt :id :name) rows)))
-    {}))
-
-(defn- load-profile-emails
-  [conn ids]
-  (if (seq ids)
-    (let [arr  (db/create-array conn "uuid" ids)
-          rows (db/exec! conn ["SELECT id, email FROM profile WHERE id = ANY(?)" arr])]
-      (into {} (map (juxt :id :email) rows)))
-    {}))
-
-(defn- maybe-assoc-team-name
-  [event team-names]
-  (let [props (:props event)]
-    (if-let [team-name (and (not (contains? props :team-name))
-                            (when-let [team-id (uuid/coerce (:team-id props))]
-                              (get team-names team-id)))]
-      (assoc-in event [:props :team-name] team-name)
-      event)))
-
-(defn- maybe-assoc-emails
-  [event emails]
-  (update event :props
-          (fn [props]
-            (reduce-kv (fn [props id-key email-key]
-                         (if-let [email (and (not (contains? props email-key))
-                                             (when-let [id (uuid/coerce (get props id-key))]
-                                               (get emails id)))]
-                           (assoc props email-key email)
-                           props))
-                       props
-                       id->email-prop))))
-
-(defn- enrich-events-for-nitrate
-  "Enrich allowlisted events before nitrate ingest:
-  1. Fill missing :team-name from team id (including soft-deleted).
-  2. Fill missing emails from matching id keys (including soft-deleted
-     profiles): :profile-email, :user-email, :member-email,
-     :user-who-send-invitation-email."
-  [{:keys [::db/conn]} events]
-  (let [team-ids    (collect-team-ids-needing-name events)
-        profile-ids (collect-profile-ids-needing-email events)
-        team-names  (load-team-names conn team-ids)
-        emails      (load-profile-emails conn profile-ids)]
-    (mapv (fn [event]
-            (-> event
-                (maybe-assoc-team-name team-names)
-                (maybe-assoc-emails emails)))
-          events)))
-
-(defn- send-to-nitrate!
-  [cfg rows]
-  (when-let [events (->> rows
-                         (filterv #(contains? event-names-for-nitrate (:name %)))
-                         (not-empty))]
-    (nitrate/call cfg :ingest-audit-log
-                  {:events (enrich-events-for-nitrate cfg events)})))
-
-(defn- mark-archived!
-  [{:keys [::db/conn]} rows]
-  (let [ids (db/create-array conn "uuid" (map :id rows))]
-    (db/exec-one! conn ["update audit_log set archived_at=now() where id = ANY(?)" ids])))
-
-(def ^:private sql:get-audit-log-chunk
-  "SELECT *
-     FROM audit_log
-    WHERE archived_at IS NULL
-    ORDER BY created_at ASC
-    LIMIT 128
-      FOR UPDATE
-     SKIP LOCKED")
-
-(defn- get-event-rows
-  [{:keys [::db/conn] :as cfg}]
-  (->> (db/exec! conn [sql:get-audit-log-chunk])
-       (not-empty)))
-
-(defn- archive-events!
-  [{:keys [::uri] :as cfg}]
-  (db/tx-run! cfg (fn [cfg]
-                    (when-let [rows (get-event-rows cfg)]
-                      (let [decoded (mapv decode-row rows)
-                            events  (mapv row->event decoded)]
-                        (l/trc :hint "archive events chunk" :uri uri :events (count events))
-
-                        (when (contains? cf/flags :admin-console)
-                          (send-to-nitrate! cfg decoded))
-
-                        ;; When :nexus is off, treat Nexus as skipped
-                        ;; success so nitrate-only still marks the chunk.
-                        ;; REPL :enabled true with neither send flag also marks
-                        ;; with no outbound send — always set at least one flag.
-                        (let [nexus-ok? (if (contains? cf/flags :nexus)
-                                          (send! cfg events)
-                                          true)]
-                          (when nexus-ok?
-                            (mark-archived! cfg rows)
-                            (count events))))))))
-
-(declare execute-audit-log-archive)
-
-(def schema:audit-log-archive-params
-  "Optional overrides for the repl invocation defaults."
-  [:map
-   ::db/pool
-   ::setup/shared-keys
-   ::http/client
-   [:app.nitrate/client {:optional true} [:maybe :map]]
-   [:enabled {:optional true} :boolean]
-   [:uri {:optional true} ::sm/uri]])
-
-(defmethod ig/init-key ::job-def
-  [_ cfg]
-  {::jobs/name      :audit-log-archive
-   ::jobs/schema    schema:audit-log-archive-params
-   ::jobs/handler
-   (fn [_context params]
-     (execute-audit-log-archive cfg params))
-   ::jobs/decoder   (sm/decoder schema:audit-log-archive-params sm/json-transformer)
-   ::jobs/validator (sm/validator schema:audit-log-archive-params)})
-
-(defn execute-audit-log-archive
-  "Plain job handler: archive the accumulated audit events in chunks
-  (heartbeat per iteration: the sent chunk batches can be long)."
-  [cfg params]
-  ;; NOTE: this let allows overwrite default configured values from
-  ;; the repl, when manually invoking the task. Prefer setting a send
-  ;; flag (:admin-console and/or :nexus); :enabled alone
-  ;; marks chunks without shipping.
-  (let [enabled (or (contains? cf/flags :nexus)
-                    (contains? cf/flags :admin-console)
-                    (:enabled params false))
-        uri     (cf/get :audit-log-archive-uri)
-        uri     (or uri (:uri params))
-        ;; Normalize to an uri object; params may carry a plain string
-        ;; on direct invocations (validation only applies on submit!).
-        uri     (u/uri uri)
-        cfg     (assoc cfg ::uri uri)]
-    (when (and (contains? cf/flags :nexus)
-               (not uri))
-      (ex/raise :type :internal
-                :code :task-not-configured
-                :hint "archive task not configured, missing uri"))
-    (when enabled
-      (loop [total 0]
-        (if-let [n (archive-events! cfg)]
-          (do
-            (jobs/heartbeat cfg)
-            (px/sleep 100)
-            (recur (+ total ^long n)))
-          (when (pos? total)
-            (l/dbg :hint "events archived" :total total)))))))

backend/src/app/loggers/audit/consumer_task.clj

@@ -0,0 +1,685 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.loggers.audit.consumer-task
+  (:require
+   [app.common.exceptions :as ex]
+   [app.common.logging :as l]
+   [app.common.schema :as sm]
+   [app.common.time :as ct]
+   [app.common.transit :as t]
+   [app.common.uri :as u]
+   [app.common.uuid :as uuid]
+   [app.config :as cf]
+   [app.db :as db]
+   [app.db.sql :as sql]
+   [app.http.client :as http]
+   [app.jobs :as jobs]
+   [app.metrics :as mtx]
+   [app.nitrate :as nitrate]
+   [app.setup :as-alias setup]
+   [integrant.core :as ig]
+   [promesa.exec :as px]))
+
+;; This task drains the audit log buffer into the long-term archives.
+;;
+;; Two separate jobs share this namespace and the queue helpers, one
+;; per destination: `:audit-log-consumer-nexus` and
+;; `:audit-log-consumer-admin-console`. Each job drains only its own queue
+;; and deletes its pendings on completion; whoever deletes the last
+;; pending also deletes the record ("last one out turns off the
+;; light"), so no separate GC task is needed. Housekeeping (reaping
+;; disabled pendings, sweeping orphans) lives in both jobs on purpose:
+;; only the enabled job runs, so it must clean up for the rest.
+
+;; Consumer ids are fixed constants, never reuse one: retired
+;; pendings are reaped by id, so a reused id would inherit the
+;; previous owner's queue.
+(def consumer-nexus 1)
+(def consumer-admin-console 2)
+
+(def ^:private consumers
+  [{:id consumer-nexus          :flag :nexus         :label "nexus"}
+   {:id consumer-admin-console :flag :admin-console :label "admin-console"}])
+
+(def ^:private xf:enabled-consumers
+  "Static transducer: keep the consumers whose send flag is on.
+  Allocated once so hot paths (every audited RPC) do not rebuild
+  it per call; the flag lookup itself still reads the live flags."
+  (comp (filter #(contains? cf/flags (:flag %)))
+        (map :id)))
+
+(defn enabled-consumers
+  "Consumers allowed to work right now, from the send flags. Public:
+  producers queue every record for exactly these consumers."
+  []
+  (into [] xf:enabled-consumers consumers))
+
+(defn queue-pendings
+  "Queue records for every enabled consumer, nothing when there is
+  nothing to queue or no consumer is on. One multi-row statement for
+  all ids instead of one per consumer: this runs inside every audited
+  RPC and the bulk frontend ingest. Runs inside the row transaction:
+  a rollback takes both."
+  [cfg ids]
+  (let [consumers (enabled-consumers)]
+    (when (and (seq ids) (seq consumers))
+      (db/insert-many! cfg :audit-log-pending
+                       [:audit-log-id :consumer]
+                       (for [id ids
+                             consumer consumers]
+                         [id consumer])
+                       {::sql/on-conflict-do-nothing true}))))
+
+(def ^:private consumer-labels
+  (into {} (map (juxt :id :label)) consumers))
+
+(defn- consumer-label
+  [consumer]
+  (get consumer-labels consumer))
+
+(defn- decode-row
+  [{:keys [props ip-addr context] :as row}]
+  (cond-> row
+    (db/pgobject? props)
+    (assoc :props (db/decode-transit-pgobject props))
+
+    (db/pgobject? context)
+    (assoc :context (db/decode-transit-pgobject context))
+
+    (db/pgobject? ip-addr "inet")
+    (assoc :ip-addr (db/decode-inet ip-addr))))
+
+(def ^:private event-keys
+  [:type
+   :name
+   :source
+   :created-at
+   :tracked-at
+   :profile-id
+   :ip-addr
+   :props
+   :context])
+
+(defn- row->event
+  [row]
+  (select-keys row event-keys))
+
+(defn- record-count
+  [metrics id labels amount]
+  (mtx/run! metrics :id id :labels labels :inc amount))
+
+(defn- truncate-body
+  [body limit]
+  (when (some? body)
+    (let [s (str body)]
+      (if (> (count s) limit)
+        (subs s 0 limit)
+        s))))
+
+;; NOTE: deliberately no SKIP LOCKED here. Batches must reach the
+;; destinations in chronological order (Nexus numbers by arrival),
+;; so every consumer takes the same total order and a second
+;; overlapping run waits for the first to commit, then sees whether
+;; it was last. The brief blocking is accepted; skipping rows could
+;; reorder archive arrival.
+(def ^:private sql:pending-chunk
+  "SELECT a.*
+     FROM audit_log_pending p
+     JOIN audit_log a ON a.id = p.audit_log_id
+    WHERE p.consumer = ?::smallint
+    ORDER BY a.created_at ASC, a.id ASC
+    LIMIT 128
+      FOR UPDATE OF a")
+
+(def ^:private sql:delete-pendings
+  "DELETE FROM audit_log_pending
+    WHERE consumer = ?::smallint
+      AND audit_log_id = ANY(?)")
+
+(def ^:private sql:delete-drained
+  "DELETE FROM audit_log a
+    WHERE a.id = ANY(?)
+      AND NOT EXISTS (
+        SELECT 1 FROM audit_log_pending p
+        WHERE p.audit_log_id = a.id
+      )")
+
+(defn- get-pending-rows
+  [{:keys [::db/conn] :as cfg} consumer]
+  (->> (db/exec! conn [sql:pending-chunk consumer])
+       (not-empty)))
+
+(defn- complete-batch
+  "Delete our pendings for the batch, then the records nobody is
+  waiting for anymore. Two statements on purpose: inside a single
+  statement the NOT EXISTS would still see the pending just deleted
+  and never delete a record."
+  [{:keys [::db/conn]} consumer rows]
+  (let [ids (db/create-array conn "uuid" (map :id rows))]
+    (db/exec! conn [sql:delete-pendings consumer ids])
+    (db/exec! conn [sql:delete-drained ids])))
+
+(defn- generate-nexus-uri
+  "Joins relative path segments to the Nexus backend URI, same shape
+  as the admin-console helper in `app.nitrate` (there the ingest path
+  is `api/audit-log`; here it is `api/audit`).
+  Segments must not start with `/`. Returns nil when no base URI is
+  configured, so the caller keeps the `:task-not-configured` guard."
+  [& segments]
+  (when-let [base (cf/get :nexus-uri)]
+    (apply cf/join-uri base segments)))
+
+(defn- send-to-nexus
+  "POST events to Nexus. Returns the number of events sent; throws on
+  transport failure or non-204 refusal, carrying the attempted count
+  as :events in the error data so the caller records the failure.
+  No metrics or logs here: the runner reports the aborted job."
+  [{:keys [::uri] :as cfg} events]
+  (let [n      (count events)
+        skey   (-> cfg ::setup/shared-keys :nexus)
+        body   (t/encode {:events events})
+        params {:uri uri
+                :timeout 12000
+                :method :post
+                :headers {"content-type" "application/transit+json"
+                          "origin" (str (cf/get :public-uri))
+                          "x-shared-key" (str "nexus " skey)}
+                :body body}
+        resp   (try
+                 (http/req cfg params {:skip-ssrf-check? true})
+                 (catch Throwable cause
+                   (throw (ex/raise :type :internal
+                                    :code :nexus-send-failed
+                                    :hint "unable to send events to nexus"
+                                    :cause cause
+                                    :events n))))]
+    (if (= (:status resp) 204)
+      n
+      (throw (ex/raise :type :internal
+                       :code :consumer-chunk-refused
+                       :hint "nexus refused archive chunk"
+                       :resp-status (:status resp)
+                       :resp-body (truncate-body (:body resp) 500)
+                       :events n)))))
+
+(def ^:private event-names-for-admin-console
+  #{"accept-organization-invitation"
+    "accept-organization-invitation-from"
+    "accept-team-invitation"
+    "accept-team-invitation-from"
+    "add-member-to-organization"
+    "add-team-to-organization"
+    "auditlog-exported"
+    "auditlog-guide-downloaded"
+    "cancel-organization-invitation"
+    "create-export-binfile-job"
+    "create-file"
+    "create-organization"
+    "create-organization-attribute"
+    "create-organization-invitation"
+    "create-project"
+    "create-team"
+    "create-team-access-request"
+    "create-team-invitation"
+    "create-team-invitations"
+    "create-webhook"
+    "delete-file"
+    "delete-font"
+    "delete-organization"
+    "delete-organization-member"
+    "delete-project"
+    "delete-team"
+    "delete-team-invitation"
+    "delete-team-member"
+    "leave-team"
+    "move-project"
+    "move-team-to-organization"
+    "organization-sso-auth-failed"
+    "organization-sso-auth-started"
+    "organization-sso-auth-succeeded"
+    "organization-sso-token-issued"
+    "organization-sso-token-revoked"
+    "permanently-delete-team-files"
+    "remove-organization-team"
+    "remove-team-from-organization"
+    "rename-organization"
+    "rename-project"
+    "restore-deleted-team-files"
+    "sso-config-deleted"
+    "sso-config-tested"
+    "success-activate-organization-sso"
+    "success-apply-organization-sso-changes"
+    "update-organization-invitation"
+    "update-organization-permissions"
+    "update-team-invitation"
+    "update-team-invitation-role"
+    "update-team-member-role"
+    "update-team-photo"
+    "verify-token"})
+
+;; Props id key -> email prop filled from profile.email (admin-console path only).
+;; Some producers store ids as UUID strings (e.g. :user-who-send-invitation).
+(def ^:private id->email-prop
+  {:profile-id                 :profile-email
+   :user-id                    :user-email
+   :member-id                  :member-email
+   :user-who-send-invitation   :user-who-send-invitation-email})
+
+(defn- collect-team-ids-needing-name
+  [events]
+  (into #{}
+        (comp (map :props)
+              (keep (fn [props]
+                      (when-let [team-id (and (not (contains? props :team-name))
+                                              (uuid/coerce (:team-id props)))]
+                        team-id))))
+        events))
+
+(defn- collect-profile-ids-needing-email
+  [events]
+  (into #{}
+        (mapcat (fn [{:keys [props]}]
+                  (keep (fn [[id-key email-key]]
+                          (when-not (contains? props email-key)
+                            (uuid/coerce (get props id-key))))
+                        id->email-prop)))
+        events))
+
+(defn- collect-actor-ids-needing-profile
+  "Actor ids (event :profile-id) that still need top-level :profile-email
+  and/or :profile-name. Skips uuid/zero and non-uuids."
+  [events]
+  (into #{}
+        (keep (fn [event]
+                (when (or (not (contains? event :profile-email))
+                          (not (contains? event :profile-name)))
+                  (when-let [id (uuid/coerce (:profile-id event))]
+                    (when-not (uuid/zero? id)
+                      id)))))
+        events))
+
+(defn- load-team-names
+  [conn ids]
+  (if (seq ids)
+    (let [arr  (db/create-array conn "uuid" ids)
+          rows (db/exec! conn ["SELECT id, name FROM team WHERE id = ANY(?)" arr])]
+      (into {} (map (juxt :id :name) rows)))
+    {}))
+
+(defn- load-profiles
+  "Batch-load profile email and fullname (including soft-deleted)."
+  [conn ids]
+  (if (seq ids)
+    (let [arr  (db/create-array conn "uuid" ids)
+          rows (db/exec! conn ["SELECT id, email, fullname FROM profile WHERE id = ANY(?)" arr])]
+      (into {} (map (juxt :id identity) rows)))
+    {}))
+
+(defn- maybe-assoc-team-name
+  [event team-names]
+  (let [props (:props event)]
+    (if-let [team-name (and (not (contains? props :team-name))
+                            (when-let [team-id (uuid/coerce (:team-id props))]
+                              (get team-names team-id)))]
+      (assoc-in event [:props :team-name] team-name)
+      event)))
+
+(defn- maybe-assoc-emails
+  [event profiles]
+  (update event :props
+          (fn [props]
+            (reduce-kv (fn [props id-key email-key]
+                         (if-let [email (and (not (contains? props email-key))
+                                             (when-let [id (uuid/coerce (get props id-key))]
+                                               (:email (get profiles id))))]
+                           (assoc props email-key email)
+                           props))
+                       props
+                       id->email-prop))))
+
+(defn- maybe-assoc-actor-profile
+  "Fill-only top-level :profile-email / :profile-name from the event
+  actor (:profile-id). Omits keys when the id is missing, zero, the
+  profile row is absent, or the corresponding column is empty
+  (fullname defaults to \"\" in the schema)."
+  [event profiles]
+  (let [id (uuid/coerce (:profile-id event))]
+    (if (or (nil? id) (uuid/zero? id))
+      event
+      (if-let [profile (get profiles id)]
+        (cond-> event
+          (and (not (contains? event :profile-email))
+               (not-empty (:email profile)))
+          (assoc :profile-email (:email profile))
+
+          (and (not (contains? event :profile-name))
+               (not-empty (:fullname profile)))
+          (assoc :profile-name (:fullname profile)))
+        event))))
+
+(defn- enrich-events-for-admin-console
+  "Enrich allowlisted events before the admin-console ingest:
+  1. Fill missing :team-name from team id (including soft-deleted).
+  2. Fill missing emails from matching id keys in props (including
+     soft-deleted profiles): :profile-email, :user-email, :member-email,
+     :user-who-send-invitation-email.
+  3. Fill missing top-level :profile-email / :profile-name from the
+     event actor (:profile-id), including soft-deleted profiles; omit
+     when there is no usable actor id or the value is empty."
+  [{:keys [::db/conn]} events]
+  (let [team-ids    (collect-team-ids-needing-name events)
+        profile-ids (into (collect-profile-ids-needing-email events)
+                          (collect-actor-ids-needing-profile events))
+        team-names  (load-team-names conn team-ids)
+        profiles    (load-profiles conn profile-ids)]
+    (mapv (fn [event]
+            (-> event
+                (maybe-assoc-team-name team-names)
+                (maybe-assoc-emails profiles)
+                (maybe-assoc-actor-profile profiles)))
+          events)))
+
+(defn- send-to-admin-console
+  "Ingest rows into the admin console. Returns {:sent :skipped}
+  (filtered-out rows count as processed for the queue); throws
+  carrying the attempted count as :events, like send-to-nexus. Enrich
+  failures propagate untouched: a local DB/decode failure is ours, not
+  the destination's, and must not count as a send failure."
+  [cfg rows]
+  (let [events   (filterv #(contains? event-names-for-admin-console (:name %)) rows)
+        total    (count rows)
+        skipped  (- total (count events))
+        uri-host (some-> (cf/get :admin-console-uri) u/uri u/get-domain)]
+    (l/dbg :hint "consumer events chunk ready"
+           :consumer "admin-console"
+           :uri-host uri-host
+           :total total
+           :skipped skipped)
+    (if (empty? events)
+      {:sent 0 :skipped skipped}
+      ;; Enrich first, outside the try: a local DB/decode failure is
+      ;; ours, not the destination's, and must not count as
+      ;; admin-console/failed.
+      (let [enriched (enrich-events-for-admin-console cfg events)
+            n        (count events)]
+        (try
+          (nitrate/call cfg :ingest-audit-log {:events enriched})
+          (l/dbg :hint "consumer events chunk sent"
+                 :consumer "admin-console"
+                 :uri-host uri-host
+                 :sent n
+                 :skipped skipped)
+          {:sent n :skipped skipped}
+          (catch Throwable cause
+            (l/dbg :hint "consumer events chunk failed"
+                   :consumer "admin-console"
+                   :uri-host uri-host
+                   :events n
+                   :code :admin-console-send-failed)
+            (throw (ex/raise :type :internal
+                             :code :admin-console-send-failed
+                             :hint "unable to archive events to the admin console"
+                             :cause cause
+                             :events n))))))))
+
+(defn- consumer-nexus-chunk
+  "Send one pending chunk to Nexus and complete it. Returns {:sent n}
+  or nil when the queue is empty; throws on refusal. Takes a
+  transactional cfg: the caller opens one transaction per chunk. No
+  metrics here: a normal return means the transaction committed, so
+  the caller records the success."
+  [cfg]
+  (db/exec! cfg ["SET LOCAL lock_timeout = '30s'"])
+  (when-let [rows (get-pending-rows cfg consumer-nexus)]
+    (let [events (mapv row->event (mapv decode-row rows))
+          ;; send-to-nexus returns the count or throws: a refusal
+          ;; aborts the job, the rows stay for the next tick.
+          n      (send-to-nexus cfg events)]
+
+      (l/trc :hint "consumer events chunk"
+             :consumer "nexus"
+             :events n)
+
+      (complete-batch cfg consumer-nexus rows)
+      {:sent n})))
+
+(defn- consumer-admin-console-chunk
+  "Send one pending chunk to the admin console and complete it. Returns
+  {:sent :skipped} or nil when the queue is empty; throws on failure.
+  Takes a transactional cfg, no metrics here: same contract as
+  consumer-nexus-chunk."
+  [cfg]
+  (db/exec! cfg ["SET LOCAL lock_timeout = '30s'"])
+  (when-let [rows (get-pending-rows cfg consumer-admin-console)]
+    (let [decoded     (mapv decode-row rows)
+          _           (l/trc :hint "consumer events chunk"
+                             :consumer "admin-console"
+                             :events (count decoded))
+          admin-console-res (send-to-admin-console cfg decoded)]
+      ;; Filtered-out rows count as processed for the
+      ;; admin-console queue: their pending is done, the
+      ;; record stays until every consumer is done.
+      (complete-batch cfg consumer-admin-console rows)
+      admin-console-res)))
+
+(defn- drain-consumer
+  "Drain one consumer queue chunk by chunk, one transaction per chunk.
+  Returns totals; a chunk the destination refuses (or that throws)
+  aborts the drain with a throw, and the committed chunks so far
+  stay sent."
+  [cfg send-chunk]
+  (loop [sent    0
+         skipped 0
+         chunks  0]
+    ;; One transaction per chunk: fetch, send and complete commit (or
+    ;; roll back) together, and the next chunk sees a fresh snapshot.
+    (if-let [res (db/tx-run! cfg send-chunk)]
+      (do
+        (jobs/heartbeat cfg)
+        (px/sleep 100)
+        (recur (+ sent ^long (:sent res))
+               (+ skipped ^long (:skipped res 0))
+               (inc chunks)))
+      {:sent sent :skipped skipped :chunks chunks})))
+
+(defn- sql:reap-pendings
+  [consumer]
+  (str "DELETE FROM audit_log_pending"
+       " WHERE consumer = " consumer
+       "   AND audit_log_id IN (SELECT audit_log_id FROM audit_log_pending"
+       " WHERE consumer = " consumer
+       " ORDER BY audit_log_id LIMIT 5000)"))
+
+(def ^:private sql:sweep-orphans
+  "DELETE FROM audit_log a
+    WHERE a.id IN (SELECT a2.id FROM audit_log a2
+                    WHERE NOT EXISTS (SELECT 1 FROM audit_log_pending p
+                                       WHERE p.audit_log_id = a2.id)
+                    ORDER BY a2.id LIMIT 5000)")
+
+(defn- delete-batched
+  "Run a batched DELETE until it removes nothing. Returns the total."
+  [cfg sql]
+  (loop [total 0]
+    (let [n (db/get-update-count (db/exec-one! cfg [sql]))]
+      (if (pos? n)
+        (do
+          (px/sleep 100)
+          (recur (+ total ^long n)))
+        total))))
+
+(defn- reap-disabled
+  "Drop the pendings of consumers that are off: a disabled destination
+  will never drain them, and without this they would pin their records
+  forever. This also absorbs any later deactivation, with no manual step.
+  Dropped pendings count as discarded: their rows may still be archived
+  by an enabled consumer, or swept unsent when nothing waits for them."
+  [cfg]
+  (let [total (->> consumers
+                   (remove #(contains? cf/flags (:flag %)))
+                   (map #(delete-batched cfg (sql:reap-pendings (:id %))))
+                   (reduce + 0))]
+    (when (pos? total)
+      (record-count (mtx/instance cfg) :audit-consumer-discarded ["reaped"] total))
+    total))
+
+(defn- sweep-orphans
+  "Delete records nobody waits for anymore: rows born while no
+  consumer was on, rows left pending-less by the reap, and leftovers
+  archived by the old task. Batched and normally a no-op. Deleted rows
+  count as discarded: on a live system they were never archived."
+  [cfg]
+  (let [total (delete-batched cfg sql:sweep-orphans)]
+    (when (pos? total)
+      (record-count (mtx/instance cfg) :audit-consumer-discarded ["swept"] total))
+    total))
+
+(declare execute-audit-log-consumer-nexus)
+(declare execute-audit-log-consumer-admin-console)
+
+(def schema:audit-log-consumer-nexus-params
+  "Overrides for the nexus job repl invocation defaults.
+  Cron submits {}; system deps (pool, shared-keys, http) live on the
+  job-def cfg closed over by the handler, not in job params.
+  Closed on purpose: a stale key (like the old :enabled) must fail
+  validation loudly instead of skipping silently."
+  [:map {:closed true}
+   [:uri {:optional true} ::sm/uri]])
+
+(def schema:audit-log-consumer-admin-console-params
+  "Params of the admin-console job: none. Cron submits {}; system deps
+  (pool, admin-console client) live on the job-def cfg closed over by
+  the handler, not in job params. Closed on purpose, same as the nexus
+  schema."
+  [:map {:closed true}])
+
+(defmethod ig/assert-key ::nexus-job-def
+  [_ cfg]
+  (assert (db/pool? (::db/pool cfg)) "expected a valid database pool")
+  (assert (http/client? (::http/client cfg)) "expected a valid http client")
+  (assert (mtx/metrics? (::mtx/metrics cfg)) "expected a valid metrics instance"))
+
+(defmethod ig/init-key ::nexus-job-def
+  [_ cfg]
+  ;; Effective destination, once at boot: per-run logs carry counts,
+  ;; the runner says the job started, this says where it sends.
+  (l/dbg :hint "audit log consumer job initialized"
+         :consumer "nexus"
+         :uri-host (some-> (cf/get :nexus-uri) u/uri u/get-domain))
+  {::jobs/name      :audit-log-consumer-nexus
+   ::jobs/schema    schema:audit-log-consumer-nexus-params
+   ::jobs/handler
+   (fn [_context params]
+     (execute-audit-log-consumer-nexus cfg params))
+   ::jobs/decoder   (sm/decoder schema:audit-log-consumer-nexus-params sm/json-transformer)
+   ::jobs/validator (sm/validator schema:audit-log-consumer-nexus-params)})
+
+(defmethod ig/assert-key ::admin-console-job-def
+  [_ cfg]
+  ;; No assert on the admin-console client: it inits to nil without the
+  ;; :admin-console flag, and the job skips on its flag anyway.
+  (assert (db/pool? (::db/pool cfg)) "expected a valid database pool")
+  (assert (mtx/metrics? (::mtx/metrics cfg)) "expected a valid metrics instance"))
+
+(defmethod ig/init-key ::admin-console-job-def
+  [_ cfg]
+  ;; Same as the nexus job-def: destination host once at boot. The
+  ;; ingest uri is resolved by nitrate/call at send time (it raises
+  ;; when missing); here it is display-only.
+  (l/dbg :hint "audit log consumer job initialized"
+         :consumer "admin-console"
+         :uri-host (some-> (cf/get :admin-console-uri) u/uri u/get-domain))
+  {::jobs/name      :audit-log-consumer-admin-console
+   ::jobs/schema    schema:audit-log-consumer-admin-console-params
+   ::jobs/handler
+   (fn [_context params]
+     (execute-audit-log-consumer-admin-console cfg params))
+   ::jobs/decoder   (sm/decoder schema:audit-log-consumer-admin-console-params sm/json-transformer)
+   ::jobs/validator (sm/validator schema:audit-log-consumer-admin-console-params)})
+
+(defn- run-single-consumer
+  "Drain one consumer queue, then housekeeping (reap disabled
+  pendings, sweep orphans) and the per-run summary. Housekeeping lives
+  in both jobs: only the enabled job runs, so it must reap the queues
+  of the consumers that are off; both halves are idempotent deletes.
+  A refused chunk (or a throw) aborts the job: the committed chunks
+  stay sent, the rest waits for the next tick, and the cron retries.
+  Every effect is rows and logs (drain metrics land here, outside
+  every chunk transaction)."
+  [cfg consumer send-chunk tpoint]
+  (let [label   (consumer-label consumer)
+        metrics (mtx/instance cfg)
+        reaped  (reap-disabled cfg)
+        ;; All drain metrics land here, outside every chunk
+        ;; transaction. Success is recorded on the totals: a normal
+        ;; chunk return means its transaction committed (a rollback
+        ;; always throws, taking the failure path instead). The send
+        ;; fns carry the attempted count as :events; anything else (a
+        ;; local DB failure) is not a destination failure and stays
+        ;; uncounted. The throw keeps rolling: a failed chunk aborts
+        ;; the job, the cron retries.
+        drained (try
+                  (drain-consumer cfg send-chunk)
+                  (catch Throwable cause
+                    (let [n (:events (ex-data cause))]
+                      (when (and (integer? n) (pos? n))
+                        (record-count metrics :audit-consumer-events [label "failed"] n)))
+                    (throw cause)))
+        _       (let [{:keys [sent skipped]} drained]
+                  (when (pos? sent)
+                    (record-count metrics :audit-consumer-events [label "archived"] sent))
+                  (when (pos? skipped)
+                    (record-count metrics :audit-consumer-events [label "skipped"] skipped)))
+        swept   (sweep-orphans cfg)
+        sent    (:sent drained)
+        skipped (:skipped drained)
+        chunks  (:chunks drained)
+        ;; Strict binding on purpose: log props evaluate lazily in
+        ;; the logger thread, so the elapsed must be frozen here.
+        elapsed (ct/format-duration (tpoint))]
+    (if (and (zero? sent)
+             (zero? skipped)
+             (zero? reaped)
+             (zero? swept))
+      (l/dbg :hint "no pending audit events"
+             :consumer label)
+      (l/dbg :hint "events archived"
+             :consumer label
+             :elapsed elapsed
+             :sent sent
+             :skipped skipped
+             :chunks chunks
+             :reaped reaped
+             :swept swept))))
+
+(defn execute-audit-log-consumer-nexus
+  "Nexus job handler: drain the nexus queue in chunks, then clean up
+  (heartbeat per iteration: the sent chunk batches can be long)."
+  [cfg params]
+  ;; NOTE: this let allows overwrite default configured values from
+  ;; the repl, when manually invoking the task.
+  (if-not (contains? cf/flags :nexus)
+    (l/dbg :hint "audit log consumer (nexus) skipped, flag off")
+    (let [uri (or (:uri params) (generate-nexus-uri "api/audit"))
+          ;; Normalize to an uri object; params may carry a plain string
+          ;; on direct invocations (validation only applies on submit!).
+          uri (u/uri uri)
+          cfg (assoc cfg ::uri uri)]
+      (when-not uri
+        (ex/raise :type :internal
+                  :code :task-not-configured
+                  :hint "consumer task not configured, missing uri"))
+      (run-single-consumer cfg consumer-nexus consumer-nexus-chunk (ct/tpoint)))))
+
+(defn execute-audit-log-consumer-admin-console
+  "Admin-console job handler: drain the admin-console queue in chunks,
+  then clean up (heartbeat per iteration: the sent chunk batches can
+  be long)."
+  [cfg _params]
+  (if-not (contains? cf/flags :admin-console)
+    (l/dbg :hint "audit log consumer (admin-console) skipped, flag off")
+    (run-single-consumer cfg consumer-admin-console consumer-admin-console-chunk (ct/tpoint))))

backend/src/app/loggers/audit/gc_task.clj

@@ -1,46 +0,0 @@
-;; This Source Code Form is subject to the terms of the Mozilla Public
-;; License, v. 2.0. If a copy of the MPL was not distributed with this
-;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
-;;
-;; Copyright (c) KALEIDOS SUBSIDIARY SL
-
-(ns app.loggers.audit.gc-task
-  (:require
-   [app.common.logging :as l]
-   [app.common.schema :as sm]
-   [app.db :as db]
-   [app.jobs :as jobs]
-   [integrant.core :as ig]))
-
-(def ^:private sql:clean-archived
-  "DELETE FROM audit_log
-    WHERE archived_at IS NOT NULL")
-
-(defn- clean-archived!
-  [{:keys [::db/pool]}]
-  (let [result (db/exec-one! pool [sql:clean-archived])
-        result (db/get-update-count result)]
-    (l/debug :hint "delete archived audit log entries" :deleted result)
-    result))
-
-(declare execute-audit-log-gc)
-
-(def schema:audit-log-gc-params
-  "Params map (no params needed; config-derived only)."
-  [:map {:closed true}])
-
-(defmethod ig/init-key ::job-def
-  [_ cfg]
-  {::jobs/name      :audit-log-gc
-   ::jobs/schema    schema:audit-log-gc-params
-   ::jobs/handler
-   (fn [_context params]
-     (execute-audit-log-gc cfg params))
-   ::jobs/decoder   (sm/decoder schema:audit-log-gc-params sm/json-transformer)
-   ::jobs/validator (sm/validator schema:audit-log-gc-params)})
-
-(defn execute-audit-log-gc
-  "Plain job handler: delete the archived audit log entries."
-  ([cfg] (execute-audit-log-gc cfg nil))
-  ([cfg _params]
-   (clean-archived! cfg)))

backend/src/app/loggers/metrics.clj

@@ -0,0 +1,46 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC Sucursal en España SL
+
+(ns app.loggers.metrics
+  "A reporter that counts the log records of this application, by logger and
+  level, and publishes them as a counter."
+  (:require
+   [app.common.logging :as l]
+   [app.metrics :as mtx]
+   [integrant.core :as ig]))
+
+(def ^:private counted-levels
+  "The levels worth counting: `info` is emitted often and says nothing about
+  the health of the application."
+  #{:warn :error :fatal})
+
+(defmethod ig/assert-key ::reporter
+  [_ params]
+  (assert (mtx/metrics? (::mtx/metrics params)) "expect valid metrics instance"))
+
+(defmethod ig/init-key ::reporter
+  [_ {::mtx/keys [metrics]}]
+  (add-watch l/log-record ::reporter
+             (fn [_ _ _ record]
+               ;; This runs inside the logging path: the watcher fires from
+               ;; the `swap!` of the log record, so an exception here would
+               ;; reach whoever was logging on the synchronous path, and on
+               ;; the async one it would die in the executor's uncaught
+               ;; handler, taking that log line out of Loki. Swallowed on
+               ;; purpose either way.
+               (try
+                 (let [level (::l/level record)]
+                   (when (contains? counted-levels level)
+                     (mtx/run! metrics
+                               :id :log-events
+                               :inc 1
+                               :labels (into-array String [(::l/logger record) (name level)]))))
+                 (catch Throwable _ nil))))
+  {::mtx/metrics metrics})
+
+(defmethod ig/halt-key! ::reporter
+  [_ _]
+  (remove-watch l/log-record ::reporter))

backend/src/app/main.clj

@@ -28,7 +28,7 @@
    [app.jobs.gc :as-alias jobs.gc]
    [app.jobs.metrics :as jobs-metrics]
    [app.loggers.webhooks :as-alias webhooks]
-   [app.metrics :as-alias mtx]
+   [app.metrics :as mtx]
    [app.metrics.definition :as-alias mdef]
    [app.msgbus :as-alias mbus]
    [app.redis :as-alias rds]
@@ -54,7 +54,13 @@
   (:gen-class))
 
 (def default-metrics
-  {:update-file-changes
+  {:log-events
+   {::mdef/name "penpot_log_events_total"
+    ::mdef/help "Log records at WARN, ERROR and FATAL emitted by the application, by logger and level."
+    ::mdef/labels ["logger" "level"]
+    ::mdef/type :counter}
+
+   :update-file-changes
    {::mdef/name "penpot_rpc_update_file_changes_total"
     ::mdef/help "A total number of changes submitted to update-file."
     ::mdef/type :counter}
@@ -64,16 +70,25 @@
     ::mdef/help "A total number of bytes processed by update-file."
     ::mdef/type :counter}
 
+   :update-file-validation-timing
+   {::mdef/name "penpot_rpc_update_file_validation_timing"
+    ::mdef/help "update-file validation step timings (milliseconds)."
+    ::mdef/labels ["step"]
+    ::mdef/buckets mtx/default-request-histogram-buckets
+    ::mdef/type :histogram}
+
    :rpc-main-timing
    {::mdef/name "penpot_rpc_main_timing"
     ::mdef/help "RPC command method call timing for main"
     ::mdef/labels ["name"]
+    ::mdef/buckets mtx/default-request-histogram-buckets
     ::mdef/type :histogram}
 
    :rpc-management-timing
    {::mdef/name "penpot_rpc_management_timing"
     ::mdef/help "RPC command method call timing for management."
     ::mdef/labels ["name"]
+    ::mdef/buckets mtx/default-request-histogram-buckets
     ::mdef/type :histogram}
 
    :websocket-active-connections
@@ -106,12 +121,20 @@
     ;; panel or alert breaks.
     ::mdef/help "Deprecated: use penpot_jobs_execution_ms, which adds a queue label."
     ::mdef/labels ["name"]
+    ::mdef/buckets mtx/default-tasks-histogram-buckets
     ::mdef/type :histogram}
 
+   :storage-gc-deleted-objects
+   {::mdef/name "penpot_storage_gc_deleted_objects_total"
+    ::mdef/help "Storage objects permanently deleted by storage-gc-deleted, by backend."
+    ::mdef/labels ["backend"]
+    ::mdef/type :counter}
+
    :redis-eval-timing
    {::mdef/name "penpot_redis_eval_timing"
     ::mdef/help "Redis EVAL commands execution timings (ms)"
     ::mdef/labels ["name"]
+    ::mdef/buckets mtx/default-redis-histogram-buckets
     ::mdef/type :histogram}
 
    :rpc-climit-waiting
@@ -130,6 +153,7 @@
    {::mdef/name "penpot_rpc_climit_timing"
     ::mdef/help "Summary of the time between queuing and executing on the CLIMIT"
     ::mdef/labels ["name"]
+    ::mdef/buckets mtx/default-request-histogram-buckets
     ::mdef/type :histogram}
 
    :audit-http-handler-queue-size
@@ -148,6 +172,7 @@
    {::mdef/name "penpot_audit_http_handler_timing"
     ::mdef/help "Summary of the time between queuing and executing on the audit log http handler"
     ::mdef/labels []
+    ::mdef/buckets mtx/default-request-histogram-buckets
     ::mdef/type :histogram}
 
    :storage-s3-requests
@@ -197,6 +222,7 @@
    {::mdef/name "penpot_http_server_dispatch_timing"
     ::mdef/help "Histogram of dispatch handler"
     ::mdef/labels []
+    ::mdef/buckets mtx/default-request-histogram-buckets
     ::mdef/type :histogram}
 
    :http-worker-queue-size
@@ -340,6 +366,48 @@
     ::mdef/help "Time spent waiting for an ephemeral job reply (milliseconds). Its _count is the request total per outcome."
     ::mdef/labels ["outcome"]
     ::mdef/type :histogram
+    ::mdef/buckets [1 10 50 100 500 1000 5000 10000 30000 60000 300000]}
+
+   :http-malformed-requests-total
+   {::mdef/name "penpot_http_malformed_requests_total"
+    ::mdef/help "Total number of requests whose body the parser could not read."
+    ::mdef/type :counter}
+
+   :audit-consumer-events
+   {::mdef/name "penpot_audit_consumer_events_total"
+    ::mdef/help "Audit log events drained by the consumer task, by destination and outcome."
+    ::mdef/labels ["destination" "outcome"]
+    ::mdef/type :counter}
+
+   :audit-consumer-discarded
+   {::mdef/name "penpot_audit_consumer_discarded_total"
+    ::mdef/help "Audit log entries discarded without being archived, by reason."
+    ::mdef/labels ["reason"]
+    ::mdef/type :counter}
+
+   :audit-ingest-events
+   {::mdef/name "penpot_audit_ingest_events_total"
+    ::mdef/help "Audit log frontend events ingested, by kind and outcome."
+    ::mdef/labels ["kind" "outcome"]
+    ::mdef/type :counter}
+
+   :file-data-offload-rows
+   {::mdef/name "penpot_file_data_offload_rows_total"
+    ::mdef/help "File data rows moved from hot to cold storage by the offload job."
+    ::mdef/labels ["type"]
+    ::mdef/type :counter}
+
+   :file-data-offload-bytes
+   {::mdef/name "penpot_file_data_offload_bytes_total"
+    ::mdef/help "File data bytes moved from hot to cold storage by the offload job."
+    ::mdef/labels ["type"]
+    ::mdef/type :counter}
+
+   :file-data-offload-timing
+   {::mdef/name "penpot_file_data_offload_timing_ms"
+    ::mdef/help "Time spent offloading one file from hot to cold storage (milliseconds)."
+    ::mdef/labels ["result"]
+    ::mdef/type :histogram
     ::mdef/buckets [1 10 50 100 500 1000 5000 10000 30000 60000 300000]}})
 
 (def system-config
@@ -460,6 +528,7 @@
     ::rpc/routes         (ig/ref ::rpc/routes)
     ::setup/props        (ig/ref ::setup/props)
     ::mtx/routes         (ig/ref ::mtx/routes)
+    ::mtx/metrics        (ig/ref ::mtx/metrics)
     ::oidc/routes        (ig/ref ::oidc/routes)
     ::mgmt/routes        (ig/ref ::mgmt/routes)
     ::http.debug/routes  (ig/ref ::http.debug/routes)
@@ -543,7 +612,8 @@
 
    :app.nitrate/client
    {::http.client/client (ig/ref ::http.client/client)
-    ::jobs/defs          (ig/ref ::jobs/defs)
+    ;; No ::jobs/defs: nitrate never submits jobs, and consumer-task's
+    ;; job-def needs this client — that would cycle through jobs/defs.
     ::setup/shared-keys  (ig/ref ::setup/shared-keys)}
 
    :app.rpc/management-methods
@@ -571,25 +641,25 @@
     ::setup/shared-keys      (ig/ref ::setup/shared-keys)}
 
    ::jobs/defs
-   {:sendmail              (ig/ref ::email/job-def)
-    :delete-object         (ig/ref :app.tasks.delete-object/job-def)
-    :demo-purge            (ig/ref :app.tasks.demo-purge/job-def)
-    :run-webhook           (ig/ref ::webhooks/run-webhook-job-def)
-    :process-webhook-event (ig/ref ::webhooks/process-webhook-event-job-def)
-    :file-gc               (ig/ref :app.tasks.file-gc/job-def)
-    :offload-file-data     (ig/ref :app.tasks.offload-file-data/job-def)
-    :objects-gc            (ig/ref :app.tasks.objects-gc/job-def)
-    :export-binfile        (ig/ref :app.tasks.export-binfile/job-def)
-    :import-binfile        (ig/ref :app.tasks.import-binfile/job-def)
-    :storage-gc-deleted    (ig/ref ::sto.gc-deleted/job-def)
-    :storage-gc-touched    (ig/ref ::sto.gc-touched/job-def)
-    :storage-pending-gc    (ig/ref ::sto.pending-gc/job-def)
-    :jobs-gc               (ig/ref :app.jobs.gc/job-def)
-    :telemetry             (ig/ref :app.tasks.telemetry/job-def)
-    :session-gc            (ig/ref ::session/job-def)
-    :file-gc-scheduler     (ig/ref :app.tasks.file-gc-scheduler/job-def)
-    :audit-log-archive     (ig/ref :app.loggers.audit.archive-task/job-def)
-    :audit-log-gc          (ig/ref :app.loggers.audit.gc-task/job-def)}
+   {:sendmail                       (ig/ref ::email/job-def)
+    :delete-object                  (ig/ref :app.tasks.delete-object/job-def)
+    :demo-purge                     (ig/ref :app.tasks.demo-purge/job-def)
+    :run-webhook                    (ig/ref ::webhooks/run-webhook-job-def)
+    :process-webhook-event          (ig/ref ::webhooks/process-webhook-event-job-def)
+    :file-gc                        (ig/ref :app.tasks.file-gc/job-def)
+    :offload-file-data              (ig/ref :app.tasks.offload-file-data/job-def)
+    :objects-gc                     (ig/ref :app.tasks.objects-gc/job-def)
+    :export-binfile                 (ig/ref :app.tasks.export-binfile/job-def)
+    :import-binfile                 (ig/ref :app.tasks.import-binfile/job-def)
+    :storage-gc-deleted             (ig/ref ::sto.gc-deleted/job-def)
+    :storage-gc-touched             (ig/ref ::sto.gc-touched/job-def)
+    :storage-pending-gc             (ig/ref ::sto.pending-gc/job-def)
+    :jobs-gc                        (ig/ref :app.jobs.gc/job-def)
+    :telemetry                      (ig/ref :app.tasks.telemetry/job-def)
+    :session-gc                     (ig/ref ::session/job-def)
+    :file-gc-scheduler              (ig/ref :app.tasks.file-gc-scheduler/job-def)
+    :audit-log-consumer-nexus        (ig/ref :app.loggers.audit.consumer-task/nexus-job-def)
+    :audit-log-consumer-admin-console (ig/ref :app.loggers.audit.consumer-task/admin-console-job-def)}
 
    ::jobs/request-pool
    {::rds/client  (ig/ref ::rds/client)
@@ -678,15 +748,16 @@
    {::db/pool     (ig/ref ::db/pool)
     ::mtx/metrics (ig/ref ::mtx/metrics)}
 
-   :app.loggers.audit.archive-task/job-def
-   {::db/pool          (ig/ref ::db/pool)
+   :app.loggers.audit.consumer-task/nexus-job-def
+   {::db/pool            (ig/ref ::db/pool)
     ::http.client/client (ig/ref ::http.client/client)
-    ::setup/shared-keys (ig/ref ::setup/shared-keys)
-    ::mtx/metrics      (ig/ref ::mtx/metrics)}
+    ::setup/shared-keys  (ig/ref ::setup/shared-keys)
+    ::mtx/metrics        (ig/ref ::mtx/metrics)}
 
-   :app.loggers.audit.gc-task/job-def
-   {::db/pool     (ig/ref ::db/pool)
-    ::mtx/metrics (ig/ref ::mtx/metrics)}
+   :app.loggers.audit.consumer-task/admin-console-job-def
+   {::db/pool            (ig/ref ::db/pool)
+    :app.nitrate/client  (ig/ref :app.nitrate/client)
+    ::mtx/metrics        (ig/ref ::mtx/metrics)}
 
    ::email/blacklist
    {}
@@ -743,6 +814,9 @@
    :app.loggers.database/reporter
    {::db/pool (ig/ref ::db/pool)}
 
+   :app.loggers.metrics/reporter
+   {::mtx/metrics (ig/ref ::mtx/metrics)}
+
    ::sto/storage
    {::db/pool      (ig/ref ::db/pool)
     ::mtx/metrics  (ig/ref ::mtx/metrics)
@@ -804,15 +878,13 @@
      {:cron #penpot/cron "0 30 */3,23 * * ?"
       :job :telemetry}
 
-     (when (or (contains? cf/flags :nexus)
-               (contains? cf/flags :admin-console))
+     (when (contains? cf/flags :nexus)
        {:cron #penpot/cron "0 */5 * * * ?" ;; every 5m
-        :job :audit-log-archive})
+        :job :audit-log-consumer-nexus})
 
-     (when (or (contains? cf/flags :nexus)
-               (contains? cf/flags :admin-console))
-       {:cron #penpot/cron "30 */5 * * * ?" ;; every 5m
-        :job :audit-log-gc})]}
+     (when (contains? cf/flags :admin-console)
+       {:cron #penpot/cron "30 */5 * * * ?" ;; every 5m, offset 30s from nexus
+        :job :audit-log-consumer-admin-console})]}
 
    ::wrk/dispatcher
    {::rds/client  (ig/ref ::rds/client)

backend/src/app/metrics.clj

@@ -148,6 +148,25 @@
     (number? value)  (str value)
     :else            fallback))
 
+;; Tail for request and wait timings (RPC, CLIMIT waits, audit handler,
+;; HTTP dispatch): covers the 30s/60s CLIMIT timeouts while keeping the
+;; sub-7.5s resolution of the default buckets untouched.
+(def default-request-histogram-buckets
+  [1 5 10 25 50 75 100 250 500 750 1000 2500 5000 7500
+   10000 15000 30000 60000])
+
+;; Tail for background tasks (GC sweeps, file workers): they run until
+;; done with no CLIMIT timeout, bounded in practice by the 300s postgres
+;; statement timeout.
+(def default-tasks-histogram-buckets
+  [1 5 10 25 50 75 100 250 500 750 1000 2500 5000 7500
+   10000 15000 30000 60000 120000 300000])
+
+;; Tail for Redis eval: sub-second in practice, one extra bucket covers
+;; the pathological (timeout) case without adding idle series.
+(def default-redis-histogram-buckets
+  [1 5 10 25 50 75 100 250 500 750 1000 2500 5000 7500 10000])
+
 (defmulti run-collector! (fn [mdef _] (::mdef/type mdef)))
 (defmulti create-collector ::mdef/type)
 

backend/src/app/migrations.clj

@@ -526,6 +526,15 @@
    {:name "0154-del-legacy-file-object-thumbnail-table"
     :fn (mg/resource "app/migrations/sql/0154-del-legacy-file-object-thumbnail-table.sql")}
 
+   {:name "0155-add-idx-to-file-change"
+    :fn (mg/resource "app/migrations/sql/0155-add-idx-to-file-change.sql")}
+
+   {:name "0156-reset-pg-storage-defaults"
+    :fn (mg/resource "app/migrations/sql/0156-reset-pg-storage-defaults.sql")}
+
+   {:name "0157-add-audit-log-pending-table"
+    :fn (mg/resource "app/migrations/sql/0157-add-audit-log-pending-table.sql")}
+
    {:name "0800-add-file-branch-table"
     :fn (mg/resource "app/migrations/sql/0800-add-file-branch-table.sql")}
 

backend/src/app/migrations/sql/0155-add-idx-to-file-change.sql

@@ -0,0 +1,38 @@
+-- Index for the version (snapshot) quota checks of app.rpc.quotes:
+-- sql:get-snapshots-per-file and sql:get-snapshots-per-team count the live
+-- versions created by a user, and this partial index answers both counts
+-- with an index only scan, without walking the whole file_change table.
+--
+-- The predicate must stay identical to the queries: the planner only uses a
+-- partial index when the query where clause implies the index predicate.
+-- Versions are told apart by label (every snapshot has one); file_change.data
+-- is a legacy column, only written by the legacy-db file data backend.
+
+CREATE INDEX IF NOT EXISTS file_change__user_snapshots__idx
+    ON file_change (file_id)
+ WHERE label IS NOT NULL
+   AND created_by = 'user'
+   AND deleted_at IS NULL;
+
+-- NOTE: on instances where file_change is large, run the statement above
+-- out of band, before deploying, with CONCURRENTLY instead: this migration
+-- runs inside a transaction, and a plain CREATE INDEX locks the table
+-- against writes for the whole build (in Penpot every file save writes a row
+-- here), while CONCURRENTLY does not. Run it on its own, not inside a
+-- transaction block:
+--
+--   CREATE INDEX CONCURRENTLY IF NOT EXISTS file_change__user_snapshots__idx
+--       ON file_change (file_id)
+--    WHERE label IS NOT NULL
+--      AND created_by = 'user'
+--      AND deleted_at IS NULL;
+--
+-- A CONCURRENTLY build interrupted midway leaves the index INVALID: drop it
+-- and retry. Verify with:
+--
+--   SELECT indexdef FROM pg_indexes
+--    WHERE indexname = 'file_change__user_snapshots__idx';
+--   SELECT indisvalid FROM pg_index
+--    WHERE indexrelid = 'file_change__user_snapshots__idx'::regclass;
+--
+-- Once the index exists, this migration finds it and does nothing.

backend/src/app/migrations/sql/0156-reset-pg-storage-defaults.sql

@@ -0,0 +1,131 @@
+-- Reset explicit `STORAGE EXTERNAL` settings back to the PostgreSQL defaults.
+--
+-- Those were set when PostgreSQL ran on top of ZFS with transparent
+-- compression (see 0041-mod-pg-storage-options.sql and the per-table
+-- migrations that followed it): skipping the TOAST compression done by
+-- PostgreSQL made sense there. On regular filesystems it only wastes
+-- space and I/O bandwidth, so restore the default storage strategy of
+-- each column type (EXTENDED for every column below: text, text[],
+-- jsonb and bytea).
+--
+-- This only touches the catalog, it does not rewrite the tables:
+-- already stored TOAST values keep their strategy until the row is
+-- rewritten, new writes use the default.
+--
+-- NOTE: `SET STORAGE DEFAULT` would express the same thing (it restores
+-- the storage mode of the column's data type), but it only exists since
+-- PostgreSQL 16 and the project also runs on PostgreSQL 15. Here every
+-- column below has a type whose default storage mode is EXTENDED
+-- (text, text[], jsonb and bytea), so `SET STORAGE EXTENDED` reaches the
+-- same catalog state on any supported version. RESET STORAGE is not
+-- valid syntax for a column.
+
+ALTER TABLE access_token
+  ALTER COLUMN name SET STORAGE EXTENDED,
+  ALTER COLUMN perms SET STORAGE EXTENDED,
+  ALTER COLUMN token SET STORAGE EXTENDED;
+
+ALTER TABLE audit_log
+  ALTER COLUMN context SET STORAGE EXTENDED,
+  ALTER COLUMN name SET STORAGE EXTENDED,
+  ALTER COLUMN props SET STORAGE EXTENDED,
+  ALTER COLUMN source SET STORAGE EXTENDED,
+  ALTER COLUMN type SET STORAGE EXTENDED;
+
+ALTER TABLE comment
+  ALTER COLUMN content SET STORAGE EXTENDED;
+
+ALTER TABLE comment_thread
+  ALTER COLUMN page_name SET STORAGE EXTENDED,
+  ALTER COLUMN participants SET STORAGE EXTENDED;
+
+ALTER TABLE file
+  ALTER COLUMN data SET STORAGE EXTENDED,
+  ALTER COLUMN data_backend SET STORAGE EXTENDED,
+  ALTER COLUMN features SET STORAGE EXTENDED,
+  ALTER COLUMN name SET STORAGE EXTENDED;
+
+ALTER TABLE file_change
+  ALTER COLUMN changes SET STORAGE EXTENDED,
+  ALTER COLUMN data SET STORAGE EXTENDED,
+  ALTER COLUMN features SET STORAGE EXTENDED,
+  ALTER COLUMN label SET STORAGE EXTENDED;
+
+ALTER TABLE file_data_fragment
+  ALTER COLUMN metadata SET STORAGE EXTENDED;
+
+ALTER TABLE file_media_object
+  ALTER COLUMN mtype SET STORAGE EXTENDED,
+  ALTER COLUMN name SET STORAGE EXTENDED;
+
+ALTER TABLE file_thumbnail
+  ALTER COLUMN data SET STORAGE EXTENDED,
+  ALTER COLUMN props SET STORAGE EXTENDED;
+
+ALTER TABLE global_complaint_report
+  ALTER COLUMN content SET STORAGE EXTENDED,
+  ALTER COLUMN type SET STORAGE EXTENDED;
+
+ALTER TABLE profile
+  ALTER COLUMN auth_backend SET STORAGE EXTENDED,
+  ALTER COLUMN email SET STORAGE EXTENDED,
+  ALTER COLUMN fullname SET STORAGE EXTENDED,
+  ALTER COLUMN lang SET STORAGE EXTENDED,
+  ALTER COLUMN password SET STORAGE EXTENDED,
+  ALTER COLUMN props SET STORAGE EXTENDED,
+  ALTER COLUMN theme SET STORAGE EXTENDED;
+
+ALTER TABLE profile_complaint_report
+  ALTER COLUMN content SET STORAGE EXTENDED,
+  ALTER COLUMN type SET STORAGE EXTENDED;
+
+ALTER TABLE project
+  ALTER COLUMN name SET STORAGE EXTENDED;
+
+ALTER TABLE scheduled_task
+  ALTER COLUMN cron_expr SET STORAGE EXTENDED,
+  ALTER COLUMN id SET STORAGE EXTENDED;
+
+ALTER TABLE server_error_report
+  ALTER COLUMN content SET STORAGE EXTENDED;
+
+ALTER TABLE server_prop
+  ALTER COLUMN content SET STORAGE EXTENDED,
+  ALTER COLUMN id SET STORAGE EXTENDED;
+
+ALTER TABLE storage_object
+  ALTER COLUMN backend SET STORAGE EXTENDED,
+  ALTER COLUMN metadata SET STORAGE EXTENDED;
+
+ALTER TABLE task
+  ALTER COLUMN error SET STORAGE EXTENDED,
+  ALTER COLUMN label SET STORAGE EXTENDED,
+  ALTER COLUMN name SET STORAGE EXTENDED,
+  ALTER COLUMN props SET STORAGE EXTENDED,
+  ALTER COLUMN queue SET STORAGE EXTENDED,
+  ALTER COLUMN status SET STORAGE EXTENDED;
+
+ALTER TABLE team
+  ALTER COLUMN name SET STORAGE EXTENDED;
+
+ALTER TABLE team_font_variant
+  ALTER COLUMN font_family SET STORAGE EXTENDED,
+  ALTER COLUMN font_style SET STORAGE EXTENDED;
+
+ALTER TABLE team_invitation
+  ALTER COLUMN email_to SET STORAGE EXTENDED,
+  ALTER COLUMN role SET STORAGE EXTENDED;
+
+ALTER TABLE usage_quote
+  ALTER COLUMN target SET STORAGE EXTENDED;
+
+ALTER TABLE webhook
+  ALTER COLUMN error_code SET STORAGE EXTENDED,
+  ALTER COLUMN mtype SET STORAGE EXTENDED,
+  ALTER COLUMN secret_key SET STORAGE EXTENDED,
+  ALTER COLUMN uri SET STORAGE EXTENDED;
+
+ALTER TABLE webhook_delivery
+  ALTER COLUMN error_code SET STORAGE EXTENDED,
+  ALTER COLUMN req_data SET STORAGE EXTENDED,
+  ALTER COLUMN rsp_data SET STORAGE EXTENDED;

backend/src/app/migrations/sql/0157-add-audit-log-pending-table.sql

@@ -0,0 +1,39 @@
+-- Work queue for the audit log consumers: one row per (record,
+-- consumer) pair still pending. Inserting a record creates its
+-- pendings; each consumer deletes its own on completion, and whoever
+-- deletes the last pending also deletes the record. Consumer ids are
+-- fixed in code and never reused: 1 = nexus, 2 = admin console.
+
+CREATE TABLE audit_log_pending (
+  -- DEFERRABLE so the test-suite reset (which deletes table by table
+  -- with SET CONSTRAINTS ALL DEFERRED) keeps working; the guard still
+  -- fires when a record with pendings is deleted. Plain DEFERRABLE
+  -- (initially immediate) is enough: every deleter removes pendings
+  -- before rows, so the check passes at each statement.
+  audit_log_id uuid     NOT NULL REFERENCES audit_log(id) DEFERRABLE,
+  consumer     smallint NOT NULL CHECK (consumer BETWEEN 1 AND 20),
+  PRIMARY KEY (consumer, audit_log_id));
+
+-- Lets the final delete ask "is anything still pending for this
+-- record?" without scanning the whole queue.
+CREATE INDEX ON audit_log_pending (audit_log_id);
+
+-- NOTE: no backfill on purpose. Rows predating this queue (sent or
+-- not) get no pendings, so the orphan sweep deletes them unsent.
+-- Losing the unarchived ones at upgrade time is accepted: this
+-- table is a minutes/hours buffer, not a store.
+
+-- The pending queue replaces the archived_at mark: whoever deletes
+-- the last pending deletes the record, so the flag column and the
+-- indexes that served the old mark-and-sweep scans go away.
+DROP INDEX IF EXISTS audit_log__created_at__idx;
+DROP INDEX IF EXISTS audit_log__archived_at__idx;
+DROP INDEX IF EXISTS audit_log_id_archived_at_idx;
+ALTER TABLE audit_log DROP COLUMN IF EXISTS archived_at;
+
+-- Serves the consumers' chronological scan (ORDER BY created_at, id):
+-- one total order for every consumer, so concurrent drains cannot
+-- deadlock on row locks. Covers the whole ORDER BY (bulk ingest
+-- shares request-at constantly, so ties are the norm).
+CREATE INDEX IF NOT EXISTS audit_log__created_at__idx
+    ON audit_log (created_at ASC, id);

backend/src/app/nitrate.clj

@@ -483,8 +483,8 @@
                       (assoc params :throw-on-error? true)))
 
 (def ^:private audit-event-keys
-  [:id :name :type :profile-id :props :context
-   :created-at :tracked-at :source :ip-addr])
+  [:id :name :type :profile-id :profile-email :profile-name
+   :props :context :created-at :tracked-at :source :ip-addr])
 
 (defn- ->audit-event
   [event]

backend/src/app/redis.clj

@@ -419,7 +419,7 @@
     (try
       (-del conn keys)
       (catch RedisCommandTimeoutException cause
-        (l/err :hint "timeout on del redis key" :key key :cause cause)
+        (l/err :hint "timeout on del redis key" :keys keys :cause cause)
         nil))))
 
 (defn ping

backend/src/app/rpc.clj

@@ -126,7 +126,15 @@
                                (-> (assoc ::auth-type :token)
                                    (assoc ::token-perms (set (::actoken/perms request #{})))))
                              (cond-> key-id
-                               (assoc ::auth-key-id key-id)))
+                               (assoc ::auth-key-id key-id))
+                             ;; Server-side only: request bodies cannot forge
+                             ;; it (qualified keys are stripped before the
+                             ;; merge). Upload maps are only trusted when they
+                             ;; arrive over multipart.
+                             (assoc ::is-multipart
+                                    (str/starts-with?
+                                     (str (yreq/get-header request "content-type"))
+                                     "multipart/")))
 
             data         (with-meta data
                            {::http/request request})
@@ -227,10 +235,23 @@
       (fn [cfg params]
         (let [request-params (-> params meta ::http/request :params decode)]
           (if (validate request-params)
-            (let [result (f cfg (merge params (d/without-qualified request-params)))]
-              (if (instance? clojure.lang.IObj result)
-                (vary-meta result assoc :encode/json encode)
-                result))
+            (let [merged (merge params (d/without-qualified request-params))]
+              ;; Upload temp files are created server-side by the multipart
+              ;; parser, so an upload map smuggled in a transit/JSON body can
+              ;; point :path at any internal server file. Methods declare
+              ;; their upload keys with ::multipart-only-params and they are
+              ;; only accepted when ::is-multipart arrived server-side with
+              ;; the request (request bodies cannot forge qualified keys).
+              (when (some #(and (contains? merged %)
+                                (not (::is-multipart merged)))
+                          (::multipart-only-params mdata))
+                (ex/raise :type :validation
+                          :code :params-validation
+                          :hint "upload requires multipart"))
+              (let [result (f cfg merged)]
+                (if (instance? clojure.lang.IObj result)
+                  (vary-meta result assoc :encode/json encode)
+                  result)))
             (ex/raise :type :validation
                       :code :params-validation
                       ::sm/explain (explain request-params))))))
@@ -429,7 +450,8 @@
   [cfg]
   (let [cfg  (assoc cfg ::module "management" ::type "command" ::metrics-id :rpc-management-timing)
         mods (cond->> (list 'app.rpc.management.exporter
-                            'app.rpc.management.jobs)
+                            'app.rpc.management.jobs
+                            'app.rpc.management.files)
                (contains? cf/flags :admin-console)
                (cons 'app.rpc.management.nitrate))]
 

backend/src/app/rpc/commands/audit.clj

@@ -16,8 +16,10 @@
    [app.db :as db]
    [app.http :as-alias http]
    [app.loggers.audit :as audit]
+   [app.loggers.audit.consumer-task :as consumer]
    [app.loggers.database :as loggers.db]
    [app.loggers.mattermost :as loggers.mm]
+   [app.metrics :as mtx]
    [app.rpc :as-alias rpc]
    [app.rpc.climit :as-alias climit]
    [app.rpc.doc :as-alias doc]
@@ -71,6 +73,9 @@
 (def ^:private xf:map-event-row
   (comp
    (map adjust-timestamp)
+   ;; Each row mints its own id: the full row and its telemetry
+   ;; shadow are distinct records that must not share it.
+   (map #(assoc % :id (uuid/next)))
    (map event->row)))
 
 (defn- prepare-events
@@ -80,8 +85,7 @@
         initiator (audit/get-client-initiator request)
         xform     (comp
                    (map (fn [event]
-                          {:id (uuid/next)
-                           :type (:type event)
+                          {:type (:type event)
                            :name (:name event)
                            :props (:props event)
                            ;; The initiator always comes from the
@@ -113,6 +117,19 @@
               (assoc :source "telemetry:frontend"))))
    (map event->row)))
 
+(defn- record-ingested
+  [metrics kind outcome amount]
+  (when (pos? amount)
+    (mtx/run! metrics :id :audit-ingest-events :labels [kind outcome] :inc amount)))
+
+(defn- insert-rows!
+  [tx rows]
+  (when (seq rows)
+    (db/insert-many! tx :audit-log event-columns rows)
+    ;; Same transaction as the rows: every ingested record is born
+    ;; pending for every enabled consumer (see consumer-task).
+    (consumer/queue-pendings tx (map first rows))))
+
 (defn- handle-events
   [{:keys [::db/pool] :as cfg} params]
   (let [events (prepare-events params)]
@@ -124,19 +141,30 @@
       (run! (partial loggers.db/emit cfg) events)
       (run! (partial loggers.mm/emit cfg) events))
 
-    (when (contains? cf/flags :audit-log)
-      ;; Process and save full audit events when audit-log flag is active
-      (when-let [rows (-> (sequence xf:map-event-row events)
-                          (not-empty))]
-        (db/insert-many! pool :audit-log event-columns rows)))
-
-    (when (contains? cf/flags :telemetry)
-      ;; Store anonymized frontend events so the telemetry task can ship them
-      ;; in batches. Runs independently from the audit-log insert above so
-      ;; both modes can be active simultaneously.
-      (when-let [rows (-> (sequence xf:map-telemetry-event-row events)
-                          (not-empty))]
-        (db/insert-many! pool :audit-log event-columns rows)))))
+    ;; Rows and their pendings commit together in one transaction: a
+    ;; row never outlives its queue. Both faces are best-effort, so a
+    ;; failure drops the whole batch (counted below as failed); the
+    ;; endpoint still answers without breaking the request.
+    (let [metrics     (mtx/instance cfg)
+          full-rows   (if (contains? cf/flags :audit-log)
+                        (into [] xf:map-event-row events)
+                        [])
+          shadow-rows (if (contains? cf/flags :telemetry)
+                        (into [] xf:map-telemetry-event-row events)
+                        [])]
+      (try
+        (db/tx-run! pool
+                    (fn [tx]
+                      (insert-rows! tx full-rows)
+                      (insert-rows! tx shadow-rows)))
+        (record-ingested metrics "full" "inserted" (count full-rows))
+        (record-ingested metrics "shadow" "inserted" (count shadow-rows))
+        (catch Throwable cause
+          ;; Whatever never landed counts as failed per attempted
+          ;; kind; the endpoint logs the cause as before.
+          (record-ingested metrics "full" "failed" (count full-rows))
+          (record-ingested metrics "shadow" "failed" (count shadow-rows))
+          (throw cause))))))
 
 (def ^:private valid-event-types
   #{"action" "identify" "trigger"})

backend/src/app/rpc/commands/binfile.clj

@@ -16,7 +16,7 @@
    [app.config :as cf]
    [app.db :as db]
    [app.http.sse :as sse]
-   [app.loggers.audit :as-alias audit]
+   [app.loggers.audit :as audit]
    [app.loggers.webhooks :as-alias webhooks]
    [app.media.validation :as media.v]
    [app.rpc :as-alias rpc]
@@ -83,7 +83,7 @@
 ;; --- Command: import-binfile
 
 (defn- import-binfile
-  [{:keys [::db/pool] :as cfg} {:keys [profile-id project-id version name file upload-id]}]
+  [{:keys [::db/pool] :as cfg} {:keys [profile-id project-id version name file upload-id] :as params}]
   (let [team
         (teams/get-team pool
                         :profile-id profile-id
@@ -116,6 +116,14 @@
                 {:id project-id}
                 {::db/return-keys false})
 
+    ;; The imported files get no event from their own command, so emit one per
+    ;; file inside a single transaction.
+    (db/tx-run! cfg
+                (fn [cfg]
+                  (doseq [file-id (:file-ids result)]
+                    (audit/submit-file-created-event cfg params file-id
+                                                     {:team-id (:id team)}))))
+
     result))
 
 (def ^:private schema:import-binfile
@@ -145,6 +153,7 @@
    ::webhooks/event? true
    ::sse/stream? true
    ::sm/params schema:import-binfile
+   ::rpc/multipart-only-params [:file]
    ::climit/id [[:import-binfile/by-profile ::rpc/profile-id]
                 [:import-binfile/global]]}
   [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id project-id version upload-id] :as params}]

backend/src/app/rpc/commands/comments.clj

@@ -489,7 +489,7 @@
     (let [params (assoc params ::file file)
           thread (db/tx-run! cfg create-comment-thread params)]
 
-      (vary-meta thread assoc ::audit/props thread))))
+      (vary-meta thread assoc ::audit/props (assoc thread :team-id team-id)))))
 
 (defn- create-comment-thread
   [{:keys [::db/conn] :as cfg}
@@ -653,7 +653,7 @@
 
       (send-comment-emails! cfg profile comment thread file)
 
-      (vary-meta comment assoc ::audit/props comment))))
+      (vary-meta comment assoc ::audit/props (assoc comment :team-id team-id)))))
 
 ;; --- COMMAND: Update Comment
 

backend/src/app/rpc/commands/files.clj

@@ -613,11 +613,21 @@
 
   (let [process-row
         (fn [{:keys [id library-file-ids]}]
-          (let [file (get-file-with-summary cfg id)]
-            (assoc file :library-file-ids (db/decode-pgarray library-file-ids #{}))))
+          ;; One shared file that fails to load (for example, a migration
+          ;; error on old data) must not break the whole list; skip it
+          ;; and report it with its id.
+          (try
+            (let [file (get-file-with-summary cfg id)]
+              (assoc file :library-file-ids (db/decode-pgarray library-file-ids #{})))
+            (catch Exception cause
+              (l/error :hint "unable to load shared file"
+                       :file-id (str id)
+                       :team-id (str team-id)
+                       :cause cause)
+              nil)))
 
         xform
-        (map process-row)]
+        (keep process-row)]
 
     (->> (db/plan conn [sql:team-shared-files team-id] {:fetch-size 1})
          (transduce xform conj #{}))))
@@ -1020,7 +1030,11 @@
     (let [file (-> (bfc/get-file cfg file-id
                                  :include-deleted? true
                                  :lock-for-update? true)
-                   (update :data ctf/absorb-assets ldata))]
+                   (update :data ctf/absorb-assets ldata)
+                   (update :data (fn [data]
+                                   (cond-> data
+                                     (cfo/effective-tokens-source? data (:id ldata))
+                                     (cfo/absorb-tokens-lib ldata)))))]
 
       (l/trc :hint "library absorbed"
              :library-id (str (:id ldata))
@@ -1035,16 +1049,21 @@
                                    :modified-at (ct/now)
                                    :has-media-trimmed false)))))
 
+(defn- get-referenced-files
+  [cfg library-id]
+  (->> (db/exec! cfg [sql:get-referenced-files library-id])
+       (into [] bfc/xf-map-id)))
+
 (defn- absorb-library*
-  "Find all files using a shared library, and absorb all library assets
-  into the file local libraries"
-  [cfg {:keys [id data] :as library}]
+  "Absorb all library assets, and its tokens lib if it is their tokens
+  source, into the local libraries of the files that use it. The files
+  are the given file-ids, or the ones linked to the library if nil."
+  [cfg {:keys [id data] :as library} file-ids]
 
   (assert (db/connection-map? cfg)
           "expected cfg with valid connection")
 
-  (let [ids (->> (db/exec! cfg [sql:get-referenced-files id])
-                 (sequence bfc/xf-map-id))]
+  (let [ids (or file-ids (get-referenced-files cfg id))]
 
     (l/trc :hint "absorbing library"
            :library-id (str id)
@@ -1054,7 +1073,9 @@
     library))
 
 (defn absorb-library
-  [{:keys [::db/conn] :as cfg} id]
+  "Absorb the library into the files that use it. The files are the given
+  file-ids, or the ones linked to the library if omitted."
+  [{:keys [::db/conn] :as cfg} id & {:keys [file-ids]}]
   (let [file (-> (bfc/get-file cfg id
                                :realize? true
                                :lock-for-update? true
@@ -1070,7 +1091,7 @@
     (-> (cfeat/get-team-enabled-features cf/flags team)
         (cfeat/check-file-features! (:features file)))
 
-    (absorb-library* cfg file)))
+    (absorb-library* cfg file file-ids)))
 
 (defn- set-file-shared
   [{:keys [::db/conn] :as cfg} {:keys [profile-id id] :as params}]
@@ -1143,7 +1164,10 @@
                           {:deleted-at (ct/in-future delay)}
                           {:id file-id}
                           {::db/return-keys [:id :name :is-shared :deleted-at
-                                             :project-id :created-at :modified-at]})]
+                                             :project-id :created-at :modified-at]})
+        ;; Files using the library, captured before their relations are removed
+        file-ids (when (:is-shared file)
+                   (get-referenced-files conn file-id))]
 
     ;; Remove all possible relations for that file
     (db/delete! conn :file-library-rel
@@ -1151,9 +1175,11 @@
 
     (jobs/submit cfg
                  {::jobs/name :delete-object
-                  ::jobs/params {:object :file
-                                 :deleted-at (:deleted-at file)
-                                 :id file-id}})
+                  ::jobs/params (cond-> {:object :file
+                                         :deleted-at (:deleted-at file)
+                                         :id file-id}
+                                  (some? file-ids)
+                                  (assoc :file-ids file-ids))})
     file))
 
 (def ^:private
@@ -1181,7 +1207,8 @@
                       :project-id (:project-id file)
                       :name (:name file)
                       :created-at (:created-at file)
-                      :modified-at (:modified-at file)}})))
+                      :modified-at (:modified-at file)
+                      :deleted-at (:deleted-at file)}})))
 
 (sv/defmethod ::delete-file
   {::doc/added "1.17"
@@ -1402,8 +1429,16 @@
     WHERE file_id = ANY(?::uuid[])")
 
 (def ^:private sql:restore-file-changes
+  ;; Only durable history comes back: a snapshot (carries `label`) or a legacy
+  ;; payload row (carries `data`, the column the new model replaces with
+  ;; `file_data`). Revisions have neither and are deliberately left alone: they
+  ;; are born already scheduled for deletion one hour later, and the collector
+  ;; only looks at rows that carry a date, so unmarking one here would keep it
+  ;; in the table for ever. The `data` half is a guard for the rows still
+  ;; pending migration, not a dependency: drop it with the last one.
   "UPDATE file_change SET deleted_at = null
-    WHERE file_id = ANY(?::uuid[])")
+    WHERE file_id = ANY(?::uuid[])
+      AND (label IS NOT NULL OR data IS NOT NULL)")
 
 (def ^:private sql:restore-file-data
   "UPDATE file_data SET deleted_at = null

backend/src/app/rpc/commands/files_create.clj

@@ -137,5 +137,11 @@
                       {:id team-id}
                       {::db/return-keys false}))))
 
-    (-> (create-file cfg params)
-        (vary-meta assoc ::audit/props {:team-id team-id}))))
+    (let [file (create-file cfg params)]
+      (vary-meta file assoc ::audit/props {:id (:id file)
+                                           :project-id project-id
+                                           :team-id team-id
+                                           :name (:name file)
+                                           :created-at (:created-at file)
+                                           :modified-at (:modified-at file)
+                                           :revn (:revn file)}))))

backend/src/app/rpc/commands/files_share.clj

@@ -12,9 +12,11 @@
    [app.common.schema :as sm]
    [app.common.uuid :as uuid]
    [app.db :as db]
+   [app.loggers.audit :as-alias audit]
    [app.rpc :as-alias rpc]
    [app.rpc.commands.files :as files]
    [app.rpc.doc :as-alias doc]
+   [app.rpc.helpers :as rph]
    [app.util.services :as sv]))
 
 ;; --- MUTATION: Create Share Link
@@ -39,7 +41,11 @@
    ::db/transaction true}
   [{:keys [::db/conn]} {:keys [::rpc/profile-id file-id] :as params}]
   (files/check-edition-permissions! conn profile-id file-id)
-  (create-share-link conn (assoc params :profile-id profile-id)))
+  (let [slink (create-share-link conn (assoc params :profile-id profile-id))]
+    ;; The consumers key share scope by file, so the event carries
+    ;; both ids; `:file-id` is already in params, `:share-id` is new.
+    (rph/with-meta slink
+      {::audit/props {:share-id (:id slink)}})))
 
 (defn create-share-link
   [conn {:keys [profile-id file-id pages who-comment who-inspect]}]
@@ -80,4 +86,8 @@
                   :hint "You can only delete share-links you created")))
 
     (db/delete! conn :share-link {:id id})
-    nil))
+    ;; The delete params carry only the link id, so resolve the file
+    ;; from the row fetched above before it is gone.
+    (rph/with-meta (rph/wrap)
+      {::audit/props {:file-id (:file-id slink)
+                      :share-id id}})))

backend/src/app/rpc/commands/files_thumbnails.clj

@@ -298,7 +298,8 @@
    ::rtry/enabled true
    ::rtry/when rtry/conflict-exception?
    ::audit/skip true
-   ::sm/params schema:create-file-object-thumbnail}
+   ::sm/params schema:create-file-object-thumbnail
+   ::rpc/multipart-only-params [:media]}
 
   [cfg {:keys [::rpc/profile-id file-id object-id media tag]}]
   (media.v/validate-media-type! media)
@@ -399,7 +400,8 @@
                 [:file-thumbnail-ops/global]]
    ::rtry/enabled true
    ::rtry/when rtry/conflict-exception?
-   ::sm/params schema:create-file-thumbnail}
+   ::sm/params schema:create-file-thumbnail
+   ::rpc/multipart-only-params [:media]}
 
   [cfg {:keys [::rpc/profile-id file-id] :as params}]
   (media.v/validate-media-type! (:media params))

backend/src/app/rpc/commands/files_update.clj

@@ -202,7 +202,8 @@
                      (assoc :file file)
                      (assoc :changes changes))
 
-        cfg      (assoc cfg ::timestamp (ct/now))]
+        cfg      (assoc cfg ::timestamp (ct/now)
+                        ::validation-timings (atom {}))]
 
     (when (not= (:vern params)
                 (:vern file))
@@ -241,9 +242,39 @@
     (binding [l/*context* (some-> (meta params)
                                   (get :app.http/request)
                                   (errors/request->context))]
-      (-> (update-file* cfg params tpoint)
-          (rph/with-defer #(let [elapsed (tpoint)]
-                             (l/trace :hint "update-file" :time (ct/format-duration elapsed))))))))
+      (try
+        (-> (update-file* cfg params tpoint)
+            (rph/with-defer #(let [elapsed (tpoint)
+                                   timings @(::validation-timings cfg)]
+                               (l/dbg :hint "update-file"
+                                      :file-id (str (:id file))
+                                      :project-id (str (:project-id file))
+                                      :team-id (str (:team-id file))
+                                      :status "success"
+                                      :time (ct/format-duration elapsed)
+                                      :resolve-libs-ms (get timings :resolve-libs-ms)
+                                      :process-changes-ms (get timings :process-changes-ms)
+                                      :soft-validate-affected-ms (get timings :soft-validate-affected-ms)
+                                      :soft-validate-schema-ms (get timings :soft-validate-schema-ms)
+                                      :validate-affected-ms (get timings :validate-affected-ms)
+                                      :validate-schema-ms (get timings :validate-schema-ms)))))
+        (catch Throwable cause
+          ;; NOTE: `with-defer` hooks only run on the success path
+          ;; (`handle-response`), so timings collected before a hard
+          ;; validation failure would be lost without this inline log.
+          (let [timings @(::validation-timings cfg)]
+            (l/dbg :hint "update-file"
+                   :file-id (str (:id file))
+                   :project-id (str (:project-id file))
+                   :team-id (str (:team-id file))
+                   :status "error"
+                   :resolve-libs-ms (get timings :resolve-libs-ms)
+                   :process-changes-ms (get timings :process-changes-ms)
+                   :soft-validate-affected-ms (get timings :soft-validate-affected-ms)
+                   :soft-validate-schema-ms (get timings :soft-validate-schema-ms)
+                   :validate-affected-ms (get timings :validate-affected-ms)
+                   :validate-schema-ms (get timings :validate-schema-ms)))
+          (throw cause))))))
 
 (defn- update-file*
   "Internal function, part of the update-file process, that encapsulates
@@ -345,11 +376,18 @@
 
         (with-meta {:revn revn :lagged (get-lagged-changes conn params)}
           {::audit/replace-props
-           (cond-> {:id         (:id file)
-                    :name       (:name file)
-                    :features   (:features file)
-                    :project-id (:project-id file)
-                    :team-id    (:team-id file)}
+           (cond-> {:id          (:id file)
+                    :name        (:name file)
+                    :features    (:features file)
+                    :project-id  (:project-id file)
+                    :team-id     (:team-id file)
+                    ;; NOTE: `file` carries the post-update revn here (`revn`
+                    ;; above is the pre-update one kept for the response), and
+                    ;; `modified-at` mirrors `persist-file!`: same `::timestamp`
+                    ;; fallback, so both agree to the millisecond when the
+                    ;; request timestamp exists.
+                    :revn        (:revn file)
+                    :modified-at (or timestamp (ct/now))}
 
              ;; The ordinary save path is untouched: the props above are
              ;; the ones it has always carried.
@@ -533,6 +571,16 @@
     (catch Throwable cause
       (l/error :hint "file schema validation error" :cause cause))))
 
+(defn- record-validation-timing
+  [cfg step-key step-label elapsed-ms]
+  (when-let [metrics (::mtx/metrics cfg)]
+    (when (mtx/metrics? metrics)
+      (mtx/run! metrics {:id :update-file-validation-timing
+                         :labels [step-label]
+                         :val (double elapsed-ms)})))
+  (when-let [timings (::validation-timings cfg)]
+    (swap! timings assoc step-key elapsed-ms)))
+
 (defn- soft-validate-file!
   [file libs changes]
   (try
@@ -543,11 +591,21 @@
 
 (defn- process-changes-and-validate
   [cfg file changes skip-validate]
-  (let [libs
-        (when (and (or (contains? cf/flags :file-validation)
-                       (contains? cf/flags :soft-file-validation))
-                   (not skip-validate))
-          (bfc/get-resolved-file-libraries cfg file))
+  (let [should-resolve-libs?
+        (and (or (contains? cf/flags :file-validation)
+                 (contains? cf/flags :soft-file-validation))
+             (not skip-validate))
+
+        tlibs
+        (ct/tpoint)
+
+        libs
+        (try
+          (when should-resolve-libs?
+            (bfc/get-resolved-file-libraries cfg file))
+          (finally
+            (when should-resolve-libs?
+              (record-validation-timing cfg :resolve-libs-ms "resolve-libs" (inst-ms (tlibs))))))
 
         ;; The main purpose of this atom is provide a contextual state
         ;; for the changes subsystem where optionally some hints can
@@ -557,12 +615,8 @@
         state
         (atom {})
 
-        file
-        (binding [cpc/*state* state]
-          (-> (files/check-version! file)
-              (update :revn inc)
-              (update :data cpc/process-changes changes)
-              (update :data d/without-nils)))
+        tproc
+        (ct/tpoint)
 
         ;; The media fix-up copies the foreign media rows into the file
         ;; and rewrites the references on the file data. A branch
@@ -573,25 +627,50 @@
         ;; row (`persist-branch-file!`), so the comparison reads the
         ;; pairing from the branch's own record.
         [file op-changes media-index]
-        (if-let [media-refs (-> @state :media-refs not-empty)]
-          (let [[file media-index] (bfc/update-media-references! cfg file media-refs)]
-            [file (bm/remap-changes changes media-index) media-index])
-          [file changes nil])]
+        (try
+          (let [file
+                (binding [cpc/*state* state]
+                  (-> (files/check-version! file)
+                      (update :revn inc)
+                      (update :data cpc/process-changes changes)
+                      (update :data d/without-nils)))]
+            (if-let [media-refs (-> @state :media-refs not-empty)]
+              (let [[file media-index] (bfc/update-media-references! cfg file media-refs)]
+                [file (bm/remap-changes changes media-index) media-index])
+              [file changes nil]))
+          (finally
+            (record-validation-timing cfg :process-changes-ms "process-changes" (inst-ms (tproc)))))]
 
     (binding [pmap/*tracked* nil]
       (when (contains? cf/flags :soft-file-validation)
-        (soft-validate-file! file libs changes))
+        (let [t (ct/tpoint)]
+          (try
+            (soft-validate-file! file libs changes)
+            (finally
+              (record-validation-timing cfg :soft-validate-affected-ms "soft-validate-affected" (inst-ms (t)))))))
 
       (when (contains? cf/flags :soft-file-schema-validation)
-        (soft-validate-file-schema! file))
+        (let [t (ct/tpoint)]
+          (try
+            (soft-validate-file-schema! file)
+            (finally
+              (record-validation-timing cfg :soft-validate-schema-ms "soft-validate-schema" (inst-ms (t)))))))
 
       (when (and (contains? cf/flags :file-validation)
                  (not skip-validate))
-        (val/validate-file-affected! file libs changes))
+        (let [t (ct/tpoint)]
+          (try
+            (val/validate-file-affected! file libs changes)
+            (finally
+              (record-validation-timing cfg :validate-affected-ms "validate-affected" (inst-ms (t)))))))
 
       (when (and (contains? cf/flags :file-schema-validation)
                  (not skip-validate))
-        (val/validate-file-schema! file)))
+        (let [t (ct/tpoint)]
+          (try
+            (val/validate-file-schema! file)
+            (finally
+              (record-validation-timing cfg :validate-schema-ms "validate-schema" (inst-ms (t))))))))
 
     [file op-changes media-index]))
 

backend/src/app/rpc/commands/jobs.clj

@@ -330,11 +330,15 @@
   "Read a job the caller created: its state and, once it is over, what it
   produced or why it failed.
 
-  The result of a finished job is the plain JSON the row holds, because a
-  result has no schema of its own like the params do: a uuid comes back as
-  the text it is on disk. The progress and the history of the job are not
-  part of the answer: a client that follows a job takes its progress from
-  the websocket, and the event log still has no read surface."
+  The result of a finished job is the JSON the row holds, decoded with the
+  result schema its job-def declares: a job that keys its result by uuid
+  (the import resolution) answers with the types it stored, and a job whose
+  job-def declares no schema answers with the JSON as it is on disk, where a
+  uuid is text and a uuid used as a map key is a keyword. A result that does
+  not conform to the declared schema never reaches the column: the substrate
+  validates it when the job is completed. The progress and the history of
+  the job are not part of the answer: a client that follows a job takes its
+  progress from the websocket, and the event log still has no read surface."
   {::doc/added "2.20"
    ::sm/params schema:get-job
    ::sm/result schema:job-detail}

backend/src/app/rpc/commands/management.clj

@@ -142,12 +142,29 @@
         (when (uuid? profile-id)
           (teams/create-project-role conn profile-id (:id project) :owner))
 
+        ;; The new entities are created by internal calls, so they carry no
+        ;; event of their own; emit one per entity for a local projection.
+        (audit/submit-create-event cfg params "create-project"
+                                   {:id (:id project)
+                                    :team-id (:team-id project)
+                                    :name (:name project)
+                                    :created-at (:created-at project)
+                                    :modified-at (:modified-at project)})
+
         (doseq [file-id files]
           (let [params (-> params
                            (dissoc :name)
                            (assoc :file-id file-id)
-                           (assoc :reset-shared-flag false))]
-            (duplicate-file cfg params)))
+                           (assoc :reset-shared-flag false))
+                file   (duplicate-file cfg params)]
+            (audit/submit-create-event cfg params "create-file"
+                                       {:id (:id file)
+                                        :project-id (:project-id file)
+                                        :team-id (:team-id project)
+                                        :name (:name file)
+                                        :created-at (:created-at file)
+                                        :modified-at (:modified-at file)
+                                        :revn (:revn file)})))
 
         project))))
 
@@ -211,7 +228,7 @@
                          (update :team-id bfc/lookup-index)
                          (assoc :created-at timestamp)
                          (assoc :modified-at timestamp))]
-          (teams/add-profile-to-team! cfg params {::db/return-keys false})))
+          (teams/add-profile-to-team! cfg params)))
 
       ;; Duplicate team fonts
       (doseq [font fonts]
@@ -444,14 +461,12 @@
                               {:id project-id}
                               {::db/return-keys false})
 
-                  (let [props (audit/clean-props params)]
-                    (doseq [file-id result]
-                      (let [props (assoc props :id file-id)
-                            event (-> (audit/event-from-rpc-params params)
-                                      (assoc :profile-id profile-id)
-                                      (assoc :name "create-file")
-                                      (assoc :props props))]
-                        (audit/submit cfg event))))))
+                  (doseq [file-id (:file-ids result)]
+                    (audit/submit-file-created-event
+                     cfg
+                     (assoc params :profile-id profile-id)
+                     file-id
+                     {:team-id (:id team)}))))
 
     result))
 

backend/src/app/rpc/commands/media.clj

@@ -40,6 +40,9 @@
 
 (declare create-file-media-object)
 
+;; TEMPORARILY DISABLED (2026-10-19): the media-storage-bytes-per-team
+;; quote check that used this lookup is disabled below. Kept active
+;; because the audit props of the upload still report the team id.
 (def ^:private sql:get-team-id-for-file
   "SELECT p.team_id
      FROM file AS f
@@ -57,6 +60,7 @@
   {::doc/added "1.17"
    ::doc/changes [["2.19" "The optional :id param is rejected with a params-validation error; the server always generates the identifier"]]
    ::sm/params schema:upload-file-media-object
+   ::rpc/multipart-only-params [:content]
    ::climit/id [[:process-image/by-profile ::rpc/profile-id]
                 [:process-image/global]]}
   [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id content] :as params}]
@@ -65,30 +69,35 @@
   (media.v/validate-media-size! content)
 
   (let [team-id (:team-id (db/exec-one! pool [sql:get-team-id-for-file file-id]))]
-    (quotes/check! cfg {::quotes/id        ::quotes/media-storage-bytes-per-team
-                        ::quotes/profile-id profile-id
-                        ::quotes/team-id    team-id
-                        ::quotes/incr       (:size content)}))
+    ;; TEMPORARILY DISABLED (2026-10-19): media-storage-bytes-per-team
+    ;; quote performs a full SUM over storage_object per upload (3-12s in
+    ;; prod on a 3.5TB DB, ~34 slow executions/hour). Disabled until we
+    ;; land a cheaper approximation (cached counter / rewritten query).
+    ;; TODO: re-enable with the new implementation.
+    ;; (quotes/check! cfg {::quotes/id        ::quotes/media-storage-bytes-per-team
+    ;;                     ::quotes/profile-id profile-id
+    ;;                     ::quotes/team-id    team-id
+    ;;                     ::quotes/incr       (:size content)})
+    (db/run! cfg (fn [{:keys [::db/conn] :as cfg}]
+                   ;; We get the minimal file for proper checking if
+                   ;; file is not already deleted
+                   (let [_      (files/get-minimal-file conn file-id)
+                         mobj   (create-file-media-object cfg params)]
 
-  (db/run! cfg (fn [{:keys [::db/conn] :as cfg}]
-                 ;; We get the minimal file for proper checking if
-                 ;; file is not already deleted
-                 (let [_      (files/get-minimal-file conn file-id)
-                       mobj   (create-file-media-object cfg params)]
+                     (db/update! conn :file
+                                 {:modified-at (ct/now)
+                                  :has-media-trimmed false}
+                                 {:id file-id}
+                                 {::db/return-keys false})
 
-                   (db/update! conn :file
-                               {:modified-at (ct/now)
-                                :has-media-trimmed false}
-                               {:id file-id}
-                               {::db/return-keys false})
-
-                   (with-meta mobj
-                     {::audit/replace-props
-                      {:name (:name params)
-                       :file-id file-id
-                       :is-local (:is-local params)
-                       :size (:size content)
-                       :mtype (:mtype content)}})))))
+                     (with-meta mobj
+                       {::audit/replace-props
+                        {:name (:name params)
+                         :file-id file-id
+                         :team-id team-id
+                         :is-local (:is-local params)
+                         :size (:size content)
+                         :mtype (:mtype content)}}))))))
 
 (defn- big-enough-for-thumbnail?
   "Checks if the provided image info is big enough for
@@ -370,6 +379,7 @@
 (sv/defmethod ::upload-chunk
   {::doc/added "2.17"
    ::sm/params schema:upload-chunk
+   ::rpc/multipart-only-params [:content]
    ::sm/result schema:upload-chunk-result}
   [{:keys [::db/pool] :as cfg}
    {:keys [::rpc/profile-id session-id index content] :as _params}]

backend/src/app/rpc/commands/profile.clj

@@ -307,6 +307,7 @@
 (sv/defmethod ::update-profile-photo
   {:doc/added "1.1"
    ::sm/params schema:update-profile-photo
+   ::rpc/multipart-only-params [:file]
    ::sm/result :nil}
   [cfg {:keys [::rpc/profile-id file] :as params}]
   ;; Validate incoming mime type
@@ -540,7 +541,8 @@
   {::doc/added "1.0"
    ::db/transaction true}
   [{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id] :as params}]
-  (let [teams      (get-owned-teams conn profile-id)
+  (let [profile    (db/get* conn :profile {:id profile-id} {::db/remove-deleted false})
+        teams      (get-owned-teams conn profile-id)
         deleted-at (ct/now)]
 
     ;; If we found owned teams with participants, we don't allow
@@ -581,6 +583,12 @@
     (session/invalidate-all cfg profile-id)
 
     (-> (rph/wrap nil)
+        (rph/with-meta {::audit/props {:id (:id profile)
+                                       :email (:email profile)
+                                       :fullname (:fullname profile)
+                                       :created-at (:created-at profile)
+                                       :modified-at (:modified-at profile)
+                                       :deleted-at deleted-at}})
         (rph/with-transform (session/delete-fn cfg)))))
 
 (def sql:get-subscription-editors

backend/src/app/rpc/commands/projects.clj

@@ -224,8 +224,9 @@
                       ::quotes/profile-id profile-id
                       ::quotes/team-id team-id})
 
-  (let [params (assoc params :profile-id profile-id)]
-    (db/tx-run! cfg create-project params)))
+  (let [params  (assoc params :profile-id profile-id)
+        project (db/tx-run! cfg create-project params)]
+    (rph/with-meta project {::audit/props {:id (:id project)}})))
 
 ;; --- MUTATION: Toggle Project Pin
 

backend/src/app/rpc/commands/teams.clj

@@ -564,7 +564,9 @@
         team     (db/tx-run! cfg create-team params)]
 
     (with-meta team
-      {::audit/props {:id (:id team)}})))
+      {::audit/props {:id (:id team)
+                      :created-at (:created-at team)
+                      :modified-at (:modified-at team)}})))
 
 
 (defn create-default-organization-team
@@ -619,10 +621,42 @@
                                     :default-team-id default-team-id}))
               default-team-id))))))))
 
+(defn- params->role
+  "Derive the team role from membership flags, inverse of
+  `types.team/permissions-for-role`. `team_profile_rel` carries no role
+  column, so no caller can pass it in `params`: derive it here so the event
+  always carries one."
+  [{:keys [is-owner is-admin can-edit]}]
+  (cond is-owner :owner is-admin :admin can-edit :editor :else :viewer))
+
+(defn- submit-membership-event
+  "Emits an audit event for a new team membership, so the event stream carries
+  every membership add, including the ones that only happen as a side effect of
+  another command (team creation, invitation accepted without email roundtrip,
+  duplicated team). `event-opts` carries what the row lacks: `:actor-id` (the
+  action owner, defaulting to the member) and `:rpc-params` (request context,
+  which sibling events get from `prepare-rpc-event`)."
+  [cfg {:keys [profile-id team-id] :as params} {:keys [actor-id rpc-params]}]
+  (when (contains? cf/flags :audit-log)
+    (audit/submit cfg
+                  (-> (audit/event-from-rpc-params (or rpc-params {}))
+                      (assoc :name "add-team-member"
+                             :type "action"
+                             :profile-id (or actor-id profile-id))
+                      (assoc :props (d/without-nils
+                                     {:team-id team-id
+                                      :member-id profile-id
+                                      :role (params->role params)
+                                      :is-owner (boolean (:is-owner params))
+                                      :is-admin (boolean (:is-admin params))
+                                      :can-edit (boolean (:can-edit params))}))))))
+
 (defn add-profile-to-team!
   ([cfg params]
-   (add-profile-to-team! cfg params nil))
-  ([{:keys [::db/conn] :as cfg} {:keys [:profile-id :team-id] :as params} options]
+   (add-profile-to-team! cfg params nil nil))
+  ([cfg params options]
+   (add-profile-to-team! cfg params options nil))
+  ([{:keys [::db/conn] :as cfg} {:keys [:profile-id :team-id] :as params} options event-opts]
    (assert (db/connection-map? cfg)
            "expected cfg with valid connection")
    (when (contains? cf/flags :admin-console)
@@ -632,7 +666,13 @@
               (some? (:organization-id membership)) ;; the team do belong to an organization
               (not (:is-member membership)))        ;; the user is not a member of the organization yet
          (initialize-user-in-organization cfg profile-id (:organization-id membership)))))
-   (db/insert! conn :team-profile-rel (assoc params :id (uuid/next)) options)))
+   (let [options (merge options {::db/return-keys [:id]})
+         result  (db/insert! conn :team-profile-rel (assoc params :id (uuid/next)) options)]
+     ;; Only emit when a row was actually inserted: with `on-conflict-do-nothing`
+     ;; an existing membership returns nil and must not be reported twice.
+     (when (some? result)
+       (submit-membership-event cfg params event-opts))
+     result)))
 
 (defn create-team
   "This is a complete team creation process, it creates the team
@@ -885,7 +925,7 @@
                     ::jobs/params {:object :team
                                    :deleted-at (:deleted-at team)
                                    :id team-id}})
-      (rph/with-meta (rph/wrap)
+      (rph/with-meta (rph/wrap team)
         {::audit/props {:organization-id organization-id
                         :team-id team-id
                         :team-name team-name}}))))
@@ -900,7 +940,19 @@
    ::sm/params schema:delete-team
    ::db/transaction true}
   [cfg {:keys [::rpc/profile-id id] :as params}]
-  (delete-team cfg {:team-id id :profile-id profile-id}))
+  (let [result (delete-team cfg {:team-id id :profile-id profile-id})
+        team   (rph/unwrap result)]
+    ;; The client keeps receiving nil; the audit event carries the deleted row.
+    ;; Both prop sets are kept: the organization ones (develop) and the entity
+    ;; identity and timestamps (staging); `with-meta` merges shallowly, so the
+    ;; maps are merged explicitly.
+    (rph/with-meta (rph/wrap)
+      {::audit/props (merge (::audit/props (meta result))
+                            {:id (:id team)
+                             :name (:name team)
+                             :created-at (:created-at team)
+                             :modified-at (:modified-at team)
+                             :deleted-at (:deleted-at team)})})))
 
 ;; --- Mutation: Team Update Role
 
@@ -967,7 +1019,13 @@
   {::doc/added "1.17"
    ::sm/params schema:update-team-member-role}
   [cfg {:keys [::rpc/profile-id] :as params}]
-  (db/tx-run! cfg update-team-member-role (assoc params :profile-id profile-id)))
+  (db/tx-run! cfg update-team-member-role (assoc params :profile-id profile-id))
+  ;; The client keeps receiving nil; when ownership moves, the audit event
+  ;; names the demoted member so a projection can keep a single owner.
+  (rph/with-meta (rph/wrap)
+    (if (= :owner (:role params))
+      {::audit/props {:prev-owner-id profile-id}}
+      {})))
 
 ;; --- Mutation: Delete Team Member
 
@@ -1036,7 +1094,8 @@
 
 (sv/defmethod ::update-team-photo
   {::doc/added "1.17"
-   ::sm/params schema:update-team-photo}
+   ::sm/params schema:update-team-photo
+   ::rpc/multipart-only-params [:file]}
   [cfg {:keys [::rpc/profile-id file] :as params}]
   ;; Validate incoming mime type
 
@@ -1045,7 +1104,7 @@
   (update-team-photo cfg (assoc params :profile-id profile-id)))
 
 (defn update-team-photo
-  [{:keys [::db/pool ::sto/storage] :as cfg} {:keys [profile-id team-id] :as params}]
+  [{:keys [::db/pool ::sto/storage] :as cfg} {:keys [profile-id team-id file] :as params}]
   (let [team  (get-team pool :profile-id profile-id :team-id team-id)
         photo (profile/upload-photo cfg params)]
 
@@ -1061,4 +1120,10 @@
                 {:photo-id (:id photo)}
                 {:id team-id})
 
-    (assoc team :photo-id (:id photo))))
+    (rph/with-meta (assoc team :photo-id (:id photo))
+      {::audit/replace-props
+       {:team-id team-id
+        :file-name (:filename file)
+        :file-size (:size file)
+        :file-path (str (:path file))
+        :file-mtype (:mtype file)}})))

backend/src/app/rpc/commands/teams_invitations.clj

@@ -202,7 +202,10 @@
           (when (contains? cf/flags :admin-console)
             (teams/initialize-user-in-organization cfg (:id member) (:id organization) email))
           ;; Insert the invited member to the team
-          (teams/add-profile-to-team! cfg params {::db/on-conflict-do-nothing? true}))
+          (teams/add-profile-to-team! cfg params
+                                      {::db/on-conflict-do-nothing? true}
+                                      {:actor-id (:id profile)
+                                       :rpc-params params}))
 
         ;; If profile is not yet verified, mark it as verified because
         ;; accepting an invitation link serves as verification.
@@ -318,7 +321,7 @@
                               :role :editor))))
 
 (defn- add-member-to-team
-  [{:keys [::db/conn] :as cfg} profile team role member]
+  [{:keys [::db/conn] :as cfg} profile team role member event-opts]
   (assert (db/connection-map? cfg)
           "expected cfg with valid connection")
 
@@ -340,7 +343,9 @@
       ::quotes/team-id team-id})
 
     ;; Insert the member to the team
-    (teams/add-profile-to-team! cfg params {::db/on-conflict-do-nothing? true})
+    (teams/add-profile-to-team! cfg params
+                                {::db/on-conflict-do-nothing? true}
+                                event-opts)
 
     ;; Delete any request
     (db/delete! conn :team-access-request
@@ -438,7 +443,9 @@
          (filter #(contains? invitation-emails (key %)))
          (map (fn [[email member]]
                 (let [role (:role (first (filter #(= (:email %) email) invitation-data)))]
-                  (add-member-to-team cfg profile team role member))))
+                  (add-member-to-team cfg profile team role member
+                                      {:actor-id (:id profile)
+                                       :rpc-params params}))))
          (doall))
 
     invitations))
@@ -593,7 +600,7 @@
                 :hint "the maximum of invitation on single request is reached"
                 :threshold max-invitations-by-request-threshold))
 
-    (let [props {:name name :features features}
+    (let [props {:id (:id team) :name name :features features}
           event (-> (audit/event-from-rpc-params params)
                     (assoc :name "create-team")
                     (assoc :props props))]
@@ -609,7 +616,8 @@
                            (map (fn [email] (assoc params :email email)))
                            (map (partial create-invitation cfg)))]
 
-      (vary-meta team assoc ::audit/props {:invitations (count invitations)}))))
+      (vary-meta team assoc ::audit/props {:id (:id team)
+                                           :invitations (count invitations)}))))
 
 ;; --- Query: get-team-invitation-token
 

backend/src/app/rpc/commands/verify_token.clj

@@ -302,21 +302,26 @@
               ;; for the same name, and tied the record to the browser finishing
               ;; the flow. Everything the copy carried is computed above.
               (when organization-id-on-add
-                (audit/submit cfg (-> (audit/event-from-rpc-params params)
-                                      (assoc :name "accept-organization-invitation")
-                                      (assoc :props
-                                             (-> props
-                                                 (assoc :organization-id organization-id-on-add)
-                                                 (assoc :invited-by (:created-by invitation))
-                                                 (assoc :profile-id (:id profile))
-                                                 (assoc :profile-email (:email profile))
-                                                 (assoc :organization-member-add-source
-                                                        organization-add-source)
-                                                 (assoc :belongs-to-team-on-add
-                                                        (boolean team-id))
-                                                 (assoc :organization-member-count-before
-                                                        organization-member-count-before)
-                                                 (audit/clean-props))))))
+                (let [org-props (-> props
+                                    (assoc :organization-id organization-id-on-add
+                                           :invited-by (:created-by invitation)
+                                           :profile-id (:id profile)
+                                           :profile-email (:email profile)
+                                           :organization-member-add-source
+                                           organization-add-source
+                                           :belongs-to-team-on-add
+                                           (boolean team-id)
+                                           :organization-member-count-before
+                                           organization-member-count-before)
+                                    (audit/clean-props))]
+                  (audit/submit cfg (-> (audit/event-from-rpc-params params)
+                                        (assoc :name "accept-organization-invitation"
+                                               :props org-props)))
+                  (when-let [created-by (:created-by invitation)]
+                    (audit/submit cfg (-> (audit/event-from-rpc-params params)
+                                          (assoc :profile-id created-by
+                                                 :name "accept-organization-invitation-from"
+                                                 :props org-props))))))
 
               (cond-> (assoc claims
                              :state :created

backend/src/app/rpc/commands/webhooks.clj

@@ -14,10 +14,12 @@
    [app.common.uuid :as uuid]
    [app.db :as db]
    [app.http.client :as http]
+   [app.loggers.audit :as-alias audit]
    [app.loggers.webhooks :as webhooks]
    [app.rpc :as-alias rpc]
    [app.rpc.commands.teams :refer [check-read-permissions!] :as t]
    [app.rpc.doc :as-alias doc]
+   [app.rpc.helpers :as rph]
    [app.rpc.permissions :as perms]
    [app.util.services :as sv]
    [cuerdas.core :as str]))
@@ -137,7 +139,8 @@
   (let [whook (-> (db/get pool :webhook {:id id}) (decode-row))]
     (check-webhook-edition-permissions! pool profile-id (:team-id whook))
     (validate-webhook! cfg whook params)
-    (update-webhook! cfg whook params)))
+    (rph/with-meta (update-webhook! cfg whook params)
+      {::audit/props {:team-id (:team-id whook)}})))
 
 (def ^:private schema:delete-webhook
   [:map {:title "delete-webhook"}
@@ -151,7 +154,8 @@
   (let [whook (-> (db/get conn :webhook {:id id}) decode-row)]
     (check-webhook-edition-permissions! conn profile-id (:team-id whook))
     (db/delete! conn :webhook {:id id})
-    nil))
+    (rph/with-meta (rph/wrap)
+      {::audit/props {:team-id (:team-id whook)}})))
 
 ;; --- Query: Webhooks
 

backend/src/app/rpc/management/exporter.clj

@@ -31,6 +31,7 @@
 (sv/defmethod ::upload-tempfile
   {::doc/added "2.12"
    ::sm/params schema:upload-tempfile-params
+   ::rpc/multipart-only-params [:content]
    ::sm/result schema:upload-tempfile-result}
   [cfg {:keys [::rpc/profile-id content]}]
   (media.v/validate-media-type! content cm/tempfile-types)

backend/src/app/rpc/management/files.clj

@@ -0,0 +1,238 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS SUBSIDIARY SL
+
+(ns app.rpc.management.files
+  "Read-only file content for the nexus file-security pipeline.
+
+  The single method `get-file-for-analysis` returns the semantic
+  projection of one file (pages, frames, text, links, interactions):
+  small, neutral data the decision model consumes. Full file binaries
+  never cross this boundary. Every call leaves an audit event like any
+  other management call, so content reads stay traceable; the volume
+  scales with analyzed files, and the nexus consumer ignores the event."
+  (:require
+   [app.binfile.common :as bfc]
+   [app.common.data :as d]
+   [app.common.exceptions :as ex]
+   [app.common.schema :as sm]
+   [app.common.time :as ct]
+   [app.common.types.text :as txt]
+   [app.db :as db]
+   [app.rpc :as-alias rpc]
+   [app.rpc.doc :as doc]
+   [app.util.services :as sv]
+   [cuerdas.core :as str]))
+
+(def projection-version 1)
+
+;; Bounds for the data that crosses to nexus on every sweep. The
+;; projection stays small by construction; anything past the caps sets
+;; `:truncated?` instead of growing the payload.
+(def ^:const max-text-chars 30000)
+(def ^:const max-links 1000)
+(def ^:const max-interactions 2000)
+
+(defn- page-shapes
+  "All shapes of one page with their page id attached."
+  [{:keys [id objects]}]
+  (let [page-id (str id)]
+    (->> (vals objects)
+         (filter map?)
+         (map #(assoc % :page-id page-id)))))
+
+(defn- shape-text
+  "Plain text of a text shape, or nil when it has none. Never throws:
+  corrupt content skips the shape instead of failing the file."
+  [shape]
+  (when (and (= :text (:type shape))
+             (some? (:content shape)))
+    (let [text (ex/ignoring (txt/content->text (:content shape)))]
+      (when-not (str/blank? text)
+        text))))
+
+(defn- open-url?
+  [{:keys [action-type url]}]
+  (and (= :open-url action-type)
+       (not (str/blank? url))))
+
+(defn file->projection
+  "Turn decoded file data into the neutral projection the decision
+  model consumes. Pure: takes the `:data` map, returns plain data
+  with all ids as strings."
+  [{:keys [pages pages-index]}]
+  (let [shapes  (mapcat page-shapes
+                        (keep #(get pages-index %) pages))
+        frames  (into []
+                      (comp (filter #(= :frame (:type %)))
+                            (map (fn [{:keys [id name page-id]}]
+                                   (d/without-nils
+                                    {:id (str id)
+                                     :page-id page-id
+                                     :name name}))))
+                      shapes)
+        texts   (into []
+                      (comp (map (juxt :id :page-id shape-text))
+                            (remove #(nil? (nth % 2)))
+                            (map (fn [[id page-id text]]
+                                   {:shape-id (str id)
+                                    :page-id page-id
+                                    :text text})))
+                      shapes)
+        ;; Texts are capped by total characters, the unit the model
+        ;; bills; links and interactions by count.
+        texts   (loop [acc []
+                       chars 0
+                       [item & rest] texts]
+                  (cond
+                    (nil? item)
+                    {:items acc :truncated? false}
+
+                    (> (+ chars (count (:text item))) max-text-chars)
+                    {:items acc :truncated? true}
+
+                    :else
+                    (recur (conj acc item)
+                           (+ chars (count (:text item)))
+                           rest)))
+        inter   (into []
+                      (comp (mapcat (fn [shape]
+                                      (->> (:interactions shape)
+                                           (filter map?)
+                                           (filter :action-type)
+                                           (filter :event-type)
+                                           (map #(assoc % :shape-id (:id shape)
+                                                        :page-id (:page-id shape))))))
+                            (map (fn [{:keys [shape-id page-id event-type action-type destination url]}]
+                                   (d/without-nils
+                                    {:shape-id (str shape-id)
+                                     :page-id page-id
+                                     :event (d/name event-type)
+                                     :action (d/name action-type)
+                                     :target (or url (some-> destination str))}))))
+                      shapes)
+        links   (into []
+                      (comp (mapcat (fn [shape]
+                                      (->> (:interactions shape)
+                                           (filter map?)
+                                           (filter open-url?)
+                                           (map #(assoc % :shape-id (:id shape)
+                                                        :page-id (:page-id shape))))))
+                            (map (fn [{:keys [shape-id page-id url]}]
+                                   {:shape-id (str shape-id)
+                                    :page-id page-id
+                                    :url url})))
+                      shapes)]
+    {:version projection-version
+     :pages (mapv (fn [page-id]
+                    (let [page (get pages-index page-id)]
+                      (d/without-nils
+                       {:id (str page-id)
+                        :name (:name page)})))
+                  pages)
+     :frames frames
+     :texts texts
+     :links (if (> (count links) max-links)
+              {:items (subvec links 0 max-links) :truncated? true}
+              {:items links :truncated? false})
+     :interactions (if (> (count inter) max-interactions)
+                     {:items (subvec inter 0 max-interactions) :truncated? true}
+                     {:items inter :truncated? false})}))
+
+(def ^:private sql:has-share-link
+  "SELECT EXISTS (SELECT 1 FROM share_link WHERE file_id = ?) AS shared;")
+
+(def ^:private sql:get-team-id
+  "SELECT team_id FROM project WHERE id = ?;")
+
+(def ^:private
+  schema:get-file-for-analysis-params
+  [:map {:title "get-file-for-analysis-params"}
+   [:file-id ::sm/uuid]])
+
+(def ^:private schema:projection-page
+  [:map
+   [:id :string]
+   [:name {:optional true} :string]])
+
+(def ^:private schema:projection-frame
+  [:map
+   [:id :string]
+   [:page-id :string]
+   [:name {:optional true} :string]])
+
+(def ^:private schema:projection-text
+  [:map
+   [:shape-id :string]
+   [:page-id :string]
+   [:text :string]])
+
+(def ^:private schema:projection-link
+  [:map
+   [:shape-id :string]
+   [:page-id :string]
+   [:url :string]])
+
+(def ^:private schema:projection-interaction
+  [:map
+   [:shape-id :string]
+   [:page-id :string]
+   [:event :string]
+   [:action :string]
+   [:target {:optional true} :string]])
+
+(def ^:private schema:projection
+  [:map {:title "file-projection"}
+   [:version ::sm/int]
+   [:pages [:vector schema:projection-page]]
+   [:frames [:vector schema:projection-frame]]
+   [:texts [:map
+            [:items [:vector schema:projection-text]]
+            [:truncated? :boolean]]]
+   [:links [:map
+            [:items [:vector schema:projection-link]]
+            [:truncated? :boolean]]]
+   [:interactions [:map
+                   [:items [:vector schema:projection-interaction]]
+                   [:truncated? :boolean]]]])
+
+(def ^:private
+  schema:get-file-for-analysis-result
+  [:map {:title "get-file-for-analysis-result"}
+   [:id ::sm/uuid]
+   [:revn ::sm/int]
+   [:modified-at ::ct/inst]
+   [:project-id ::sm/uuid]
+   [:team-id ::sm/uuid]
+   [:has-share-link :boolean]
+   [:projection-version ::sm/int]
+   [:projection schema:projection]])
+
+(sv/defmethod ::get-file-for-analysis
+  "Get the semantic projection of one file for security analysis.
+
+  Read-only: returns the file identity, revision, share state and the
+  neutral projection (pages, frames, text, links, interactions).
+  Called by nexus with the shared-key identity; takes no profile.
+  Every call leaves an audit event like any other management call."
+  {::doc/added "2.20"
+   ::sm/params schema:get-file-for-analysis-params
+   ::sm/result schema:get-file-for-analysis-result
+   ::rpc/auth false}
+  [cfg {:keys [file-id]}]
+  ;; `realize?` loads the whole object tree eagerly: the projection
+  ;; walks every shape, so lazy pointers (no `load-fn` bound here)
+  ;; would fail on first touch. Same choice as the file exporter.
+  (let [file       (bfc/get-file cfg file-id :realize? true)
+        shared     (:shared (db/exec-one! cfg [sql:has-share-link file-id]))
+        team-id    (:team-id (db/exec-one! cfg [sql:get-team-id (:project-id file)]))]
+    {:id (:id file)
+     :revn (:revn file)
+     :modified-at (:modified-at file)
+     :project-id (:project-id file)
+     :team-id team-id
+     :has-share-link (boolean shared)
+     :projection-version projection-version
+     :projection (file->projection (:data file))}))

backend/src/app/rpc/management/jobs.clj

@@ -106,7 +106,10 @@
    [:job-id ::sm/uuid]
    ;; Per-job result: any JSON-encodable value, nil when the job returns
    ;; nothing. Nested maps and vectors are allowed; unserializable
-   ;; values are dropped to nil by jobs/encode-result with a warning.
+   ;; values are dropped to nil by jobs/encode-result with a warning. A
+   ;; job whose job-def declares a result schema has its result validated
+   ;; against it: one that does not conform is rejected and the job is
+   ;; not completed.
    [:result [:maybe :any]]
    ;; Storage object the worker produced for this job. A resource is only
    ;; set, never replaced: a job that already has one is rejected.
@@ -117,6 +120,13 @@
    [:action [:enum :run :skip]]])
 
 (sv/defmethod ::complete-job
+  "Complete a job on behalf of the worker that ran it, with the result it
+  produced and, when it wrote one, the resource it stores.
+
+  The result of a job whose job-def declares a result schema is validated
+  against it before it is stored: one that does not conform is rejected and
+  the job is not completed. A job that is no longer completable is skipped,
+  and nothing is validated for it."
   {::doc/added "2.20"
    ::sm/params schema:complete-job-params
    ::sm/result schema:complete-job-result

backend/src/app/rpc/management/nitrate.clj

@@ -194,6 +194,7 @@
   collection when replacing an existing one."
   {::doc/added "2.18"
    ::sm/params schema:upload-organization-logo
+   ::rpc/multipart-only-params [:content]
    ::sm/result schema:upload-organization-logo-result
    ::nitrate/sso false}
   [{:keys [::sto/storage]} {:keys [content organization-id previous-id]}]
@@ -1059,10 +1060,20 @@ RETURNING id, deleted_at;")
                 [:updated-props ::sm/boolean]
                 [:announce-activation ::sm/boolean]]
    ::rpc/auth false}
-  [{:keys [::db/pool] :as cfg} {:keys [organization-id updated-props announce-activation]}]
+  [{:keys [::db/pool] :as cfg}
+   {:keys [organization-id updated-props announce-activation] :as params}]
   (when updated-props
     (rpc/invalidate-organization-sso-cache-by-organization! organization-id)
-    (session/clear-organization-sso-sessions! pool organization-id))
+    (session/clear-organization-sso-sessions! pool organization-id)
+    (let [request (-> params meta ::http/request)
+          context (-> (audit/prepare-context-from-request request)
+                      (assoc :request-id (::rpc/request-id params)))]
+      (audit/submit cfg {:type "action"
+                         :name "organization-sso-token-revoked"
+                         :profile-id uuid/zero
+                         :ip-addr (::rpc/ip-addr params)
+                         :props {:organization-id organization-id}
+                         :context (d/without-nils context)})))
   (notifications/notify-organization-change-sso cfg organization-id)
   (when announce-activation
     (neh/send-organization-setup-sso-emails! cfg organization-id))

backend/src/app/rpc/quotes.clj

@@ -428,7 +428,7 @@
     WHERE fc.file_id = ?
       AND fc.created_by = 'user'
       AND fc.deleted_at IS NULL
-      AND fc.data IS NOT NULL")
+      AND fc.label IS NOT NULL")
 
 (defmethod check-quote ::snapshots-per-file
   [{:keys [::profile-id ::file-id ::team-id ::project-id ::target] :as cfg}]
@@ -461,7 +461,7 @@
     WHERE p.team_id = ?
       AND fc.created_by = 'user'
       AND fc.deleted_at IS NULL
-      AND fc.data IS NOT NULL")
+      AND fc.label IS NOT NULL")
 
 (defmethod check-quote ::snapshots-per-team
   [{:keys [::profile-id ::team-id ::target] :as cfg}]

backend/src/app/scripts/decode_file_data.clj

@@ -0,0 +1,300 @@
+;; This Source Code Form is subject to the terms of the Mozilla Public
+;; License, v. 2.0. If a copy of the MPL was not distributed with this
+;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
+;;
+;; Copyright (c) KALEIDOS INC
+
+(ns app.scripts.decode-file-data
+  "Decode a file's `file_data` binary blob to `jq`-readable JSON.
+
+  The content of `file_data.data` (and `file.data` on the `legacy-db`
+  backend, or the `file-data` object on storage) is not text: it is a blob
+  encoded with `app.util.blob` (versions 1, 3, 4 and 5; transit+json or
+  fressian plus lz4/zstd). That is why it cannot be inspected with SQL
+  directly.
+
+  Two uses:
+
+  1. Offline (no DB, just the blob): dump the `bytea` to a file with
+     `psql` (`\\copy ...` or `encode(data, 'base64')`) and pass it with `--input`.
+     Needs no system and no `file_data` read permissions.
+
+     backend/scripts/decode-file-data --input /tmp/blob.bin --format summary | jq .
+     backend/scripts/decode-file-data --input /tmp/blob.b64 --from-base64 --format pages | jq .
+
+  2. Inside the penpot context (nREPL with the backend running, where
+     `sys/system` already exists): call `get-file-json` or `get-snapshot-json`.
+     They resolve the backend (`legacy-db`, `db`, `storage`), realize the
+     pointer-maps and objects-maps and return JSON.
+
+     (require '[app.scripts.decode-file-data :as dfd] :reload)
+     (spit \"/tmp/maquette.json\"
+       (dfd/get-file-json sys/system #uuid \"9e056cfe-f83f-81f8-8008-cc885732f00d\"
+                      :format :full))
+     (spit \"/tmp/snap-518.json\"
+       (dfd/get-snapshot-json sys/system #uuid \"9e056cfe-f83f-81f8-8008-cc885732f00d\"
+                          #uuid \"<snapshot-id>\" :format :summary))
+
+  Then filter with `jq`: `.pages`, `.data[\"pages-index\"] | keys`,
+  `[.data[\"pages-index\"][] | {id, name}]`, etc."
+  (:require
+   [app.binfile.common :as bfc]
+   [app.common.json :as json]
+   [app.db :as db]
+   [app.features.file-snapshots :as fsnap]
+   [app.util.blob :as blob]
+   [app.util.objects-map :as omap-legacy]
+   [app.util.pointer-map :as pmap]
+   [clojure.java.io :as io]
+   [clojure.string :as str])
+  (:import
+   java.util.Base64))
+
+(set! *warn-on-reflection* true)
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; JSON sanitization
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn- sanitize-key
+  [k]
+  (cond
+    (keyword? k) (subs (str k) 1)
+    (symbol? k)  (str k)
+    (uuid? k)    (str k)
+    :else        (str k)))
+
+(declare sanitize)
+
+(defn- sanitize-map
+  [m]
+  (persistent!
+   (reduce-kv (fn [acc k v]
+                (assoc! acc (sanitize-key k) (sanitize v)))
+              (transient {})
+              m)))
+
+(defn sanitize
+  "Convert a blob-decoded value to plain JSON-ready data.
+
+  - Unrealized pointers become `{\"$pointer\": \"<uuid>\"}`.
+  - `objects-map` instances are materialized to a plain map.
+  - Bytes become `{\"$bytes\": \"<base64>\"}`.
+  - UUIDs, instants, keywords and symbols become strings."
+  [x]
+  (cond
+    (nil? x)     nil
+    (string? x)  x
+    (boolean? x) x
+
+    (number? x)
+    (if (ratio? x) (double x) x)
+
+    (keyword? x) (subs (str x) 1)
+    (symbol? x)  (str x)
+    (uuid? x)    (str x)
+    (inst? x)    (str x)
+
+    (bytes? x)
+    {"$bytes" (.encodeToString (Base64/getEncoder) ^bytes x)}
+
+    (pmap/pointer-map? x)
+    {"$pointer" (str (pmap/get-id x))}
+
+    (omap-legacy/objects-map? x)
+    (sanitize (into {} x))
+
+    (map? x)
+    (sanitize-map x)
+
+    (set? x)
+    (mapv sanitize x)
+
+    (coll? x)
+    (mapv sanitize x)
+
+    :else
+    (str x)))
+
+(defn data->json
+  "Encode sanitizable data to a JSON string."
+  [data]
+  (json/encode (sanitize data)))
+
+(defn blob->json
+  "Decode a `file_data` blob (`app.util.blob`) and return it as JSON."
+  [^bytes blob-bytes]
+  (data->json (blob/decode blob-bytes)))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; jq-oriented investigation summary
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn summarize-data
+  "Small, stable summary for `jq`: pages (in `:pages` order) with object
+  counts, plus component, color and typography counters. Takes the already
+  decoded `:data` (not the JSON)."
+  [data]
+  (let [pages       (:pages data)
+        pages-index (:pages-index data)
+        page-entry  (fn [pid]
+                      {:id      (str pid)
+                       :name    (get-in pages-index [pid :name])
+                       :objects (count (get-in pages-index [pid :objects]))})]
+    {:pages (mapv page-entry (or pages []))
+     :stats {:pages        (count (or pages []))
+             :components   (count (:components data))
+             :colors       (count (:colors data))
+             :typographies (count (:typographies data))}}))
+
+(defn file-data-summary-json
+  [data]
+  (data->json (summarize-data data)))
+
+(defn file-data-pages-json
+  [data]
+  (data->json (:pages (summarize-data data))))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; DB access (for use from nREPL with the backend running)
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(defn- write-output
+  [^String payload output]
+  (if output
+    (do (spit output payload) output)
+    (print payload))
+  payload)
+
+(defn get-file-json
+  "Return a file as JSON (realized). For nREPL.
+
+  `format` can be `:full` (whole file), `:data` (`:data` only),
+  `:summary` or `:pages`. With `:output` it also writes it to a file."
+  [system file-id & {:keys [format output] :or {format :full}}]
+  (let [file (db/run! system bfc/get-file file-id :realize? true)
+        data (case format
+               :full    file
+               :data    (:data file)
+               :summary (summarize-data (:data file))
+               :pages   (:pages (summarize-data (:data file))))]
+    (write-output (data->json data) output)))
+
+(defn get-snapshot-json
+  "Return a snapshot as JSON (realized). For nREPL.
+
+  Takes the same `format` and `output` as `get-file-json`."
+  [system file-id snapshot-id & {:keys [format output] :or {format :full}}]
+  (let [snapshot (db/run! system
+                          (fn [cfg]
+                            (fsnap/get-snapshot cfg file-id snapshot-id)))
+        data   (case format
+                 :full    snapshot
+                 :data    (:data snapshot)
+                 :summary (summarize-data (:data snapshot))
+                 :pages   (:pages (summarize-data (:data snapshot))))]
+    (write-output (data->json data) output)))
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Offline CLI: decode a file holding the blob
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+(def ^:private cli-usage
+  (str "Usage: decode-file-data --input PATH [--output PATH] [--format full|data|summary|pages]\n"
+       "                       [--from-base64|--from-hex]\n"
+       "\n"
+       "Decode a file_data blob (app.util.blob) to JSON for inspection with jq.\n"
+       "\n"
+       "Examples:\n"
+       "  psql -c \"\\\\copy (select data from file_data where file_id='...' and id='...') to '/tmp/blob.bin' with binary\"\n"
+       "  decode-file-data --input /tmp/blob.bin --format summary | jq .\n"
+       "  decode-file-data --input /tmp/blob.b64 --from-base64 --format pages | jq .\n"))
+
+(defn- parse-args
+  [args]
+  (loop [args args
+         opts {:format "full"}]
+    (if (empty? args)
+      opts
+      (let [[a b & rest-args] args]
+        (cond
+          (contains? #{"-h" "--help"} a)
+          (assoc opts :help true)
+
+          (= a "--input")
+          (recur rest-args (assoc opts :input b))
+
+          (= a "--output")
+          (recur rest-args (assoc opts :output b))
+
+          (= a "--format")
+          (recur rest-args (assoc opts :format b))
+
+          (= a "--from-base64")
+          (recur (cons b rest-args) (assoc opts :from :base64))
+
+          (= a "--from-hex")
+          (recur (cons b rest-args) (assoc opts :from :hex))
+
+          :else
+          (assoc opts :error (str "unknown argument: " a)))))))
+
+(defn- hex->bytes
+  [^String s]
+  (let [s   (str/trim s)
+        s   (cond
+              (str/starts-with? s "\\x") (subs s 2)
+              (str/starts-with? s "0x") (subs s 2)
+              :else s)
+        s   (str/replace s #"\s+" "")
+        n   (/ (count s) 2)
+        out (byte-array n)]
+    (dotimes [i n]
+      (aset-byte out i (unchecked-byte (Integer/parseInt (subs s (* i 2) (+ (* i 2) 2)) 16))))
+    out))
+
+(defn- read-input-bytes
+  [{:keys [input from]}]
+  (cond
+    (nil? input)
+    (throw (ex-info "missing --input" {}))
+
+    (= from :base64)
+    (.decode (Base64/getDecoder) ^String (str/trim (slurp (io/file input))))
+
+    (= from :hex)
+    (hex->bytes (slurp (io/file input)))
+
+    :else
+    (java.nio.file.Files/readAllBytes (.toPath (io/file input)))))
+
+(defn -main
+  [& args]
+  (let [{:keys [help error input format output] :as opts} (parse-args (vec args))]
+    (cond
+      help
+      (do (print cli-usage) nil)
+
+      error
+      (do (binding [*out* *err*] (println error) (println cli-usage)) (System/exit 2))
+
+      (nil? input)
+      (do (binding [*out* *err*] (println "missing --input") (println cli-usage)) (System/exit 2))
+
+      (not (contains? #{"full" "data" "summary" "pages"} (or format "full")))
+      (do (binding [*out* *err*] (println (str "unknown format: " format))) (System/exit 2))
+
+      :else
+      (let [blob-bytes (read-input-bytes opts)
+            decoded    (blob/decode blob-bytes)
+            payload    (case format
+                         "full"    decoded
+                         "data"    (:data decoded (:data decoded decoded))
+                         "summary" (summarize-data (:data decoded decoded))
+                         "pages"   (:pages (summarize-data (:data decoded decoded))))
+            ;; NOTE: the file_data blob already IS the file's :data; but if
+            ;; someone dumps the whole serialized row, tolerate both.
+            out        (data->json payload)]
+        (if output
+          (spit output out)
+          (println out))))))

backend/src/app/srepl/main.clj

@@ -783,7 +783,7 @@
                    :type "action"
                    :props {:id team-id}
                    :context {:triggered-by "srepl"
-                             :cause "explicit call to delete-profile!"}
+                             :cause "explicit call to delete-team!"}
                    :tracked-at tnow})
 
     (jobs/invoke (-> sys/system
@@ -852,6 +852,7 @@
     (audit/insert sys/system
                   {:name "delete-profile"
                    :type "action"
+                   :props {:id profile-id}
                    :context {:triggered-by "srepl"
                              :cause "explicit call to delete-profile!"}
                    :tracked-at tnow})
@@ -1040,4 +1041,4 @@
                       (let [params (-> rel
                                        (assoc :id (uuid/next))
                                        (assoc :team-id (:id team)))]
-                        (teams/add-profile-to-team! cfg params {::db/return-keys false}))))))))
+                        (teams/add-profile-to-team! cfg params))))))))

backend/src/app/storage/gc_deleted.clj

@@ -19,201 +19,175 @@
    [app.common.time :as ct]
    [app.db :as db]
    [app.jobs :as jobs]
+   [app.metrics :as mtx]
    [app.storage :as sto]
    [app.storage.impl :as impl]
-   [clojure.set :as set]
    [integrant.core :as ig]))
 
-(def ^:private max-attempts
-  "Maximum number of deletion attempts before giving up and accepting
-  the orphan blob."
-  7)
-
-(def ^:private chunk-size
-  "Pending objects deleted in each transaction."
-  25)
-
-(def ^:private sql:lock-sobjects
-  "SELECT id FROM storage_object
-    WHERE id = ANY(?::uuid[])
-      FOR UPDATE
-     SKIP LOCKED")
-
-(defn- lock-ids
-  "Perform a select before delete for proper object locking and
-  prevent concurrent operations and we proceed only with successfully
-  locked objects."
-  [conn ids]
-  (let [ids (db/create-array conn "uuid" ids)]
-    (->> (db/exec! conn [sql:lock-sobjects ids])
-         (into #{} (map :id))
-         (not-empty))))
-
 (def ^:private sql:delete-sobjects
   "DELETE FROM storage_object
     WHERE id = ANY(?::uuid[])")
 
-(defn- delete-sobjects
-  [conn ids]
-  (let [ids (db/create-array conn "uuid" ids)]
-    (-> (db/exec-one! conn [sql:delete-sobjects ids])
-        (db/get-update-count))))
+(defn- delete-by-backend
+  "Deletes the locked objects of one backend group: the database rows and
+  their backend content. Runs inside the caller's transaction. Returns the
+  number of deleted rows; reporting it (metrics) is the caller's job, once
+  the transaction has committed."
+  [{:keys [::db/conn ::sto/storage]} backend-id ids]
+  (let [ids-array (db/create-array conn "uuid" ids)
+        total     (-> (db/exec-one! conn [sql:delete-sobjects ids-array])
+                      (db/get-update-count))]
+    (-> (impl/resolve-backend storage backend-id)
+        (impl/del-objects-in-bulk ids))
 
-(def ^:private sql:delete-upload-session-chunks
-  "DELETE FROM upload_session_chunk
-    WHERE object_id = ANY(?::uuid[])")
-
-(defn- delete-upload-session-chunks
-  "Remove the chunk mappings for the given storage object ids. This must run
-  before the storage_object rows are deleted: the upload_session_chunk
-  foreign keys are ON DELETE NO ACTION."
-  [conn ids]
-  (let [ids (db/create-array conn "uuid" ids)]
-    (db/exec-one! conn [sql:delete-upload-session-chunks ids])))
-
-(def ^:private sql:increment-attempts-and-defer
-  "UPDATE storage_object
-      SET deletion_attempts = deletion_attempts + 1,
-          deleted_at = NOW() + INTERVAL '1 day'
-    WHERE id = ANY(?::uuid[])")
-
-(defn- increment-attempts-and-defer
-  [conn ids]
-  (let [ids (db/create-array conn "uuid" ids)]
-    (db/exec-one! conn [sql:increment-attempts-and-defer ids])))
-
-(def ^:private sql:delete-give-up
-  "DELETE FROM storage_object
-    WHERE id = ANY(?::uuid[])
-      AND deletion_attempts >= ?")
-
-(defn- delete-give-up
-  [conn ids]
-  (let [ids (db/create-array conn "uuid" ids)]
-    (db/exec-one! conn [sql:delete-give-up ids max-attempts])))
-
-(defn- process-chunk
-  "Attempt to delete a chunk of storage objects from a specific backend.
-
-  This function runs inside the chunk's own transaction (see
-  `delete-in-bulk`) — it does NOT open its own transaction. The rows are
-  locked via FOR UPDATE SKIP LOCKED before deleting.
-
-  Returns the number of successfully deleted objects, or 0 if no rows
-  could be locked."
-  [conn storage backend-id ids]
-  (if-let [locked-ids (lock-ids conn ids)]
-    (let [fail-ids (try
-                     (-> (impl/resolve-backend storage backend-id)
-                         (impl/del-objects-in-bulk locked-ids))
-                     (catch Throwable cause
-                       (l/err :hint "error on physical deletion, will retry"
-                              :ids locked-ids
-                              :cause cause)
-                       locked-ids))
-          ok-ids   (set/difference locked-ids fail-ids)]
-
-      (doseq [id ok-ids]
-        (l/dbg :hint "permanently delete storage object"
-               :id (str id)
-               :backend (name backend-id)))
-
-      (when (seq ok-ids)
-        ;; NOTE: the chunk mappings must be removed before the
-        ;; storage_object rows (NO ACTION foreign keys). It only affects
-        ;; objects of the upload-session bucket; for any other bucket the
-        ;; delete matches no rows.
-        (delete-upload-session-chunks conn ok-ids)
-        (delete-sobjects conn ok-ids))
-
-      (when (seq fail-ids)
-        (increment-attempts-and-defer conn fail-ids)
-        ;; NOTE: same NO ACTION ordering as above: the give-up DELETE below
-        ;; removes storage_object rows, so chunk mappings must go first.
-        ;; Deferred objects keep their rows; only the mapping of a
-        ;; permanently given-up object disappears early, and that object is
-        ;; already deleted-marked.
-        (delete-upload-session-chunks conn fail-ids)
-        (let [given-up (delete-give-up conn fail-ids)]
-          (when (pos? (db/get-update-count given-up))
-            (l/wrn :hint "giving up on orphan blob after max attempts"
-                   :ids fail-ids
-                   :max-attempts max-attempts))))
-
-      (count ok-ids))
-    0))
-
-(defn- delete-in-bulk
-  [cfg backend-id ids]
-  ;; Each chunk runs in its own transaction, on its own connection: a chunk
-  ;; that fails (for example, an object that a foreign key still references) is
-  ;; logged and skipped, and it cannot poison the cursor that feeds the rest.
-  (try
-    (db/tx-run! cfg
-                (fn [{:keys [::db/conn ::sto/storage]}]
-                  (process-chunk conn storage backend-id ids)))
-    (catch Throwable cause
-      (l/err :hint "unexpected error on bulk deletion, skipping chunk"
-             :ids ids
-             :cause cause)
-      0)))
+    (doseq [id ids]
+      (l/dbg :hint "permanently delete storage object"
+             :id (str id)
+             :backend (name backend-id)))
+    total))
 
 (defn- group-by-backend
-  [items]
-  (d/group-by (comp keyword :backend) :id #{} items))
+  [rows]
+  (d/group-by (comp keyword :backend) :id #{} rows))
+
+(defn- process-chunk-deletion
+  "Deletes the locked rows of one chunk, grouped by backend. Runs inside the
+  caller's transaction: any failure here aborts it, so the whole chunk is
+  rolled back (nothing is deleted and the locks are released) and skipped by
+  the caller. Returns a map of backend id to deleted row count."
+  [tx rows]
+  (->> (group-by-backend rows)
+       (map (fn [[backend-id ids]]
+              [backend-id (delete-by-backend tx backend-id ids)]))
+       (into {})))
 
 (def ^:private sql:get-deleted-sobjects
-  "SELECT id, backend
-     FROM storage_object
-    WHERE deleted_at IS NOT NULL
-      AND deleted_at <= ?
-      AND status = 'valid'
-    ORDER BY deleted_at ASC")
+  "SELECT s.id, s.backend, s.deleted_at
+     FROM storage_object AS s
+    WHERE s.deleted_at IS NOT NULL
+      AND s.deleted_at <= ?
+      AND s.status = 'valid'
+      AND (s.deleted_at, s.id) > (?, ?)
+    ORDER BY s.deleted_at ASC, s.id ASC
+    LIMIT ?
+      FOR UPDATE SKIP LOCKED")
 
-(defn- pending-row
-  "Takes the columns the GC needs from a `db/plan` row.
+(def ^:private sql:get-first-deleted-sobjects
+  "SELECT s.id, s.backend, s.deleted_at
+     FROM storage_object AS s
+    WHERE s.deleted_at IS NOT NULL
+      AND s.deleted_at <= ?
+      AND s.status = 'valid'
+    ORDER BY s.deleted_at ASC, s.id ASC
+    LIMIT ?
+      FOR UPDATE SKIP LOCKED")
 
-  A plan row is a view over the open result set, valid only while the reduction
-  is running, so it cannot be stored for later: this has to happen inside the
-  reducing function."
-  [row]
-  {:id (:id row)
-   :backend (:backend row)})
+(def ^:private chunk-size
+  "Pending objects read, locked and deleted in each transaction."
+  25)
+
+(def ^:private throttle-every
+  "Deleted objects between throttle pauses."
+  5000)
+
+(def ^:private throttle-ms
+  "Pause applied every `throttle-every` deleted objects, letting the
+  database (and its replicas) breathe on huge backlogs."
+  5000)
+
+(defn- throttle
+  "Sleeps `throttle-ms` when the deleted count crosses a `throttle-every`
+  boundary. Runs between chunk transactions, never holding one open."
+  [prev-total new-total]
+  (when (> (quot new-total throttle-every)
+           (quot prev-total throttle-every))
+    (l/inf :hint "throttling storage gc"
+           :deleted new-total
+           :pause-ms throttle-ms)
+    (Thread/sleep (long throttle-ms))))
+
+(defn- fetch-and-lock-chunk
+  "Reads and locks one chunk of pending objects in `(deleted_at, id)` order.
+  Objects already locked by another process are skipped, so two concurrent
+  runs never take the same chunk. Runs inside the caller's transaction."
+  [conn last-deleted-at last-id]
+  (let [[sql params] (if (some? last-deleted-at)
+                       [sql:get-deleted-sobjects
+                        [(ct/now) last-deleted-at last-id chunk-size]]
+                       [sql:get-first-deleted-sobjects
+                        [(ct/now) chunk-size]])]
+    (db/exec! conn (into [sql] params))))
 
 (defn- clean-deleted
-  "Walks the pending objects with `db/plan`, a server-side cursor that streams
-  the rows instead of materialising them, and deletes them chunk by chunk.
+  "Processes the pending objects chunk by chunk with keyset pagination: each
+  chunk is one short transaction that reads and locks the next `chunk-size`
+  objects after the `(deleted_at, id)` cursor and deletes them (database rows
+  and backend content). Only `valid` objects are read: pending ones (still
+  being written, see the status lifecycle) stay untouched even when already
+  marked as deleted.
 
-  The rows are read in their own read-only transaction and every chunk of
-  `chunk-size` objects is deleted in its own one (see `delete-in-bulk`), so a
-  chunk that fails is logged and skipped instead of aborting the whole run, and
-  the work of the previous chunks is already committed. A physical-deletion
-  failure inside a chunk defers its rows one day and gives up after
-  `max-attempts` attempts (see `process-chunk`)."
-  [cfg]
-  (let [xform (comp (map pending-row)
-                    (partition-all chunk-size)
-                    (mapcat group-by-backend))
-        rf    (fn
-                ([total] total)
-                ([total [backend-id ids]]
-                 (let [deleted (or (delete-in-bulk cfg backend-id ids) 0)]
-                   ;; Beat the job after every chunk: each chunk commits on its
-                   ;; own, so a long sweep neither loses work on a late failure
-                   ;; nor outruns the orphan lease.
-                   (jobs/heartbeat cfg)
-                   (+ total deleted))))]
-    (db/tx-run! (assoc cfg ::db/read-only true)
-                (fn [{:keys [::db/conn]}]
-                  (transduce xform rf 0
-                             (db/plan conn [sql:get-deleted-sobjects (ct/now)]))))))
+  A chunk that fails aborts its transaction: nothing of the chunk is deleted,
+  its locks are released, and it is logged and skipped. The cursor advances
+  to the `(deleted_at, id)` of the last row of each read chunk regardless of
+  the deletion result, so a failed chunk is not visited again in the same
+  run. There is no long-lived read transaction: each transaction lives only
+  as long as one chunk, letting the vacuum proceed between chunks. Rows
+  locked by a concurrent run are skipped and deferred to the next run; a
+  chunk that fails after a partial backend delete self-heals on the next
+  run (physical deletes are idempotent). Every `throttle-every` deleted
+  objects the run pauses `throttle-ms` (see `throttle`)."
+  [{:keys [::mtx/metrics] :as cfg}]
+  (loop [total           0
+         last-deleted-at nil
+         last-id         nil]
+
+    (let [[rows counts]
+          (db/tx-run! cfg
+                      (fn [{:keys [::db/conn] :as tx}]
+                        (let [rows (fetch-and-lock-chunk conn last-deleted-at last-id)]
+                          (try
+                            [rows (process-chunk-deletion tx rows)]
+                            (catch Throwable cause
+                              ;; Explicit rollback, then return normally (do
+                              ;; not rethrow): the caller needs `rows` to
+                              ;; advance the keyset cursor past the failed
+                              ;; chunk. The commit that `tx-run!` performs
+                              ;; afterwards is a JDBC no-op on the already
+                              ;; rolled-back connection.
+                              (db/rollback! conn)
+                              (l/err :hint "unexpected error on chunk deletion"
+                                     :ids (map :id rows)
+                                     :cause cause)
+                              [rows {}])))))]
+
+      ;; Reported only after the chunk commits: a rolled-back chunk leaves
+      ;; an empty count map, so retried rows are never counted twice.
+      (doseq [[backend-id n] counts]
+        (mtx/run! metrics
+                  :id :storage-gc-deleted-objects
+                  :inc n
+                  :labels [(name backend-id)]))
+
+      ;; Beat the job after every chunk: each chunk commits on its own, so a
+      ;; long sweep neither loses work on a late failure nor outruns the
+      ;; orphan lease.
+      (jobs/heartbeat cfg)
+
+      (if-let [cursor (peek rows)]
+        (let [new-total (+ total (reduce + 0 (vals counts)))]
+          (throttle total new-total)
+          (recur new-total
+                 (:deleted-at cursor)
+                 (:id cursor)))
+        total))))
 
 (declare execute-storage-gc-deleted)
 
 (defmethod ig/assert-key ::job-def
   [_ params]
   (assert (sto/valid-storage? (::sto/storage params)) "expect valid storage")
-  (assert (db/pool? (::db/pool params)) "expect valid db pool"))
+  (assert (db/pool? (::db/pool params)) "expect valid db pool")
+  (assert (mtx/metrics? (::mtx/metrics params)) "expect valid metrics instance"))
 
 (def schema:storage-gc-deleted-params
   "Params map (no params needed; cfg-provided config only)."

backend/src/app/storage/gc_touched.clj

@@ -96,6 +96,15 @@
   (-> (db/exec-one! conn [sql:has-job-resource-refs id])
       (get :has-refs)))
 
+(def ^:private
+  sql:has-upload-session-chunk-refs
+  "SELECT EXISTS (SELECT 1 FROM upload_session_chunk WHERE object_id = ?) AS has_refs")
+
+(defn- has-upload-session-chunk-refs?
+  [conn {:keys [id]}]
+  (-> (db/exec-one! conn [sql:has-upload-session-chunk-refs id])
+      (get :has-refs)))
+
 (def sql:exists-file-data-refs
   "SELECT EXISTS (
      SELECT 1 FROM file_data
@@ -165,6 +174,31 @@
                :id (str id))
         sto/default-bucket)))
 
+(def ^:private sql:lock-storage-objects
+  "SELECT id FROM storage_object WHERE id = ANY(?::uuid[]) ORDER BY id FOR UPDATE")
+
+(defn- verify-frozen
+  "Re-checks, holding the row locks, that the about-to-freeze objects are
+  still referenced. A concurrent purge (mapping delete plus touch) either
+  commits before the check and the object moves to deletion, or blocks on
+  the lock and its touch wins last, so no touch is ever swallowed by a
+  stale freeze. Ids are locked in order to avoid lock-order inversions
+  between workers. Returns [still-frozen-ids now-deletable-ids]."
+  [conn has-refs? bucket objects]
+  (let [by-id  (into {} (map (juxt :id identity)) objects)
+        ids    (db/create-array conn "uuid" (sort (keys by-id)))
+        locked (into #{} (map :id) (db/exec! conn [sql:lock-storage-objects ids]))]
+    (reduce (fn [[frozen deletable] {:keys [id] :as object}]
+              (if (and (contains? locked id) (has-refs? conn object))
+                [(conj frozen id) deletable]
+                (do
+                  (l/dbg :id (str id)
+                         :status "delete-after-recheck"
+                         :bucket bucket)
+                  [frozen (conj deletable id)])))
+            [#{} #{}]
+            objects)))
+
 (defn- process-objects
   [conn has-refs? bucket objects]
   (loop [to-freeze #{}
@@ -176,17 +210,24 @@
           (l/dbg :id (str id)
                  :status "freeze"
                  :bucket bucket)
-          (recur (conj to-freeze id) to-delete (rest objects)))
+          (recur (conj to-freeze object) to-delete (rest objects)))
         (do
           (l/dbg :id (str id)
                  :status "delete"
                  :bucket bucket)
           (recur to-freeze (conj to-delete id) (rest objects))))
-      (do
-        (some->> (seq to-freeze) (mark-freeze-in-bulk conn))
-        (some->> (seq to-delete) (mark-delete-in-bulk conn))
-        [(count to-freeze) (count to-delete)]))))
+      (let [[frozen deletable] (if (seq to-freeze)
+                                 (verify-frozen conn has-refs? bucket to-freeze)
+                                 [#{} #{}])
+            doomed (into to-delete deletable)]
+        (some->> (seq frozen) (mark-freeze-in-bulk conn))
+        (some->> (seq doomed) (mark-delete-in-bulk conn))
+        [(count frozen) (count doomed)]))))
 
+;; NOTE: upload-session chunks are 1:1 with their mapping and never
+;; shared, but the scan owns the liveness decision anyway: a referenced
+;; chunk freezes, so gc-deleted never meets a live NO ACTION reference.
+;; The freeze path re-verifies under row locks (see verify-frozen).
 (defn- process-bucket
   [conn bucket objects]
   (cond
@@ -197,7 +238,7 @@
     (= bucket "profile")                 (process-objects conn has-profile-refs? bucket objects)
     (= bucket "file-data")               (process-objects conn has-file-data-refs? bucket objects)
     (= bucket sto/tempfile-bucket)       (process-objects conn (constantly false) sto/tempfile-bucket objects)
-    (= bucket sto/upload-session-bucket) (process-objects conn (constantly false) sto/upload-session-bucket objects)
+    (= bucket sto/upload-session-bucket) (process-objects conn has-upload-session-chunk-refs? sto/upload-session-bucket objects)
     (= bucket sto/job-resource-bucket)   (process-objects conn has-job-resource-refs? bucket objects)
     (= bucket "organization")            (process-objects conn (constantly false) bucket objects)
     :else

backend/src/app/tasks/delete_object.clj

@@ -13,6 +13,7 @@
    [app.db :as db]
    [app.db.sql :as-alias sql]
    [app.jobs :as jobs]
+   [app.loggers.audit :as audit]
    [app.rpc.commands.files :as files]
    [app.rpc.commands.profile :as profile]
    [app.storage :as sto]
@@ -20,6 +21,29 @@
 
 (def ^:dynamic *team-deletion* false)
 
+(def ^:dynamic *cascade*
+  "Accumulates the ids marked deleted by the current cascade execution,
+  grouped by entity. Bound by the task entry point; nil on direct calls
+  (e.g. objects-gc, whose purges are announced by their own procs)."
+  nil)
+
+(defn- record
+  [kind id]
+  (when (some? *cascade*)
+    (swap! *cascade* update kind conj id)))
+
+(defn- top-exists?
+  "The cascade marks blindly, so an unknown top id would report phantom
+  rows. Every queue submission and SREPL call names a real row; only
+  check, never change behavior: a missing row just skips the run."
+  [{:keys [::db/conn]} {:keys [object id]}]
+  (let [object (when (some? object) (keyword (name object)))]
+    (if (contains? #{:team :project :file :profile} object)
+      (some? (db/get* conn object {:id id}
+                      {::db/remove-deleted false
+                       ::sql/columns [:id]}))
+      true)))
+
 (defmulti delete-object
   (fn [_ props] (:object props)))
 
@@ -45,7 +69,7 @@
       AND deleted_at IS NULL")
 
 (defmethod delete-object :file
-  [{:keys [::db/conn] :as cfg} {:keys [id deleted-at]}]
+  [{:keys [::db/conn] :as cfg} {:keys [id deleted-at file-ids]}]
   (when-let [file (db/get* conn :file {:id id}
                            {::db/remove-deleted false
                             ::sql/columns [:id :is-shared]})]
@@ -103,16 +127,20 @@
                (not *team-deletion*))
       ;; NOTE: we don't prevent file deletion on absorb operation failure
       (try
-        (db/tx-run! cfg files/absorb-library id)
+        (db/tx-run! cfg files/absorb-library id :file-ids file-ids)
         (catch Throwable cause
           (l/warn :hint "error on absorbing library"
                   :file-id id
                   :cause cause))))
 
-    ;; Mark file change to be deleted
+    ;; Mark file change to be deleted, but only the rows that do not carry a
+    ;; date yet. A revision is born already scheduled for deletion one hour
+    ;; later, and rewriting its date here would push it out to the trash
+    ;; lifetime; the collector would still take it, just much later than it
+    ;; should.
     (db/update! conn :file-change
                 {:deleted-at deleted-at}
-                {:file-id id}
+                {:file-id id :deleted-at nil}
                 {::db/return-keys false})
 
     ;; Mark file data fragment to be deleted
@@ -136,7 +164,9 @@
     (db/update! conn :file-tagged-object-thumbnail
                 {:deleted-at deleted-at}
                 {:file-id id}
-                {::db/return-keys false})))
+                {::db/return-keys false})
+
+    (record :files id)))
 
 (defmethod delete-object :project
   [{:keys [::db/conn] :as cfg} {:keys [id deleted-at]}]
@@ -153,7 +183,9 @@
                          {::db/columns [:id :deleted-at]})]
     (delete-object cfg (assoc file
                               :object :file
-                              :deleted-at deleted-at))))
+                              :deleted-at deleted-at)))
+
+  (record :projects id))
 
 (defmethod delete-object :team
   [{:keys [::db/conn] :as cfg} {:keys [id deleted-at]}]
@@ -175,7 +207,9 @@
                               {::db/columns [:id :deleted-at]})]
       (delete-object cfg (assoc project
                                 :object :project
-                                :deleted-at deleted-at)))))
+                                :deleted-at deleted-at))))
+
+  (record :teams id))
 
 (defmethod delete-object :profile
   [{:keys [::db/conn] :as cfg} {:keys [id deleted-at]}]
@@ -202,13 +236,30 @@
    [:object [:enum :snapshot :team :project :profile :file]]
    [:deleted-at ::ct/inst]
    [:id ::sm/uuid]
-   [:file-id {:optional true} ::sm/uuid]])
+   [:file-id {:optional true} ::sm/uuid]
+   [:file-ids {:optional true} [:vector ::sm/uuid]]])
 
 (defn execute-delete-object
   "Plain job handler: run the delete-object multimethod on the provided
-  params inside a single transaction."
+  params inside a single transaction, tracking the cascade and reporting
+  it as an audit event when anything was marked."
   [cfg params]
-  (db/tx-run! cfg delete-object params))
+  (db/tx-run! cfg
+              (fn [cfg]
+                (when (top-exists? cfg params)
+                  (binding [*cascade* (atom {:teams [] :projects [] :files []})]
+                    (delete-object cfg params)
+                    (let [{:keys [teams projects files]} @*cascade*]
+                      (when (or (seq teams) (seq projects) (seq files))
+                        (audit/submit cfg {:name "delete-object-with-cascade"
+                                           :type "action"
+                                           :props {:object (:object params)
+                                                   :id (:id params)
+                                                   :deleted-at (:deleted-at params)
+                                                   :cascade {:team-ids teams
+                                                             :project-ids projects
+                                                             :file-ids files}}
+                                           :context {:triggered-by "delete-object"}}))))))))
 
 (defmethod ig/assert-key ::job-def
   [_ params]

backend/src/app/tasks/file_gc_scheduler.clj

@@ -30,7 +30,7 @@
       FOR UPDATE OF f
      SKIP LOCKED")
 
-(defn- schedule!
+(defn- schedule-files
   [{:keys [::db/conn] :as cfg} threshold]
   (let [total (reduce (fn [total {:keys [id modified-at revn]}]
                         (let [params {:file-id id :revn revn}]
@@ -84,4 +84,4 @@
                       (ct/in-past))]
     (-> cfg
         (assoc ::db/rollback (:rollback? params))
-        (db/tx-run! schedule! threshold))))
+        (db/tx-run! schedule-files threshold))))