* 🐛 Confine upload paths to temp dirs
Reject upload maps whose path leaves the server temp dirs.
Transit bodies can forge a Path such as /etc/passwd and the
handlers open it directly. Centralize the realpath prefix
check in media validation and apply it to all nine upload
fields. Legit multipart files under /tmp/undertow/ or
/tmp/penpot/ still pass.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Rename upload validators to check-*
Drop the validate prefix and the bang from the media
validation helpers for a consistent check naming. Move
the temp dir confinement into validation and cover all
nine upload fields. Add unit tests for each helper.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix svg upload tests to use penpot tmp dir
SVG upload tests wrote tempfiles to the system tmp dir,
outside the allowed upload prefixes, so check-upload-path
rejected them. Point write-temp-svg at /tmp/penpot like
the other upload tests.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Rework the import dialog
The import wizard paints a Check/Upload/Import stepper derived from the
dialog status, plus a phase title, a description and a progress bar per
phase, and its rows follow the new design: folder icon, file name and a
size subtitle. While checking, the dialog shows no rows and no spinner,
because the phase header already carries the message.
The wizard writes the job milestone once, in the header
(`jp/current-milestone-text`), since every file of a container reports
the same one. A row keeps its name and its size, says "Ready" once its
container finished uploading, and shows the error text when it fails;
the button that drops a file only appears while the wizard still offers
its import step. The `jobs.queued` label now reads "Waiting job slot".
The bar measures the whole import: determinate while the containers are
uploaded, weighted by their size (`imp/upload-pct`), and animated
otherwise, because the stages of a job are not comparable. The phase
title counts the files, and `imp/imported-count` counts them done with
the job's own counter, so the number moves while a container rebuilds
its files one by one.
The analysis reads the entries of a version 3 package in one pass: a
file shows what it occupies inside its container, its own entries plus
the storage objects its media and thumbnails reference, while what the
container weighs stays in `:container-size`, the size the upload bar is
weighed on. The ready step waits for the end of the analysis
(`imp/ready-to-import?`), not for every known file, so Continue never
leaves out the files that arrive later.
Tests cover the file sizes, the upload percentage, the done counter,
the wizard step and the milestone in course. The change also adds
`-XX:NativeMemoryTracking=summary` to the backend devenv so `jcmd`
shows native memory, and documents the wizard, the promise call styles
(`mem:clojure/promesa`) and two CSS pitfalls: a logical `inline-size`
wins over an inline physical `width`, and Rumext leaves a plain
`:style` map raw, so inline styles leave CLJS as `#js` objects.
AI-assisted-by: glm-5.3-flash
AI-assisted-by: deepseek-v4.1-flash
* ✨ Rework the export dialog
Every exported file is its own job, so each row reports its own state:
folder icon, name and a size-plus-`.penpot` subtitle; waiting for a
worker; running with the milestone of the job and an animated bar; or
exported with a tick. A failed row paints red, takes the close icon and
shows the hint of the job, or a generic label, through
`files-export-modal.export-error`.
Success, failure and cancellation replace each other, so a row never
reports two outcomes at once. The ids of the messages are canonicalized
to string before they match a row, because they cross serialization
boundaries. The export result now carries `:size`, so the subtitle says
the format until the job produces the artifact and the size once it
exists; a transport failure builds a synthetic error, so a nil
`ex-data` no longer leaves the row hanging.
The header counts the files and says whether the export runs or ended;
since every file has its own job, it paints no global bar.
Closing changes too: the Close button is gone. While files are in
flight the footer offers Cancel export and the X cancels as well, which
marks the loading rows cancelled and leaves them in place, so their
outcome can be read; once every row reached a terminal state the footer
offers Done, which dismisses the dialog.
The retry button of the finished design is left out, because the client
has no retry mechanism yet. Styles come along: the rows scroll while
the header and the footer stay put, and the new phase header, row and
bar follow the tokens of the import wizard. The title key becomes
`files-export-modal.title`, and the new keys cover the header, the row
states and the buttons.
AI-assisted-by: muse-spark-1.3
* ♻️ Inline the binfile job adapters into their task handlers
The wrappers in app.binfile.jobs only forwarded to the core, so each
task handler now composes its own run: frozen cfg, active check,
progress bridge and format dispatch. export-cfg freezes the whole
export (file-ids, export-type, output) and v3/export-files! takes it
as its single argument; ::bfc/ids becomes ::bfc/file-ids. The bridge
keeps one job: bfj/run forwards the cfg to its callback, and
check-active answers with it to thread the run.
AI-assisted-by: muse-spark-1.3
* 🐛 Migrate binfile tests to the single-arg export-files!
The refactor made v3/export-files! take a single cfg carrying
::bfc/file-ids and ::bfc/output, but six calls in binfile_test
(added by the tokens handling change) still used the old two-arg
form with ::bfc/ids. Move them to the new form, as the rest of
the file already does.
AI-assisted-by: muse-spark-1.3
* 🐛 Fix export artifact expectations to include file size
The export flow now reports the artifact size, but the stale
expectations dropped it and failed the frontend suite. Include the
size in both assertions so they match the emitted message.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Migrate remaining export-files! test callers to the single-arg form
rpc_binfile_test and offload_file_data_test still passed the output apart
and used ::bfc/ids; move both into the cfg as ::bfc/file-ids/::bfc/output.
* 🐛 Hide the downloads hint when no file was exported
After Cancel or when every job failed, the export header said
"0 files exported · Check your downloads folder."; drop the hint then.
* ♻️ Read the object of a container entry once
The index of a container asked twice for the object of the same zip
entry: once to test it and once as the key of the update. Resolve it
once, in the branch that owns it, so the entries of a file never pay
for a question that does not apply to them.
Addresses a review note on #12192.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Never size an import row with its container's weight
A row of a version 3 package took the size of the container whenever the
container held no entry for its file, while the comment beside it
promised the opposite. An unknown size now stays nil, which the row
already understands: it paints no subtitle instead of a number that
belongs to the whole package.
The test carries a file the manifest lists and the container does not
hold, and the memory records what the row does then.
Addresses a review note on #12192.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Retry pool stats read until snapshot is consistent
Hikari exposes each pool counter through a separate MXBean call, so a
borrow or return between the reads can leave one snapshot off by one.
The test read once and failed in CI with total 2 vs active + idle 3.
Read up to 20 times with a 10ms pause until total equals active +
idle, then assert on the stable snapshot. The invariant stays strict;
only the transient race is tolerated.
AI-assisted-by: muse-spark-1.3-contributor
---------
Co-authored-by: Elena Torro <elenatorro@gmail.com>
* 🚑 Reject forged upload paths outside multipart
Transit bodies can smuggle upload maps whose path points at any
server file; the schema only checks the shape and handlers open
the path directly.
app.rpc now injects a server-side multipart flag that bodies cannot
forge, and methods declare their upload keys with the new
multipart-only-params metadata, enforced centrally after schema
validation with a params-validation error. Covers all nine upload
methods, including chunked and management flows.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Sanitize raw explain in generic validation error bodies
The generic :validation branch returned ex-data untouched, so errors
carrying the raw malli explain (live schema objects, e.g. storage
invalid-storage-metadata) broke JSON encoding mid-stream and clients
got truncated bodies. Apply the same clean treatment the known codes
already use: drop the raw explain and attach its humanized text.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Add e2e path traversal tests for binfile and media upload
Add two e2e suites that hand-write transit bodies with a forged
["~#path", ...] upload path (no transit library needed) and expect
400 validation/params-validation. They cover import-binfile :file,
upload-file-media-object :content and upload-chunk :content, plus a
plain-JSON string-path control case. Before the multipart-path fix
they failed (500 io-exception, invalid-image, 200 stored chunk);
after it they pass.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Use multipart helper in binfile SVG test setup
The SVG import setup uploaded through plain command!, so the new
multipart-only guard rejected it with params-validation. Switch
the setup to multipart-command! like the other upload tests.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Use multipart helper in remaining upload tests
The multipart-only upload guard rejects plain RPC calls carrying
upload maps. Five tests with real files still used command! and
failed with params-validation. Switch them to multipart-command!
while keeping the forged-content tests on plain command!.
AI-assisted-by: muse-spark-1.3-contributor
* ✨ Enrich backend audit events with team and entity ids
Backend audit events now carry the data a consumer needs to rebuild
state without joining production tables:
- prepare-rpc-event adds :team-id when the event carries a project or
file id; commands that already loaded the team set it themselves.
- add-profile-to-team! emits add-team-member on every real insert.
- create-project and create-file emit the new row id.
- clone-template, import-binfile and duplicate-project emit one event
per created entity.
- both binfile versions return {:file-ids [...]} from import-files!.
Closes#12219
AI-assisted-by: deepseek-v4.1-flash
* ✨ Emit delete and hard-delete audit events with entity data
The event stream must let Nexus rebuild local state without joining
production tables. This completes the enrichment with the delete paths:
- delete-team keeps returning nil but now carries the deleted row
(id, name, timestamps) in the audit props, like delete-project.
- create-team adds created-at to its audit props.
- delete-profile carries id, email, fullname and deleted-at; its
props used to be empty because every param is namespaced.
- objects-gc emits hard-delete-team/project/file per purged batch,
with the deleted ids, only when the audit log is on.
- delete-team-member needs no change: the delete is physical and the
params already carry team-id and member-id.
The nitrate push-audit-events tests now drop the membership events
emitted by their own profile setup before counting.
Closes#12219
AI-assisted-by: muse-spark-1.3-free
* ✨ Announce cascade deletes and missing ids in audit events
The :delete-object worker task marked teams, projects and files without
leaving any event, so an event-sourced projection kept them alive for
weeks. It now emits one cascade-deleted event per execution with every
affected id, in the same transaction, only with :audit-log on. Rows
purged by objects-gc and snapshot cascades stay silent on purpose.
Also: create-team-with-invitations carries the new team id, and
update-team-member-role names the demoted member on owner transfers.
Closes#12219
AI-assisted-by: muse-spark-1.3-free
* 🐛 Identify the SREPL profile delete in audit events
The SREPL delete-profile! emitted no profile-id and empty props, so the
row could not be attributed by any consumer. It now carries the profile
id in both fields. delete-team! named the wrong cause in its context.
Closes#12219
AI-assisted-by: muse-spark-1.3-free
* 🐛 Identify the SREPL profile restore in audit events
The SREPL restore-profile! emitted no profile-id, so the consumer
keyed on the zero profile instead of the restored one. It now carries
the profile id like a normal profile event, and no consumer special
case is needed.
Closes#12219
AI-assisted-by: muse-spark-1.3-free
* 🐛 Stop attributing SREPL events to the subject profile
profile-id names the actor, and SREPL calls have none: setting the
subject id blamed the user for an admin action and would have leaked
those rows into PostHog as user activity. The subject id stays in
props, where the consumer resolves it, and the event profile stays
zero.
Closes#12219
AI-assisted-by: muse-spark-1.3-free
* ✨ Expose file analysis data for the nexus pipeline
The nexus file-security pipeline needs file content and scheduling
state from the backend. This adds a read-only management method,
`get-file-for-analysis`, returning the semantic projection of one
file (pages, frames, texts, links, interactions; ids as strings;
capped with a truncated flag), plus identity, revn, timestamps and
share state. No prior projection code existed, so the method ships
its own pure `file->projection`.
Audit events now carry what the `:local` consumer schedules from:
revn and modified-at on update-file, name and timestamps on every
create-file emitter, deleted-at on delete-file, and new
create/delete-share-link events with file and link ids.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Rename deletion audit events to action names
Event names are actions, not past tense: `cascade-deleted` becomes
`delete-object-with-cascade` with the same shape and transaction.
The three `hard-delete-*` events merge into `hard-delete-objects`:
one event per purged chunk carrying only that chunk's entity key
(`:team-ids`, `:project-ids` or `:file-ids`), because each GC
section purges in its own chunk transactions and a single event
could never share them all. Every submit stays inside its delete
transaction, `triggered-by` stays annotated, and silent paths stay
silent.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Read worker parallelism knobs with plain config keys
The runner wiring looked up `:app.main/worker-*-parallelism`
(namespaced) while the schema and the environment provide plain
keys, so `PENPOT_WORKER_DEFAULT_PARALLELISM` and
`PENPOT_WORKER_WEBHOOK_PARALLELISM` never applied and workers
always ran with the fallback. Read the keys the config holds.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix binfile audit test with a real file row
`submit-file-created-event` reads the light columns back from the
row, so the mocked importer must return an existing file id: a
phantom id emits no event and the test saw zero `create-file`
events. Create the file first and use its id. The audit-log memory
now tells both bulk-event helpers apart.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix audit event fidelity from review findings
The review of the audit enrichment found data-fidelity gaps; fix
them while the branch is fresh.
Membership events now belong to the actor: the flows where the
owner adds somebody else pass an explicit actor id, the role is
derived from the flags (the table has no role column), and the
event carries the request context like its siblings.
Also: team-id resolution skips read events, duplicate-project
emits the project name and timestamps, delete-profile is
idempotent again, the GC tests assert physical purge, and
get-file-for-analysis leaves an audit trace.
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Nest cascade ids under :cascade in delete event
The cascade event carried every affected id flat next to the
object identity; nest them under `:cascade` so the payload reads
as header plus affected set. The event is an action like its
siblings, not a command. Drop the `:audit-log` flag re-check:
`submit` already fans out per flag.
AI-assisted-by: muse-spark-1.3-contributor
* 📎 Update copyright name on file header
The company name changed; a single header in this branch still
carried the old one. Align it with the current form.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Fix nil event-id on webhook trigger audit rows
process-event generated the row id inside append-audit-entry, so the
:process-webhook-event task never received it and the trigger row
stored a nil :event-id. Assign the id up-front and share it between
the stored row and the worker params; the telemetry shadow row keeps
its own id to avoid a PK collision. Add regression tests covering
the handoff and the end-to-end trigger row.
AI-assisted-by: muse-spark-1.3-contributor
`resolve-and-link-libraries` asserts `::bfc/team-id`, and
`enable-backend-asserts` is now enabled in `scripts/ci` and
`tests-backend.yml`. Without a team-id, the v3 import tests
error instead of passing. Production always provides the
team (RPC import, management and debug routes); mirror that
in the tests.
Update `mem:backend/testing` and `mem:scripts/ci` with the
`enable-backend-asserts` behavior.
Verified with `./scripts/ci backend`: 1235 tests, 6384
assertions, 0 failures.
AI-assisted-by: deepseek-v4.1-flash
* 🐛 Sanitize smuggled SVG on binfile import
Route imported SVG storage objects through the SVG sanitizer on
all three binfile paths (v3, v1, v2), closing the second bypass
of GHSA-ffhp-m958-qxvr. Integrity checks still run on the raw
bundle bytes first; only the persisted copy is the sanitized one.
Adds a shared sanitize-imported-svg helper with unit tests and a
v3 export-tamper-import test proving smuggled scripts no longer
survive the import.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Normalize SVG content-type on binfile import
Close the residual GHSA-ffhp-m958-qxvr bypass where a crafted
content-type spelling (uppercase, parameters) skipped the import
sanitizer. Detection now uses a shared case-insensitive predicate,
stored values are canonicalized, and the v3 re-read honors the
import size limit.
Extends the tamper tests to obfuscated spellings and adds
exhaustive v1 coverage, including the tempfile branch.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Gate binfile content-type on known storage types
Define the complete set of storage content-types in
app.common.media and enforce it on the binfile import schema, so
unknown types fail closed instead of passing through. Detection
keeps a single normalization at the boundary with an exact
predicate, and the string helpers use cuerdas.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Keep only produced types in storage-object-types
Drop apng, avif, penpot and plain text from the set: no flow
stores them, they only exist in the extension mapping table.
Every member must have a producing code path; unproduced types
stay out until some flow actually stores them.
AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Reject unknown content-type on v1 binfile import
Enforce the storage content-type allowlist on the v1 import
path, like v3 already does via schema. Bundles declaring types
outside the set, or none at all, fail closed with the same
media-type-not-allowed error as uploads. The sanitize branch is
explicit: SVG bytes are sanitized, anything else passes through.
AI-assisted-by: muse-spark-1.3-contributor
