New code grows under `exporter.*` instead of extending `app.*`:
`exporter.main` holds the future entry point, `exporter.utils.system`
is an async-aware Integrant-style lifecycle (`init-key`/`halt-key`
multimethods, `ref`s, dependency ordering, `^:async` + `await`, no
promesa chains), and `exporter.browser` ports the whole browser
namespace as a pool service with an injectable factory. Each piece
ships with its tests, registered in the test runner.
AI-assisted-by: muse-spark-1.3-contributor
Three requires nobody reads (`app.common.uuid` in the consumer
worker, `app.common.exceptions` in the api test, `node:path` in the
worker test), the `original-fetch`/`original-render` vars left over
from before the fakes returned their own restore thunks, and the
`port` and `terminated` atoms no test ever reads.
AI-assisted-by: muse-spark-1.3-contributor
Mechanical rename, no behavior change: every function the exporter
defines loses the `!`, at the definition and at every call site, in
`src` and in the tests. Two names could not just drop it: the
mutating `merge!` shares the namespace with a persistent `merge` of
another contract, so it becomes `merge-into`; the test restore thunks
take the role as their name (`fetch-restore`, `render-restore`). Two
dead pieces go away with their bangs: the never-called `rmdir!` and
an unused `mark!` binding in the cancel test. Untouched on purpose:
the stdlib (`swap!`, `run!`, `try!`, …), the `set!` special form,
`common` (`serialize-shape!`, `write-shape-*`), and the shared-key
config the deploy still reads.
AI-assisted-by: glm-5.3-flash
The widget could never say queued: the creation answers the row
status (`pending`) and the widget only knew `queued`, so every export
opened on the generic exporting wording. The widget now names a fresh
job queued, and only the first milestone moves it to running — the
worker sends that first breath right after the claim, before the
session roundtrip. The figure is a percentage once the worker said how
much there is, and nothing while it has not: a milestone without
counters names the stage but no longer blanks the figures, and the
queued and terminal states show no counter at all.
AI-assisted-by: glm-5.3-flash
A cancelled export is a normal ending, but the run treated it like a
failure: an error log with a full trace for something the user asked
for. Worse, the watchdog interval outlived the run — its stopper was
bound inside the scope the failure unwinds, so a cancelled or failed
job kept beating on a dead row forever, one HTTP call a second that
only ever answers skip. The stopper now travels in an atom the catch
reaches, the run stops beating on every ending, and a cancel logs a
single info line where a failure keeps its error with the cause.
AI-assisted-by: glm-5.3-flash
The claim carries the job row JSON: every id a string. The run handed
them to the renderer as-is, and the render spec only takes uuid
objects, so every export died on `render-params`. The run now parses
the item ids at the normalize boundary, threads the job-id through
the plan (it never arrived there: the render saw nil), and omits an
absent share-id instead of naming nil.
AI-assisted-by: glm-5.3-flash
Every poller turn died on `claim-job` with `missing-key` for `job-id`
and `scheduled-at`: the JSON calls never declared their content type,
so the backend never parsed the body as transit and validated an empty
map. The header goes on the transit calls only; the multipart settle
keeps the content type its own FormData settles.
AI-assisted-by: glm-5.3-flash
The exporter is a consumer's process now, so everything that only the
HTTP surface held goes away with it: the server, the router and the
cookie auth; the export handlers (the runner up past their pieces
through `app.consumer.plan`, and `handlers/resources` keeps only the
local artifacts); the scheduler with its admission caps; the redis
job store, publish and cancel-sweeper; and with the store gone, the
redis engine too — the pollers own plain ioredis connections.
`app.jobs` is the local cancel arm only: a runtime record the claimed
job is registered under, the flag the parts of a run read between
renders, the shared-array signal a render thread reads inside a render,
the terminate callbacks, and a `mark-cancelled` that marks the record
terminal before the callbacks run. The row of the job was never this
process's business.
The pools size themselves to the worker concurrency, which is what a
process of K pollers can run at once; there is no independent pool cap
nor scheduler admission left. Docker: the nginx export proxy and the
exporter uri die with the endpoints they pointed at; the compose keeps
the exporter env thin and honest, the README describes the worker.
AI-assisted-by: glm-5.3-flash
Nothing of the old exporter deserves preserving: once the plan ends
there is no HTTP surface, so the default of `PENPOT_EXPORTER_ROLES`
stops being `http` and becomes `worker`. The process that dev and
deploy have always started (the tmux window's recipe and the compose
service, both untouched) now boots the K pollers of the backend queue;
`http` stays selectable by env only until the deletion task takes the
legacy surfaces away.
AI-assisted-by: glm-5.3-flash
A cancellation reaches the worker through its beats: a `skip` answer
from `report-job-progress` says the job can no longer hear it. The
runner raises `:job-cancelled` between units of work (the same
exception the legacy surfaces raised), and the render mid-Skia dies by
terminate: the watchdog of the beats fires the cancel port the wasm
pool already serves (`jobs/mark-cancelled` raises the flag, arms the
shared-array signal and runs the terminate callbacks
`renderer.wasm/with-scope` registered), and the run settles as
`fail-job` with the session id, which the backend answers skip on a row
it already cancelled.
The settle is what ends the render session's life, whatever the write
says: complete and fail now close the named session even when they
lose the write, so no settle of the worker leaves a session behind.
The process boots by roles (`PENPOT_EXPORTER_ROLES`): `http` boots the
server and its pools exactly as today, `worker` boots K pollers of the
backend queue, and both roles coexist during the transition sharing
the engines; the shutdown unwinds the worker before redis, so no poller
pops on a closing connection.
AI-assisted-by: glm-5.3-flash
The `:exporter` queue gets its consumer: K pollers (the new
`:exporter-worker-concurrency`), each one with its own Redis
connection, so the blocking BLPOP is both the semaphore of the slot and
the backpressure: a poller claims one job and does not pop again until
the job settles. The dispatcher's `[job-id, scheduled-at]` payloads are
read per pop; a corrupt payload is logged and dropped, a Redis hiccup
is a failed turn that the loop retries after a small delay, and no
test exercises the forever loop directly.
The run of the `:export-assets` job ports the legacy export machinery
to the substrate: the item transducers and partitioning of
export-shapes, the zip assembly and the pdfunite join of export-frames,
rendering as the session the backend minted. The progress contract
dresses the beats (`:preparing`, `:rendering`, `:packaging` with the
`objects` and `pages` counters, throttled per object) and the settle is
one `complete-job` multipart that stores the artifact and closes the
session; an export failure settles as `fail-job` with the session id,
self-contained, so the loop only ever sees a resolved run.
AI-assisted-by: glm-5.3-flash
The worker side of the exporter gets its config: `PENPOT_EXPORTER_ROLES`
(`http` by default, `http,worker` during the transition) and
`PENPOT_EXPORTER_WORKER_CONCURRENCY` (K pollers, the concurrency of the
process, floored at one), both read from the live config, never at
import time; every accessor is a plain function so the tests do not
journey through globals. The queue key it polls is
`penpot.worker.queue:<tenant>:exporter`, the same one the dispatcher
pushes.
The management client (`app.consumer.api`) is the only place of the
worker that speaks HTTP: claim, progress, session, complete and fail,
all POSTs with `X-Shared-Key: exporter <key>`, transit+json bodies and
transit answers. The artifact form of `complete-job` rides multipart
(the undici FormData the uploads already use); a non-200 answer arrives
as the failure of the call, carrying the status and the error body.
AI-assisted-by: glm-5.3-flash
* ♻️ Share structural batch upload through common helper
- Merge svg-filters and svg-fills to app.common.render-wasm.svg-derived
- Add serialize-shapes-batch! in common, shared by the sync and chunked
workspace paths
- Add a routing test for the helper and wires the svg-filters test.
AI-assisted-by: muse-spark, GLM 5.3
* 🐛 Derive SVG effects inside single-shape serializer
- Single and batch paths: one svg effect derivation step
owned by shared serializers.
- set-object forwards the derived shape to its host attrs,
and the exporter reads the derived fills, so SVG-attr
fills, blur and shadow render as in the frontend.
- Adds regression test to the exporter.
AI-assisted-by: muse-spark, GLM 5.3, deepseek-flash
* ✨ Add export job model, store and scheduler to exporter
* ✨ Render wasm exports on pooled worker threads
* ✨ Add export job REST API to exporter
* ✨ Use export job API and allow cancelling wasm exports
* 🔧 Show export jobs in the internal debug panel
* 🔧 Pass flags and export job settings to the exporter container
* 📚 Document the exporter job API and its redis layout
* 🐛 Use gradient type instead of export type in SVG renderer
data->gradient-def was comparing the render `type` parameter (:svg,
:png, :pdf) against "linear" to decide between linearGradient and
radialGradient elements. Since the export type is never "linear",
the comparison always fell through to radialGradient, causing all
linear gradients to be exported as radial in SVG output.
Read the gradient type from the data map instead:
(get-in data ["gradient" "type"])
Closes#5972
* 🐛 Add SVG gradient export regression test
Extract SVG gradient definition generation from the renderer so it can
be tested directly. Add exporter test build wiring and cover both
linear and radial gradient output.
AI-assisted-by: gpt-5.6-luna
* ✨ Standardize exporter testing workflow
Align exporter scripts with the frontend testing pattern. Add a
dedicated GitHub Actions workflow and document the canonical exporter
commands in Serena memories.
AI-assisted-by: gpt-5.6-luna
* ✨ Add focused exporter test execution
Mirror frontend test-runner behavior for focused namespaces and test
vars. Support --focus, --log-level, and --help, and document the
commands.
AI-assisted-by: gpt-5.6-luna
* 🐛 Replace shell exec with execFile in exporter
Replace child_process.exec with execFile to eliminate shell
interpretation. Add hex color validation in exporter and frontend
to reject malformed input before command construction.
This fixes GHSA-4f36-m4hj-cv86 (CVSS 9.9 Critical), an authenticated
OS command injection vulnerability where malicious fill-color values
could execute arbitrary commands in the exporter container.
Defense in depth:
- Layer 1: execFile passes arguments directly without shell parsing
- Layer 2: Exporter validates colors with strict hex regex
- Layer 3: Frontend filters invalid colors before DOM emission
All three independent reporters' attack vectors are addressed:
- Quote breakout (lyhtheori)
- Command substitution (B1gN0Se)
- Path traversal (KimiSecurityTeam)
AI-assisted-by: qwen3.7-plus
* 🐛 Use existing hex-color-string? and fix test path mismatch
Address code review feedback:
- Replace duplicated hex-color-rx and valid-hex-color? with existing
hex-color-string? from app.common.types.color
- Fix RCE test to use marker path in payload instead of hardcoded /tmp/pwned
AI-assisted-by: qwen3.7-plus
---------
Co-authored-by: Sumit Ridhal <sridhal@redhat.com>
