github/penpot
1
0
Fork 0
mirror of https://github.com/penpot/penpot.git synced 2026-08-29 08:08:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

📚 Document the exporter job API and its redis layout

Browse Source
This commit is contained in:
Elena Torro 2026-08-26 11:26:49 +02:00
parent 166234be47
commit 11e2841ebd
1 changed files with 98 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

98
exporter/README.md Normal file
View File

@ -0,0 +1,98 @@
# Exporter
Node service that renders shapes and files to bitmap, SVG and PDF. Wasm exports
are **jobs**: created over HTTP, admitted by a scheduler with bounded
concurrency, and persisted in Redis so their state can be queried and cancelled.
The legacy entry point, which is what the browser backend still goes through,
runs the export as soon as it is asked for, with no admission control.
## HTTP API
Mounted under `/api/export` (the router matches on the path *after* that prefix,
so it also works when the process is hit directly on `/`).
| Method | Path | Description |
|----------|-----------------|----------------------------------------------------|
| `POST` | `/` | Legacy command multiplex; runs unscheduled |
| `POST` | `/jobs` | Create an export job |
| `GET` | `/jobs/{id}` | Job record |
| `DELETE` | `/jobs/{id}` | Request cancellation |
Job states: `queued` -> `running` -> `ended` | `error` | `cancelled`. The last
three are terminal.
## Redis layout
Every key is namespaced with the tenant (`PENPOT_TENANT`, `default` in code but
set to the workspace name in devenv, e.g. `devenv-ws0`), matching the prefix
used for pub/sub topics.
```
{tenant}.export.job.{job-id} hash field: data (transit blob of the whole
record)
{tenant}.export.job-cancel pubsub payload: the job id, one line
```
There is no index: the keyspace is one self-expiring hash per job and nothing
else. Each hash carries the same TTL as the exported file
(`PENPOT_EXPORT_JOB_TTL`, default 3600s), refreshed on every write and never
after the job settles.
## Inspecting Redis
Redis is not published on the host, so `redis-cli` from your machine gets
connection refused. Run it **inside the devenv container**, against the `valkey`
host on database 0:
```bash
redis-cli -h valkey -n 0
```
`redis-cli -u "$PENPOT_REDIS_URI"` does the same and follows whatever the env is
set to (`redis://valkey/0` in devenv).
Keys are prefixed with the tenant, which in devenv is the **workspace name**
(`$PENPOT_TENANT`, e.g. `devenv-ws0`), not `default`. From the prompt:
```
# every job record
KEYS devenv-ws0.export.job.*
# the whole record, transit-json in the `data` field
HGET devenv-ws0.export.job.<job-id> data
# seconds left before the record expires
TTL devenv-ws0.export.job.<job-id>
# watch cancellations as they are published (blocks the connection)
SUBSCRIBE devenv-ws0.export.job-cancel
# drop one record
DEL devenv-ws0.export.job.<job-id>
```
`KEYS` is fine here -- the keyspace is a handful of job hashes. On a real
deployment use `SCAN 0 MATCH <tenant>.export.job.* COUNT 100` instead. Do not
`FLUSHDB`: the backend shares this database.
The backend debug UI also renders these records: `/dbg` has an *Export jobs*
section, with a `?job-id=` filter.
## Configuration
| Variable | Default | Description |
|---------------------------------------|---------|--------------------------------------|
| `PENPOT_REDIS_URI` | `redis://redis/0` | Job store and cancel topic |
| `PENPOT_TENANT` | `default` | Key and topic prefix |
| `PENPOT_EXPORT_JOB_TTL` | `3600` | Lifetime of a job record, in seconds |
| `PENPOT_EXPORT_MAX_CONCURRENT_JOBS` | `4` | Admission limit |
| `PENPOT_EXPORT_MAX_JOBS_PER_PROFILE` | `2` | Per-profile admission limit |
| `PENPOT_EXPORT_QUEUE_MAX` | `64` | Queue cap; over it, `429 :queue-full` |
| `PENPOT_WASM_WORKER_POOL_MAX` | `2` | Headless render worker threads; min 1 |
| `PENPOT_WASM_WORKER_POOL_MIN` | `1` | Workers kept warm; clamped to the max |
| `PENPOT_WASM_RENDER_IDLE_TIMEOUT` | `300` | Silence before a worker is terminated, in seconds |
| `PENPOT_WASM_IMAGE_CACHE_MB` | `128` | Per-worker image cache budget |
A headless job leases one render worker for its whole run, so it is admitted
only when a worker is free: `PENPOT_WASM_WORKER_POOL_MAX` is the real limit for
them, and `PENPOT_EXPORT_MAX_CONCURRENT_JOBS` bounds the browser ones alongside.