An object whose :storage-target id is not in the S3 routes configuration
can no longer be located. Reads and serving now fail with
:invalid-storage-target instead of silently using the default bucket,
and garbage collection refuses to delete the row: it logs an error and
parks the row (deleted_at +1 day, no deletion attempts, no give-up)
until the target is configured again. Legacy rows (no target) and the
default target keep working and deleting normally.
Also addresses review findings: per-target dedup isolation and
stale-repair coverage, legacy-row GC coverage through the real task
paths, a single declared-target schema, closing already-built S3
clients when target init fails, and loader edge pins. Docs and memory
updated.
Closes#11630
AI-assisted-by: deepseek-flash
Introduce optional named S3 targets and route internal semantic buckets
(for example tempfile) to them, keeping the fs/s3 backend choice and the
storage_object.backend value unchanged.
Targets and routes are read from an EDN file referenced by
PENPOT_OBJECTS_STORAGE_S3_ROUTES_FILE. The chosen target id is stored in
the object metadata as :storage-target and resolved on reads, URL
signing, deduplication, individual deletes and both GC tasks. The
implicit :default target is built from PENPOT_OBJECTS_STORAGE_S3_*, so
behavior is unchanged when the file is absent. Bulk deletion now carries
the target, and one S3 client/presigner pair is shared per distinct
region/endpoint.
Closes#11630
AI-assisted-by: deepseek-flash
Add the frontend MCP variables PENPOT_MCP_URI and PENPOT_MCP_URI_WS,
the PENPOT_INTERNAL_RESOLVER variable used by nginx to resolve the
internal service names, and the enable-mcp flag to the configuration
guide. All three were used by the frontend docker image but were
missing from the docs.
Closes#11318
AI-assisted-by: claude-opus-5
Signed-off-by: Shreyash Agare <envisiontechdevelopers@gmail.com>
Corrected a typo in the configuration documentation regarding the auto-file-snapshot timeout setting.
Signed-off-by: Sebastien MALOT <sebastien.malot@pm.gouv.fr>
Add PENPOT_INTERNAL_URI environment variable to the exporter. This allows
separating the URI used for internal communication (headless browser to
frontend) from the public URI used for resource references in exported SVGs.
Previously, PENPOT_PUBLIC_URI served both purposes, which caused exported
SVGs to contain broken font URLs when the internal Docker address was used.
Changes:
- Add :internal-uri to exporter config schema with fallback to :public-uri
- Add get-internal-uri helper function
- Use internal-uri for browser navigation in SVG/PDF/bitmap renderers
- Post-process SVG output to replace internal URI with public URI
- Use internal-uri for backend API calls in resource handler
- Log both URIs on startup
- Update docker-compose.yaml to use both variables
- Document the new variable in configuration.md
Closes#10627
AI-assisted-by: mimo-v2.5-pro
* 🐳 Add ImageMagick policy.xml resource limits to backend Docker image
Add a restrictive policy.xml to the backend Docker image that caps
ImageMagick resource usage: 256MiB memory, 512MiB map, 128MP area,
30s time limit, 16KP max dimensions. Blocks PS/EPS/PDF/XPS coders
to prevent Ghostscript attack surface.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add timeout support to shell/exec!
Add optional :timeout parameter (in seconds) that uses
Process.waitFor(long, TimeUnit). On timeout, the process is
destroyed forcibly and an :internal/:process-timeout exception
is raised. Stdout/stderr readers handle IOException from closed
streams when the process is killed.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Rename ::wrk/netty-executor to ::wrk/executor with cached pool
Replace DefaultEventExecutorGroup (fixed Netty thread pool) with a
cached thread pool (px/cached-executor) for general async task
offloading. The cached pool creates threads on demand and reuses
idle ones, which is more appropriate for blocking I/O workloads
(shell commands, message bus, rate limiting, etc.).
Changes:
- Rename ::wrk/netty-executor to ::wrk/executor in worker/executor.clj
- Switch implementation from DefaultEventExecutorGroup to px/cached-executor
- Update all ig/ref wiring in main.clj (msgbus, tmp cleaner, climit, rlimit, rpc)
- Remove ::wrk/netty-executor from redis.clj (let lettuce create its own
eventExecutorGroup instead of sharing a Netty executor)
- Assert executor is present in shell/exec! to prevent silent nil usage
- Remove executor-threads config (no longer needed for cached pool)
The ::wrk/netty-io-executor (NioEventLoopGroup) remains unchanged as it
handles actual non-blocking network I/O for Redis and S3.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove im4java dependency and replace with direct ImageMagick CLI calls
- Replace im4java Java library with direct 'magick' CLI calls via shell/exec!
- Add PENPOT_IMAGEMAGICK_* config env vars for resource limits (thread, memory, map, area, disk, time, width, height)
- Use configurable ImageMagick environment with sensible defaults matching policy.xml
- Remove -Dim4java.useV7=true JVM flag from startup scripts
- Remove org.im4java/im4java from deps.edn
- All ImageMagick commands now use shell/exec! with 60s timeout and resource limits
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 💄 Rename imagemagick env functions and optimize config reads
- Rename imagemagick-defaults -> imagemagick-default-env
- Rename imagemagick-env -> get-imagemagick-env
- Optimize to avoid double cf/get calls per config key
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add tests for shell/exec! timeout and media processing
- Add shell_test.clj: tests for exec! timeout, env vars, stdin, stderr
- Add media_test.clj: tests for info, generic-thumbnail, profile-thumbnail
- Fix generic-process to prefer explicit format over input mtype
- Fix shell/exec! to use cached executor when system has no executor
- Fix reduce-kv accumulator in set-env (must return penv)
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Refactor media/process to take system as first argument
- Change (defmulti process :cmd) -> (defmulti process (fn [_system params] (:cmd params)))
- Change (run params) -> (run system params)
- All process methods now receive [system params]
- Update all callers: rpc/commands/media, profile, auth, fonts
- Revert shell/exec! to require system with executor (no fallback)
- Fix lint warnings and formatting
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused app.svgo namespace
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove Node.js from backend Docker image
- Delete unused svgo-cli.js script
- Remove Node.js installation from Dockerfile.backend
- Remove svgo-cli.js copy from backend build script
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused process-error multimethod
- Remove process-error multimethod and its default handler
- Simplify media/run to directly call process
- Fix alignment in main.clj
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 📚 Add ImageMagick resource limits configuration to technical guide
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
---------
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
