Embed compact shapes in page entries for library exports, add geometry
verification and structure tests for backend, and remove the redundant
:format field from the manifest — version alone determines format.
AI-assisted-by: deepseek-v4-pro
Add feature-flagged compact export format where pages embed shapes
as a single JSON entry instead of one ZIP entry per shape. Shapes
go through compact-shape (prune defaults/nils/identity transforms)
and round-values (round floats to 4 decimals) before encoding.
Export: one JSON per page with embedded objects, manifest v2,
no pretty-printing. Gated by :binfile-v3-compact flag (opt-in).
Import: detect manifest version 2, expand compact shapes
(restore transforms, recompute selrect/points) on read.
AI-assisted-by: deepseek-v4-pro
* 📎 Update serena documentation about creating-prs workflow
* 🐛 Handle unrecognized JSON escape sequences as malformed-json
When clojure.data.json's read-escaped-char encounters an unrecognized
escape sequence (e.g. a backslash followed by '}', or other case
fall-throughs in the parser) in a JSON request body, it throws a bare
IllegalArgumentException. Previously this fell through to the generic
RuntimeException branch in wrap-parse-request's handle-error, which
unwrapped and recurred without matching, eventually reaching the
internal-error handler and producing HTTP 500 + an error report — even
though the root cause was malformed client input, not a server bug.
The fix converts any IllegalArgumentException raised in the JSON parse
path into a `:validation`/`:malformed-json` error by raising a new
ex-info (which is caught by the top-level error handler in
`app.http/router-handler`). The result is an HTTP 400 response with a
descriptive hint, and no error report is generated. This addresses
~10% of all error reports received.
The new IAE branch is placed before the RuntimeException branch in
the cond (since IllegalArgumentException IS-A RuntimeException) and
uses the throw-style (ex/raise) to match the existing
RequestTooBigException / EOFException branches. A comment above the
handle-error cond documents why raising is intentional and is caught
by the top-level app.http error handler, not by the per-route
wrap-errors middleware.
Test suite changes:
- Extend the existing `DummyRequest` defrecord in
`http_middleware_test.clj` from 2 fields to 12 fields, implementing
every IRequest method, and add a private `make-dummy-request`
constructor that accepts an options map with every key optional and
sensible `:or` defaults. Future fields added to DummyRequest won't
break existing call sites as long as the `:or` defaults are kept in
sync.
- Remove the now-redundant `JsonRequest` defrecord and migrate all 11
`->DummyRequest` call sites to `make-dummy-request`.
- Add 6 new deftest cases:
- parse-request-illegal-argument-exception: malformed JSON body
(containing `\}`) is converted to `:malformed-json`.
- parse-request-request-too-big-exception: RequestTooBigException
is converted to `:request-body-too-large`.
- parse-request-eof-exception: java.io.EOFException is converted
to `:malformed-json`.
- parse-request-runtime-exception-with-cause: a wrapped
RuntimeException recurses on ex-cause and dispatches to the
matching specific branch.
- parse-request-runtime-exception-without-cause: a bare
RuntimeException falls through to errors/handle, returning 500
with :type :server-error :code :unexpected.
- parse-request-non-runtime-throwable: java.io.IOException (a
non-RuntimeException Throwable) is handled by the dedicated
handle-exception method, returning 500 with :code :io-exception.
Together, the new tests cover all 6 branches of wrap-parse-request's
handle-error cond.
Refs #10804.
AI-assisted-by: minimax-m3
The audit event validation was failing when processing error reports that
contain string profile-id values. The error report storage converts
profile-id to string format, but the audit schema expects a UUID.
Changes:
- Modified prepare-rpc-event to convert string profile-id to UUID using
uuid/parse* (exception-safe parsing)
- Updated access token middleware to set ::id and ::type on request so
audit context includes token identification
- Added tests for profile-id conversion and token context population
Closes#10897
AI-assisted-by: qwen3.7-plus
* 🐛 Fix several issues in RPC command handlers
- Reject circular library references in link-file-to-library
- Add explicit team permission check in search-files
- Constrain search-term max length to 250 chars
- Include :deleted-at in file ETag for COND caching
- Move storage I/O outside DB transaction in create-file-thumbnail
AI-assisted-by: deepseek-v4-pro
* 📎 Check perms before circular link checks
* 🐛 Handle circular library reference error
Catch :circular-library-reference error from backend when linking
files to libraries. Show user-friendly toast notification instead of
propagating unhandled error. Add English and Spanish translations.
AI-assisted-by: qwen3.7-plus
Server changes:
- Switch list ordering from DESC to ASC (oldest first)
- Flip cursor direction to > for forward pagination
- Add 'until' param for server-side upper-bound filtering
CLI changes:
- Add --from/--to flags mapping to server's since/until
- Streaming output for --all and --format ndjson
- Add --format ndjson option (one JSON object per line)
- Add --normalize-hints flag to strip dynamic values
- Add --output flag to write list results to file
- Add 'stats' subcommand with aggregations (signature, host,
tenant, version, source, kind, hour) reading from API, file, stdin
- stats input supports JSON, JSON array, and NDJSON formats
Test changes:
- Fix pagination assertions for ASC ordering
AI-assisted-by: mimo-v2.5-pro
Implement RPC methods for querying server error reports with pagination
and filtering. Add CLI tool (tools/error-reports.mjs) for convenient
access with table and JSON output formats. Extract profile-id from audit
events and logging context for better error categorization. Build
improved HREF using request path when available.
AI-assisted-by: qwen3.7-plus
When ImageMagick fails to process an uploaded image (e.g., corrupted PNG
with invalid IHDR data), the backend was raising :type :internal with
:code :imagemagick-error, which mapped to HTTP 500. The frontend treated
this as a server error and displayed the full error page.
Changed exec-magick! to raise :type :validation with :code :invalid-image
instead. This flows through the existing :invalid-image handler in
errors.clj which returns HTTP 400. The frontend's handle-media-error and
process-error now catch this code and show a notification banner.
AI-assisted-by: qwen3.7-plus
* 🐛 Demote unable-to-retrieve-user-info OIDC error to warning level
401 responses from the OIDC userinfo endpoint (e.g. expired/revoked GitHub
token) are normal auth failures, not server errors. Logging at :error level
triggers the database and Mattermost error reporters unnecessarily.
AI-assisted-by: deepseek-v4-flash
* ✨ Add pure function tests for OIDC auth module
Add tests for: int-in-range?, valid-info?, qualify-prop-key, qualify-props,
provider-has-email-verified?, profile-has-provider-props?, redirect-response,
redirect-with-error, redirect-to-verify-token, and build-redirect-uri.
AI-assisted-by: deepseek-v4-flash
* ✨ Add HTTP-mock tests for fetch-user-info and fetch-access-token
Replace with-redefs with binding (cf/config is ^:dynamic).
Add tests for: fetch-user-info (success, 401, 500, request structure),
fetch-access-token (success, 400 error).
AI-assisted-by: deepseek-v4-flash
* ✨ Add get-info integration tests with partial mocking
Test all branches: token/userinfo/auto info sources, incomplete info,
role checks (satisfied and insufficient), state props merge,
sso-session-id from claims, and sso-provider-id for uuid providers.
AI-assisted-by: deepseek-v4-flash
* ✨ Add callback-handler integration tests with real tokens and session
Tests all main branches: error param, no profile (registration disabled),
profile blocked, provider mismatch, inactive profile, success flow,
and graceful handling of unable-to-retrieve-user-info exception.
Uses real tokens/generate, tokens/verify, and session/inmemory-manager.
AI-assisted-by: deepseek-v4-flash
* ✨ Add font processing resource limits via prlimit
Font processing tools (fontforge, sfnt2woff, woff2sfnt, woff2_decompress)
were invoked via clojure.java.shell/sh with no timeouts or resource limits.
This adds process-level resource limits using prlimit(1) and the shell/exec!
infrastructure from the ImageMagick hardening work.
shell/exec! changes:
- Add :prlimit parameter that prepends prlimit(1) to the command
- :prlimit takes {:mem <MiB> :cpu <seconds>} for address space and CPU time
limits, enforced by the kernel's RLIMIT subsystem
- prlimit-cmd builds the prlimit command prefix (private helper)
Font processing changes:
- Replace all clojure.java.shell/sh calls with shell/exec! via exec-font!
- exec-font! applies font-prlimit (512 MiB, 30s CPU, 60s wall-clock)
- All 5 conversion functions (ttf->otf, otf->ttf, ttf-or-otf->woff,
woff->sfnt, woff2->sfnt) use try/finally for explicit temp file cleanup
- Remove clojure.java.shell require from media.clj
Tests:
- Add exec-prlimit-normal, exec-prlimit-cpu, exec-prlimit-memory tests
Closes#10234
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Make font processing resource limits configurable
Replace hardcoded font-prlimit map and wall-clock timeout with
config-driven values under the PENPOT_FONT_PROCESS_* namespace.
The prlimit implementation detail is not exposed in config keys.
Co-authored-by: deepseek-v4-flash <deepseek-v4-flash@penpot.app>
---------
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
Co-authored-by: deepseek-v4-flash <deepseek-v4-flash@penpot.app>
* 🐳 Add ImageMagick policy.xml resource limits to backend Docker image
Add a restrictive policy.xml to the backend Docker image that caps
ImageMagick resource usage: 256MiB memory, 512MiB map, 128MP area,
30s time limit, 16KP max dimensions. Blocks PS/EPS/PDF/XPS coders
to prevent Ghostscript attack surface.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add timeout support to shell/exec!
Add optional :timeout parameter (in seconds) that uses
Process.waitFor(long, TimeUnit). On timeout, the process is
destroyed forcibly and an :internal/:process-timeout exception
is raised. Stdout/stderr readers handle IOException from closed
streams when the process is killed.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Rename ::wrk/netty-executor to ::wrk/executor with cached pool
Replace DefaultEventExecutorGroup (fixed Netty thread pool) with a
cached thread pool (px/cached-executor) for general async task
offloading. The cached pool creates threads on demand and reuses
idle ones, which is more appropriate for blocking I/O workloads
(shell commands, message bus, rate limiting, etc.).
Changes:
- Rename ::wrk/netty-executor to ::wrk/executor in worker/executor.clj
- Switch implementation from DefaultEventExecutorGroup to px/cached-executor
- Update all ig/ref wiring in main.clj (msgbus, tmp cleaner, climit, rlimit, rpc)
- Remove ::wrk/netty-executor from redis.clj (let lettuce create its own
eventExecutorGroup instead of sharing a Netty executor)
- Assert executor is present in shell/exec! to prevent silent nil usage
- Remove executor-threads config (no longer needed for cached pool)
The ::wrk/netty-io-executor (NioEventLoopGroup) remains unchanged as it
handles actual non-blocking network I/O for Redis and S3.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove im4java dependency and replace with direct ImageMagick CLI calls
- Replace im4java Java library with direct 'magick' CLI calls via shell/exec!
- Add PENPOT_IMAGEMAGICK_* config env vars for resource limits (thread, memory, map, area, disk, time, width, height)
- Use configurable ImageMagick environment with sensible defaults matching policy.xml
- Remove -Dim4java.useV7=true JVM flag from startup scripts
- Remove org.im4java/im4java from deps.edn
- All ImageMagick commands now use shell/exec! with 60s timeout and resource limits
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 💄 Rename imagemagick env functions and optimize config reads
- Rename imagemagick-defaults -> imagemagick-default-env
- Rename imagemagick-env -> get-imagemagick-env
- Optimize to avoid double cf/get calls per config key
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add tests for shell/exec! timeout and media processing
- Add shell_test.clj: tests for exec! timeout, env vars, stdin, stderr
- Add media_test.clj: tests for info, generic-thumbnail, profile-thumbnail
- Fix generic-process to prefer explicit format over input mtype
- Fix shell/exec! to use cached executor when system has no executor
- Fix reduce-kv accumulator in set-env (must return penv)
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Refactor media/process to take system as first argument
- Change (defmulti process :cmd) -> (defmulti process (fn [_system params] (:cmd params)))
- Change (run params) -> (run system params)
- All process methods now receive [system params]
- Update all callers: rpc/commands/media, profile, auth, fonts
- Revert shell/exec! to require system with executor (no fallback)
- Fix lint warnings and formatting
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused app.svgo namespace
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove Node.js from backend Docker image
- Delete unused svgo-cli.js script
- Remove Node.js installation from Dockerfile.backend
- Remove svgo-cli.js copy from backend build script
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused process-error multimethod
- Remove process-error multimethod and its default handler
- Simplify media/run to directly call process
- Fix alignment in main.clj
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 📚 Add ImageMagick resource limits configuration to technical guide
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
---------
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Batch multiple thumbnail deletions into a single RPC call
Replace the old per-object immediate thumbnail deletion with a
debounced batched approach. The frontend queues object-ids in state
and waits 200ms before sending a single RPC request with up to 200
object-ids. The backend deletes all matching thumbnails in one SQL
statement with a single RETURNING clause, then touches the affected
media objects.
This reduces RPC overhead when rapidly clearing thumbnails (e.g.
navigating pages) and makes deletions more efficient.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* 📎 Fix missing issues
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add dedicated concurrency limit for restore-file-snapshot
This adds a dedicated climit configuration for the restore-file-snapshot
RPC method with :permits 1 per profile (plus queue of 2 and 60s timeout)
and a global limit of 3. Previously the method only used the generic
root/by-profile and root/global limits, allowing up to 7 concurrent
restore operations per profile which caused database row lock contention
on FOR UPDATE and connection pool exhaustion.
* ✨ Skip locking on restore! to avoid blocking other operations
Changes the row lock acquisition in restore! from a blocking FOR UPDATE
to FOR UPDATE SKIP LOCKED. If the file row is already locked by another
concurrent operation (e.g., another restore or an update-file), the query
returns no rows and the caller fails fast with a clear conflict error
instead of blocking indefinitely holding a database connection.
* ✨ Add queue and timeout limits to root/by-profile concurrency limit
Previously root/by-profile had no queue limit (unbounded Integer/MAX_VALUE)
and no timeout, allowing requests to pile up indefinitely behind a profile
whose permits were exhausted by long-running operations. This could lead
to memory pressure and cascading failures. Now limited to 30 queued
requests with a 30-second timeout so excess requests fail fast.
* ✨ Move backup snapshot creation outside restore transaction
The backup snapshot (fsnap/create!) is now created in its own short-lived
connection before the actual restore transaction begins. This ensures the
backup is persisted independently of the restore outcome and reduces the
restore transaction window.
The restore itself runs inside a db/tx-run! block with an optimistic
locking check: it reads the file with FOR UPDATE and compares its revn
against the value captured at backup time. If the file was edited
concurrently, the restore aborts with a conflict error to prevent data
loss.
Co-dependent with the SKIP LOCKED change in restore! — the FOR UPDATE
acquired here is in the same transaction as restore!, so the SKIP LOCKED
inside restore! correctly sees the row as unlocked (same transaction).
* ♻️ Remove unused private function get-minimal-file
The local get-minimal-file function in file_snapshots.clj is no longer
used since restore! switched to direct exec-one! with FOR UPDATE SKIP
LOCKED. The sql:get-minimal-file SQL constant is still used directly.
* ✨ Add minor improvements on db connection management
* ♻️ Refactor create-file-snapshot to use explicit transaction management
Remove automatic transaction wrapping (`::db/transaction true`) and
pass `cfg` through the call chain instead of destructured `conn`.
Wrap `fsnap/create!` in an explicit `db/tx-run!` for clearer
transaction boundaries.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add dedicated concurrency limit for create-file-snapshot
This adds a dedicated climit configuration for the create-file-snapshot
RPC method with :permits 1 per profile (plus queue of 2 and 60s timeout)
and a global limit of 3. Previously the method only used the generic
root/by-profile and root/global limits, allowing up to 10 concurrent
snapshot creation operations per profile which could cause database
contention and connection pool exhaustion.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Add :uri and :scheme/:host keys to exceptions raised by
`validate-uri` for better error diagnostics. Also fix a bug
where (str url) was used instead of (str uri) in the
host-missing exception path.
Update the existing blocked-target test to verify the new :uri
key, and add three new tests covering scheme rejection, missing
host, and DNS failure error paths. All 27 tests pass with 60
assertions and 0 failures.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The /api/main/doc endpoint was returning HTML content with a
text/plain content-type header instead of text/html. This caused
browsers to render the response as plain text.
Added content-type: text/html; charset=utf-8 header to the
response in the doc handler and added a regression test to
verify the fix.
Closes#9680
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
- Add ::setup/props and ::db/pool to :app.http.assets/routes config
so session renewal works correctly for asset requests.
- Add actoken/authz middleware to the assets middleware chain so
access tokens are properly recognized.
- Add authenticated? helper that checks both ::session/profile-id
and ::actoken/profile-id, fixing 401 errors when accessing
protected assets with a valid access token.
- Add comprehensive test suite for assets auth scenarios.
Closes#9677
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Add a shared `schema:font-family` whitelist validator in
app.common.types.font that only allows letters, digits, spaces,
hyphens, underscores, and dots in font family names. Apply the schema
to create-font-variant and update-font RPC endpoints on the
backend, and add client-side validation in the dashboard fonts UI.
Include unit tests for the schema and integration tests for the RPC
handlers.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
