* ⬆️ Upgrade MCP SDK to v2 and remove HTTP sessions
MCP's per-request protocol removes the need to retain HTTP sessions.
Use the v2 handler to manage each request's transport and lifecycle,
so requests can reach any server instance without session affinity
or the workaround that adopts sessions through private SDK fields.
Keep legacy SSE support and the shared plugin and Redis bridges.
Remove the shared expiry checker, including legacy SSE idle expiry;
SSE connections now remain until disconnection or server shutdown.
Verify stateless requests, token isolation, and legacy compatibility.
Resolves#11827
AI-assisted-by: gpt-6-astra
* 🔥 Remove legacy MCP SSE support
Use Streamable HTTP as the sole MCP client transport so the server no
longer needs a separate SSE connection registry or lifecycle.
Remove /sse and /messages, their nginx routes, and the server-legacy
dependency. Legacy SSE clients must switch to /mcp; older Streamable
HTTP clients remain supported. Document the migration and verify that
the removed endpoints return 404.
Resolves#11846
AI-assisted-by: gpt-6-astra
A compose port mapping delivers traffic to the container address,
never to loopback, so `listen 127.0.0.1:8082` made
`ports: <host>:8082` fail from the host. Both configs (image
template and devenv) now use `listen 8082`, which binds every
interface and matches the implicit bind of the public
`listen 8080 default_server`.
Rewrite both block comments to state the new bind and who decides
access from outside the host. The scrape URI stays on
127.0.0.1:8082: it still reaches the socket.
AI-assisted-by: mimo-v2.6-flash-free
* 🐳 Add upstream diagnostics to nginx access log
Enrich every access-log line with the internal journey of the request:
the status the backend answered (us), the time spent connecting to it
(uct), the time spent waiting for its answer (urt) and the internal
address that served the request (ua).
A plain 502 line used to say nothing about where the request died. With
this format, the tail of the line classifies the failure: connection
rejected, backend accepted and hung (uct + urt under 1s), or backend
stuck until read timeout. This was the missing witness in the Sep 20
incident, where nginx received connection resets with zero timeouts and
zero rejections.
Applied both to the production image template and the devenv config.
With proxy_pass on variables there is no upstream keepalive, so uct
measures one real TCP connection per request.
Parsing the new fields (us, uct, urt, ua) on the log shipper is left to
ops, so they can be filtered in Loki.
AI-assisted-by: glm-5.3-flash
* 🐳 Add stub_status endpoint for nginx metrics
Add a dedicated localhost-only server (listen 127.0.0.1:8082) exposing
/stub_status next to every other location of the public server. Ops can
run the official nginx-prometheus-exporter as a sidecar against
http://127.0.0.1:8082/stub_status and get nginx_connections_active,
accepted vs handled, reading/writing/waiting and request rates in
Prometheus.
Binding it to localhost and its own server keeps it unreachable from
outside the host and out of the public surface, and access_log off
avoids polluting Loki with one line per Prometheus scrape. The base
image already ships stub_status compiled in, so no image rebuild is
needed.
Applied both to the production image template and the devenv config.
AI-assisted-by: glm-5.3-flash
* ✨ Expose http server gate metrics (worker and connector)
The backend already measured dispatch latency but nothing reported the
state of the "house door": the xnio worker queue and threads, and the
monitor-level listener counters. This was the exact blind spot of the
Sep 20 incident, where the server kept answering health checks while it
accepted connections and dropped them without response.
Add a periodic metrics sampler that lives and dies with the http
server (single daemon thread, 15s interval, each sample guarded so an
unexpected error does not cancel subsequent runs) and publishes:
- worker (xnio MXBean gauges): penpot_http_worker_queue_size,
busy_threads, pool_size and max_pool_size. Negative samples are
discarded: the MXBean transiently reports -1 on the busy thread
count (verified live), and a stale negative would read as zero.
- listener (Undertow connector statistics, enabled via the new
:server/statistics yetti option): penpot_http_connector_active*
_connections gauge and requests_total / errors_total counters.
Undertow exposes absolute totals, so the sampler keeps a watermark
atom and publishes deltas, skipping (and moving forward past) a
counter reset.
The connector-level part depends on yetti v11.11, which now accepts
a :server/statistics server option (patch authored and released
upstream; before it, ListenerInfo#getConnectorStatistics always
returned nil).
New tests cover the samplers with fake MXBean/collector statistics
against real prometheus collectors, including the negative-sample
filter, the delta/watermark logic and the sampler lifecycle.
AI-assisted-by: glm-5.3-flash
* 🐛 Include jdk.management in the backend runtime JRE
The production image builds a trimmed JRE with jlink and omitted
jdk.management. Without that module the OS MXBean is
sun.management.BaseOperatingSystemImpl, which has no
getProcessCpuTime, getOpenFileDescriptorCount nor
getMaxFileDescriptorCount. The prometheus client StandardExports
reads those getters reflectively and collect() swallows the
NoSuchMethodException, so process_open_fds, process_max_fds and
process_cpu_seconds_total silently disappeared from /metrics while
the other process_* families kept flowing.
Verified against Prometheus: the app job only ever exposed
process_start_time_seconds, process_virtual_memory_bytes and
process_resident_memory_bytes; the fd and cpu families were absent.
Reproduced locally by running the backend metrics registry on a JRE
built with the same jlink module list (false/false/false) and on one
with jdk.management added (true/true/true).
Add the module to --add-modules and pin the metric contract with
backend-tests.metrics-test.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Build the http metrics sampler on promesa.exec
Replace the hand-rolled ScheduledThreadPoolExecutor and ThreadFactory
with promesa.exec primitives: px/scheduled-executor with a daemon
thread factory, and a px/schedule chain that reschedules the next
sample when the current one finishes.
Beyond fitting the existing periodic-task pattern (worker/cron,
rpc/rlimit), the chained schedule makes the docstring promise real:
with scheduleAtFixedRate an exception escaping the runnable cancelled
the following executions, while the reschedule now happens in a
finally block.
The sampler shutdown uses px/shutdown-now (shutdown! is deprecated in
promesa 12.0.0) to cancel the pending sample, keeping the previous
halt semantics.
The lifecycle test moves to the promesa predicates and a new test
covers the error-resilience promise: the first sample runs, throws,
and the next one is still scheduled.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Tighten the http metrics samplers
The samplers are leaf functions: they receive what they need and
publish it. Drop the internal nil guards (if there is no metrics
instance or no mxbean there is nothing to call them for) and move the
checks to the boundary, where the optional data is resolved:
sample-http-metrics now short-circuits with some-> and when-let.
Write the four worker gauges as four static operations instead of a
vector of pairs walked by doseq: the set is fixed, so the collection
only adds an allocation and hides each operation.
Drop the ! suffix from the sample-*-metrics family: ! marks a function
whose contract is to mutate state, while these report, and the mutation
happens in the mtx/run! they call. The constant true return, which only
existed so the removed guard tests could assert it, goes away too.
Tests follow the move: the internal-guard tests are replaced by one
boundary test (a nil server publishes nothing).
AI-assisted-by: deepseek-v4.1-flash
* 📚 Add the function design rules memory
Document the rules that came out of the http metrics sampler review:
preconditions are checked at the boundary instead of re-checked in the
core, optional-by-design data is guarded where the optionality is born,
a fixed set of operations is written statically, ! marks mutation and
not reporting, and production code is not shaped for tests.
Also state in the memory maintenance guide that memories must not use
manual line wrapping.
Linked from critical-info so it is read when designing a solution or
an API, not only when touching the samplers.
AI-assisted-by: deepseek-v4.1-flash
* 📚 Unwrap the critical-info memory lines
The memory maintenance guide forbids manual line wrapping, so rewrite
critical-info with one line per bullet and paragraph. A stray `*` at
the start of one continuation line is dropped.
AI-assisted-by: deepseek-v4.1-flash
* ♻️ Drop the redundant guard in the http server halt
create-metrics-sampler always returns the scheduler, so the sampler is
always present when integrant calls halt-key!; the nil check was dead
code, same as the yt/stop! call next to it.
AI-assisted-by: deepseek-v4.1-flash
* ✨ Add srepl helper to delete profiles by email
Add `delete-profiles-by-email!` to app.srepl.main. It accepts a
single email, a comma separated list of emails or a coll of emails,
resolves each profile, logs it to audit and enqueues the
delete-object task. The deleted-at is backdated with the configured
deletion-delay so profiles and their owned teams are purged on the
next gc pass.
Extract the per-email deletion logic into a private fn and reuse it
from `delete-profiles-in-bulk!`. Add tests for the new
`parse-emails` helper.
AI-assisted-by: glm-5.3-flash
The frontend build now emits the sha256 hashes of the inline scripts of every page it writes into resources/public, the image moves them out of the document root, and the entrypoint splices them into the default script-src. This removes one of the two reasons why enforcing mode was not usable.
The hashes are computed on the rendered output rather than on the mustache templates, since the digest covers the exact bytes served between the script tags. All four served pages contribute, not just index.html: challenge.html handles the redirect, render.html is loaded by the exporter in a headless browser, and rasterizer.html is initialised by the frontend itself, so leaving any of them out would have broken those paths under enforcing mode. The storybook previews are excluded because that container does not serve them.
A bundle predating this change yields no hashes and the policy stays as it was, so older bundles keep building.
The three external locations were also passing through the security headers of their upstreams. raw.githubusercontent.com returns its own Content-Security-Policy and both it and fonts.googleapis.com return Strict-Transport-Security. Browsers enforce the intersection of every policy they receive, so the upstream one takes precedence on those responses, and the HSTS one lands on our own host, meaning a deployment that deliberately disables HSTS would get it set anyway by a third party. Hide all three at the proxy.
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
The three locations of nginx-external-locations.conf define their own add_header directives, which under nginx's inheritance rules discards every add_header from the enclosing server block. They were therefore served without any of the four security headers already shipped, and would equally have been served without the new CSP and HSTS ones.
Include the common file in all three, which is the pattern the development environment already follows for the equivalent locations.
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
Ship both headers from the image so that every deployment starts from a sensible default instead of each installation deriving its own policy. Report-only mode never blocks a request, so this changes no behaviour for existing deployments, and HSTS stays absent unless PENPOT_PUBLIC_URI declares an https scheme.
The policy can be narrow because the frontend already reverse proxies its own external dependencies, so 'self' covers them. What it must permit beyond that comes from the code: 'wasm-unsafe-eval' for the render engine, 'unsafe-inline' styles for the inline style attributes of the UI, and blob:/data: for thumbnails, exports and fonts.
Closes#11374
Signed-off-by: David Barragán Merino <david.barragan@kaleidos.net>
AI-assisted-by: Claude
On self hosted installs /js/config.js is regenerated from PENPOT_FLAGS
on every container start, but nginx served it with the same
`public, max-age=604800` used for build assets, and index.html versions
it only by the build. A flags only change therefore leaves the URL
untouched, so a browser that had already loaded the app kept using its
cached copy for up to a week: enabling a flag such as
enable-login-with-google had no visible effect for returning users
until the cache expired or they cleared their site data.
Serve that one file with the same no-store headers already used for
index.html, which is the other file whose contents change without its
URL changing. Every other static asset keeps the long lived cache.
Fixes#10556.
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
The global `proxy_set_header Host $http_host;` forwarded the client-facing
Host to internal proxy_pass calls (backend/exporter), breaking mTLS routing
in service-mesh setups (e.g. Istio STRICT mode), which match outbound
requests to a cluster based on Host/:authority.
Explicitly set `Host $proxy_host` on /api, /assets, /api/export, /readyz
and /ws/notifications so these calls always target the correct internal
service host, independent of the client's original Host header.
Fixes#10835
Signed-off-by: Sebastien MALOT <sebastien.malot@pm.gouv.fr>
Co-authored-by: Sebastien MALOT <sebastien.malot@pm.gouv.fr>
The generated /etc/nginx/overrides/server.d/mcp-locations.conf used a
plain proxy_pass target (e.g. `proxy_pass http://penpot-mcp:4402;`)
where $PENPOT_MCP_URI/$PENPOT_MCP_URI_WS are shell variables substituted
once by envsubst in nginx-entrypoint.sh at container startup, not nginx
variables. nginx resolves a literal proxy_pass hostname once when the
config loads and never re-checks it, so the existing
`resolver 127.0.0.11 valid=10s;` directive in
overrides/http.d/resolvers.conf has no effect on these three locations
- it only applies to nginx variables evaluated per-request.
In multi-container deployments where the penpot-mcp container restarts
or is recreated independently of penpot-frontend (image update, OOM,
orchestrator reschedule), it gets a new IP from Docker's/the
orchestrator's DNS, and the frontend's nginx keeps forwarding to the
old, now-dead address until penpot-frontend itself is restarted. This
surfaces to users as `wss://<host>/mcp/ws` failing to connect from the
browser after enabling the MCP plugin, with
`connect() failed (111: Connection refused)` in the frontend's nginx
logs.
Route each location through a `set $var ...; proxy_pass $var;` pair so
proxy_pass evaluates a real nginx variable, letting the pre-existing
resolver directive re-resolve penpot-mcp within its 10s TTL instead of
caching the address for the container's lifetime.
For /mcp/stream and /mcp/sse, the set value also appends
$is_args$args explicitly: when proxy_pass targets a variable AND that
variable's value includes a URI/path component, nginx does not
automatically forward the original request's query string the way it
does for a static proxy_pass target - it must be appended by hand, or
the userToken query parameter used for multi-user authentication is
silently dropped before reaching the MCP server. /mcp/ws has no path
component in its target so it isn't affected by this and needed no
such change.
Verified locally: force-recreated the penpot-mcp container onto a
different IP while leaving penpot-frontend untouched; the /mcp/ws
WebSocket upgrade kept returning 101 Switching Protocols throughout,
both immediately and after the resolver's TTL window. Separately
verified /mcp/stream: a POST with ?userToken=... now shows up
server-side as userTokenFp=<redacted first 8 chars> instead of <none>,
and an actual MCP client (Claude Code) using this proxy can now call
authenticated tools like execute_code successfully.
Signed-off-by: Jules LaPrairie <jules@lucidbox.ca>
* ♻️ Rename nitrate config to admin-console
Rename user-facing configuration from 'nitrate' to 'admin-console':
- Feature flags: :nitrate -> :admin-console, :nitrate-bulk-create-profiles -> :admin-console-bulk-create-profiles
- Config keys: :nitrate-shared-key -> :admin-console-shared-key, :nitrate-backend-uri -> :admin-console-uri
- Shared-keys map entry: :nitrate -> :admin-console (setup.clj + main.clj)
- Env vars: PENPOT_NITRATE_SHARED_KEY -> PENPOT_ADMIN_CONSOLE_SHARED_KEY, PENPOT_NITRATE_BACKEND_URI removed (consolidated into PENPOT_ADMIN_CONSOLE_URI)
- Docker/nginx: PENPOT_NITRATE_URI -> PENPOT_ADMIN_CONSOLE_URI
Code namespaces, file paths, CSS classes, and i18n keys stay as-is.
AI-assisted-by: mimo-v2.5-pro
* ♻️ Rename initialize-user-in-nitrate-organization to initialize-user-in-organization
Part of the nitrate -> admin-console rename series. The function and all 9 references across 6 files have been renamed.
* ♻️ Rename :nitrate-bulk-create-profiles-not-allowed to :bulk-create-profiles-not-allowed
* ♻️ Inline nitrate-permissions into app.common.types.organization
- Delete app.common.types.nitrate-permissions and its test
- Move permission rules (allowed?, can-send-invitations?, etc.) into organization.cljc
- Harmonize all consumers to use alias cto for app.common.types.organization
- Update test runner and create organization_test.cljc
When nginx follows a backend 307 redirect to a presigned S3 URL, it was
forwarding the client's Authorization header to S3. Production S3 rejects
this because it sees two auth mechanisms (presigned URL signature +
Authorization header). MinIO in devenv is more lenient and ignores the
extra header.
Fix: add proxy_set_header Authorization "" in the @handle_redirect block.
Fixes#10776
AI-assisted-by: mimo-v2.5-pro
* 🐳 Add ImageMagick policy.xml resource limits to backend Docker image
Add a restrictive policy.xml to the backend Docker image that caps
ImageMagick resource usage: 256MiB memory, 512MiB map, 128MP area,
30s time limit, 16KP max dimensions. Blocks PS/EPS/PDF/XPS coders
to prevent Ghostscript attack surface.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add timeout support to shell/exec!
Add optional :timeout parameter (in seconds) that uses
Process.waitFor(long, TimeUnit). On timeout, the process is
destroyed forcibly and an :internal/:process-timeout exception
is raised. Stdout/stderr readers handle IOException from closed
streams when the process is killed.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Rename ::wrk/netty-executor to ::wrk/executor with cached pool
Replace DefaultEventExecutorGroup (fixed Netty thread pool) with a
cached thread pool (px/cached-executor) for general async task
offloading. The cached pool creates threads on demand and reuses
idle ones, which is more appropriate for blocking I/O workloads
(shell commands, message bus, rate limiting, etc.).
Changes:
- Rename ::wrk/netty-executor to ::wrk/executor in worker/executor.clj
- Switch implementation from DefaultEventExecutorGroup to px/cached-executor
- Update all ig/ref wiring in main.clj (msgbus, tmp cleaner, climit, rlimit, rpc)
- Remove ::wrk/netty-executor from redis.clj (let lettuce create its own
eventExecutorGroup instead of sharing a Netty executor)
- Assert executor is present in shell/exec! to prevent silent nil usage
- Remove executor-threads config (no longer needed for cached pool)
The ::wrk/netty-io-executor (NioEventLoopGroup) remains unchanged as it
handles actual non-blocking network I/O for Redis and S3.
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove im4java dependency and replace with direct ImageMagick CLI calls
- Replace im4java Java library with direct 'magick' CLI calls via shell/exec!
- Add PENPOT_IMAGEMAGICK_* config env vars for resource limits (thread, memory, map, area, disk, time, width, height)
- Use configurable ImageMagick environment with sensible defaults matching policy.xml
- Remove -Dim4java.useV7=true JVM flag from startup scripts
- Remove org.im4java/im4java from deps.edn
- All ImageMagick commands now use shell/exec! with 60s timeout and resource limits
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 💄 Rename imagemagick env functions and optimize config reads
- Rename imagemagick-defaults -> imagemagick-default-env
- Rename imagemagick-env -> get-imagemagick-env
- Optimize to avoid double cf/get calls per config key
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ✨ Add tests for shell/exec! timeout and media processing
- Add shell_test.clj: tests for exec! timeout, env vars, stdin, stderr
- Add media_test.clj: tests for info, generic-thumbnail, profile-thumbnail
- Fix generic-process to prefer explicit format over input mtype
- Fix shell/exec! to use cached executor when system has no executor
- Fix reduce-kv accumulator in set-env (must return penv)
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* ♻️ Refactor media/process to take system as first argument
- Change (defmulti process :cmd) -> (defmulti process (fn [_system params] (:cmd params)))
- Change (run params) -> (run system params)
- All process methods now receive [system params]
- Update all callers: rpc/commands/media, profile, auth, fonts
- Revert shell/exec! to require system with executor (no fallback)
- Fix lint warnings and formatting
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused app.svgo namespace
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove Node.js from backend Docker image
- Delete unused svgo-cli.js script
- Remove Node.js installation from Dockerfile.backend
- Remove svgo-cli.js copy from backend build script
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 🔥 Remove unused process-error multimethod
- Remove process-error multimethod and its default handler
- Simplify media/run to directly call process
- Fix alignment in main.clj
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
* 📚 Add ImageMagick resource limits configuration to technical guide
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
---------
Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
