From 6961767881eaca0ac073bf27e50bebf34c1e86ef Mon Sep 17 00:00:00 2001 From: Andrey Antukh Date: Wed, 7 Oct 2026 13:22:46 +0000 Subject: [PATCH] :sparkles: Abstract SVG sanitization via DOMPurify in media-processor Route SVG sanitization through app.media/sanitize-svg, the same local/remote switch the rest of media processing uses. Without :remote-media-processing the existing ad-hoc blocklist filter runs locally; with it enabled, the SVG is posted to media-processor (POST /api/svg/sanitize) and sanitized by DOMPurify's allowlist, which fails closed. When the remote backend is on but the service fails, the upload or import fails instead of silently falling back to the weaker filter. Both entrances go through it: the media upload command and the binfile import (v1, v2, v3). The import job-def now carries the http client and shared keys so it can reach the service. The service creates a fresh jsdom window per sanitization and closes it, because DOMPurify keeps state on its window and reusing one leaks memory in a long-lived process. The XML declaration and DOCTYPE are stripped before parsing. AI-assisted-by: deepseek-v4.1-flash --- .serena/memories/backend/core.md | 3 +- .../memories/backend/media-sanitization.md | 39 ++ .serena/memories/media-processor/core.md | 25 +- backend/src/app/binfile/common.clj | 10 +- backend/src/app/binfile/v1.clj | 2 +- backend/src/app/binfile/v2.clj | 2 +- backend/src/app/binfile/v3.clj | 2 +- backend/src/app/main.clj | 10 +- backend/src/app/media.clj | 11 + backend/src/app/media/local.clj | 8 + backend/src/app/media/remote.clj | 29 +- backend/src/app/rpc/commands/media.clj | 7 +- backend/src/app/tasks/import_binfile.clj | 8 +- .../test/backend_tests/binfile_svg_test.clj | 46 ++- .../test/backend_tests/media_remote_test.clj | 54 +++ backend/test/backend_tests/media_test.clj | 18 + backend/test/backend_tests/rpc_media_test.clj | 66 ++++ media-processor/esbuild.config.mjs | 2 +- media-processor/package.json | 3 + media-processor/pnpm-lock.yaml | 357 +++++++++++++++++- media-processor/src/config.ts | 2 + media-processor/src/index.ts | 5 + media-processor/src/routes/svg.ts | 43 +++ media-processor/src/services/svg.ts | 78 ++++ media-processor/src/types.ts | 1 + media-processor/test/svg.test.ts | 305 +++++++++++++++ media-processor/tsconfig.json | 2 +- 27 files changed, 1097 insertions(+), 41 deletions(-) create mode 100644 .serena/memories/backend/media-sanitization.md create mode 100644 media-processor/src/routes/svg.ts create mode 100644 media-processor/src/services/svg.ts create mode 100644 media-processor/test/svg.test.ts diff --git a/.serena/memories/backend/core.md b/.serena/memories/backend/core.md index 1c4dc27119..8892b9e959 100644 --- a/.serena/memories/backend/core.md +++ b/.serena/memories/backend/core.md @@ -5,6 +5,7 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m ## Focused memories - Cross-cutting backend subtleties (RPC, DB, workers, cron, HTTP/sessions, storage, file data): `mem:backend/subtleties` +- SVG sanitization on upload/import and its local/remote dispatch: `mem:backend/media-sanitization`. - Storage abstraction, logical buckets, object lifecycle, deduplication, access, and garbage collection: `mem:backend/storage`. - Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment` - Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`. @@ -62,7 +63,7 @@ Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` i - Business params live in the `params` column (plain JSON, decoded with the job-def decoder). `submit` takes optional `::jobs/profile-id`, `::jobs/resource-id` (both validated as UUIDs) and `::jobs/expires-at` (an instant: the retention the jobs GC sweeps by), all stored as columns; nothing is inferred from `params`. The binfile creation commands are the callers that pass a profile-id and a resource. - A job-def may carry optional metadata keys read from the registry by its consumers, never by the substrate: `::jobs/family` (e.g. `:export`) groups the jobs of a family for quotes, `::jobs/resource-role` (`:input` or `:output`) says whether the job's `resource_id` is what it consumes or what it produces, and `::jobs/queue-name` names the queue the creation command routes the job to (`submit-job` falls back to `:binfile` when the def does not set it). - Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object. -- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids. +- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg; the import def also carries the http client and shared keys, because it sanitizes imported SVGs through `app.media` (`mem:backend/media-sanitization`). Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids. - Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it, because the job-def has no result schema of its own. - The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default. - Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner. diff --git a/.serena/memories/backend/media-sanitization.md b/.serena/memories/backend/media-sanitization.md new file mode 100644 index 0000000000..9716183a6e --- /dev/null +++ b/.serena/memories/backend/media-sanitization.md @@ -0,0 +1,39 @@ +# Backend Media Sanitization (SVG) + +Uploading and importing an SVG both run it through `app.media/sanitize-svg`, +the single dispatch point: + +- without `:remote-media-processing`, `app.media.local/process :sanitize-svg` + runs the ad-hoc filter `app.media.svg/sanitize-svg` (blocklist: `script`, + `foreignObject`, `set`, `animate*`, `on*` attributes, `javascript:` hrefs); +- with `:remote-media-processing`, `app.media.remote/process :sanitize-svg` + posts the SVG to media-processor (`POST /api/svg/sanitize`), where DOMPurify + sanitizes it with an allowlist. + +Call sites: + +- upload: `app.rpc.commands.media/process-main-image` (writes the cleaned SVG to + a tempfile before `sto/put-object!`); +- binfile import: `app.binfile.common/sanitize-imported-svg` (returns the cleaned + `:bytes` with their `:size` and blake2b `:hash`), called from v3 + `import-storage-objects`, v2 `read-storage-object!` and v1 + `read-section :v1/sobjects`. + +Rules: + +- Fail closed: when the remote backend is enabled but the service fails, the + exception propagates and the upload/import fails; there is no silent fallback + to the weaker local filter. +- Text in, text out: the sanitizer takes and returns the SVG as a UTF-8 string. +- `:app.tasks.import-binfile/job-def` carries `::http.client/client` and + `::setup/shared-keys` (the RPC cfg already does), so the import job — not only + the RPC path — can reach media-processor. +- The two modes do not produce identical output: the local filter re-serializes + with `clojure.xml`, DOMPurify uses its own serializer and a different + allowlist (e.g. it also drops ``). Do not assert byte equality across + modes. +- Scope: only sanitization is delegated. SVG info extraction + (`app.media.svg/parse-svg`, `get-basic-info-from-svg`) stays in the JVM. + +Service endpoint, its DOMPurify config and its per-call jsdom window: +`mem:media-processor/core`. diff --git a/.serena/memories/media-processor/core.md b/.serena/memories/media-processor/core.md index 209d4b1a37..296c58dca3 100644 --- a/.serena/memories/media-processor/core.md +++ b/.serena/memories/media-processor/core.md @@ -9,6 +9,7 @@ Stateless HTTP service for Penpot image and font processing. Handles image info - Framework: Express - Image processing: sharp (libvips) - Font processing: FontForge (TTF/OTF), sfnt2woff, woff2_decompress +- SVG sanitization: DOMPurify (allowlist) over jsdom (temporary DOM for Node) - Upload handling: multer (hybrid storage: memory for small, disk for large) - Logging: pino (with optional Loki transport) - Config validation: Zod @@ -33,10 +34,12 @@ media-processor/ │ ├── routes/ │ │ ├── health.ts # GET /api/health │ │ ├── image.ts # POST /api/image/info, /api/image/thumbnail -│ │ └── font.ts # POST /api/font/convert +│ │ ├── font.ts # POST /api/font/convert +│ │ └── svg.ts # POST /api/svg/sanitize │ └── services/ │ ├── image.ts # sharp-based image info/thumbnail generation │ ├── font.ts # FontForge/woff-tools font conversion +│ ├── svg.ts # DOMPurify SVG sanitization over jsdom │ └── errors.ts # throwValidation, throwRestriction, throwProcessing ├── test/ # Vitest test files ├── vitest.config.ts # Test configuration @@ -54,6 +57,7 @@ media-processor/ ### Resource Limits - Image: max pixels, max width/height enforced before processing +- SVG: max input size enforced before parsing (default 30MB, `PENPOT_MEDIA_PROCESSOR_SVG_MAX_SIZE`); an oversized input is rejected with 413, never truncated - Font: prlimit wraps FontForge processes with memory (AS) and CPU time limits - Concurrency: p-queue limits concurrent requests (default 10) - Upload: hybrid storage — memory for files < 10MB, disk for larger; configurable via `PENPOT_MEDIA_PROCESSOR_MEMORY_THRESHOLD` @@ -74,6 +78,25 @@ media-processor/ - SFNT type detected via magic bytes (0x4f54544f = OTF, 0x00010000 = TTF) - Temp files cleaned up in finally blocks (best-effort) +### SVG Sanitization +- `POST /api/svg/sanitize` (multipart field `file`) returns the sanitized SVG + bytes with `Content-Type: image/svg+xml`; it is the remote backend of + `app.media/sanitize-svg` in the JVM. +- DOMPurify is given `USE_PROFILES: {svg: true, svgFilters: true}` and + `NAMESPACE: "http://www.w3.org/2000/svg"`, so it parses the document as XML/SVG + the way a browser parses a standalone `.svg`. Its default allowlist fails closed + and already drops `script`, `foreignObject`, `set`, `animate` and `use`; do not + add `ADD_TAGS`/`ADD_ATTR`/`FORBID_TAGS`, and do not use `IN_PLACE`, `setConfig` + or hooks. To fix a bypass, upgrade DOMPurify (it is a fast-moving security + dependency; pin the latest patched 3.x). +- A fresh jsdom `window` is created per call and closed in `finally`. Reusing one + window in a long-lived process leaks memory and degrades latency without bound. +- The XML declaration and DOCTYPE are stripped before parsing (the XML parser + bails on them; stripping the DOCTYPE also keeps its entities undeclared). An + input that does not produce an ` (zip-entry-stream input (get-zip-entry input path)) limit (size-limiting-stream limit (atom 0) path))] (io/read istream))] - (bfc/sanitize-imported-svg object raw))) + (bfc/sanitize-imported-svg cfg object raw))) content (if-let [{:keys [bytes hash]} clean] (-> (sto/content bytes) (sto/wrap-with-hash hash)) diff --git a/backend/src/app/main.clj b/backend/src/app/main.clj index 87d0b31f29..d82700bc9d 100644 --- a/backend/src/app/main.clj +++ b/backend/src/app/main.clj @@ -652,10 +652,12 @@ ::mbus/msgbus (ig/ref ::mbus/msgbus)} :app.tasks.import-binfile/job-def - {::db/pool (ig/ref ::db/pool) - ::sto/storage (ig/ref ::sto/storage) - ::mtx/metrics (ig/ref ::mtx/metrics) - ::mbus/msgbus (ig/ref ::mbus/msgbus)} + {::db/pool (ig/ref ::db/pool) + ::sto/storage (ig/ref ::sto/storage) + ::mtx/metrics (ig/ref ::mtx/metrics) + ::mbus/msgbus (ig/ref ::mbus/msgbus) + ::http.client/client (ig/ref ::http.client/client) + ::setup/shared-keys (ig/ref ::setup/shared-keys)} :app.storage.gc-deleted/job-def {::db/pool (ig/ref ::db/pool) diff --git a/backend/src/app/media.clj b/backend/src/app/media.clj index c00ac00b24..000d3a7fd7 100644 --- a/backend/src/app/media.clj +++ b/backend/src/app/media.clj @@ -37,6 +37,17 @@ (media.remote/process system params) (media.local/process system params))) +(defn sanitize-svg + "Sanitize SVG `content` (a UTF-8 string) with the active media backend: + the local ad-hoc filter, or DOMPurify in media-processor when remote + media processing is enabled. Returns the sanitized SVG as text. + + When the remote backend is up but the service fails, the exception + propagates: the upload or import fails instead of silently falling back + to the weaker local filter." + [system content] + (:content (run system {:cmd :sanitize-svg :content content}))) + ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; IMAGE HELPERS ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; diff --git a/backend/src/app/media/local.clj b/backend/src/app/media/local.clj index f3811e83aa..cadcb94063 100644 --- a/backend/src/app/media/local.clj +++ b/backend/src/app/media/local.clj @@ -196,6 +196,14 @@ :size (fs/size path) :ts (ct/now)))))) +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +;; SVG SANITIZATION +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + +(defmethod process :sanitize-svg + [_system {:keys [content] :as params}] + (assoc params :content (svg/sanitize-svg content))) + ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; FONTS ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; diff --git a/backend/src/app/media/remote.clj b/backend/src/app/media/remote.clj index aaeec36cb0..506cb81e71 100644 --- a/backend/src/app/media/remote.clj +++ b/backend/src/app/media/remote.clj @@ -121,17 +121,25 @@ (defn- service-multipart-request "Send a multipart request to the media-processor service. - Accepts a file from disk via :path. The file stream is closed - after the HTTP request completes (success or failure)." - [system {:keys [endpoint path mtype query timeout]}] + + The file payload comes from `:path` (a file on disk) or from `:data` + (bytes already in memory); `:path` wins when both are present. The + stream is closed after the HTTP request completes (success or failure)." + [system {:keys [endpoint path data mtype query timeout]}] (let [shared-key (get-shared-key system) boundary (multipart-boundary) ctype (or mtype "application/octet-stream") base-url (service-base-url) request-uri (cond-> (uri/join base-url endpoint) (seq query) - (str "?" (uri/map->query-string query)))] - (with-open [file-stream (io/input-stream path)] + (str "?" (uri/map->query-string query))) + input (cond + (some? path) (io/input-stream path) + (some? data) (ByteArrayInputStream. ^bytes data) + :else (ex/raise :type :internal + :code :missing-multipart-input + :hint "service-multipart-request needs :path or :data"))] + (with-open [file-stream input] (let [body (build-multipart-stream boundary ctype file-stream)] (service-request system {:method :post @@ -211,6 +219,17 @@ (finally (.close body))))))) +(defmethod process :sanitize-svg + [system {:keys [content] :as params}] + (let [resp (service-multipart-request system {:endpoint "api/svg/sanitize" + :data (.getBytes ^String content "UTF-8") + :mtype "image/svg+xml"}) + body (:body resp)] + (try + (assoc params :content (slurp body :encoding "UTF-8")) + (finally + (.close body))))) + (defn- thumbnail-request "Shared implementation for generic-thumbnail and profile-thumbnail." [system params mode] diff --git a/backend/src/app/rpc/commands/media.clj b/backend/src/app/rpc/commands/media.clj index e914ff3302..29e49c634c 100644 --- a/backend/src/app/rpc/commands/media.clj +++ b/backend/src/app/rpc/commands/media.clj @@ -16,7 +16,6 @@ [app.db :as db] [app.loggers.audit :as-alias audit] [app.media :as media] - [app.media.svg :as svg] [app.media.validation :as media.v] [app.rpc :as-alias rpc] [app.rpc.climit :as climit] @@ -126,12 +125,12 @@ ;; inverse, soft referential integrity). (defn- process-main-image - [info] + [cfg info] (let [path (:path info) mtype (:mtype info) path (if (= mtype "image/svg+xml") (let [content (slurp path) - sanitized (svg/sanitize-svg content) + sanitized (media/sanitize-svg cfg content) temp-path (tmp/tempfile :prefix "penpot-svg-" :suffix ".svg" :min-age "5m")] (spit (str temp-path) sanitized) temp-path) @@ -168,7 +167,7 @@ (assoc ::thumb (process-thumb-image cfg info)) :always - (assoc ::image (process-main-image info))))) + (assoc ::image (process-main-image cfg info))))) (defn- create-file-media-object [{:keys [::sto/storage ::db/conn] :as cfg} diff --git a/backend/src/app/tasks/import_binfile.clj b/backend/src/app/tasks/import_binfile.clj index 5dc286db08..84bc43c69f 100644 --- a/backend/src/app/tasks/import_binfile.clj +++ b/backend/src/app/tasks/import_binfile.clj @@ -12,11 +12,13 @@ [app.common.exceptions :as ex] [app.common.schema :as sm] [app.db :as db] + [app.http.client :as http.client] [app.jobs :as jobs] [app.jobs.storage :as js] [app.metrics :as mtx] [app.rpc.commands.projects :as projects] [app.rpc.commands.teams :as teams] + [app.setup :as-alias setup] [app.storage :as sto] [datoteka.fs :as fs] [integrant.core :as ig])) @@ -112,7 +114,11 @@ [_ params] (assert (db/pool? (::db/pool params)) "expected a valid database pool") (assert (sto/valid-storage? (::sto/storage params)) "expected valid storage to be provided") - (assert (mtx/metrics? (::mtx/metrics params)) "expected valid metrics")) + (assert (mtx/metrics? (::mtx/metrics params)) "expected valid metrics") + ;; The import sanitizes SVGs through `app.media`, which talks to + ;; media-processor when remote media processing is enabled. + (assert (http.client/client? (::http.client/client params)) "expected a valid http client") + (assert (map? (::setup/shared-keys params)) "expected shared keys to be provided")) (defmethod ig/init-key ::job-def [_ cfg] diff --git a/backend/test/backend_tests/binfile_svg_test.clj b/backend/test/backend_tests/binfile_svg_test.clj index 41d8d212ad..07270b588a 100644 --- a/backend/test/backend_tests/binfile_svg_test.clj +++ b/backend/test/backend_tests/binfile_svg_test.clj @@ -18,8 +18,10 @@ [app.common.features :as cfeat] [app.common.types.shape :as cts] [app.common.uuid :as uuid] + [app.config :as cf] [app.db :as db] [app.rpc :as-alias rpc] + [app.setup :as-alias setup] [app.storage :as sto] [app.storage.tmp :as tmp] [backend-tests.helpers :as th] @@ -28,7 +30,8 @@ [clojure.string :as str] [clojure.test :as t] [datoteka.fs :as fs] - [datoteka.io :as io]) + [datoteka.io :as io] + [mockery.core :refer [with-mocks]]) (:import java.io.ByteArrayInputStream java.util.zip.ZipEntry @@ -56,7 +59,7 @@ (String. data "UTF-8")) (t/deftest sanitize-imported-svg-removes-script - (let [result (bfc/sanitize-imported-svg {:content-type "image/svg+xml" :bucket "file-media-object"} + (let [result (bfc/sanitize-imported-svg th/*system* {:content-type "image/svg+xml" :bucket "file-media-object"} (utf8bytes evil-svg))] (t/is (some? result)) (t/is (not (str/includes? (bytes-str (:bytes result)) "")))))) +(t/deftest sanitize-imported-svg-routes-through-media-processor + (t/testing "with remote media processing on, the import sanitizes through the service" + (let [served ""] + (with-mocks [mock {:target 'app.media.remote/service-request + :return {:status 200 + :body (ByteArrayInputStream. (utf8bytes served))}}] + (with-redefs [cf/flags #{:remote-media-processing} + cf/get (th/config-get-mock {:media-processing-service-uri "http://localhost:6065" + :media-processing-service-timeout 5000})] + (let [system (assoc th/*system* ::setup/shared-keys {:media-processor "test-key"}) + result (bfc/sanitize-imported-svg system + {:content-type "image/svg+xml" :bucket "file-media-object"} + (utf8bytes evil-svg))] + (t/is (some? result)) + (t/is (= served (bytes-str (:bytes result)))) + (t/is (= 1 (:call-count @mock))))))))) + (t/deftest check-storage-content-type-allows-known-types (doseq [ctype ["image/svg+xml" "image/jpeg" "font/woff2" "application/octet-stream"]] (t/is (nil? (bfc/check-storage-content-type {:content-type ctype})) @@ -295,7 +315,7 @@ _ (spit (str path) evil-svg :encoding "UTF-8") raw (with-open [istream (jio/input-stream path)] (io/read istream)) - result (bfc/sanitize-imported-svg {:content-type "image/svg+xml"} raw)] + result (bfc/sanitize-imported-svg th/*system* {:content-type "image/svg+xml"} raw)] (t/is (some? result)) (t/is (not (str/includes? (bytes-str (:bytes result)) ""] + (with-mocks [mock {:target 'app.media.remote/service-request + :return {:status 200 + :body (ByteArrayInputStream. (.getBytes clean "UTF-8"))}}] + (with-redefs [cf/get (th/config-get-mock config-mock)] + (let [result (media.remote/process (mk-system) + {:cmd :sanitize-svg + :content ""})] + (t/is (= clean (:content result))) + (t/is (= 1 (:call-count @mock))) + (let [[_ req-map] (:call-args @mock)] + (t/is (= :post (:method req-map))) + (t/is (str/includes? (str (:uri req-map)) "api/svg/sanitize")) + (t/is (= "test-shared-key" (get-in req-map [:headers "x-shared-key"]))) + (t/is (str/starts-with? (get-in req-map [:headers "Content-Type"]) + "multipart/form-data"))))))))) + +(t/deftest sanitize-svg-closes-response-stream + (t/testing "sanitize-svg closes the response stream after reading it" + (let [{:keys [stream closed]} (tracking-stream (.getBytes "" "UTF-8"))] + (with-mocks [mock {:target 'app.media.remote/service-request + :return {:status 200 :body stream}}] + (with-redefs [cf/get (th/config-get-mock config-mock)] + (media.remote/process (mk-system) {:cmd :sanitize-svg :content ""}) + (t/is @closed)))))) + +(t/deftest sanitize-svg-service-unavailable + (t/testing "sanitize-svg propagates the service error instead of falling back" + (with-mocks [mock {:target 'app.media.remote/service-request + :throw (ex-info "Cannot connect to media-processor service" + {:type :internal + :code :media-processor-unavailable})}] + (with-redefs [cf/get (th/config-get-mock config-mock)] + (let [err (ex/try! (media.remote/process (mk-system) + {:cmd :sanitize-svg :content ""}))] + (t/is (ex/error? err)) + (t/is (= :media-processor-unavailable (:code (ex-data err))))))))) + +(t/deftest sanitize-svg-dispatch-remote + (t/testing "app.media/sanitize-svg uses the remote backend when the flag is on" + (with-mocks [mock {:target 'app.media.remote/service-request + :return {:status 200 + :body (ByteArrayInputStream. (.getBytes "REMOTE" "UTF-8"))}}] + (with-redefs [cf/flags #{:remote-media-processing} + cf/get (th/config-get-mock config-mock)] + (t/is (= "REMOTE" (media/sanitize-svg (mk-system) ""))))))) diff --git a/backend/test/backend_tests/media_test.clj b/backend/test/backend_tests/media_test.clj index 60334ebf77..107317488e 100644 --- a/backend/test/backend_tests/media_test.clj +++ b/backend/test/backend_tests/media_test.clj @@ -7,6 +7,7 @@ (ns backend-tests.media-test (:require [app.common.exceptions :as ex] + [app.config :as cf] [app.media :as media] [app.media.svg :as svg] [backend-tests.helpers :as th] @@ -203,6 +204,23 @@ (t/is (clojure.string/includes? result "xml:space")) (t/is (clojure.string/includes? result "hola"))))) +(t/deftest sanitize-svg-dispatch-local + (t/testing "app.media/sanitize-svg delegates to the local filter without the remote flag" + (with-redefs [cf/flags #{}] + (let [result (media/sanitize-svg th/*system* + "")] + (t/is (not (clojure.string/includes? result ""))] + (t/is (ex/error? err)) + (t/is (= :validation (:type (ex-data err)))) + (t/is (= :invalid-svg-file (:code (ex-data err)))))))) + (t/deftest info-invalid-image (t/testing "info on invalid image raises error" (let [path (fs/create-tempfile :prefix "penpot-test-" :suffix ".jpg")] diff --git a/backend/test/backend_tests/rpc_media_test.clj b/backend/test/backend_tests/rpc_media_test.clj index fb34396a09..74c5ff318c 100644 --- a/backend/test/backend_tests/rpc_media_test.clj +++ b/backend/test/backend_tests/rpc_media_test.clj @@ -7,17 +7,20 @@ (ns backend-tests.rpc-media-test (:require [app.common.uuid :as uuid] + [app.config :as cf] [app.db :as db] [app.http.client :as http] [app.media :as media] [app.rpc :as-alias rpc] [app.storage :as sto] [backend-tests.helpers :as th] + [clojure.string :as str] [clojure.test :as t] [datoteka.fs :as fs] [datoteka.io :as io] [mockery.core :refer [with-mocks]]) (:import + java.io.ByteArrayInputStream java.io.RandomAccessFile)) (t/use-fixtures :once th/state-init) @@ -1172,6 +1175,69 @@ (t/is (= :not-found (:type error-data))) (t/is (= :object-not-found (:code error-data)))))) +(defn- write-svg-tempfile + [^String text] + (let [path (fs/create-tempfile :prefix "penpot-upload-svg-" :suffix ".svg")] + (spit (str path) text :encoding "UTF-8") + path)) + +(def ^:private evil-upload-svg + "") + +(t/deftest upload-svg-sanitizes-script + (t/testing "uploading an SVG strips scripts before storing it" + (let [prof (th/create-profile* 1) + file (th/create-file* 1 {:profile-id (:id prof) + :project-id (:default-project-id prof) + :is-shared false}) + path (write-svg-tempfile evil-upload-svg) + out (th/command! {::th/type :upload-file-media-object + ::rpc/profile-id (:id prof) + :file-id (:id file) + :is-local true + :name "evil.svg" + :content {:filename "evil.svg" + :path path + :mtype "image/svg+xml" + :size (alength (.getBytes evil-upload-svg "UTF-8"))}})] + (t/is (nil? (:error out))) + (let [storage (:app.storage/storage th/*system*) + object (sto/get-object storage (get-in out [:result :media-id])) + stored (String. ^bytes (sto/get-object-bytes storage object) "UTF-8")] + (t/is (not (str/includes? stored "" + path (write-svg-tempfile evil-upload-svg)] + (with-mocks [mock {:target 'app.media.remote/service-request + :return {:status 200 + :body (ByteArrayInputStream. (.getBytes served "UTF-8"))}}] + (with-redefs [cf/flags #{:remote-media-processing} + cf/get (th/config-get-mock {:media-processing-service-uri "http://localhost:6065" + :media-processing-service-timeout 5000})] + (let [out (th/command! {::th/type :upload-file-media-object + ::rpc/profile-id (:id prof) + :file-id (:id file) + :is-local true + :name "evil.svg" + :content {:filename "evil.svg" + :path path + :mtype "image/svg+xml" + :size (alength (.getBytes evil-upload-svg "UTF-8"))}})] + (t/is (nil? (:error out))) + (let [storage (:app.storage/storage th/*system*) + object (sto/get-object storage (get-in out [:result :media-id])) + stored (String. ^bytes (sto/get-object-bytes storage object) "UTF-8")] + (t/is (= served stored)) + (t/is (= 1 (:call-count @mock)))))))))) + (t/deftest upload-file-media-object-rejects-client-id (let [prof (th/create-profile* 1) file (th/create-file* 1 {:profile-id (:id prof) diff --git a/media-processor/esbuild.config.mjs b/media-processor/esbuild.config.mjs index 698edc38d4..c3534fbf02 100644 --- a/media-processor/esbuild.config.mjs +++ b/media-processor/esbuild.config.mjs @@ -7,7 +7,7 @@ await build({ target: "node24", format: "esm", outfile: "dist/index.js", - external: ["sharp", "pino", "pino-pretty", "pino-loki"], + external: ["sharp", "pino", "pino-pretty", "pino-loki", "dompurify", "jsdom"], banner: { js: ` import { createRequire } from 'node:module'; diff --git a/media-processor/package.json b/media-processor/package.json index f5373a591f..acf9c29112 100644 --- a/media-processor/package.json +++ b/media-processor/package.json @@ -16,7 +16,9 @@ }, "packageManager": "pnpm@12.8.1+sha512.f64ba907507f5ceafe06c8d38e6052d0179444580ec1279ddd5bfc11cb48aa8a2644b66598e07e761da84872a9fc57d5f902b87fa49d024198d558612aabbe45", "dependencies": { + "dompurify": "^3.4.16", "express": "^5.2.1", + "jsdom": "^30.1.2", "multer": "^2.4.0", "p-queue": "^9.3.3", "pino": "^10.4.0", @@ -27,6 +29,7 @@ }, "devDependencies": { "@types/express": "^5.0.6", + "@types/jsdom": "^30.0.0", "@types/multer": "^2.3.0", "@types/node": "^26.6.4", "@types/supertest": "^7.2.1", diff --git a/media-processor/pnpm-lock.yaml b/media-processor/pnpm-lock.yaml index 56410eee5c..90633cd2bb 100644 --- a/media-processor/pnpm-lock.yaml +++ b/media-processor/pnpm-lock.yaml @@ -166,9 +166,15 @@ importers: .: dependencies: + dompurify: + specifier: ^3.4.16 + version: 3.4.16 express: specifier: ^5.2.1 version: 5.2.1 + jsdom: + specifier: ^30.1.2 + version: 30.1.2(@noble/hashes@1.8.0) multer: specifier: ^2.4.0 version: 2.4.0 @@ -194,6 +200,9 @@ importers: '@types/express': specifier: ^5.0.6 version: 5.0.6 + '@types/jsdom': + specifier: ^30.0.0 + version: 30.0.0 '@types/multer': specifier: ^2.3.0 version: 2.3.0 @@ -220,10 +229,58 @@ importers: version: 7.0.2 vitest: specifier: ^5.0.3 - version: 5.0.3(@types/node@26.6.4)(vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15)) + version: 5.0.3(@types/node@26.6.4)(jsdom@30.1.2(@noble/hashes@1.8.0))(vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15)) packages: + '@asamuzakjp/css-color@7.1.3': + resolution: {integrity: sha512-1t1U8Cm3RBl9vh3RAZhjkPaduzLSvQJRToRrxpGR/eDRW99mOREleuu+mWfiFa795bUzjwht7NgphCKTs2ianQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@asamuzakjp/dom-selector@9.2.4': + resolution: {integrity: sha512-YnVzxDxLaqL0t7Q4wfcgHZjg55Wm6y5EaO7Bhu3q9J7wq/8iw3PhsnWiOQhYekx4VexnbkgWpTT6n+QIV7kczw==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + '@bramus/specificity@2.4.3': + resolution: {integrity: sha512-0GEz8qJWokkUVYQYRukmkSHIFNoBH8QTreeHrAFbvz1XFYc3Y+puhMs8PQ5x8GNugiC2l5jvDBXKLbBIByeSBw==} + hasBin: true + + '@csstools/color-helpers@6.1.2': + resolution: {integrity: sha512-grhRy3OKmniaAEKXMjua5z/EODX0MSqBGjunw8+j/3HQjOnahs2AGhvEOIYVUWcU6ScApbhLhVrQTX8XqrMrow==} + engines: {node: '>=20.19.0'} + + '@csstools/css-calc@3.4.3': + resolution: {integrity: sha512-iex20d8CHVkyvg6B7UKV7uHnI2Bqo9g+EFfT9E0y+GvTvhZ/DwONJ+9aKb1dlqm0ZiGsL5RXjp0fCoJYnkeDjA==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-parser-algorithms': ^4.0.2 + '@csstools/css-tokenizer': ^4.0.2 + + '@csstools/css-color-parser@4.2.6': + resolution: {integrity: sha512-iiPQ3iRWwnJkeEn6RIu6SJPr7hYrLz6XZ9s/QZl+2/LI5KQVjpl2fdmDSZKuD4xP6GMmMPgHFFXg6k1Wkz0Trg==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-parser-algorithms': ^4.0.2 + '@csstools/css-tokenizer': ^4.0.2 + + '@csstools/css-parser-algorithms@4.0.2': + resolution: {integrity: sha512-40cSKyMvK+tq4qz6Awrlye2WGuOKt3FwPgtGg6KTfbHOWNw+Rk1rzbAtZnZ6IBhsY491HLRnDXwoyBAijmmILA==} + engines: {node: '>=20.19.0'} + peerDependencies: + '@csstools/css-tokenizer': ^4.0.2 + + '@csstools/css-syntax-patches-for-csstree@1.1.15': + resolution: {integrity: sha512-J0u7HkVl2nzSlhsiTOp4AmwcUQ3D+mGEEKfBy/7To5/y7F2OHwyLrXfrhR0SMgr4p5Lo+eaMVSeai24zUcBIxA==} + peerDependencies: + css-tree: ^3.2.1 + peerDependenciesMeta: + css-tree: + optional: true + + '@csstools/css-tokenizer@4.0.2': + resolution: {integrity: sha512-OoKoR0f76dCY666JlcbhmVTs2drYj1GUXZTYTcbUgJjh9Nv41aFfZ21bPQTERm5+L5cBDo466NltB2lplS5GBw==} + engines: {node: '>=20.19.0'} + '@emnapi/runtime@1.11.3': resolution: {integrity: sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==} @@ -383,6 +440,15 @@ packages: cpu: [x64] os: [win32] + '@exodus/bytes@1.16.0': + resolution: {integrity: sha512-IcpW84uEn3N7ETtNZMlxKhfl6Pec8rUNGOTBtWbK1FKhJxIFAptZyVrvVRVBimAJxJCgc3PxepxkdWWG4DVzfA==} + engines: {node: ^20.19.0 || ^22.12.0 || >=24.0.0} + peerDependencies: + '@noble/hashes': ^1.8.0 || ^2.0.0 + peerDependenciesMeta: + '@noble/hashes': + optional: true + '@img/colour@1.1.0': resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} engines: {node: '>=18'} @@ -733,6 +799,9 @@ packages: '@types/http-errors@2.0.5': resolution: {integrity: sha512-r8Tayk8HJnX0FztbZN7oVqGccWgw98T/0neJphO91KkmOzug1KkofZURD4UaD5uH8AqcFLfdPErnBod0u71/qg==} + '@types/jsdom@30.0.0': + resolution: {integrity: sha512-uAHGxujGE0cDaKGdK28zgDotFtNA7MKq5DXl8LrfdxdCI8VHcg15oJz+amHTChPNI5JpgEPQWc2xFdrw3em/nQ==} + '@types/methods@1.1.4': resolution: {integrity: sha512-ymXWVrDiCxTBE3+RIrrP533E70eA+9qu7zdWoHuOmGujkYtzf4HQF96b8nwHLqhuf4ykX61IGRIB38CC6/sImQ==} @@ -760,6 +829,12 @@ packages: '@types/supertest@7.2.1': resolution: {integrity: sha512-4CbBvoYVLHL7+yhbYrZET0vsvuyXTC05aRe7dNQkwMzm56auceoy6Yu3K50uZmwfHna1os3CMSgM/3QVkUtPTw==} + '@types/tough-cookie@4.0.5': + resolution: {integrity: sha512-/Ad8+nIOV7Rl++6f1BdKxFSMgmoqEoYbHRpPcx3JEfv8VRsQe9Z4mCXeJBzxs7mbHY/XOZZuXlRNfhpVPbs6ZA==} + + '@types/trusted-types@2.0.7': + resolution: {integrity: sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==} + '@typescript/typescript-aix-ppc64@7.0.2': resolution: {integrity: sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==} engines: {node: '>=16.20.0'} @@ -915,6 +990,9 @@ packages: resolution: {integrity: sha512-kNOjDqAh7px0XWNI+4QbzoiR/nTkHAWNud2uvnJquD1/x5a7EQZMJT0AczqK0Qn67oY/TTQ1LbUKajZpp3I9tQ==} engines: {node: '>=8.0.0'} + bidi-js@1.1.0: + resolution: {integrity: sha512-fX1Onk0tdVPC7obPWB5EbJ1z7NVhLq4m2xZLq2YXBkxzMXIGRpNMU88n0EPgWseKl12J7zXs7qrDxPK4sRs2fg==} + body-parser@2.3.0: resolution: {integrity: sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==} engines: {node: '>=18'} @@ -972,6 +1050,14 @@ packages: cookiejar@2.1.4: resolution: {integrity: sha512-LDx6oHrK+PhzLKJU9j5S7/Y3jM/mUHvD/DeI1WQmJn652iPC5Y4TBzC9l+5OMOXlyTTA+SmVUPm0HQUwpD5Jqw==} + css-tree@3.2.1: + resolution: {integrity: sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA==} + engines: {node: ^10 || ^12.20.0 || ^14.13.0 || >=15.0.0} + + data-urls@8.0.0: + resolution: {integrity: sha512-JQUCCK+/CAGbGGx618PlFZMXH81Jtj7t73CHv9dd/zVWvxgGT2zLFRjnKb+Ng46GIwcy+f7f88qd4duqllK+WQ==} + engines: {node: ^22.14.0 || >=24.0.0} + dateformat@4.6.3: resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==} @@ -984,6 +1070,9 @@ packages: supports-color: optional: true + decimal.js@10.6.0: + resolution: {integrity: sha512-YpgQiITW3JXGntzdUmyUR1V812Hn8T1YVXhCu+wO3OpS4eU9l4YdD3qjyiKdV6mvV29zapkMeD390UVEf2lkUg==} + delayed-stream@1.0.0: resolution: {integrity: sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==} engines: {node: '>=0.4.0'} @@ -999,6 +1088,9 @@ packages: dezalgo@1.0.4: resolution: {integrity: sha512-rXSP0bf+5n0Qonsb+SVVfNfIsimO4HEtmnIpPHY8Q1UCzKlQrDMfdobr8nJOOsRgWCyMRqeSBQzmWUMq7zvVig==} + dompurify@3.4.16: + resolution: {integrity: sha512-sqo+pNp3qRhCIpbgRi1y8Tgk27Bo2Ry7w0dC1NBeNTdZChWjz9Xb/KOoZbRP/R6pQZ80Qw8YhXw13hWWBbMRnQ==} + dunder-proto@1.0.1: resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==} engines: {node: '>= 0.4'} @@ -1013,6 +1105,10 @@ packages: end-of-stream@1.4.5: resolution: {integrity: sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==} + entities@8.1.0: + resolution: {integrity: sha512-kxL7msIffSuh9aaFAMD7rxAIuTRMAHMeBtgHW2yUdWw732ZNh4MehkF2gdjvtdmikkaIP9bFDDJOPlsvm7avrA==} + engines: {node: '>=20.19.0'} + es-define-property@1.0.1: resolution: {integrity: sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==} engines: {node: '>= 0.4'} @@ -1128,6 +1224,10 @@ packages: help-me@5.0.0: resolution: {integrity: sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg==} + html-encoding-sniffer@7.0.0: + resolution: {integrity: sha512-UikN5yr7xsCDAq87Or5or0PAlD3HJJOKVzM05az588WnpDJ4Ux7a2A53Qi6gofGg2/EtvF/H4hCi/TXfCW4Y6w==} + engines: {node: ^22.13.0 || >=24.0.0} + http-errors@2.0.1: resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==} engines: {node: '>= 0.8'} @@ -1143,6 +1243,9 @@ packages: resolution: {integrity: sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==} engines: {node: '>= 0.10'} + is-potential-custom-element-name@1.0.1: + resolution: {integrity: sha512-bCYeRA2rVibKZd+s2625gGnGF/t7DSqDs4dP7CrLA1m7jKWz6pps0LpYLJN8Q64HtmPKJ1hrN3nzPNKFEKOUiQ==} + is-promise@4.0.0: resolution: {integrity: sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==} @@ -1150,6 +1253,19 @@ packages: resolution: {integrity: sha512-34wB/Y7MW7bzjKRjUKTa46I2Z7eV62Rkhva+KkopW7Qvv/OSWBqvkSY7vusOPrNuZcUG3tApvdVgNB8POj3SPw==} engines: {node: '>=10'} + jsdom@30.1.2: + resolution: {integrity: sha512-0FFE/jE1rppmVfUrJUgxqXjcwolZYIGtAgj1pTussMhNZxIxi7W/PvfWaMNroGi2B36X1IKHbexpZ9DhkoOPiQ==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + peerDependencies: + canvas: ^3.2.3 + peerDependenciesMeta: + canvas: + optional: true + + lru-cache@11.5.3: + resolution: {integrity: sha512-U4N8FgzmWxc8k1VH8Kr6lQg18U7Fjvby6wXHVRX/ZZ7IwWbRMgrRbP0Wrb5q5NVinryp4SQampHKdvtecItxUg==} + engines: {node: 20 || >=22} + magic-string@1.4.3: resolution: {integrity: sha512-z12OxmaPGE0F4xlpGdjuAUckipLpQlTAuAmyGhNoD7ODpYUzvDfvtxUbjM/jwznyP178oju/KDdyi220wNJ0RQ==} @@ -1157,6 +1273,9 @@ packages: resolution: {integrity: sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==} engines: {node: '>= 0.4'} + mdn-data@2.27.1: + resolution: {integrity: sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ==} + media-typer@0.3.0: resolution: {integrity: sha512-dq+qelQ9akHpcOl/gUVRTxVIOkAJ1wR3QAvb4RsVjS8oVoFjDGTc679wJYmUmknUF5HwMLOgb5O+a3KxfWapPQ==} engines: {node: '>= 0.6'} @@ -1240,6 +1359,9 @@ packages: resolution: {integrity: sha512-AxTM2wDGORHGEkPCt8yqxOTMgpfbEHqF51f/5fJCmwFC3C/zNcGT63SymH2ttOAaiIws2zVg4+izQCjrakcwHg==} engines: {node: '>=20'} + parse5@8.0.1: + resolution: {integrity: sha512-z1e/HMG90obSGeidlli3hj7cbocou0/wa5HacvI3ASx34PecNjNQeaHNo5WIZpWofN9kgkqV1q5YvXe3F0FoPw==} + parseurl@1.3.3: resolution: {integrity: sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==} engines: {node: '>= 0.8'} @@ -1296,6 +1418,10 @@ packages: pump@3.0.4: resolution: {integrity: sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==} + punycode@2.3.1: + resolution: {integrity: sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==} + engines: {node: '>=6'} + qs@6.15.3: resolution: {integrity: sha512-O9gl3zCl5h5blw1KGUzQKhA5oUXSl8rwUIM5o0S3nCXMliSvy5Dzx7/DJcI+SwgICv+IneSZwhBh1oSyEHA71A==} engines: {node: '>=0.6'} @@ -1314,6 +1440,10 @@ packages: real-require@1.0.0: resolution: {integrity: sha512-P4nbQYQfePJxRSmY+v/KINxVucm4NF3p3s7pJveMTtom52FR4YGltUQLB8idDXwDDWW+eYrWDFbuzUnjoWHF7g==} + require-from-string@2.0.2: + resolution: {integrity: sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw==} + engines: {node: '>=0.10.0'} + rollup@4.62.2: resolution: {integrity: sha512-RFnrW4lhXA3s3eqHDZvN654g8OTjzRfqpIRJYczCGB6HzphckVAi/Qh4tbPUbRuDi7s1Llv8g/NspLkttY3gTA==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} @@ -1330,6 +1460,10 @@ packages: safer-buffer@2.1.2: resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==} + saxes@6.0.0: + resolution: {integrity: sha512-xAg7SOnEhrm5zI3puOOKyy1OMcMlIJZYNJY7xLBwSze0UjhPLnWfj2GF2EpT0jmzaJKIWKHLsaSSajf35bcYnA==} + engines: {node: '>=v12.22.7'} + secure-json-parse@4.1.0: resolution: {integrity: sha512-l4KnYfEyqYJxDwlNVyRfO2E4NTHfMKAWdUuA8J0yve2Dz/E/PdBepY03RvyJpssIpRFwJoCD55wA+mEDs6ByWA==} @@ -1427,10 +1561,25 @@ packages: resolution: {integrity: sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==} engines: {node: '>=12.0.0'} + tldts-core@7.4.16: + resolution: {integrity: sha512-MDolfaSJtlSK5Y0A1xl3277ekubZwobpBjugknDizI9O5Rm60a1m8k4ICK+MRsCDzPygT81mp3BBf5RKDlFRfA==} + + tldts@7.4.16: + resolution: {integrity: sha512-QwBER5KMR86IIjpIiO7H/Z3IMJPsZ1A6RKPAqzTTgOyUQUSt9FdnKcqhTaJmkY6HVrgouZHZR0ncK5QxvmnQeg==} + hasBin: true + toidentifier@1.0.1: resolution: {integrity: sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==} engines: {node: '>=0.6'} + tough-cookie@6.0.2: + resolution: {integrity: sha512-exgYmnmL/sJpR3upZfXG5PoatXQii55xAiXGXzY+sROLZ/Y+SLcp9PgJNI9Vz37HpQ74WvDcLT8eqm+kV3FzrA==} + engines: {node: '>=16'} + + tr46@7.0.0: + resolution: {integrity: sha512-BcDSDW+SIGWVOns1Zm6Yg91q9AEj7eaID0QDY9oIqJD1A8dscNwMcl9Sms2DeuWCNov1X0/oXanHGYQBSLwZsg==} + engines: {node: ^22.14.0 || >=24.0.0} + tslib@2.8.1: resolution: {integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==} @@ -1455,6 +1604,10 @@ packages: undici-types@8.9.0: resolution: {integrity: sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==} + undici@8.11.2: + resolution: {integrity: sha512-u4UB2/IrKdU6lFxumHmmo1a3fCQO5tzQllRorfoRS63txhrB7xTpSn1PftwC4qEHkOaqP95fCWW4lJzwErwzhQ==} + engines: {node: '>=22.19.0'} + unpipe@1.0.0: resolution: {integrity: sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==} engines: {node: '>= 0.8'} @@ -1544,6 +1697,22 @@ packages: jsdom: optional: true + w3c-xmlserializer@6.0.0: + resolution: {integrity: sha512-4Nsy8K5Tr6SPDH9jhKJOHf7ChDrc1zufZTVSF7x72hwuEXBqxqk9G6cK+K2NRUtB3iELRJqjXb4JPDMBjMTl2Q==} + engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} + + webidl-conversions@8.0.1: + resolution: {integrity: sha512-BMhLD/Sw+GbJC21C/UgyaZX41nPt8bUTg+jWyDeg7e7YN4xOM05YPSIXceACnXVtqyEw/LMClUQMtMZ+PGGpqQ==} + engines: {node: '>=20'} + + whatwg-mimetype@5.0.0: + resolution: {integrity: sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw==} + engines: {node: '>=20'} + + whatwg-url@17.2.0: + resolution: {integrity: sha512-IhLIHNcBrzIixC5XXsbr4DxdjajyA5vQ5RwGZMtrUp+GbtE5Nrsge9QKau7jXrC0aTXLYGV0xiW7Tb4YS2aomw==} + engines: {node: ^22.14.0 || >=24.0.0} + why-is-node-running@3.2.1: resolution: {integrity: sha512-Tb2FUhB4vUsGQlfSquQLYkApkuPAFQXGFzxWKHHumVz2dK+X1RUm/HnID4+TfIGYJ1kTcwOaCk/buYCEJr6YjQ==} engines: {node: '>=20.11'} @@ -1552,11 +1721,61 @@ packages: wrappy@1.0.2: resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==} + xml-name-validator@5.0.0: + resolution: {integrity: sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg==} + engines: {node: '>=18'} + + xmlchars@2.2.0: + resolution: {integrity: sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==} + zod@4.6.5: resolution: {integrity: sha512-v5l/aFXZQeai4awLbOpSoHecE9UiMrnfx75tEXLjNonXVARxQ5mOeipTjROUchszUNCqnE+hqAMujRsRHsut2Q==} snapshots: + '@asamuzakjp/css-color@7.1.3': + dependencies: + '@csstools/css-calc': 3.4.3(@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2))(@csstools/css-tokenizer@4.0.2) + '@csstools/css-color-parser': 4.2.6(@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2))(@csstools/css-tokenizer@4.0.2) + '@csstools/css-parser-algorithms': 4.0.2(@csstools/css-tokenizer@4.0.2) + '@csstools/css-tokenizer': 4.0.2 + lru-cache: 11.5.3 + + '@asamuzakjp/dom-selector@9.2.4': + dependencies: + bidi-js: 1.1.0 + css-tree: 3.2.1 + is-potential-custom-element-name: 1.0.1 + lru-cache: 11.5.3 + + '@bramus/specificity@2.4.3': + dependencies: + css-tree: 3.2.1 + + '@csstools/color-helpers@6.1.2': {} + + '@csstools/css-calc@3.4.3(@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2))(@csstools/css-tokenizer@4.0.2)': + dependencies: + '@csstools/css-parser-algorithms': 4.0.2(@csstools/css-tokenizer@4.0.2) + '@csstools/css-tokenizer': 4.0.2 + + '@csstools/css-color-parser@4.2.6(@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2))(@csstools/css-tokenizer@4.0.2)': + dependencies: + '@csstools/color-helpers': 6.1.2 + '@csstools/css-calc': 3.4.3(@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2))(@csstools/css-tokenizer@4.0.2) + '@csstools/css-parser-algorithms': 4.0.2(@csstools/css-tokenizer@4.0.2) + '@csstools/css-tokenizer': 4.0.2 + + '@csstools/css-parser-algorithms@4.0.2(@csstools/css-tokenizer@4.0.2)': + dependencies: + '@csstools/css-tokenizer': 4.0.2 + + '@csstools/css-syntax-patches-for-csstree@1.1.15(css-tree@3.2.1)': + optionalDependencies: + css-tree: 3.2.1 + + '@csstools/css-tokenizer@4.0.2': {} + '@emnapi/runtime@1.11.3': dependencies: tslib: 2.8.1 @@ -1640,6 +1859,10 @@ snapshots: '@esbuild/win32-x64@0.28.2': optional: true + '@exodus/bytes@1.16.0(@noble/hashes@1.8.0)': + optionalDependencies: + '@noble/hashes': 1.8.0 + '@img/colour@1.1.0': {} '@img/sharp-darwin-arm64@0.35.5': @@ -1875,6 +2098,13 @@ snapshots: '@types/http-errors@2.0.5': {} + '@types/jsdom@30.0.0': + dependencies: + '@types/node': 26.6.4 + '@types/tough-cookie': 4.0.5 + parse5: 8.0.1 + undici-types: 8.9.0 + '@types/methods@1.1.4': {} '@types/multer@2.3.0': @@ -1910,6 +2140,11 @@ snapshots: '@types/methods': 1.1.4 '@types/superagent': 8.1.11 + '@types/tough-cookie@4.0.5': {} + + '@types/trusted-types@2.0.7': + optional: true + '@typescript/typescript-aix-ppc64@7.0.2': optional: true @@ -1996,6 +2231,10 @@ snapshots: atomic-sleep@1.0.0: {} + bidi-js@1.1.0: + dependencies: + require-from-string: 2.0.2 + body-parser@2.3.0: dependencies: bytes: 3.1.2 @@ -2048,12 +2287,26 @@ snapshots: cookiejar@2.1.4: {} + css-tree@3.2.1: + dependencies: + mdn-data: 2.27.1 + source-map-js: 1.2.1 + + data-urls@8.0.0(@noble/hashes@1.8.0): + dependencies: + whatwg-mimetype: 5.0.0 + whatwg-url: 17.2.0(@noble/hashes@1.8.0) + transitivePeerDependencies: + - '@noble/hashes' + dateformat@4.6.3: {} debug@4.4.3: dependencies: ms: 2.1.3 + decimal.js@10.6.0: {} + delayed-stream@1.0.0: {} depd@2.0.0: {} @@ -2065,6 +2318,10 @@ snapshots: asap: 2.0.6 wrappy: 1.0.2 + dompurify@3.4.16: + optionalDependencies: + '@types/trusted-types': 2.0.7 + dunder-proto@1.0.1: dependencies: call-bind-apply-helpers: 1.0.2 @@ -2079,6 +2336,8 @@ snapshots: dependencies: once: 1.4.0 + entities@8.1.0: {} + es-define-property@1.0.1: {} es-errors@1.3.0: {} @@ -2178,6 +2437,10 @@ snapshots: optionalDependencies: picomatch: 4.0.5 + fdir@6.5.0(picomatch@4.0.7): + optionalDependencies: + picomatch: 4.0.7 + finalhandler@2.1.1: dependencies: debug: 4.4.3 @@ -2244,6 +2507,12 @@ snapshots: help-me@5.0.0: {} + html-encoding-sniffer@7.0.0(@noble/hashes@1.8.0): + dependencies: + '@exodus/bytes': 1.16.0(@noble/hashes@1.8.0) + transitivePeerDependencies: + - '@noble/hashes' + http-errors@2.0.1: dependencies: depd: 2.0.0 @@ -2260,16 +2529,47 @@ snapshots: ipaddr.js@1.9.1: {} + is-potential-custom-element-name@1.0.1: {} + is-promise@4.0.0: {} joycon@3.1.1: {} + jsdom@30.1.2(@noble/hashes@1.8.0): + dependencies: + '@asamuzakjp/css-color': 7.1.3 + '@asamuzakjp/dom-selector': 9.2.4 + '@bramus/specificity': 2.4.3 + '@csstools/css-syntax-patches-for-csstree': 1.1.15(css-tree@3.2.1) + '@exodus/bytes': 1.16.0(@noble/hashes@1.8.0) + css-tree: 3.2.1 + data-urls: 8.0.0(@noble/hashes@1.8.0) + decimal.js: 10.6.0 + html-encoding-sniffer: 7.0.0(@noble/hashes@1.8.0) + is-potential-custom-element-name: 1.0.1 + lru-cache: 11.5.3 + parse5: 8.0.1 + saxes: 6.0.0 + tough-cookie: 6.0.2 + undici: 8.11.2 + w3c-xmlserializer: 6.0.0 + webidl-conversions: 8.0.1 + whatwg-mimetype: 5.0.0 + whatwg-url: 17.2.0(@noble/hashes@1.8.0) + xml-name-validator: 5.0.0 + transitivePeerDependencies: + - '@noble/hashes' + + lru-cache@11.5.3: {} + magic-string@1.4.3: dependencies: '@jridgewell/sourcemap-codec': 1.6.0 math-intrinsics@1.1.0: {} + mdn-data@2.27.1: {} + media-typer@0.3.0: {} media-typer@1.1.0: {} @@ -2327,6 +2627,10 @@ snapshots: p-timeout@7.0.1: {} + parse5@8.0.1: + dependencies: + entities: 8.1.0 + parseurl@1.3.3: {} path-to-regexp@8.4.2: {} @@ -2398,6 +2702,8 @@ snapshots: end-of-stream: 1.4.5 once: 1.4.0 + punycode@2.3.1: {} + qs@6.15.3: dependencies: es-define-property: 1.0.1 @@ -2416,6 +2722,8 @@ snapshots: real-require@1.0.0: {} + require-from-string@2.0.2: {} + rollup@4.62.2: dependencies: '@types/estree': 1.0.9 @@ -2461,6 +2769,10 @@ snapshots: safer-buffer@2.1.2: {} + saxes@6.0.0: + dependencies: + xmlchars: 2.2.0 + secure-json-parse@4.1.0: {} semver@7.8.5: {} @@ -2608,8 +2920,22 @@ snapshots: fdir: 6.5.0(picomatch@4.0.5) picomatch: 4.0.5 + tldts-core@7.4.16: {} + + tldts@7.4.16: + dependencies: + tldts-core: 7.4.16 + toidentifier@1.0.1: {} + tough-cookie@6.0.2: + dependencies: + tldts: 7.4.16 + + tr46@7.0.0: + dependencies: + punycode: 2.3.1 + tslib@2.8.1: optional: true @@ -2655,6 +2981,8 @@ snapshots: undici-types@8.9.0: {} + undici@8.11.2: {} + unpipe@1.0.0: {} vary@1.1.2: {} @@ -2662,8 +2990,8 @@ snapshots: vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15): dependencies: esbuild: 0.28.2 - fdir: 6.5.0(picomatch@4.0.5) - picomatch: 4.0.5 + fdir: 6.5.0(picomatch@4.0.7) + picomatch: 4.0.7 postcss: 8.5.20 rollup: 4.62.2 tinyglobby: 0.2.17 @@ -2672,7 +3000,7 @@ snapshots: fsevents: 2.3.3 tsx: 4.23.15 - vitest@5.0.3(@types/node@26.6.4)(vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15)): + vitest@5.0.3(@types/node@26.6.4)(jsdom@30.1.2(@noble/hashes@1.8.0))(vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15)): dependencies: '@types/chai': 5.2.3 '@vitest/mocker': 5.0.3(vite@7.3.6(@types/node@26.6.4)(tsx@4.23.15)) @@ -2690,11 +3018,32 @@ snapshots: why-is-node-running: 3.2.1 optionalDependencies: '@types/node': 26.6.4 + jsdom: 30.1.2(@noble/hashes@1.8.0) transitivePeerDependencies: - msw + w3c-xmlserializer@6.0.0: + dependencies: + xml-name-validator: 5.0.0 + + webidl-conversions@8.0.1: {} + + whatwg-mimetype@5.0.0: {} + + whatwg-url@17.2.0(@noble/hashes@1.8.0): + dependencies: + '@exodus/bytes': 1.16.0(@noble/hashes@1.8.0) + tr46: 7.0.0 + webidl-conversions: 8.0.1 + transitivePeerDependencies: + - '@noble/hashes' + why-is-node-running@3.2.1: {} wrappy@1.0.2: {} + xml-name-validator@5.0.0: {} + + xmlchars@2.2.0: {} + zod@4.6.5: {} diff --git a/media-processor/src/config.ts b/media-processor/src/config.ts index c6229ae343..8d3218fffe 100644 --- a/media-processor/src/config.ts +++ b/media-processor/src/config.ts @@ -12,6 +12,7 @@ const envSchema = z.object({ PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_PIXELS: z.coerce.number().int().positive().default(128_000_000), PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_WIDTH: z.coerce.number().int().positive().default(16384), PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_HEIGHT: z.coerce.number().int().positive().default(16384), + PENPOT_MEDIA_PROCESSOR_SVG_MAX_SIZE: z.coerce.number().int().positive().default(31457280), // 30 MB PENPOT_MEDIA_PROCESSOR_FONT_PROCESS_MEM: z.coerce.number().int().positive().default(512), PENPOT_MEDIA_PROCESSOR_FONT_PROCESS_CPU_TIME: z.coerce.number().int().positive().default(30), PENPOT_MEDIA_PROCESSOR_FONT_TIMEOUT: z.coerce.number().int().positive().default(120000), @@ -51,6 +52,7 @@ export function loadConfig(): AppConfig { imageMaxPixels: parsed.PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_PIXELS, imageMaxWidth: parsed.PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_WIDTH, imageMaxHeight: parsed.PENPOT_MEDIA_PROCESSOR_IMAGE_MAX_HEIGHT, + svgMaxSize: parsed.PENPOT_MEDIA_PROCESSOR_SVG_MAX_SIZE, fontProcessMem: parsed.PENPOT_MEDIA_PROCESSOR_FONT_PROCESS_MEM, fontProcessCpuTime: parsed.PENPOT_MEDIA_PROCESSOR_FONT_PROCESS_CPU_TIME, fontTimeout: parsed.PENPOT_MEDIA_PROCESSOR_FONT_TIMEOUT, diff --git a/media-processor/src/index.ts b/media-processor/src/index.ts index ac05201659..484ef67818 100644 --- a/media-processor/src/index.ts +++ b/media-processor/src/index.ts @@ -4,6 +4,7 @@ import { initLogger, logger, logActiveTransports } from "./logger.js"; import { healthRoutes } from "./routes/health.js"; import { createImageRoutes } from "./routes/image.js"; import { createFontRoutes } from "./routes/font.js"; +import { createSvgRoutes } from "./routes/svg.js"; import { errorHandler } from "./middleware/error-handler.js"; import { timeoutMiddleware } from "./middleware/timeout.js"; import { sharedKeyAuth } from "./middleware/auth.js"; @@ -11,6 +12,7 @@ import { createQueueMiddleware } from "./middleware/queue.js"; import { loggingMiddleware } from "./middleware/logging.js"; import { configureImageLimits } from "./services/image.js"; import { configureFontLimits } from "./services/font.js"; +import { configureSvgLimits } from "./services/svg.js"; import { configureUploadLimits } from "./upload.js"; import sharp from "sharp"; @@ -39,6 +41,8 @@ configureFontLimits({ timeout: config.fontTimeout, }); +configureSvgLimits({ maxSize: config.svgMaxSize }); + configureUploadLimits({ maxFileSize: config.maxFileSize, memoryThreshold: config.memoryThreshold }); const queueMiddleware = createQueueMiddleware(config.maxConcurrentRequests); @@ -49,6 +53,7 @@ app.use(loggingMiddleware); app.get("/api/health", healthRoutes); app.use("/api/image", sharedKeyAuth(config.sharedKey), queueMiddleware, createImageRoutes()); app.use("/api/font", sharedKeyAuth(config.sharedKey), queueMiddleware, createFontRoutes()); +app.use("/api/svg", sharedKeyAuth(config.sharedKey), queueMiddleware, createSvgRoutes()); app.use(errorHandler); app.listen(config.port, config.host, () => { diff --git a/media-processor/src/routes/svg.ts b/media-processor/src/routes/svg.ts new file mode 100644 index 0000000000..dd83fb9489 --- /dev/null +++ b/media-processor/src/routes/svg.ts @@ -0,0 +1,43 @@ +import { Router, type IRouter, type Request, type Response, type NextFunction } from "express"; +import { getUpload, getFileBuffer } from "../upload.js"; +import { sanitizeSvg, getSvgMaxSize } from "../services/svg.js"; +import { throwValidation, throwRestriction } from "../services/errors.js"; +import { cleanupMiddleware } from "../middleware/cleanup.js"; + +export function createSvgRoutes(): IRouter { + const router: IRouter = Router(); + const upload = getUpload(); + + router.post( + "/sanitize", + upload.single("file"), + cleanupMiddleware, + async (req: Request, res: Response, next: NextFunction) => { + const releaseQueue = (res as any).locals?.releaseQueue; + try { + if (!req.file) { + throwValidation("invalid-svg-file", "No file uploaded"); + } + + if (req.file!.size > getSvgMaxSize()) { + throwRestriction( + "svg-too-large", + `SVG size ${req.file!.size} exceeds the maximum of ${getSvgMaxSize()} bytes` + ); + } + + res.locals.opMeta = `size=${req.file!.size}`; + const clean = sanitizeSvg((await getFileBuffer(req.file!)).toString("utf8")); + + res.setHeader("Content-Type", "image/svg+xml"); + res.send(Buffer.from(clean, "utf8")); + } catch (err) { + next(err); + } finally { + if (releaseQueue) releaseQueue(); + } + } + ); + + return router; +} diff --git a/media-processor/src/services/svg.ts b/media-processor/src/services/svg.ts new file mode 100644 index 0000000000..a0f3a304d8 --- /dev/null +++ b/media-processor/src/services/svg.ts @@ -0,0 +1,78 @@ +import createDOMPurify, { type WindowLike } from "dompurify"; +import { JSDOM } from "jsdom"; +import { throwValidation, throwRestriction } from "./errors.js"; +import { ProcessingError } from "../middleware/error-handler.js"; + +const SVG_NAMESPACE = "http://www.w3.org/2000/svg"; + +let svgMaxSize = 30 * 1024 * 1024; + +export function configureSvgLimits(opts: { maxSize: number }): void { + svgMaxSize = opts.maxSize; +} + +export function getSvgMaxSize(): number { + return svgMaxSize; +} + +// A well-formed SVG document may start with an XML declaration or a DOCTYPE. +// DOMPurify parses the SVG as XML, and both make the XML parser bail out and +// the sanitizer return nothing, so they are stripped first. Stripping the +// DOCTYPE is also what keeps its entities from ever being declared. A prolog +// the stripper misses still fails closed: DOMPurify returns an empty document +// and `sanitizeSvg` rejects it. +function stripProlog(svg: string): string { + return svg + .replace(/^\uFEFF/, "") + .replace(/^\s*<\?xml[\s\S]*?\?>/, "") + .replace(/^\s*[]*(\[[\s\S]*?\])?[^>]*>/i, "") + .trimStart(); +} + +const SVG_ROOT = /^]/i; + +/** + * Sanitize an SVG document with DOMPurify over jsdom and return the clean + * SVG as UTF-8 text. + * + * DOMPurify works from an allowlist: every element and attribute it does not + * explicitly allow is removed, so it fails closed. It is given the SVG profile + * and the SVG namespace, so it parses (and therefore judges) the document as an + * SVG document, the way a browser does when it loads a standalone `.svg`. + * + * A fresh jsdom window is created for every call and closed afterwards: + * DOMPurify keeps state on its window, and reusing one window in a long-lived + * process leaks memory and degrades latency without bound. + * + * Throws a `ProcessingError` (413 `svg-too-large`) when the input exceeds the + * configured limit, and (400 `invalid-svg-file`) when the input is not a + * well-formed SVG document. + */ +export function sanitizeSvg(svg: string): string { + const size = Buffer.byteLength(svg, "utf8"); + if (size > svgMaxSize) { + throwRestriction("svg-too-large", `SVG size ${size} exceeds the maximum of ${svgMaxSize} bytes`); + } + + const dom = new JSDOM(""); + try { + const purify = createDOMPurify(dom.window as unknown as WindowLike); + const clean = purify.sanitize(stripProlog(svg), { + USE_PROFILES: { svg: true, svgFilters: true }, + NAMESPACE: SVG_NAMESPACE, + }); + + const normalized = clean.trimStart(); + if (!SVG_ROOT.test(normalized)) { + throwValidation("invalid-svg-file", "SVG sanitization produced no svg root"); + } + return normalized; + } catch (err) { + if (err instanceof ProcessingError) { + throw err; + } + throwValidation("invalid-svg-file", `Failed to sanitize svg: ${(err as Error).message}`); + } finally { + dom.window.close(); + } +} diff --git a/media-processor/src/types.ts b/media-processor/src/types.ts index 22c702f5f7..681af6a619 100644 --- a/media-processor/src/types.ts +++ b/media-processor/src/types.ts @@ -10,6 +10,7 @@ export interface AppConfig { imageMaxPixels: number; imageMaxWidth: number; imageMaxHeight: number; + svgMaxSize: number; fontProcessMem: number; fontProcessCpuTime: number; fontTimeout: number; diff --git a/media-processor/test/svg.test.ts b/media-processor/test/svg.test.ts new file mode 100644 index 0000000000..c376e75a80 --- /dev/null +++ b/media-processor/test/svg.test.ts @@ -0,0 +1,305 @@ +import { describe, it, expect, beforeAll, afterAll, afterEach } from "vitest"; +import request from "supertest"; +import express from "express"; +import { sanitizeSvg, configureSvgLimits } from "../src/services/svg.js"; +import { createSvgRoutes } from "../src/routes/svg.js"; +import { errorHandler, ProcessingError } from "../src/middleware/error-handler.js"; +import { timeoutMiddleware } from "../src/middleware/timeout.js"; +import { sharedKeyAuth } from "../src/middleware/auth.js"; +import { createQueueMiddleware } from "../src/middleware/queue.js"; +import { configureUploadLimits } from "../src/upload.js"; + +const SVG_NS = "http://www.w3.org/2000/svg"; +const DEFAULT_MAX_SIZE = 30 * 1024 * 1024; + +function svg(inner: string, attrs = `xmlns="${SVG_NS}" width="100" height="100"`): string { + return `${inner}`; +} + +// Outputs must never carry an event handler, a javascript: URL or a script +// element, whatever the input was. +function expectNoDangerousOutput(out: string): void { + expect(out).not.toMatch(/\son\w+\s*=/i); + expect(out).not.toMatch(/`), + forbidden: /`), + forbidden: /foreignObject/i, + }, + { + name: "set element", + svg: svg(``), + forbidden: /]/i, + }, + { + name: "animate element", + svg: svg(``), + forbidden: /]/i, + }, + { + name: "use element", + svg: svg(``), + forbidden: /]/i, + }, +]; + +describe("sanitizeSvg: removes dangerous content", () => { + it.each(MALICIOUS)("removes $name", ({ svg: dirty, forbidden }) => { + const clean = sanitizeSvg(dirty); + expect(clean).toMatch(/^]/i); + expect(clean).not.toMatch(forbidden); + expectNoDangerousOutput(clean); + }); +}); + +// --------------------------------------------------------------------------- +// sanitizeSvg: legitimate content survives +// --------------------------------------------------------------------------- + +describe("sanitizeSvg: keeps legitimate SVG", () => { + it("keeps gradients, filters, clipPath, mask, style and text", () => { + const dirty = svg( + `` + + `` + + `` + + `` + + `` + + `` + + `` + + `` + + `hi`, + `xmlns="${SVG_NS}" viewBox="0 0 10 10"` + ); + const clean = sanitizeSvg(dirty); + expect(clean).toMatch(/^]/i); + expect(clean).toContain(""); + expect(clean).toContain(" { + const clean = sanitizeSvg(svg(``)); + expect(clean).toContain('href="https://example.com/a.png"'); + }); + + it("accepts a document with a leading UTF-8 BOM", () => { + const clean = sanitizeSvg(`\uFEFF${svg(``)}`); + expect(clean).toMatch(/^]/i); + }); + + it("accepts a document with an XML declaration and leading whitespace", () => { + const clean = sanitizeSvg( + `\n` + ); + expect(clean).toMatch(/^]/i); + expect(clean).toContain(" { + const dirty = `${svg(``)}`; + const clean = sanitizeSvg(dirty); + expect(clean).toMatch(/^]/i); + expect(clean).toContain(" { + const dirty = `]>${svg(``)}`; + const clean = sanitizeSvg(dirty); + expect(clean).not.toContain("passwd"); + expect(clean).not.toContain("&xxe;"); + }); +}); + +// --------------------------------------------------------------------------- +// sanitizeSvg: rejects what is not a well-formed SVG document +// --------------------------------------------------------------------------- + +describe("sanitizeSvg: rejects invalid input", () => { + const INVALID: Array<{ name: string; svg: string }> = [ + { name: "empty input", svg: "" }, + { name: "plain text", svg: "this is not an svg" }, + { name: "malformed xml", svg: "" }, + { name: "html document", svg: "hi" }, + { name: "text before the root", svg: `hello${svg(``)}` }, + { name: "undefined entity", svg: svg(`&xxe;`) }, + ]; + + it.each(INVALID)("rejects $name with invalid-svg-file", ({ svg: dirty }) => { + expectProcessingError(() => sanitizeSvg(dirty), 400, "invalid-svg-file"); + }); +}); + +// --------------------------------------------------------------------------- +// sanitizeSvg: resource limits and stability +// --------------------------------------------------------------------------- + +describe("sanitizeSvg: limits and stability", () => { + afterEach(() => configureSvgLimits({ maxSize: DEFAULT_MAX_SIZE })); + + it("rejects an input larger than the configured maximum", () => { + configureSvgLimits({ maxSize: 100 }); + const big = svg(`${"".repeat(20)}`); + expectProcessingError(() => sanitizeSvg(big), 413, "svg-too-large"); + }); + + it("stays stable over many consecutive calls", () => { + const dirty = svg(``); + for (let i = 0; i < 150; i++) { + expect(sanitizeSvg(dirty)).toMatch(/^]/i); + } + }); +}); + +// --------------------------------------------------------------------------- +// POST /api/svg/sanitize +// --------------------------------------------------------------------------- + +function createTestApp() { + const app = express(); + // Low threshold to exercise the disk-storage path of the upload. + configureUploadLimits({ maxFileSize: 10 * 1024 * 1024, memoryThreshold: 10 }); + const queueMiddleware = createQueueMiddleware(10); + + app.use(timeoutMiddleware(5000)); + app.use("/api/svg", sharedKeyAuth("test-key"), queueMiddleware, createSvgRoutes()); + app.use(errorHandler); + + return app; +} + +describe("POST /api/svg/sanitize", () => { + let app: ReturnType; + + beforeAll(() => { + app = createTestApp(); + }); + + afterAll(() => { + configureSvgLimits({ maxSize: DEFAULT_MAX_SIZE }); + }); + + it("returns 200 with the sanitized SVG", async () => { + const dirty = svg(``); + const res = await request(app) + .post("/api/svg/sanitize") + .set("x-shared-key", "test-key") + .attach("file", Buffer.from(dirty, "utf8"), { filename: "x.svg", contentType: "image/svg+xml" }); + + expect(res.status).toBe(200); + expect(res.headers["content-type"]).toContain("image/svg+xml"); + const body = Buffer.isBuffer(res.body) ? res.body.toString("utf8") : res.text; + expect(body).toMatch(/^]/i); + expectNoDangerousOutput(body); + expect(body).toContain(" { + const res = await request(app) + .post("/api/svg/sanitize") + .attach("file", Buffer.from(svg(``), "utf8"), { filename: "x.svg", contentType: "image/svg+xml" }); + + expect(res.status).toBe(403); + }); + + it("returns 400 without a file", async () => { + const res = await request(app).post("/api/svg/sanitize").set("x-shared-key", "test-key"); + expect(res.status).toBe(400); + expect(res.body.code).toBe("invalid-svg-file"); + }); + + it("returns 400 when the svg is malformed", async () => { + const res = await request(app) + .post("/api/svg/sanitize") + .set("x-shared-key", "test-key") + .attach("file", Buffer.from("this is not an svg", "utf8"), { + filename: "x.svg", + contentType: "image/svg+xml", + }); + + expect(res.status).toBe(400); + expect(res.body.code).toBe("invalid-svg-file"); + }); + + it("returns 413 when the svg is larger than the configured maximum", async () => { + configureSvgLimits({ maxSize: 10 }); + try { + const res = await request(app) + .post("/api/svg/sanitize") + .set("x-shared-key", "test-key") + .attach("file", Buffer.from(svg(``), "utf8"), { + filename: "x.svg", + contentType: "image/svg+xml", + }); + + expect(res.status).toBe(413); + expect(res.body.code).toBe("svg-too-large"); + } finally { + configureSvgLimits({ maxSize: DEFAULT_MAX_SIZE }); + } + }); +}); diff --git a/media-processor/tsconfig.json b/media-processor/tsconfig.json index b5edd7fa16..408d9ad23c 100644 --- a/media-processor/tsconfig.json +++ b/media-processor/tsconfig.json @@ -3,7 +3,7 @@ "target": "ES2022", "module": "ESNext", "moduleResolution": "bundler", - "lib": ["ES2022"], + "lib": ["ES2022", "DOM"], "outDir": "./dist", "rootDir": "./src", "strict": true,