penpot/backend/src/app/media.clj
Andrey Antukh c39e3e8301
✨ Strictly validate uploads against file bytes on disk (#12801)
* ✨ Strictly validate uploads against file bytes on disk

Sniff the real content type of every upload with simplemagic magic
bytes instead of trusting the client-supplied mtype header. Declared
types outside the allowlist keep the not-allowed code; allowed
declarations whose bytes differ are rejected as mismatch. Covers SVG
by probing its root tag over leading bytes (magic never matches XML
text), TTF/OTF by magic name and WOFF/WOFF2 by four-cc.

Read the real file size from disk instead of trusting the declared
:size. Files over the limit are rejected even when the declared size
lies downwards, and small files pass even when it lies upwards.

Run the path check first everywhere so forged server paths keep the
params-validation code. Recheck from-url downloads after the write,
with a declared-only fast fail at header time. Thread the assembled
path through font validation.

Refresh the affected tests around real and sparse fixture files,
including a PNG-as-font chunked-upload rejection case.

Closes #12796

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Accept BOM-prefixed SVG uploads and cover WOFF/WOFF2/OTF/WEBP

Address review comments on #12801:

- SVG files saved with a UTF-8 BOM were rejected as
  :media-type-mismatch: read-head kept U+FEFF and the <svg probe
  never ran. A leading BOM is now stripped before the probe, and a
  generic XML sniff (application/xml, text/xml) is treated as
  "no magic evidence" for declared image/svg+xml, so the root-tag
  probe decides whether the file opens with an <svg root.
- sniff-match?/svg-file? docstrings claimed the magic database never
  matches SVG; simplemagic 1.17 does match an SVG with an XML prolog
  as image/svg+xml. Reworded to match reality.
- Added matching-pass tests for WOFF, WOFF2, OTF and WEBP (they hit
  the read-four-cc, magic-name and direct magic branches) and three
  BOM SVG regression cases.

AI-assisted-by: inkling-2-small-free
2026-11-10 15:51:48 +01:00

130 lines
5.5 KiB
Clojure

;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns app.media
"Media & Font postprocessing.
This namespace is the dispatch layer only. Processing implementations
live in two separate namespaces, each owning their own defmulti:
app.media.local — shell/ImageMagick/FontForge implementations
app.media.remote — HTTP delegation to media-processor service
Validation and schemas live in app.media.validation (leaf namespace,
no circular dep). When adding a new :cmd type, add defmethods in
BOTH local and remote."
(:require
[app.common.data :as d]
[app.common.exceptions :as ex]
[app.common.media :as cm]
[app.config :as cf]
[app.db :as-alias db]
[app.http.client :as http]
[app.media.local :as media.local]
[app.media.remote :as media.remote]
[app.media.sanitize :as sanitize]
[app.media.validation :as validation]
[app.storage :as-alias sto]
[app.storage.tmp :as tmp]
[cuerdas.core :as str]
[datoteka.io :as io]))
(defn run
[system params]
(if (contains? cf/flags :remote-media-processing)
(media.remote/process system params)
(media.local/process system params)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; IMAGE HELPERS
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn download-image
"Download an image from the provided URI and return the media input object"
[{:keys [::http/client]} uri]
(letfn [(parse-and-validate [{:keys [status headers] :as response}]
(let [size (some-> (get headers "content-length") d/parse-integer)
mtype (get headers "content-type")]
(when-not (<= 200 status 299)
(ex/raise :type :validation
:code :unable-to-download-image
:hint (str/ffmt "unable to download image from '%': unexpected status code %" uri status)))
(when-not size
(ex/raise :type :validation
:code :unknown-size
:hint "seems like the url points to resource with unknown size"))
;; Fast fail on the declared header type and size; the
;; downloaded bytes are strictly verified after the write
;; (there is no file on disk yet to measure or sniff).
(when-not (contains? cm/image-types mtype)
(ex/raise :type :validation
:code :media-type-not-allowed
:hint "seems like you are uploading an invalid media object"))
(when (> size (cf/get :media-max-file-size))
(ex/raise :type :restriction
:code :media-max-file-size-reached
:hint (str/ffmt "the uploaded file size % is greater than the maximum %"
size
(cf/get :media-max-file-size))))
{:size size :mtype mtype}))]
(let [{:keys [body] :as response}
(try
(http/req-with-redirects
client
{:method :get :uri uri}
{:response-type :input-stream
:max-redirects 3})
(catch java.net.ConnectException cause
(ex/raise :type :validation
:code :unable-to-download-image
:hint (str/ffmt "unable to download image from '%': connection refused or host unreachable" uri)
:cause cause))
(catch java.net.http.HttpConnectTimeoutException cause
(ex/raise :type :validation
:code :unable-to-download-image
:hint (str/ffmt "unable to download image from '%': connection timeout" uri)
:cause cause))
(catch java.net.http.HttpTimeoutException cause
(ex/raise :type :validation
:code :unable-to-download-image
:hint (str/ffmt "unable to download image from '%': request timeout" uri)
:cause cause))
(catch java.io.IOException cause
(ex/raise :type :validation
:code :unable-to-download-image
:hint (str/ffmt "unable to download image from '%': I/O error" uri)
:cause cause)))]
(if body
(with-open [body body]
(let [{:keys [size mtype]} (parse-and-validate response)
path (tmp/tempfile :prefix "penpot.media.download.")
written (io/write* path body :size size)]
(when (not= written size)
(ex/raise :type :internal
:code :mismatch-write-size
:hint "unexpected state: unable to write to file"))
;; Sanitize: strip trailing data after image EOF markers
(let [new-size (sanitize/truncate-after-eof path mtype)
upload {:path path :mtype mtype :size new-size}]
;; Strict: the downloaded bytes must be what the remote
;; header claimed, same rule as direct uploads.
(validation/check-media-type upload)
{:path path
:mtype mtype
:size new-size})))
;; No body - validation will raise appropriate error
(parse-and-validate response)))))