mirror of
https://github.com/penpot/penpot.git
synced 2026-07-28 00:48:22 +00:00
b4532486e3
* 🐳 Add ImageMagick policy.xml resource limits to backend Docker image Add a restrictive policy.xml to the backend Docker image that caps ImageMagick resource usage: 256MiB memory, 512MiB map, 128MP area, 30s time limit, 16KP max dimensions. Blocks PS/EPS/PDF/XPS coders to prevent Ghostscript attack surface. Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * ✨ Add timeout support to shell/exec! Add optional :timeout parameter (in seconds) that uses Process.waitFor(long, TimeUnit). On timeout, the process is destroyed forcibly and an :internal/:process-timeout exception is raised. Stdout/stderr readers handle IOException from closed streams when the process is killed. Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * ♻️ Rename ::wrk/netty-executor to ::wrk/executor with cached pool Replace DefaultEventExecutorGroup (fixed Netty thread pool) with a cached thread pool (px/cached-executor) for general async task offloading. The cached pool creates threads on demand and reuses idle ones, which is more appropriate for blocking I/O workloads (shell commands, message bus, rate limiting, etc.). Changes: - Rename ::wrk/netty-executor to ::wrk/executor in worker/executor.clj - Switch implementation from DefaultEventExecutorGroup to px/cached-executor - Update all ig/ref wiring in main.clj (msgbus, tmp cleaner, climit, rlimit, rpc) - Remove ::wrk/netty-executor from redis.clj (let lettuce create its own eventExecutorGroup instead of sharing a Netty executor) - Assert executor is present in shell/exec! to prevent silent nil usage - Remove executor-threads config (no longer needed for cached pool) The ::wrk/netty-io-executor (NioEventLoopGroup) remains unchanged as it handles actual non-blocking network I/O for Redis and S3. Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 🔥 Remove im4java dependency and replace with direct ImageMagick CLI calls - Replace im4java Java library with direct 'magick' CLI calls via shell/exec! - Add PENPOT_IMAGEMAGICK_* config env vars for resource limits (thread, memory, map, area, disk, time, width, height) - Use configurable ImageMagick environment with sensible defaults matching policy.xml - Remove -Dim4java.useV7=true JVM flag from startup scripts - Remove org.im4java/im4java from deps.edn - All ImageMagick commands now use shell/exec! with 60s timeout and resource limits Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 💄 Rename imagemagick env functions and optimize config reads - Rename imagemagick-defaults -> imagemagick-default-env - Rename imagemagick-env -> get-imagemagick-env - Optimize to avoid double cf/get calls per config key Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * ✨ Add tests for shell/exec! timeout and media processing - Add shell_test.clj: tests for exec! timeout, env vars, stdin, stderr - Add media_test.clj: tests for info, generic-thumbnail, profile-thumbnail - Fix generic-process to prefer explicit format over input mtype - Fix shell/exec! to use cached executor when system has no executor - Fix reduce-kv accumulator in set-env (must return penv) Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * ♻️ Refactor media/process to take system as first argument - Change (defmulti process :cmd) -> (defmulti process (fn [_system params] (:cmd params))) - Change (run params) -> (run system params) - All process methods now receive [system params] - Update all callers: rpc/commands/media, profile, auth, fonts - Revert shell/exec! to require system with executor (no fallback) - Fix lint warnings and formatting Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 🔥 Remove unused app.svgo namespace Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 🔥 Remove Node.js from backend Docker image - Delete unused svgo-cli.js script - Remove Node.js installation from Dockerfile.backend - Remove svgo-cli.js copy from backend build script Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 🔥 Remove unused process-error multimethod - Remove process-error multimethod and its default handler - Simplify media/run to directly call process - Fix alignment in main.clj Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> * 📚 Add ImageMagick resource limits configuration to technical guide Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app> --------- Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
127 lines
5.5 KiB
Clojure
127 lines
5.5 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS INC Sucursal en España SL
|
|
|
|
(ns backend-tests.media-test
|
|
(:require
|
|
[app.common.exceptions :as ex]
|
|
[app.media :as media]
|
|
[backend-tests.helpers :as th]
|
|
[clojure.test :as t]
|
|
[datoteka.fs :as fs]))
|
|
|
|
(t/use-fixtures :once th/state-init)
|
|
|
|
(t/deftest info-jpeg
|
|
(t/testing "info on valid JPEG returns dimensions and mime type"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.jpg")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/jpeg"}})]
|
|
(t/is (pos? (:width info)))
|
|
(t/is (pos? (:height info)))
|
|
(t/is (= "image/jpeg" (:mtype info)))
|
|
(t/is (pos? (:size info)))
|
|
(t/is (some? (:ts info))))))
|
|
|
|
(t/deftest info-png
|
|
(t/testing "info on valid PNG returns dimensions and mime type"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.png")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/png"}})]
|
|
(t/is (pos? (:width info)))
|
|
(t/is (pos? (:height info)))
|
|
(t/is (= "image/png" (:mtype info))))))
|
|
|
|
(t/deftest info-webp
|
|
(t/testing "info on valid WebP returns dimensions and mime type"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.webp")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/webp"}})]
|
|
(t/is (pos? (:width info)))
|
|
(t/is (pos? (:height info)))
|
|
(t/is (= "image/webp" (:mtype info))))))
|
|
|
|
(t/deftest info-svg
|
|
(t/testing "info on valid SVG returns dimensions from viewBox"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample1.svg")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/svg+xml"}})]
|
|
(t/is (pos? (:width info)))
|
|
(t/is (pos? (:height info))))))
|
|
|
|
(t/deftest info-invalid-image
|
|
(t/testing "info on invalid image raises error"
|
|
(let [path (fs/create-tempfile :prefix "penpot-test-" :suffix ".jpg")]
|
|
;; Write garbage data
|
|
(spit (str path) "not an image")
|
|
(try
|
|
(media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/jpeg"}})
|
|
(t/is false "should have thrown")
|
|
(catch Exception e
|
|
(let [data (ex-data e)]
|
|
;; Could be validation or imagemagick-error depending on what magick does
|
|
(t/is (contains? #{:validation :internal} (:type data)))))
|
|
(finally
|
|
(fs/delete path))))))
|
|
|
|
(t/deftest generic-thumbnail
|
|
(t/testing "generic-thumbnail produces a file of expected format"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.jpg")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/jpeg"}})
|
|
thumb (media/run th/*system* {:cmd :generic-thumbnail
|
|
:input info
|
|
:format :jpeg
|
|
:quality 80
|
|
:width 200
|
|
:height 200})]
|
|
(t/is (some? (:data thumb)))
|
|
(t/is (pos? (:size thumb)))
|
|
(t/is (= :jpeg (:format thumb)))
|
|
(t/is (= "image/jpeg" (:mtype thumb)))
|
|
;; Verify the thumbnail file exists
|
|
(t/is (fs/exists? (:data thumb))))))
|
|
|
|
(t/deftest profile-thumbnail
|
|
(t/testing "profile-thumbnail produces a center-cropped file"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.jpg")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/jpeg"}})
|
|
thumb (media/run th/*system* {:cmd :profile-thumbnail
|
|
:input info
|
|
:format :jpeg
|
|
:quality 85
|
|
:width 128
|
|
:height 128})]
|
|
(t/is (some? (:data thumb)))
|
|
(t/is (pos? (:size thumb)))
|
|
(t/is (= :jpeg (:format thumb)))
|
|
(t/is (= "image/jpeg" (:mtype thumb)))
|
|
;; Verify the thumbnail file exists
|
|
(t/is (fs/exists? (:data thumb))))))
|
|
|
|
(t/deftest generic-thumbnail-webp
|
|
(t/testing "generic-thumbnail can produce WebP format"
|
|
(let [path (th/tempfile "backend_tests/test_files/sample.jpg")
|
|
info (media/run th/*system* {:cmd :info
|
|
:input {:path path
|
|
:mtype "image/jpeg"}})
|
|
thumb (media/run th/*system* {:cmd :generic-thumbnail
|
|
:input info
|
|
:format :webp
|
|
:quality 80
|
|
:width 200
|
|
:height 200})]
|
|
(t/is (= :webp (:format thumb)))
|
|
(t/is (= "image/webp" (:mtype thumb))))))
|