mirror of
https://github.com/penpot/penpot.git
synced 2026-08-27 15:19:49 +00:00
aeedb96260
* ✨ Add media-processor service for image and font processing Externalizes ImageMagick and FontForge subprocess invocations into a separate Node.js HTTP service (media-processor/). Backend dispatches via feature flag :use-remote-media-processing. Key changes: - media-processor module (TypeScript, Express 5, Sharp, FontForge/woff) - POST /api/image/info, /api/image/thumbnail, /api/font/generate - Resource limits: 128MP rejection, prlimit (512MB + 30s CPU) - Streaming multipart via SequenceInputStream - app.media split into validation (leaf), local (shell impls), remote (HTTP) - Schema enforcement: :upload and :input schemas in validation namespace - Configurable timeout (PENPOT_MEDIA_PROCESSING_SERVICE_TIMEOUT) - 78 tests across 4 files (image, font, middleware, config) - FontForge path escaping for command injection prevention - Parallel font variant conversions with Promise.all AI-assisted-by: mimo-v2.5-pro * 🐳 Revert docker-compose changes from media-processor commit Remove docker-compose.yaml modifications that were part of the media-processor service commit. The media-processor service definition, flags, and environment variables are reverted to their previous state. AI-assisted-by: qwen3.7-plus * ⬆️ Update dependencies * 🐛 Fix PR review issues in media-processor - Font path bug: sfntToWoff and woff2ToSfnt now copy input to temp dir when input is a file path, ensuring output lands in expected location - Error preservation: execCommand preserves killed/signal/code properties from child process errors for OOM detection - Content-Length: service-multipart-request calculates and includes Content-Length header for streaming multipart requests AI-assisted-by: qwen3.7-plus * 🐛 Fix code review issues in media-processor - Rename PENPOT_MEDIA_PROCESSOR_SECRET_KEY to PENPOT_MEDIA_PROCESSOR_SHARED_KEY in devenv to match backend config key - Fix timeout middleware to destroy request AFTER response finishes, preventing truncated 504 responses - Fix quality=0 parsing to preserve explicit zero (was silently overridden to 85) - Replace require('fs') with proper ES module import in upload-storage.ts - Refactor font conversion temp-dir boilerplate into withTempInput helper - Document FontForge escaping limitations (single quotes only) - Fix misleading comment in image.ts about sharp metadata decoding AI-assisted-by: qwen3.7-plus * 🐛 Fix code review issues in media-processor (round 2) - Fix queue middleware to skip next() when response already ended, preventing orphaned work after timeout - Fix hybrid storage to use disk when Content-Length is absent (chunked transfer), preventing unbounded memory allocation - Add source image format validation in generateThumbnail to reject unsupported formats (TIFF, BMP, etc.) with 400 instead of 500 - Remove dead code in convertFont for unreachable woff→woff path - Remove unused isEnabled() method from LokiLogTransport - Fix sfntToWoff to use correct extension (.ttf/.otf) based on source type - Extract queue middleware to separate file for testability - Add comprehensive tests for queue middleware and upload storage AI-assisted-by: qwen3.7-plus * 🐛 Fix code review issues in media-processor (round 3) - Fix disk-backed upload cleanup after successful requests by adding cleanup middleware that removes temp files on response finish/close - Wrap sharp metadata/decoding errors as 400 validation errors instead of 500 internal errors - Only apply flatten() for JPEG output to preserve alpha channel in PNG and WebP outputs AI-assisted-by: qwen3.7-plus * ✨ Add comprehensive tests for media-processor Phase 1 - Cleanup verification: - Add cleanup middleware unit tests (6 tests) - Add HTTP upload cleanup integration tests (5 tests) Phase 2 - Error handling & alpha preservation: - Add sharp error wrapping tests (4 tests) - Add HTTP malformed image tests (2 tests) - Add alpha preservation tests (3 tests) Phase 3 - Edge cases: - Add upload storage edge case tests (3 tests) - Add queue middleware edge case tests (4 tests) Phase 4 - Backend mock verification: - Fix backend mocks to include :mtype field in image info responses - Verify all error codes match actual service behavior Total: 27 new tests added (160 tests passing) AI-assisted-by: qwen3.7-plus * 🐛 Fix code review issues in media-processor (round 4) - Add Zod validation constraints for config values (int, positive, min) - Fix auth middleware to compare Buffer byte lengths instead of string lengths - Validate requested output dimensions in generateThumbnail (crop mode) - Change queue middleware to release slot via callback in finally block - Add comprehensive tests for all fixes AI-assisted-by: qwen3.7-plus * 🐛 Close HTTP response streams in backend media remote - Wrap stream consumption in try/finally with .close() calls - Add tests to verify stream closure for info, font-convert, and thumbnail AI-assisted-by: qwen3.7-plus * 🐛 Fix queue slot leak on upload failures Make releaseQueue idempotent and attach fallback listener to release slot when response finishes. This covers Multer errors that bypass the route handler's finally block, preventing permanent queue stall. AI-assisted-by: qwen3.7-plus * 🐛 Cancel processing on timeout Create AbortController in timeout middleware and abort signal when timeout fires. Pass signal to Sharp and FontForge to cancel ongoing processing and release resources when request is cancelled. AI-assisted-by: qwen3.7-plus * 🐛 Fix code review issues in media-processor (round 6) - Error handler: check headersSent before writing response to prevent ERR_HTTP_HEADERS_SENT when timeout already sent 504 - Timeout config: increase default requestTimeout from 60s to 180s to match font processing timeout (120s) and backend request timeout - Image processing: check abort signal before starting Sharp operations to cancel processing when timeout fires - Queue lifecycle: remove res.on('close', release) fallback to hold queue slot until processing completes, preventing concurrency limit violation when client disconnects AI-assisted-by: qwen3.7-plus * 🐛 Close HTTP response stream in download-image Wrap response body in with-open to ensure stream is closed after writing to temp file, preventing HTTP connection leaks on repeated URL imports. AI-assisted-by: qwen3.7-plus * 🐛 Close HTTP response stream on validation errors in download-image Move with-open to wrap the entire validation and processing block, ensuring the response body stream is closed even when validation fails (non-2xx status, missing size, invalid media type). This prevents HTTP connection leaks on repeated failed downloads. Add test to verify stream closure on validation errors. AI-assisted-by: qwen3.7-plus * 🐛 Pass abort signal to Sharp toBuffer for timeout cancellation Wrap Sharp's toBuffer() with Promise.race to check abort signal during processing. This ensures large thumbnails stop processing when the request times out, preventing wasted CPU/memory and queue capacity. Add test to verify abort during toBuffer operation. AI-assisted-by: qwen3.7-plus * 🐛 Hold queue slot until Sharp completes and handle client disconnect - Remove Promise.race from generateThumbnail — Sharp processing now completes fully before queue slot is released, preventing concurrency limit violations under timeout conditions - Remove res.on("finish", release) fallback from queue middleware — error handler now explicitly calls releaseQueue in all error paths - Add res.on("close") handler in timeout middleware to abort signal when client disconnects, ensuring processing stops early - Add tests for client disconnect handling and queue slot lifecycle AI-assisted-by: qwen3.7-plus * 🐛 Address round 9 review findings - Document Sharp 0.35.3 cancellation limitation in image.ts - Add integration test for timeout cleanup with large images - Fix font tools (sfntToWoff, woffToSfnt, woff2ToSfnt) to throw ProcessingError on resource limit kills instead of returning null - Validate font signatures for same-format conversions to prevent arbitrary files from being persisted as valid fonts - Fix concurrent mkdtemp race in upload-storage by using shared initialization promise AI-assisted-by: qwen3.7-plus * 🐛 Address round 10 review findings - Add tmpdir assertion in font.ts to prevent path injection - Preserve original error in queue middleware catch handler - Change auth middleware response type from "internal" to "authorization" - Add cleanup flag to prevent double cleanup in cleanup middleware - Move quality clamping into parseQuality function for consistency - Add integration tests for quality parameter clamping at route level - Update existing tests to match new auth response type AI-assisted-by: qwen3.7-plus * 🐛 Address round 11 review findings - Extract releaseSlot helper in error-handler to reduce duplication - Remove redundant try/catch in font.ts withTempDir cleanup - Improve font path validation error message for clarity - Move path validation before try/catch to prevent swallowing - Add debug logging for cleanup failures in cleanup middleware - Inline TransportTargetSpec type alias in logger.ts - Extract logging middleware to separate file for consistency - Remove duplicate MIME validation in image thumbnail route - Add test for font path validation (outside tmpdir rejection) - Add tests for error handler queue release across all branches AI-assisted-by: qwen3.7-plus * 🐛 Remove Content-Length header from multipart requests The JDK's HttpClient rejects Content-Length as a restricted header, causing IllegalArgumentException when sending multipart requests to the media-processor. Remove the explicit Content-Length header and let the JDK use chunked transfer encoding. The media-processor will use disk storage for all multipart requests (safe default behavior). Remove unused size computations (file-size, header-bytes, footer-bytes, total-size) that were only used for Content-Length. Update test to verify Content-Length is not present in request headers. AI-assisted-by: qwen3.7-plus * 🐛 Fix pino ESM bundling for media-processor Mark pino and its transports (pino-pretty, pino-loki) as external to avoid bundling issues with worker thread modules that reference __dirname (not available in ES modules). AI-assisted-by: qwen3.7-plus
450 lines
18 KiB
Clojure
450 lines
18 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS INC Sucursal en España SL
|
|
|
|
(ns app.rpc.commands.files-thumbnails
|
|
(:require
|
|
[app.binfile.common :as bfc]
|
|
[app.common.data :as d]
|
|
[app.common.data.macros :as dm]
|
|
[app.common.features :as cfeat]
|
|
[app.common.files.helpers :as cfh]
|
|
[app.common.geom.shapes :as gsh]
|
|
[app.common.schema :as sm]
|
|
[app.common.thumbnails :as thc]
|
|
[app.common.time :as ct]
|
|
[app.common.types.shape-tree :as ctt]
|
|
[app.config :as cf]
|
|
[app.db :as db]
|
|
[app.db.sql :as-alias sql]
|
|
[app.loggers.audit :as-alias audit]
|
|
[app.loggers.webhooks :as-alias webhooks]
|
|
[app.media.validation :as media.v]
|
|
[app.rpc :as-alias rpc]
|
|
[app.rpc.climit :as-alias climit]
|
|
[app.rpc.commands.files :as files]
|
|
[app.rpc.commands.teams :as teams]
|
|
[app.rpc.cond :as-alias cond]
|
|
[app.rpc.doc :as-alias doc]
|
|
[app.rpc.retry :as rtry]
|
|
[app.storage :as sto]
|
|
[app.util.pointer-map :as pmap]
|
|
[app.util.services :as sv]
|
|
[cuerdas.core :as str]))
|
|
|
|
;; --- FEATURES
|
|
|
|
(def long-cache-duration
|
|
(ct/duration {:days 7}))
|
|
|
|
;; --- COMMAND QUERY: get-file-object-thumbnails
|
|
|
|
(defn- get-object-thumbnails-by-tag
|
|
[conn file-id tag]
|
|
(let [sql (str/concat
|
|
"select object_id, media_id, tag "
|
|
" from file_tagged_object_thumbnail"
|
|
" where file_id=? and tag=? and deleted_at is null")
|
|
res (db/exec! conn [sql file-id tag])]
|
|
(->> res
|
|
(d/index-by :object-id :media-id)
|
|
(d/without-nils))))
|
|
|
|
(defn- get-object-thumbnails
|
|
([conn file-id]
|
|
(let [sql (str/concat
|
|
"select object_id, media_id, tag "
|
|
" from file_tagged_object_thumbnail"
|
|
" where file_id=? and deleted_at is null")
|
|
res (db/exec! conn [sql file-id])]
|
|
(->> res
|
|
(d/index-by :object-id :media-id)
|
|
(d/without-nils))))
|
|
|
|
([conn file-id object-ids]
|
|
(let [sql (str/concat
|
|
"select object_id, media_id, tag "
|
|
" from file_tagged_object_thumbnail"
|
|
" where file_id=? and object_id = ANY(?) and deleted_at is null")
|
|
ids (db/create-array conn "text" (seq object-ids))
|
|
res (db/exec! conn [sql file-id ids])]
|
|
|
|
(->> res
|
|
(d/index-by :object-id :media-id)
|
|
(d/without-nils)))))
|
|
|
|
(sv/defmethod ::get-file-object-thumbnails
|
|
"Retrieve a file object thumbnails."
|
|
{::doc/added "1.17"
|
|
::doc/module :files
|
|
::sm/params [:map {:title "get-file-object-thumbnails"}
|
|
[:file-id ::sm/uuid]
|
|
[:tag {:optional true} [:string {:max 50}]]]
|
|
::sm/result [:map-of [:string {:max 250}] [:string {:max 250}]]}
|
|
[{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id tag] :as params}]
|
|
(dm/with-open [conn (db/open pool)]
|
|
(files/check-read-permissions! cfg profile-id file-id)
|
|
(if tag
|
|
(get-object-thumbnails-by-tag conn file-id tag)
|
|
(get-object-thumbnails conn file-id))))
|
|
|
|
;; --- COMMAND QUERY: get-file-data-for-thumbnail
|
|
|
|
;; We need to improve how we set frame for thumbnail in order to avoid
|
|
;; loading all pages into memory for find the frame set for thumbnail.
|
|
|
|
(defn get-file-data-for-thumbnail
|
|
[{:keys [::db/conn] :as cfg} {:keys [data id] :as file} strip-frames-with-thumbnails]
|
|
(letfn [;; function responsible on finding the frame marked to be
|
|
;; used as thumbnail; the returned frame always have
|
|
;; the :page-id set to the page that it belongs.
|
|
(get-thumbnail-frame [{:keys [data]}]
|
|
(d/seek #(or (:use-for-thumbnail %)
|
|
(:use-for-thumbnail? %)) ; NOTE: backward comp (remove on v1.21)
|
|
(for [page (-> data :pages-index vals)
|
|
frame (-> page :objects ctt/get-frames)]
|
|
(assoc frame :page-id (:id page)))))
|
|
|
|
;; function responsible to filter objects data structure of
|
|
;; all unneeded shapes if a concrete frame is provided. If no
|
|
;; frame, the objects is returned untouched.
|
|
(filter-objects [objects frame-id]
|
|
(d/index-by :id (cfh/get-children-with-self objects frame-id)))
|
|
|
|
;; function responsible of assoc available thumbnails
|
|
;; to frames and remove all children shapes from objects if
|
|
;; thumbnails is available
|
|
(assoc-thumbnails [objects page-id thumbnails]
|
|
(loop [objects objects
|
|
frames (filter cfh/frame-shape? (vals objects))]
|
|
|
|
(if-let [frame (-> frames first)]
|
|
(let [frame-id (:id frame)
|
|
object-id (thc/fmt-object-id (:id file) page-id frame-id "frame")
|
|
|
|
frame (if-let [media-id (get thumbnails object-id)]
|
|
(-> frame
|
|
(assoc :thumbnail-id media-id)
|
|
(assoc :shapes []))
|
|
(dissoc frame :thumbnail))
|
|
|
|
children-ids
|
|
(cfh/get-children-ids objects frame-id)
|
|
|
|
bounds
|
|
(when (:show-content frame)
|
|
(gsh/shapes->rect (cons frame (map (d/getf objects) children-ids))))
|
|
|
|
frame
|
|
(cond-> frame
|
|
(some? bounds)
|
|
(assoc :children-bounds bounds))]
|
|
|
|
(if (:thumbnail frame)
|
|
(recur (-> objects
|
|
(assoc frame-id frame)
|
|
(d/without-keys children-ids))
|
|
(rest frames))
|
|
(recur (assoc objects frame-id frame)
|
|
(rest frames))))
|
|
|
|
objects)))]
|
|
|
|
(let [frame (get-thumbnail-frame file)
|
|
frame-id (:id frame)
|
|
page-id (or (:page-id frame)
|
|
(-> data :pages first))
|
|
|
|
page (dm/get-in data [:pages-index page-id])
|
|
page (cond-> page (pmap/pointer-map? page) deref)
|
|
frame-ids (if (some? frame) (list frame-id) (map :id (ctt/get-frames (:objects page))))
|
|
|
|
obj-ids (map #(thc/fmt-object-id (:id file) page-id % "frame") frame-ids)
|
|
thumbs (get-object-thumbnails conn id obj-ids)]
|
|
|
|
(cond-> page
|
|
;; If we have frame, we need to specify it on the page level
|
|
;; and remove the all other unrelated objects.
|
|
(some? frame-id)
|
|
(-> (assoc :thumbnail-frame-id frame-id)
|
|
(update :objects filter-objects frame-id))
|
|
|
|
;; Assoc the available thumbnails and prune not visible shapes
|
|
;; for avoid transfer unnecessary data.
|
|
strip-frames-with-thumbnails
|
|
(update :objects assoc-thumbnails page-id thumbs)))))
|
|
|
|
(def ^:private
|
|
schema:get-file-data-for-thumbnail
|
|
[:map {:title "get-file-data-for-thumbnail"}
|
|
[:file-id ::sm/uuid]])
|
|
|
|
(def ^:private
|
|
schema:partial-file
|
|
[:map {:title "PartialFile"}
|
|
[:id ::sm/uuid]
|
|
[:revn {:min 0} ::sm/int]
|
|
[:page [:map-of :keyword ::sm/any]]
|
|
[:strip-frames-with-thumbnails {:optional true} ::sm/boolean]])
|
|
|
|
(sv/defmethod ::get-file-data-for-thumbnail
|
|
"Retrieves the data for generate the thumbnail of the file. Used
|
|
mainly for render thumbnails on dashboard."
|
|
{::doc/added "1.17"
|
|
::doc/module :files
|
|
::sm/params schema:get-file-data-for-thumbnail
|
|
::sm/result schema:partial-file}
|
|
[cfg {:keys [::rpc/profile-id file-id strip-frames-with-thumbnails] :as params}]
|
|
(db/run! cfg (fn [cfg]
|
|
(files/check-read-permissions! cfg profile-id file-id)
|
|
(let [team (teams/get-team cfg
|
|
:profile-id profile-id
|
|
:file-id file-id)
|
|
file (bfc/get-file cfg file-id
|
|
:include-deleted? true
|
|
:realize? true
|
|
:read-only? true)
|
|
strip-frames-with-thumbnails
|
|
(or (nil? strip-frames-with-thumbnails) ;; if not present, default to true
|
|
(true? strip-frames-with-thumbnails))]
|
|
|
|
(-> (cfeat/get-team-enabled-features cf/flags team)
|
|
(cfeat/check-file-features! (:features file)))
|
|
|
|
{:file-id file-id
|
|
:revn (:revn file)
|
|
:page (get-file-data-for-thumbnail cfg file strip-frames-with-thumbnails)}))))
|
|
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
;; MUTATION COMMANDS
|
|
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
|
|
|
(def sql:get-file-object-thumbnail
|
|
"SELECT * FROM file_tagged_object_thumbnail
|
|
WHERE file_id = ? AND object_id = ? AND tag = ?
|
|
FOR UPDATE")
|
|
|
|
(def sql:create-file-object-thumbnail
|
|
"INSERT INTO file_tagged_object_thumbnail (file_id, object_id, tag, media_id)
|
|
VALUES (?, ?, ?, ?)
|
|
ON CONFLICT (file_id, object_id, tag)
|
|
DO UPDATE SET updated_at=?, media_id=?, deleted_at=?
|
|
RETURNING *")
|
|
|
|
(defn- persist-thumbnail!
|
|
[storage media created-at]
|
|
(let [path (:path media)
|
|
mtype (:mtype media)
|
|
hash (sto/calculate-hash path)
|
|
data (-> (sto/content path)
|
|
(sto/wrap-with-hash hash))]
|
|
|
|
(sto/put-object! storage
|
|
{::sto/content data
|
|
::sto/deduplicate? true
|
|
::sto/touched-at created-at
|
|
:content-type mtype
|
|
:bucket "file-object-thumbnail"})))
|
|
|
|
(defn- create-file-object-thumbnail!
|
|
[{:keys [::sto/storage] :as cfg} file object-id media tag]
|
|
(let [file-id (:id file)
|
|
timestamp (ct/now)
|
|
media (persist-thumbnail! storage media timestamp)
|
|
[th1 th2] (db/tx-run! cfg (fn [{:keys [::db/conn]}]
|
|
(let [th1 (db/exec-one! conn [sql:get-file-object-thumbnail file-id object-id tag])
|
|
th2 (db/exec-one! conn [sql:create-file-object-thumbnail
|
|
file-id object-id tag
|
|
(:id media)
|
|
timestamp
|
|
(:id media)
|
|
(:deleted-at file)])]
|
|
[th1 th2])))]
|
|
|
|
(when (and (some? th1)
|
|
(not= (:media-id th1)
|
|
(:media-id th2)))
|
|
(sto/touch-object! storage (:media-id th1)))
|
|
|
|
th2))
|
|
|
|
(def ^:private
|
|
schema:create-file-object-thumbnail
|
|
[:map {:title "create-file-object-thumbnail"}
|
|
[:file-id ::sm/uuid]
|
|
[:object-id [:string {:max 250}]]
|
|
[:media media.v/schema:upload]
|
|
[:tag {:optional true} [:string {:max 50}]]])
|
|
|
|
(sv/defmethod ::create-file-object-thumbnail
|
|
{::doc/added "1.19"
|
|
::doc/module :files
|
|
::climit/id [[:file-thumbnail-ops/by-profile ::rpc/profile-id]
|
|
[:file-thumbnail-ops/global]]
|
|
::rtry/enabled true
|
|
::rtry/when rtry/conflict-exception?
|
|
::audit/skip true
|
|
::sm/params schema:create-file-object-thumbnail}
|
|
|
|
[cfg {:keys [::rpc/profile-id file-id object-id media tag]}]
|
|
(media.v/validate-media-type! media)
|
|
(media.v/validate-media-size! media)
|
|
|
|
(db/run! cfg files/check-edition-permissions! profile-id file-id)
|
|
(when-let [file (files/get-minimal-file cfg file-id {::db/check-deleted false})]
|
|
(create-file-object-thumbnail! cfg file object-id media (or tag "frame"))))
|
|
|
|
;; --- MUTATION COMMAND: delete-file-object-thumbnail
|
|
|
|
(defn- delete-file-object-thumbnail!
|
|
[{:keys [::db/conn ::sto/storage]} file-id object-id]
|
|
(when-let [{:keys [media-id tag]} (db/get* conn :file-tagged-object-thumbnail
|
|
{:file-id file-id
|
|
:object-id object-id}
|
|
{::sql/for-update true})]
|
|
(sto/touch-object! storage media-id)
|
|
(db/update! conn :file-tagged-object-thumbnail
|
|
{:deleted-at (ct/now)}
|
|
{:file-id file-id
|
|
:object-id object-id
|
|
:tag tag})))
|
|
|
|
(defn- delete-file-object-thumbnails!
|
|
"Soft-deletes multiple object thumbnails in a single UPDATE statement
|
|
with RETURNING, then touches all returned media objects."
|
|
[{:keys [::db/conn ::sto/storage]} object-ids]
|
|
(let [ids (db/create-array conn "text" (seq object-ids))
|
|
sql (str/concat
|
|
"UPDATE file_tagged_object_thumbnail"
|
|
" SET deleted_at = now()"
|
|
" WHERE object_id = ANY(?)"
|
|
" AND deleted_at IS NULL"
|
|
" RETURNING media_id")
|
|
rows (db/exec! conn [sql ids])]
|
|
(doseq [{:keys [media-id]} rows]
|
|
(sto/touch-object! storage media-id))))
|
|
|
|
(def ^:private schema:delete-file-object-thumbnail
|
|
[:map {:title "delete-file-object-thumbnail"}
|
|
[:file-id ::sm/uuid]
|
|
[:object-id [:string {:max 250}]]])
|
|
|
|
(def ^:private schema:delete-file-object-thumbnails
|
|
[:map {:title "delete-file-object-thumbnails"}
|
|
[:object-ids [:vector {:max 200} [:string {:max 250}]]]])
|
|
|
|
(sv/defmethod ::delete-file-object-thumbnail
|
|
{::doc/added "1.19"
|
|
::doc/module :files
|
|
::sm/params schema:delete-file-object-thumbnail
|
|
::audit/skip true}
|
|
[cfg {:keys [::rpc/profile-id file-id object-id]}]
|
|
(files/check-edition-permissions! cfg profile-id file-id)
|
|
(db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
|
|
(-> cfg
|
|
(update ::sto/storage sto/configure conn)
|
|
(delete-file-object-thumbnail! file-id object-id))
|
|
nil)))
|
|
|
|
(sv/defmethod ::delete-file-object-thumbnails
|
|
{::doc/added "1.19"
|
|
::doc/module :files
|
|
::climit/id [[:file-thumbnail-ops/by-profile ::rpc/profile-id]
|
|
[:file-thumbnail-ops/global]]
|
|
::sm/params schema:delete-file-object-thumbnails
|
|
::audit/skip true}
|
|
[cfg {:keys [::rpc/profile-id object-ids]}]
|
|
(when (seq object-ids)
|
|
;; Extract unique file-ids from object-ids for permission checks
|
|
(let [file-ids (->> object-ids
|
|
(map thc/get-file-id)
|
|
(into #{}))]
|
|
;; Check permissions for each unique file using a single connection
|
|
(db/run! cfg (fn [{:keys [::db/conn]}]
|
|
(doseq [file-id file-ids]
|
|
(files/check-edition-permissions! conn profile-id file-id))))
|
|
;; Delete all matching thumbnails in one transaction
|
|
(db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
|
|
(-> cfg
|
|
(update ::sto/storage sto/configure conn)
|
|
(delete-file-object-thumbnails! object-ids))
|
|
nil)))))
|
|
|
|
;; --- MUTATION COMMAND: create-file-thumbnail
|
|
|
|
(def ^:private
|
|
schema:create-file-thumbnail
|
|
[:map {:title "create-file-thumbnail"}
|
|
[:file-id ::sm/uuid]
|
|
[:revn ::sm/int]
|
|
[:media media.v/schema:upload]])
|
|
|
|
(sv/defmethod ::create-file-thumbnail
|
|
"Creates or updates the file thumbnail. Mainly used for paint the
|
|
grid thumbnails."
|
|
{::doc/added "1.19"
|
|
::doc/module :files
|
|
::audit/skip true
|
|
::climit/id [[:file-thumbnail-ops/by-profile ::rpc/profile-id]
|
|
[:file-thumbnail-ops/global]]
|
|
::rtry/enabled true
|
|
::rtry/when rtry/conflict-exception?
|
|
::sm/params schema:create-file-thumbnail}
|
|
|
|
[cfg {:keys [::rpc/profile-id file-id] :as params}]
|
|
(media.v/validate-media-type! (:media params))
|
|
(media.v/validate-media-size! (:media params))
|
|
|
|
(db/run! cfg files/check-edition-permissions! profile-id file-id)
|
|
|
|
(when-not (db/read-only? (::db/pool cfg))
|
|
(let [storage (::sto/storage cfg)
|
|
file (bfc/get-file cfg file-id :include-deleted? true :load-data? false)
|
|
props (db/tjson (or (:props params) {}))
|
|
{:keys [path mtype]} (:media params)
|
|
hash (sto/calculate-hash path)
|
|
data (-> (sto/content path)
|
|
(sto/wrap-with-hash hash))
|
|
tnow (ct/now)
|
|
|
|
media (sto/put-object! storage
|
|
{::sto/content data
|
|
::sto/deduplicate? true
|
|
::sto/touched-at tnow
|
|
:content-type mtype
|
|
:bucket "file-thumbnail"})
|
|
|
|
revn (:revn params)
|
|
|
|
result (db/tx-run! cfg
|
|
(fn [{:keys [::db/conn]}]
|
|
(let [thumb (db/get* conn :file-thumbnail
|
|
{:file-id file-id :revn revn}
|
|
{::db/remove-deleted false
|
|
::sql/for-update true})]
|
|
(if (some? thumb)
|
|
(do
|
|
(when (not= (:id media) (:media-id thumb))
|
|
(sto/touch-object! storage (:media-id thumb)))
|
|
(db/update! conn :file-thumbnail
|
|
{:media-id (:id media)
|
|
:deleted-at (:deleted-at file)
|
|
:updated-at tnow
|
|
:props props}
|
|
{:file-id file-id :revn revn}))
|
|
(db/insert! conn :file-thumbnail
|
|
{:file-id file-id
|
|
:revn revn
|
|
:created-at tnow
|
|
:updated-at tnow
|
|
:deleted-at (:deleted-at file)
|
|
:props props
|
|
:media-id (:id media)}))
|
|
media)))]
|
|
|
|
(when result
|
|
{:uri (files/resolve-public-uri (:id result))
|
|
:id (:id result)}))))
|