84 Commits

Author SHA1 Message Date
Andrey Antukh
c39e3e8301
✨ Strictly validate uploads against file bytes on disk (#12801)
* ✨ Strictly validate uploads against file bytes on disk

Sniff the real content type of every upload with simplemagic magic
bytes instead of trusting the client-supplied mtype header. Declared
types outside the allowlist keep the not-allowed code; allowed
declarations whose bytes differ are rejected as mismatch. Covers SVG
by probing its root tag over leading bytes (magic never matches XML
text), TTF/OTF by magic name and WOFF/WOFF2 by four-cc.

Read the real file size from disk instead of trusting the declared
:size. Files over the limit are rejected even when the declared size
lies downwards, and small files pass even when it lies upwards.

Run the path check first everywhere so forged server paths keep the
params-validation code. Recheck from-url downloads after the write,
with a declared-only fast fail at header time. Thread the assembled
path through font validation.

Refresh the affected tests around real and sparse fixture files,
including a PNG-as-font chunked-upload rejection case.

Closes #12796

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Accept BOM-prefixed SVG uploads and cover WOFF/WOFF2/OTF/WEBP

Address review comments on #12801:

- SVG files saved with a UTF-8 BOM were rejected as
  :media-type-mismatch: read-head kept U+FEFF and the <svg probe
  never ran. A leading BOM is now stripped before the probe, and a
  generic XML sniff (application/xml, text/xml) is treated as
  "no magic evidence" for declared image/svg+xml, so the root-tag
  probe decides whether the file opens with an <svg root.
- sniff-match?/svg-file? docstrings claimed the magic database never
  matches SVG; simplemagic 1.17 does match an SVG with an XML prolog
  as image/svg+xml. Reworded to match reality.
- Added matching-pass tests for WOFF, WOFF2, OTF and WEBP (they hit
  the read-four-cc, magic-name and direct magic branches) and three
  BOM SVG regression cases.

AI-assisted-by: inkling-2-small-free
2026-11-10 15:51:48 +01:00
Andrey Antukh
610813281b Merge remote-tracking branch 'origin/main-2.18.x' into staging 2026-11-03 15:06:32 +01:00
Andrey Antukh
afe6c1b99c
🐛 Confine upload paths to temp dirs (#12772)
* 🐛 Confine upload paths to temp dirs

Reject upload maps whose path leaves the server temp dirs.
Transit bodies can forge a Path such as /etc/passwd and the
handlers open it directly. Centralize the realpath prefix
check in media validation and apply it to all nine upload
fields. Legit multipart files under /tmp/undertow/ or
/tmp/penpot/ still pass.

AI-assisted-by: muse-spark-1.3-contributor

* ♻️ Rename upload validators to check-*

Drop the validate prefix and the bang from the media
validation helpers for a consistent check naming. Move
the temp dir confinement into validation and cover all
nine upload fields. Add unit tests for each helper.

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Fix svg upload tests to use penpot tmp dir

SVG upload tests wrote tempfiles to the system tmp dir,
outside the allowed upload prefixes, so check-upload-path
rejected them. Point write-temp-svg at /tmp/penpot like
the other upload tests.

AI-assisted-by: muse-spark-1.3-contributor
2026-11-03 12:07:52 +01:00
Andrey Antukh
81c3b3cd56
📎 Update copyright name on file header (#11346) 2026-08-25 11:55:10 +02:00
Andrey Antukh
aeedb96260
✨ Add media-processor service for image and font processing (#10767)
* ✨ Add media-processor service for image and font processing

Externalizes ImageMagick and FontForge subprocess invocations into a
separate Node.js HTTP service (media-processor/). Backend dispatches
via feature flag :use-remote-media-processing.

Key changes:
- media-processor module (TypeScript, Express 5, Sharp, FontForge/woff)
  - POST /api/image/info, /api/image/thumbnail, /api/font/generate
  - Resource limits: 128MP rejection, prlimit (512MB + 30s CPU)
  - Streaming multipart via SequenceInputStream
- app.media split into validation (leaf), local (shell impls), remote (HTTP)
- Schema enforcement: :upload and :input schemas in validation namespace
- Configurable timeout (PENPOT_MEDIA_PROCESSING_SERVICE_TIMEOUT)
- 78 tests across 4 files (image, font, middleware, config)
- FontForge path escaping for command injection prevention
- Parallel font variant conversions with Promise.all

AI-assisted-by: mimo-v2.5-pro

* 🐳 Revert docker-compose changes from media-processor commit

Remove docker-compose.yaml modifications that were part of the media-processor
service commit. The media-processor service definition, flags, and environment
variables are reverted to their previous state.

AI-assisted-by: qwen3.7-plus

* ⬆️ Update dependencies

* 🐛 Fix PR review issues in media-processor

- Font path bug: sfntToWoff and woff2ToSfnt now copy input to temp dir
  when input is a file path, ensuring output lands in expected location
- Error preservation: execCommand preserves killed/signal/code properties
  from child process errors for OOM detection
- Content-Length: service-multipart-request calculates and includes
  Content-Length header for streaming multipart requests

AI-assisted-by: qwen3.7-plus

* 🐛 Fix code review issues in media-processor

- Rename PENPOT_MEDIA_PROCESSOR_SECRET_KEY to PENPOT_MEDIA_PROCESSOR_SHARED_KEY
  in devenv to match backend config key
- Fix timeout middleware to destroy request AFTER response finishes,
  preventing truncated 504 responses
- Fix quality=0 parsing to preserve explicit zero (was silently overridden to 85)
- Replace require('fs') with proper ES module import in upload-storage.ts
- Refactor font conversion temp-dir boilerplate into withTempInput helper
- Document FontForge escaping limitations (single quotes only)
- Fix misleading comment in image.ts about sharp metadata decoding

AI-assisted-by: qwen3.7-plus

* 🐛 Fix code review issues in media-processor (round 2)

- Fix queue middleware to skip next() when response already ended,
  preventing orphaned work after timeout
- Fix hybrid storage to use disk when Content-Length is absent (chunked
  transfer), preventing unbounded memory allocation
- Add source image format validation in generateThumbnail to reject
  unsupported formats (TIFF, BMP, etc.) with 400 instead of 500
- Remove dead code in convertFont for unreachable woff→woff path
- Remove unused isEnabled() method from LokiLogTransport
- Fix sfntToWoff to use correct extension (.ttf/.otf) based on source type
- Extract queue middleware to separate file for testability
- Add comprehensive tests for queue middleware and upload storage

AI-assisted-by: qwen3.7-plus

* 🐛 Fix code review issues in media-processor (round 3)

- Fix disk-backed upload cleanup after successful requests by adding
  cleanup middleware that removes temp files on response finish/close
- Wrap sharp metadata/decoding errors as 400 validation errors instead
  of 500 internal errors
- Only apply flatten() for JPEG output to preserve alpha channel in
  PNG and WebP outputs

AI-assisted-by: qwen3.7-plus

* ✨ Add comprehensive tests for media-processor

Phase 1 - Cleanup verification:
- Add cleanup middleware unit tests (6 tests)
- Add HTTP upload cleanup integration tests (5 tests)

Phase 2 - Error handling & alpha preservation:
- Add sharp error wrapping tests (4 tests)
- Add HTTP malformed image tests (2 tests)
- Add alpha preservation tests (3 tests)

Phase 3 - Edge cases:
- Add upload storage edge case tests (3 tests)
- Add queue middleware edge case tests (4 tests)

Phase 4 - Backend mock verification:
- Fix backend mocks to include :mtype field in image info responses
- Verify all error codes match actual service behavior

Total: 27 new tests added (160 tests passing)

AI-assisted-by: qwen3.7-plus

* 🐛 Fix code review issues in media-processor (round 4)

- Add Zod validation constraints for config values (int, positive, min)
- Fix auth middleware to compare Buffer byte lengths instead of string lengths
- Validate requested output dimensions in generateThumbnail (crop mode)
- Change queue middleware to release slot via callback in finally block
- Add comprehensive tests for all fixes

AI-assisted-by: qwen3.7-plus

* 🐛 Close HTTP response streams in backend media remote

- Wrap stream consumption in try/finally with .close() calls
- Add tests to verify stream closure for info, font-convert, and thumbnail

AI-assisted-by: qwen3.7-plus

* 🐛 Fix queue slot leak on upload failures

Make releaseQueue idempotent and attach fallback listener to release
slot when response finishes. This covers Multer errors that bypass
the route handler's finally block, preventing permanent queue stall.

AI-assisted-by: qwen3.7-plus

* 🐛 Cancel processing on timeout

Create AbortController in timeout middleware and abort signal when
timeout fires. Pass signal to Sharp and FontForge to cancel ongoing
processing and release resources when request is cancelled.

AI-assisted-by: qwen3.7-plus

* 🐛 Fix code review issues in media-processor (round 6)

- Error handler: check headersSent before writing response to prevent
  ERR_HTTP_HEADERS_SENT when timeout already sent 504
- Timeout config: increase default requestTimeout from 60s to 180s to
  match font processing timeout (120s) and backend request timeout
- Image processing: check abort signal before starting Sharp operations
  to cancel processing when timeout fires
- Queue lifecycle: remove res.on('close', release) fallback to hold
  queue slot until processing completes, preventing concurrency limit
  violation when client disconnects

AI-assisted-by: qwen3.7-plus

* 🐛 Close HTTP response stream in download-image

Wrap response body in with-open to ensure stream is closed after
writing to temp file, preventing HTTP connection leaks on repeated
URL imports.

AI-assisted-by: qwen3.7-plus

* 🐛 Close HTTP response stream on validation errors in download-image

Move with-open to wrap the entire validation and processing block,
ensuring the response body stream is closed even when validation fails
(non-2xx status, missing size, invalid media type). This prevents
HTTP connection leaks on repeated failed downloads.

Add test to verify stream closure on validation errors.

AI-assisted-by: qwen3.7-plus

* 🐛 Pass abort signal to Sharp toBuffer for timeout cancellation

Wrap Sharp's toBuffer() with Promise.race to check abort signal during
processing. This ensures large thumbnails stop processing when the
request times out, preventing wasted CPU/memory and queue capacity.

Add test to verify abort during toBuffer operation.

AI-assisted-by: qwen3.7-plus

* 🐛 Hold queue slot until Sharp completes and handle client disconnect

- Remove Promise.race from generateThumbnail — Sharp processing now
  completes fully before queue slot is released, preventing concurrency
  limit violations under timeout conditions
- Remove res.on("finish", release) fallback from queue middleware —
  error handler now explicitly calls releaseQueue in all error paths
- Add res.on("close") handler in timeout middleware to abort signal
  when client disconnects, ensuring processing stops early
- Add tests for client disconnect handling and queue slot lifecycle

AI-assisted-by: qwen3.7-plus

* 🐛 Address round 9 review findings

- Document Sharp 0.35.3 cancellation limitation in image.ts
- Add integration test for timeout cleanup with large images
- Fix font tools (sfntToWoff, woffToSfnt, woff2ToSfnt) to throw
  ProcessingError on resource limit kills instead of returning null
- Validate font signatures for same-format conversions to prevent
  arbitrary files from being persisted as valid fonts
- Fix concurrent mkdtemp race in upload-storage by using shared
  initialization promise

AI-assisted-by: qwen3.7-plus

* 🐛 Address round 10 review findings

- Add tmpdir assertion in font.ts to prevent path injection
- Preserve original error in queue middleware catch handler
- Change auth middleware response type from "internal" to "authorization"
- Add cleanup flag to prevent double cleanup in cleanup middleware
- Move quality clamping into parseQuality function for consistency
- Add integration tests for quality parameter clamping at route level
- Update existing tests to match new auth response type

AI-assisted-by: qwen3.7-plus

* 🐛 Address round 11 review findings

- Extract releaseSlot helper in error-handler to reduce duplication
- Remove redundant try/catch in font.ts withTempDir cleanup
- Improve font path validation error message for clarity
- Move path validation before try/catch to prevent swallowing
- Add debug logging for cleanup failures in cleanup middleware
- Inline TransportTargetSpec type alias in logger.ts
- Extract logging middleware to separate file for consistency
- Remove duplicate MIME validation in image thumbnail route
- Add test for font path validation (outside tmpdir rejection)
- Add tests for error handler queue release across all branches

AI-assisted-by: qwen3.7-plus

* 🐛 Remove Content-Length header from multipart requests

The JDK's HttpClient rejects Content-Length as a restricted header,
causing IllegalArgumentException when sending multipart requests to the
media-processor. Remove the explicit Content-Length header and let the
JDK use chunked transfer encoding. The media-processor will use disk
storage for all multipart requests (safe default behavior).

Remove unused size computations (file-size, header-bytes, footer-bytes,
total-size) that were only used for Content-Length.

Update test to verify Content-Length is not present in request headers.

AI-assisted-by: qwen3.7-plus

* 🐛 Fix pino ESM bundling for media-processor

Mark pino and its transports (pino-pretty, pino-loki) as external to
avoid bundling issues with worker thread modules that reference
__dirname (not available in ES modules).

AI-assisted-by: qwen3.7-plus
2026-08-05 09:41:48 +02:00
Andrey Antukh
26c4ec18fe
🐛 Return 400 instead of 500 when ImageMagick fails on invalid images (#10643)
When ImageMagick fails to process an uploaded image (e.g., corrupted PNG
with invalid IHDR data), the backend was raising :type :internal with
:code :imagemagick-error, which mapped to HTTP 500. The frontend treated
this as a server error and displayed the full error page.

Changed exec-magick! to raise :type :validation with :code :invalid-image
instead. This flows through the existing :invalid-image handler in
errors.clj which returns HTTP 400. The frontend's handle-media-error and
process-error now catch this code and show a notification banner.

AI-assisted-by: qwen3.7-plus
2026-07-20 11:56:07 +02:00
Andrey Antukh
9e52bb40d0
✨ Add process-level resource limits to font processing tools (#10274)
* ✨ Add font processing resource limits via prlimit

Font processing tools (fontforge, sfnt2woff, woff2sfnt, woff2_decompress)
were invoked via clojure.java.shell/sh with no timeouts or resource limits.
This adds process-level resource limits using prlimit(1) and the shell/exec!
infrastructure from the ImageMagick hardening work.

shell/exec! changes:
- Add :prlimit parameter that prepends prlimit(1) to the command
- :prlimit takes {:mem <MiB> :cpu <seconds>} for address space and CPU time
  limits, enforced by the kernel's RLIMIT subsystem
- prlimit-cmd builds the prlimit command prefix (private helper)

Font processing changes:
- Replace all clojure.java.shell/sh calls with shell/exec! via exec-font!
- exec-font! applies font-prlimit (512 MiB, 30s CPU, 60s wall-clock)
- All 5 conversion functions (ttf->otf, otf->ttf, ttf-or-otf->woff,
  woff->sfnt, woff2->sfnt) use try/finally for explicit temp file cleanup
- Remove clojure.java.shell require from media.clj

Tests:
- Add exec-prlimit-normal, exec-prlimit-cpu, exec-prlimit-memory tests

Closes #10234

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* ✨ Make font processing resource limits configurable

Replace hardcoded font-prlimit map and wall-clock timeout with
config-driven values under the PENPOT_FONT_PROCESS_* namespace.
The prlimit implementation detail is not exposed in config keys.

Co-authored-by: deepseek-v4-flash <deepseek-v4-flash@penpot.app>

---------

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
Co-authored-by: deepseek-v4-flash <deepseek-v4-flash@penpot.app>
2026-06-19 11:30:48 +02:00
Andrey Antukh
b4532486e3
✨ Add configurable resource usage limits for imagemagick (#10240)
* 🐳 Add ImageMagick policy.xml resource limits to backend Docker image

Add a restrictive policy.xml to the backend Docker image that caps
ImageMagick resource usage: 256MiB memory, 512MiB map, 128MP area,
30s time limit, 16KP max dimensions. Blocks PS/EPS/PDF/XPS coders
to prevent Ghostscript attack surface.

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* ✨ Add timeout support to shell/exec!

Add optional :timeout parameter (in seconds) that uses
Process.waitFor(long, TimeUnit). On timeout, the process is
destroyed forcibly and an :internal/:process-timeout exception
is raised. Stdout/stderr readers handle IOException from closed
streams when the process is killed.

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* ♻️ Rename ::wrk/netty-executor to ::wrk/executor with cached pool

Replace DefaultEventExecutorGroup (fixed Netty thread pool) with a
cached thread pool (px/cached-executor) for general async task
offloading. The cached pool creates threads on demand and reuses
idle ones, which is more appropriate for blocking I/O workloads
(shell commands, message bus, rate limiting, etc.).

Changes:
- Rename ::wrk/netty-executor to ::wrk/executor in worker/executor.clj
- Switch implementation from DefaultEventExecutorGroup to px/cached-executor
- Update all ig/ref wiring in main.clj (msgbus, tmp cleaner, climit, rlimit, rpc)
- Remove ::wrk/netty-executor from redis.clj (let lettuce create its own
  eventExecutorGroup instead of sharing a Netty executor)
- Assert executor is present in shell/exec! to prevent silent nil usage
- Remove executor-threads config (no longer needed for cached pool)

The ::wrk/netty-io-executor (NioEventLoopGroup) remains unchanged as it
handles actual non-blocking network I/O for Redis and S3.

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 🔥 Remove im4java dependency and replace with direct ImageMagick CLI calls

- Replace im4java Java library with direct 'magick' CLI calls via shell/exec!
- Add PENPOT_IMAGEMAGICK_* config env vars for resource limits (thread, memory, map, area, disk, time, width, height)
- Use configurable ImageMagick environment with sensible defaults matching policy.xml
- Remove -Dim4java.useV7=true JVM flag from startup scripts
- Remove org.im4java/im4java from deps.edn
- All ImageMagick commands now use shell/exec! with 60s timeout and resource limits

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 💄 Rename imagemagick env functions and optimize config reads

- Rename imagemagick-defaults -> imagemagick-default-env
- Rename imagemagick-env -> get-imagemagick-env
- Optimize to avoid double cf/get calls per config key

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* ✨ Add tests for shell/exec! timeout and media processing

- Add shell_test.clj: tests for exec! timeout, env vars, stdin, stderr
- Add media_test.clj: tests for info, generic-thumbnail, profile-thumbnail
- Fix generic-process to prefer explicit format over input mtype
- Fix shell/exec! to use cached executor when system has no executor
- Fix reduce-kv accumulator in set-env (must return penv)

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* ♻️ Refactor media/process to take system as first argument

- Change (defmulti process :cmd) -> (defmulti process (fn [_system params] (:cmd params)))
- Change (run params) -> (run system params)
- All process methods now receive [system params]
- Update all callers: rpc/commands/media, profile, auth, fonts
- Revert shell/exec! to require system with executor (no fallback)
- Fix lint warnings and formatting

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 🔥 Remove unused app.svgo namespace

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 🔥 Remove Node.js from backend Docker image

- Delete unused svgo-cli.js script
- Remove Node.js installation from Dockerfile.backend
- Remove svgo-cli.js copy from backend build script

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 🔥 Remove unused process-error multimethod

- Remove process-error multimethod and its default handler
- Simplify media/run to directly call process
- Fix alignment in main.clj

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

* 📚 Add ImageMagick resource limits configuration to technical guide

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>

---------

Co-authored-by: mimo-v2.5-pro <mimo-v2.5-pro@penpot.app>
2026-06-18 17:52:01 +02:00
Andrey Antukh
c5de4c27b0 Merge remote-tracking branch 'origin/main' into staging 2026-06-01 12:57:39 +02:00
Yamila Moreno
ddba2ffa75
📎 Update Kaleidos Copyright (#9929) 2026-05-29 11:24:58 +02:00
Andrey Antukh
d78074307f Merge remote-tracking branch 'origin/main' into staging 2026-05-14 11:07:42 +02:00
Andrey Antukh
947f6d392d
🎉 Add chunked upload support for font variants (#9551)
* ✨ Add additional logging and validation for image upload

* 🎉 Add chunked upload support for font variants

Extend the font variant upload flow across frontend, backend, and common
to support the standardized chunked upload protocol.

**Backend:**
- Add \`:font-max-file-size\` config default (30 MiB) and schema entry
- Add \`validate-font-size!\` in \`media.clj\` (mirrors
  \`validate-media-size!\`, raises \`:font-max-file-size-reached\`)
- Extend \`schema:create-font-variant\` to accept either \`:data\`
  (legacy bytes or chunk-vector) or \`:uploads\` (new chunked session
  map), with a validator requiring exactly one
- Add \`prepare-font-data-from-uploads\`: assembles each chunked
  session via \`cmedia/assemble-chunks\`, validates type+size
- Add \`prepare-font-data-from-legacy\`: normalises legacy byte/chunk
  entries, writing to a tempfile (joining via SequenceInputStream),
  validates type+size
- Add structured logging ("init"/"end") with \`:size\`, \`:mtypes\`,
  and \`:elapsed\` in \`create-font-variant\`

**Frontend:**
- \`upload-blob-chunked\` accepts a per-caller \`:chunk-size\` option
- Add \`font-upload-chunk-size\` (10 MiB) and \`upload-font-variant\`
  fn that uploads each mtype as a separate chunked session
- \`on-upload*\` in dashboard fonts now calls \`upload-font-variant\`
  instead of issuing \`create-font-variant\` RPC directly
- \`process-upload\` stores raw ArrayBuffer instead of chunking
  client-side

**Common:**
- Replace \`"font/opentype"\` with \`"font/woff2"\` in \`font-types\`

**Tests:**
- 25 tests / 224 assertions covering all three upload paths (direct
  bytes, legacy chunk-vector, new chunked sessions), size validation,
  and media type validation

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

* 📎 Add a script for check the commit format locally

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-05-12 18:30:19 +02:00
Andrey Antukh
10a23a6869 Merge remote-tracking branch 'origin/main' into staging 2026-05-10 09:16:41 +02:00
Andrey Antukh
279231240d
🐛 Harden outbound HTTP requests against SSRF and restrict assets handlers (#9390)
* ⬆️ Update root deps

* 🐛 Harden outbound HTTP requests against SSRF and restrict unauthenticated asset access

- Add app.util.ssrf URL/host validator that resolves hostnames and blocks
  loopback, link-local, site-local, cloud metadata, and operator-supplied CIDRs
- Add app.media.sanitize image EOF truncator that strips trailing data after
  PNG IEND, JPEG EOI, GIF trailer, and WebP RIFF markers
- Disable HTTP client auto-redirect; add req-with-redirects! helper that
  revalidates every redirect hop against the SSRF blocklist
- Wire SSRF validation and EOF sanitization into media/download-image
- Validate webhook URLs and OIDC profile picture URLs against SSRF
- Restrict /assets/by-id to require authentication for non-public buckets
  (profile) while keeping public access for file-media-object,
  file-object-thumbnail, team-font-variant, and file-data-fragment
- Add config knobs: ssrf-protection-enabled, ssrf-allowed-hosts,
  ssrf-extra-blocked-cidrs

Signed-off-by: Andrey Antukh <niwi@niwi.nz>

---------

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-05-08 09:18:22 +02:00
Andrey Antukh
35f8e1b084 Merge remote-tracking branch 'origin/main-staging' into staging 2026-04-24 14:09:21 +02:00
Andrey Antukh
d380efdb0c
⬆️ Update devenv dependencies (#9142)
* ⬆️ Update devenv dependencies

* ✨ Fix formatting issues

* 📎 Fix linter issues
2026-04-24 14:07:51 +02:00
Andrey Antukh
ab90500ec8 🐛 Fix download-image to properly handle network errors and non-2xx responses (#8554)
The download-image function in app.media silently succeeded when the
remote image URL was unreachable or returned an error status code,
causing create-file-media-object-from-url to report success with no
actual image stored.

Add exception handling for connection refused, timeouts, and I/O errors
around the HTTP request, and validate the HTTP status code in
parse-and-validate before processing the response body.

Fixes #8499

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-03-10 15:01:23 +01:00
Andrey Antukh
70030fa9e3
🐛 Fix download-image to properly handle network errors and non-2xx responses (#8554)
The download-image function in app.media silently succeeded when the
remote image URL was unreachable or returned an error status code,
causing create-file-media-object-from-url to report success with no
actual image stored.

Add exception handling for connection refused, timeouts, and I/O errors
around the HTTP request, and validate the HTTP status code in
parse-and-validate before processing the response body.

Fixes #8499

Signed-off-by: Andrey Antukh <niwi@niwi.nz>
2026-03-10 10:04:07 +01:00
Melvin Laplanche
6cfaeb8a44 🎉 Add woff2 support on user uploaded fonts
Signed-off-by: Melvin Laplanche <noreply@melvin.la>
2026-02-17 10:29:05 +01:00
Andrey Antukh
fbf1240998 ⚡ Add several optimizations for fonts zip download
Mainly prevent hold the whole zip in memory and uses an
unified response type, leavin frontend fetching the blob
data from the assets/storage subsystem.
2026-02-16 10:14:50 +01:00
Alejandro Alonso
3d20fc508d
🐛 Fix image magick info call (#8300) 2026-02-09 12:26:42 +01:00
Andrey Antukh
363b4e3778
♻️ Make the SSO code more modular (#7575)
* 📎 Disable by default social auth on devenv

* 🎉 Add the ability to import profile picture from SSO provider

* 📎 Add srepl helper for insert custom sso config

* 🎉 Add custom SSO auth flow
2025-11-12 12:49:10 +01:00
Alejandro Alonso
643621a389
Merge pull request #7119 from penpot/niwinz-develop-type-hints
⚡ Add several performance enhancements
2025-08-20 12:58:43 +02:00
Andrey Antukh
575342b3bb ♻️ Use direct schemas instead of references
Only a very common use, basic types schemas should be used as
reference (with namespaced keywords)
2025-08-20 12:33:07 +02:00
Andrey Antukh
17ec360720 ⚡ Add several missing type hints for avoid reflection and autoboxing 2025-08-18 13:03:10 +02:00
Andrey Antukh
283eb0419c ♻️ Refactor time related namespaces
Mainly removes the custom app.util.time namespace
from frontend and backend and normalize all to use
the app.common.time namespace
2025-08-01 11:20:01 +02:00
Alejandro Alonso
3cb0e1b6ee
🐛 Fix exif rotation detection when auto-rotation isn't supported (#6818) 2025-07-02 10:31:05 +02:00
Alejandro Alonso
8f00292f8f
🎉 Support for exim rotated images (#6767) 2025-06-25 10:20:37 +02:00
Andrey Antukh
8f774a3611 ✨ Simplify media mime types structs 2025-06-04 12:46:49 +02:00
Andrey Antukh
c8a211742a 🔥 Remove already broken and unused internal components-v2 migration code 2025-05-20 13:06:06 +02:00
Andrey Antukh
88fb5e7ab5 ♻️ Update integrant to latest version
This upgrade also includes complete elimination of use spec
from the backend codebase, completing the long running migration
to fully use malli for validation and decoding.
2024-11-13 19:09:19 +01:00
Andrey Antukh
f949649ba3 ⬆️ Update backend dependencies 2024-10-22 20:23:38 +02:00
Andrey Antukh
cacee40d11 🎉 Add proper schema encoding/decoding mechanism
this allows almost all api operations to success usin application/json
encoding with the exception of the update-file, which we need to
approach a bit differently;

the reason update-file is different, is because the operations vector
is right now defined without the context of shape type, so we are just
unable to properly parse the value to correct type using the schema
decoding mechanism
2024-08-21 11:27:36 +02:00
Andrey Antukh
0e92bcc0de 🎉 Add file-data offload mechanism 2024-08-09 14:28:18 +02:00
Andrey Antukh
9174bb140b ♻️ Refactor configuration validation
Replace spec with schema
2024-07-10 15:16:28 +02:00
Andrey Antukh
d2bedec59c ✨ Unify naming of schema registering functions 2024-07-04 12:30:28 +02:00
Andrey Antukh
fede8c9975 ✨ Setup better media max file-size on devenv 2024-01-10 14:00:31 +01:00
Andrey Antukh
00afb841ac ✨ Add svg optimization on components migration process 2023-11-07 12:48:31 +01:00
Andrey Antukh
81facd58c9 ✨ Improve error report of invalid image 2023-08-08 12:57:49 +02:00
Andrey Antukh
d1a7c58c53 ✨ Report error on something goes wrong on image processing 2023-07-10 15:07:17 +02:00
Andrey Antukh
5ca3d01ea1 🎉 Add malli based validation and coersion subsystem 2023-05-17 16:05:29 +02:00
Andrey Antukh
e8ffcbae69 🎉 Add support for multipart upload of thumbnails
and improve the thumbnails storage to offloading it
to the storage subsystem
2023-05-05 17:00:35 +02:00
Andrey Antukh
aafbf6bc15 ♻️ Refactor cocurrency model on backend
Mainly the followin changes:

- Pass majority of code to the old and plain synchronous style
  and start using virtual threads for the RPC (and partially some
  HTTP server middlewares).
- Make some improvements on how CLIMIT is handled, simplifying code
- Improve considerably performance reducing the reflection and
  unnecesary funcion calls on the whole stack-trace of an RPC call.
- Improve efficiency reducing considerably the total threads number.
2023-03-14 12:30:27 +01:00
Andrey Antukh
ab3b9cba45 ♻️ Refactor storage and assets related modules
- improve internal error handling
- add more specs and more asserts
2023-02-07 18:16:55 +01:00
luz paz
e30bea0b6f 🔧 Fix typos in source code
Found via `codespell -q 3 -S *.po,./frontend/yarn.lock -L childs,clen,fpr,inflight,ody,ot,ro,te,trys,ue`
2022-10-04 10:40:34 +02:00
Alejandro Alonso
c3f0657652 Merge remote-tracking branch 'origin/staging' into develop 2022-09-26 15:56:21 +02:00
Alejandro Alonso
f67c4ddca0 🐛 Fix woff2 generation 2022-09-26 15:25:42 +02:00
Andrey Antukh
41134f22e9 📎 Update license header 2022-09-20 23:23:22 +02:00
Andrey Antukh
e657c1bbfa ✨ Start use datoteka.io ns 2022-09-09 12:19:49 +02:00
Andrey Antukh
ec3651d85b 🎉 Add optional rate limit support for RPC calls 2022-08-31 12:55:48 +02:00