Binfile import ran the whole import in a single transaction with
idle_in_transaction_session_timeout disabled (= 0), so a stalled
import could retain a connection pool slot indefinitely. Set a
finite 20 minutes ceiling via SET LOCAL instead (a compile-time
constant interpolated into the SQL; PostgreSQL does not accept bind
parameters on SET).
The v3 importer also located each file data by rescanning the full
zip entry collection once per manifest file and once per page,
making the cost close to quadratic on large files. Replace the
per-file regex matchers with a single classification pass that
groups entries by their raw path shape; consumers now lookup their
entries per file and page. As a deliberate tightening, the .json
suffix is matched literally: the previous regexes left the dot
unescaped, so crafted paths like files/<f>/tokensXjson or
objects/x-json matched by accident and are now ignored.
Closes#11579
AI-assisted-by: omen-alpha
* 🐛 Bound decompressed size of JSON entries on binfile v3 import
Every JSON/text zip entry (manifest, files, pages, shapes, colors, components, typographies, tokens, plugin-data) was decompressed without any size limit, letting a small .penpot archive exhaust the backend heap (GHSA-qcw7-v626-g6cf). Only binary storage blobs were guarded. Reuse the existing size-limiting-stream guard on the text path: 20 MiB cap per entry, 200 MiB cumulative budget per import job, plus a cheap declared-size pre-check. Both limits are configurable and wired through the binfile, management and debug entry points. Adds zip-bomb regression tests for the file entry, the synchronous manifest read and the cumulative budget. Closes#11606
AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Uniform binfile import limits behind init-limits
Move the binfile import limits to a single source of truth in app.binfile.common (default-* vars) and drop the duplicated entries from config/default; env overrides keep working through the schema. Resolve all limits once per job with init-limits (::max-size, ::total-max, ::current-size, ::max-object-size, ::max-zip-entries) instead of rebuilding the map per zip entry. Thread cfg as the first arg through the v3 readers, collapse read-plain-entry into read-entry, and give size-limiting-stream a single explicit-counter arity. v1 keeps using the compiled default (mechanical rename only). No behavior change. AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Rename binfile limits to text-entry/binary-entry terms
Use text-entry/binary-entry vocabulary consistently across config keys, bfc input keys, default-* vars and the limits resolved by setup-limits (::max-text-entry-size, ::max-text-total-size, ::current-text-size, ::max-binary-entry-size). Rename init-limits to setup-limits. No behavior change. AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Rename cumulative text counter and document binary limit
Rename ::current-text-size to ::accumulated-total-text-size for clarity and expand the default-max-binary-entry-size comment to match the other limit vars. No behavior change. AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Harden binfile guards and prove budget accumulation
Add a regression test that only passes when text bytes accumulate across entries (budget between largest entry and summed total; verified red against a per-entry atom). Include the entry name in streaming-guard errors, count skipped bytes against the budget with a direct unit test, and forward all four limit keys in get-manifest. No behavior change. AI-assisted-by: muse-spark-1.3-contributor
* ♻️ Thread cfg through get-manifest
get-manifest now takes the caller cfg and resolves limits with setup-limits like the import job itself, instead of building a single-use mini-cfg from cf/get. No behavior change. AI-assisted-by: muse-spark-1.3-contributor
* 🐛 Add configurable limits for ZIP entry count and object size in v3 import
Add binfile-import-max-zip-entries (default 500,000) and
binfile-import-max-object-size (default 100 MiB) config entries.
Both are configurable via PENPOT_BINFILE_IMPORT_MAX_ZIP_ENTRIES and
PENPOT_BINFILE_IMPORT_MAX_OBJECT_SIZE env vars.
Entry count is checked before processing begins. Per-object size is
checked after each storage object content is resolved.
AI-assisted-by: mimo-v2.5-pro
* 🐛 Enforce actual decompressed byte limits on v3 import
The previous object-size check trusted the ZIP entry header's declared
size (ZipEntry.getSize()), which a malicious zip-bomb can forge. The
check would pass, then the full decompressed payload would be read
anyway during hashing and storage persistence.
Add size-limiting-stream, a FilterInputStream wrapper that counts
actual bytes read and raises :validation :max-file-size-reached when
the configured limit is exceeded. Wire it into zip-entry-storage-content
so both the hash calculation and storage write paths are bounded by
real decompressed bytes, not declared header size.
Also wire import limits into management.clj (clone-template) and
debug.clj (import-handler + clone path) for defense-in-depth, and
add a test that exercises the object-size limit with a real storage
object in the exported ZIP.
AI-assisted-by: mimo-v2.5-pro
* ✨ Auto-link libraries during import based on slugified name
When a Penpot file is exported without bundled libraries and then
imported into a different environment, external library links are
broken because library UUIDs differ across environments.
This feature adds a heuristic to auto-relink libraries by matching
slugified library names against shared files in the target team:
- Export: embed external library metadata (id, name, slug, used-by)
in the manifest when libraries are not included in the export.
- Import: resolve external libraries by slugifying shared file names
in the destination team and matching against manifest slugs.
- Single match: auto-link silently (creates file-library-rel row).
- Multiple matches: emit SSE event so the frontend shows a selection
dialog for the user to pick the correct library.
- No match: import continues without linking (current behavior).
Backend changes:
- Extended manifest schema with optional :external-libraries field
- Added slugify-name, get-files-names, get-shared-files-for-team,
find-shared-files-by-slug helpers in app.binfile.common
- Threaded team-id into import cfg from RPC layer
- Added resolve-external-libraries and auto-link-libraries in v3
- Emit :library-candidates SSE event for multi-match cases
Frontend changes:
- Worker captures library-candidates SSE events and forwards them
- Import dialog shows auto-link notification and multi-match
selection UI with select dropdowns
- Added link-files-to-library! RPC helper for user selections
- Added en/es translations for new UI strings
Closes#9263
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ✨ Add UI for the auto-link plumbing
* ♻️ Consolidate auto-link libraries with unified export-type and fix ref integrity
Consolidates the auto-link libraries feature into a single coherent implementation:
- Unify boolean flags (embed-assets, include-libraries, link-later)
into single ::bfc/export-type parameter
- Fix critical reference-integrity bug: pre-resolution no longer remaps
:component-file refs when no link is created (multi-match / no-permission)
- compute-link-decisions as single source of truth for auto-link logic
- 80+ backend tests covering round-trip, cross-team, permissions,
edge cases, and reference integrity
AI-assisted-by: longcat-2.0
---------
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Co-authored-by: Eva Marco <evamarcod@gmail.com>
Persist binfile manifest metadata in file_data on import so file
statistics are available at open-workspace time. Emit a new
open-workspace-file audit event enriched with file statistics:
page count, shape count, component count, linked libraries,
design tokens, and whether the file is a shared library.
Closes#11106
AI-assisted-by: mimo-v2.5-pro
Prevent unbounded memory allocation when a crafted binfile specifies
an excessively large object size. Apply the same 100 MiB limit that
read-stream! already enforces.
AI-assisted-by: mimo-v2.5
Importing a .penpot file left every svg-raw subtree broken: the parent's
:shapes vector came back holding plain strings instead of uuids, so the
child ids no longer resolved against the page objects map. The next
persisted change touching that page then failed referential integrity
validation with :child-not-found, surfaced to the client as an HTTP 400
:referential-integrity error, which in practice bricks the file.
An svg-raw shape can be a container: importing an SVG builds a tree of
svg-raw shapes, and cfh/group-like-shape? explicitly treats an svg-raw
with children as group-like. But schema:svg-raw-attrs was an empty map.
Frame, group and bool all declare :shapes as a vector of uuid; svg-raw
did not, so the JSON decoder used by binfile had no type information for
those ids and left them as strings.
Declare :shapes on schema:svg-raw-attrs, optional because a leaf svg-raw
shape has no children, so the child ids decode back to uuids.
Closes#10496.
Signed-off-by: Filip Sajdak <filip.sajdak@siili.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
