- Add session ID tracking to RPC layer (backend and frontend)
- Send session ID header with RPC requests for request correlation
- Rename file-restore to file-restored for consistency
- Extract initialize-file function from initialize-workspace flow
- Improve file restoration initialization with wait-for-persistence
- Extract initialize-version event handler for version restoration
- Fix viewport key generation with file version numbers for proper re-renders
- Update layout item schema and constraints to use internal sizing state
- Add v-sizing state retrieval in layout-size-constraints component
- Refactor file-change notifications stream handling with rx/map
- Fix team-id lookup in restore-version-from-plugins
Improves request traceability across frontend/backend sessions and streamlines
the workspace initialization flow for file restoration scenarios.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
Introduce a purpose-agnostic three-step session-based upload API that
allows uploading large binary blobs (media files and .penpot imports)
without hitting multipart size limits.
Backend:
- Migration 0147: new `upload_session` table (profile_id, total_chunks,
created_at) with indexes on profile_id and created_at.
- Three new RPC commands in media.clj:
* `create-upload-session` – allocates a session row; enforces
`upload-sessions-per-profile` and `upload-chunks-per-session`
quota limits (configurable in config.clj, defaults 5 / 20).
* `upload-chunk` – stores each slice as a storage object;
validates chunk index bounds and profile ownership.
* `assemble-file-media-object` – reassembles chunks via the shared
`assemble-chunks!` helper and creates the final media object.
- `assemble-chunks!` is a public helper in media.clj shared by both
`assemble-file-media-object` and `import-binfile`.
- `import-binfile` (binfile.clj): accepts an optional `upload-id` param;
when provided, materialises the temp file from chunks instead of
expecting an inline multipart body, removing the 200 MiB body limit
on .penpot imports. Schema updated with an `:and` validator requiring
either `:file` or `:upload-id`.
- quotes.clj: new `upload-sessions-per-profile` quota check.
- Background GC task (`tasks/upload_session_gc.clj`): deletes stalled
(never-completed) sessions older than 1 hour; scheduled daily at
midnight via the cron system in main.clj.
- backend/AGENTS.md: document the background-task wiring pattern.
Frontend:
- New `app.main.data.uploads` namespace: generic `upload-blob-chunked`
helper drives steps 1–2 (create session + upload all chunks with a
concurrency cap of 2) and emits `{:session-id uuid}` for callers.
- `config.cljs`: expose `upload-chunk-size` (default 25 MiB, overridable
via `penpotUploadChunkSize` global).
- `workspace/media.cljs`: blobs ≥ chunk-size go through the chunked path
(`upload-blob-chunked` → `assemble-file-media-object`); smaller blobs
use the existing direct `upload-file-media-object` path.
`handle-media-error` simplified; `on-error` callback removed.
- `worker/import.cljs`: new `import-blob-via-upload` helper replaces the
inline multipart approach for both binfile-v1 and binfile-v3 imports.
- `repo.cljs`: `:upload-chunk` derived as a `::multipart-upload`;
`form-data?` removed from `import-binfile` (JSON params only).
Tests:
- Backend (rpc_media_test.clj): happy path, idempotency, permission
isolation, invalid media type, missing chunks, session-not-found,
chunk-index out-of-range, and quota-limit scenarios.
- Frontend (uploads_test.cljs): session creation and chunk-count
correctness for `upload-blob-chunked`.
- Frontend (workspace_media_test.cljs): direct-upload path for small
blobs, chunked path for large blobs, and chunk-count correctness for
`process-blobs`.
- `helpers/http.cljs`: shared fetch-mock helpers (`install-fetch-mock!`,
`make-json-response`, `make-transit-response`, `url->cmd`).
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
* ♻️ Handle fetch-error gracefully with toast instead of full-page error
Network-level failures (lost connectivity, DNS failure, etc.) on RPC
calls were propagating as :internal/:fetch-error to the global error
handler, which replaced the entire UI with a full-page error screen.
Now the :internal handler distinguishes :fetch-error from other internal
errors and shows a non-intrusive toast notification instead, allowing
the user to continue working.
* ✨ Add automatic retry with backoff for idempotent RPC requests
Idempotent (GET) RPC requests are now automatically retried up to 3
times with exponential back-off (1s, 2s, 4s) when a transient error
occurs. Retryable errors include: network-level failures
(:fetch-error), 502 Bad Gateway, 503 Service Unavailable, and browser
offline (status 0).
Mutation (POST) requests are never retried to avoid unintended
side-effects. Non-transient errors (4xx client errors, auth errors,
validation errors) propagate immediately without retry.
* ♻️ Make retry helpers public with configurable parameters
Make retryable-error? and with-retry public functions, and replace
private constants with a default-retry-config map. with-retry now
accepts an optional config map (:max-retries, :base-delay-ms) enabling
callers and tests to customize retry behavior.
* ✨ Add tests for RPC retry mechanism
Comprehensive tests for the retry helpers in app.main.repo:
- retryable-error? predicate: covers all retryable types (fetch-error,
bad-gateway, service-unavailable, offline) and non-retryable types
(validation, authentication, authorization, plain errors)
- with-retry observable wrapper: verifies immediate success, recovery
after transient failures, max-retries exhaustion, no retry for
non-retryable errors, fetch-error retry, custom config, and mixed
error scenarios
* ♻️ Introduce :network error type for fetch-level failures
Replace the awkward {:type :internal :code :fetch-error} combination
with a proper {:type :network} type in app.util.http/fetch. This makes
the error taxonomy self-explanatory and removes the special-case branch
in the :internal handler.
Consequences:
- http.cljs: emit {:type :network} instead of {:type :internal :code :fetch-error}
- errors.cljs: add a dedicated ptk/handle-error :network method (toast);
restore :internal handler to its original unconditional full-page error form
- repo.cljs: simplify retryable-types and retryable-error? — :network
replaces the former :internal special-case, no code check needed
- repo_test.cljs: update tests to use {:type :network}
* 📚 Add comment explaining the use of bit-shift-left
Include request URI and status in frontend handle-response error data,
and add request path/context to backend IOException handler logs and
response body. Previously these errors had no identifying information
about which endpoint or request caused the failure.
* ✨ Improve error handling and exception formatting
- Enhance exception formatting with visual separators and cause chaining
- Add new handler for :internal error type
- Refine error types: change assertion-related errors to :assertion type
- Improve error messages and hints consistency
- Clean up error handling in zip utilities and HTTP modules
* 🐛 Properly handle AbortError on fetch request unsubscription
When a fetch request in-flight is cancelled due to RxJS unsubscription
(e.g. navigating away from the workspace while thumbnail loads are
pending), the AbortController.abort() call triggers a catch handler
that previously relied solely on a @unsubscribed? flag to suppress the
error.
This was unreliable: nested observables spawned inside rx/mapcat (such
as datauri->blob-uri conversions within get-file-object-thumbnails)
could abort independently, with their own AbortController instances,
meaning the outer unsubscribed? flag was never set and the AbortError
propagated as an unhandled exception.
Add an explicit AbortError name check as a disjunctive condition so
that abort errors originating from any observable in the chain are
suppressed at the source, regardless of subscription state.
Signed-off-by: Andrey Antukh <niwi@niwi.nz>
The current binfile export process uses a streaming technique. The
major problem with the streaming approach is the case when an error
happens on the middle of generation, because we have no way to
notify the user about the error (because the response is already
is sent and contents are streaming directly to the user
client/browser).
This commit replaces the streaming with temporal files and SSE
encoded response for emit the export progress events; once the
exportation is finished, a temporal uri to the exported artifact
is emited to the user via "end" event and the frontend code
will automatically trigger the download.
Using the SSE approach removes possible transport timeouts on export
large files by sending progress data over the open connection.
This commit also removes obsolete code related to old binfile
formats.
Mainly the followin changes:
- Pass majority of code to the old and plain synchronous style
and start using virtual threads for the RPC (and partially some
HTTP server middlewares).
- Make some improvements on how CLIMIT is handled, simplifying code
- Improve considerably performance reducing the reflection and
unnecesary funcion calls on the whole stack-trace of an RPC call.
- Improve efficiency reducing considerably the total threads number.
