mirror of
https://github.com/penpot/penpot.git
synced 2026-11-17 07:30:35 +00:00
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
69c33c4877
|
♻️ Wait on signals instead of event-loop turns on frontend tests (#12986)
* ♻️ Wait on signals instead of event-loop turns `wait-for` counted its budget in event-loop turns (50 x `setTimeout 0`, about 54 ms idle), so a delivery that travelled through the RxJS async scheduler could arrive after the wait gave up. It now accepts a `:signal`: the predicate is re-checked on every emission and the wait resolves on the event that changes the state. The ceiling (5000 ms) only bounds failure when a signal is given, and stays a poll budget for the waits that have nothing to listen to. `wait-for-store` wraps the store's input stream, which notifies its subscribers after applying the update. The five explicit `:max-ticks` sites move to `:timeout-ms` in the same change, so the old option never lives in a state where destructuring silently ignores it. New `frontend-tests.helpers.async-test` pins the contract: a late delivery resolves, a store signal resolves on the event without a poll, an already-true predicate does not yield, a never-true predicate records one failure within the ceiling, and a resolved wait stops listening. AI-assisted-by: deepseek-v4.1-flash * ♻️ Wait for the persistence requests on their own stream The persistence tests waited for `(= N (count @requests))` by polling the atom at every event-loop turn. The mocked transport now pushes each request into a `requests$` subject, and those 59 waits pass it as `:signal`, so they resolve on the request that makes the count true. The two predicates that mix the request count with the store state merge both signals. The wait that flaked (`the other file's edit is sent next`, in the rollback test) is one of these: it no longer depends on a turn budget, and it cannot lose the delivery. AI-assisted-by: deepseek-v4.1-flash * ♻️ Wait for the store event in the store backed tests 153 waits whose predicate reads the store now go through `wait-for-store`, so they resolve on the store event that changes the state instead of polling at every event-loop turn. The count is 153, not the 162 the plan estimated: nine of them read the store through a helper (`fill-color`, `shape-name`, `copy-children`) rather than `@store`. The five sites that carried an explicit ceiling drop it, since with an event the ceiling only bounds failure. Resolving on the event exposes the places where the poll was really a pause for a side effect to advance: - `tokens-source-test` emitted `unlink-file-from-library`, a watch that chains a second `rp/cmd!` after an async hop. The store signal fires before the chain runs (the event itself dissocs the library), so the mock scope closed in flight and the second call reached the real HTTP layer. The test now waits for that second call, through the mock. - `persistence-rollback-test` drives the persistence pipeline step by step; two of its waits are the pause before the next emit, not an effect, so they keep the poll. AI-assisted-by: deepseek-v4.1-flash * 📚 Document the wait contract and what still polls `mem:frontend/testing` described `wait-for` as an immediate check plus a bounded poll. It now waits on a signal, and the ceiling only bounds failure; `wait-for-store` and the persistence `requests$` subject are how a wait resolves on the event that changes the state. The waits with nothing to listen to keep the poll, and the memory names them. The two traps that cost a red suite are recorded too: a wait that means "the side effect advanced" has no event and keeps the poll, and a store-state wait can close the mock scope before the flow it observes ends, so the chain's last RPC call has to be awaited through the mock. AI-assisted-by: deepseek-v4.1-flash * ♻️ Give the wait wrapper a single signal `wait-for-store` built its options with `assoc`, so it replaced the `:signal` a caller passed. The two waits whose predicate reads both the store and the request counter relied on a merged signal that was being discarded; they now pass it to `wait-for` themselves, and the wrapper takes no options. That merge was the safety net for a request counted after the store event, and it was inert. The helper also reports why a wait failed (ceiling, ended signal, or an error with its cause) instead of calling all three a timeout, counts the predicate evaluations in the late-delivery case to prove the signal path does not poll, reads the clock through `app.common.time` in the helper and the test, and the test's turn probe cancels its timer. The copyright header of `helpers/async.cljs` stays as the previous commit left it, matching the majority spelling. AI-assisted-by: deepseek-v4.1-flash * ♻️ Give the cycle counter a signal too Two waits in the retry tests read the store and the `cycles` atom, which the `slow-cycle-timer` double fills inside the persistence watch. They passed only the store signal, so a counter change could not wake them: the same latent race the previous commit removed from the request count. The double now also emits each new cycle into a `cycles$` subject, and both waits pass the merged signal to `wait-for`. The public `wait-for` docstring names the three failure reasons, and the message for a failed signal bounds the cause text: the exception message when there is one, otherwise the printed value inside a try, cut at 500 characters. AI-assisted-by: deepseek-v4.1-flash * ♻️ Stop the error handlers flooding the test log Four tests in `main-errors-test` run a real error handler end to end. Those handlers print the cause with `ex/print-throwable` before notifying, which is what a developer wants in the app; in a test it only writes console noise. One of them replays 10,000 offline errors, so a run carried 10,006 dumps: 190,010 of its 192,682 lines, 13.8 MB, well past the ~4 MB where GitHub truncates a step log and takes the summary and any failing assertion with it. The tests stub `ex/print-throwable` with a no-op for the duration of their mock scope. The handlers still run, the assertions do not change, and the 10,000-error loop still proves the governor emits once. A full run now logs 2,583 lines and 384 KB. The single block that remains comes from the re-entrancy guard, which prints its own notice on purpose. AI-assisted-by: deepseek-v4.1-flash * ♻️ Silence the deliberate WASM panic in its test `get-svg-markup` logs a WASM panic with `console.error` before returning empty, and `get-svg-markup-swallows-wasm-errors` sends two panics through it on purpose. That was two 35-line dumps in which the exception printed its own `data`, `arr` and protocol masks, and it reads like a failure that the suite is hiding. The test replaces `(.-error js/console)` for its body and restores it in a `finally`. The assertions and the helper stay as they are: the same panic answered with a different stack would still show up, because only this test silences the console. AI-assisted-by: deepseek-v4.1-flash * 🐛 Use real timer for wait-poll ceiling wait-poll read the app clock (ct/now), so tests freezing that clock never reached the deadline and hung instead of failing. Track expiry with a real setTimeout flag, pass timeout-ms through wait-signal and wait-for, and cover the frozen-clock case with a regression test. AI-assisted-by: muse-spark-1.3-contributor |
||
|
|
d57cdb4588 |
⏪ Backport persistence related fixes from staging (#12306)
* ✨ Add async test helpers and the with-mocks* macro Backport of the test scaffolding that the persistence retry work needs, taken from staging. Purely additive: no existing helper changes behavior, so the suite result is unchanged (740 tests, 0 failures). - frontend-tests.helpers.async: ->promise, await-response, settle, observe and wait-for. wait-for checks immediately and then polls one macrotask at a time, bounded, so it fails instead of hanging. - mock/with-mocks*: installs mocks with set! and evaluates its body in a generated ^:async fn, so the test reads sequentially and there is no done to thread. Evaluates to a promise. - mock/asap and mock/run-mocked: run-mocked awaits the body and restores the originals always, in that order, reporting a rejection or a non-promise return as an :error. The matching change that made with-mocks defer its body via asap is NOT backported: it would force all 58 existing uses across 14 files into done-chained contexts, which is a separate refactor. AI-assisted-by: space-bunny-free * ✨ Add an error report governor with compact payloads One browser session could emit an unbounded number of identical error reports, and a dropped connection was reported as an application defect with the full payload. Both are fixed here, in one place, because the persistence retry work that follows depends on the new submit-report contract. - A report governor in app.main.errors. Each report carries a fingerprint (report-name|type|code|hint|first stack frame); the first occurrence is always emitted, repeats inside a 2 minute window are counted, and the next emitted report carries them as :occurrences. The cache is bounded by evicting the oldest entry, so memory stays fixed. - flash reserves the report before generating it, so a suppressed occurrence never pays the report-building cost. Its whole body now runs inside one ts/schedule callback, a reporting or notification failure is logged instead of propagating, and it returns a total promise so tests can await it. - generate-report accepts :format :compact|:full. The compact payload keeps the context header plus type, code and uri, and skips the stack, the ex-data dump (it can carry request headers) and the last-events list. - environment-error-types is derived from rp/retryable-types, so the transport classification has a single source of truth. Environment failures are audit-only handled-exceptions and never reach the exception page or the alerts. :offline gets its own handler instead of falling through to :default. :occurrences travels inside the audit event :props, which the backend schema accepts as a free-form map, so no backend change is needed. The dedup governor test file is new: 7 tests over the fingerprint, the pure reservation decision, and the flash pipeline. Relates to #11726, #11743 AI-assisted-by: space-bunny-free * ♻️ Answer commit refusals with an empty stream commit-changes used to return nil when it refused to build a commit, which forced every caller that wanted to observe the outcome to special-case nil. It now returns an empty stream, so a refusal and an empty result look the same and callers can subscribe uniformly. Only the refusal branch changes: the accepted branch already answered with (rx/of commit). Potok treats a nil watch result as "nothing", so both spellings were already correct at the store level. The 57 call sites that wrap this event in rx/of are unaffected. They hold the ::commit-changes event itself, not the value its watch returns, so they keep processing exactly as before. AI-assisted-by: space-bunny-free * ✨ Retry transient saves with backoff and a reconnect notice Any failed update-file landed on the same path: the queue halted in :error, the retained changes were flushed away, and the user was told to reload. A dropped connection and a revoked permission are not the same thing, so they now take different paths. transient-error? splits them, checking :type and :cause-type because persistence-failed wraps the original cause. Transient failures enter a :retrying episode: the head commit stays queued and resends with backoff at 2s, 8s and 20s, then falls through to today's terminal path. Everything else is terminal and keeps the exact :error path. - Resends rotate the ::request-id stamp only once the old request has left active-requests, carry the same :commit-id, and never double send while one request is in flight. - Retry timers carry an episode token; a timer whose episode was superseded stays silent instead of sending or failing. - One tagged reconnect notice per episode, hidden by tag on save and on terminal failure. Recovery is silent. - The browser online event re-enters the runner, but only for a live :retrying episode, guarded by exists? js/window. - wait-persisted-or-error waits through :retrying instead of cutting the wait short. The episode keeps its :run-id, so a new local edit only joins the queue. Re-entering the runner on every edit would bypass the backoff and spend one attempt per batched commit. The retry test namespace is new: 13 tests over the classification, the backoff schedule, stamp rotation, stale tokens and the notice lifecycle. Relates to #11724 AI-assisted-by: space-bunny-free * 🐛 Fix stalled saves that never answer and edits lost on leave Two holes the retry work did not close. An update-file request that never gets an answer kept its attempt stamp forever, so the queue could not move and the user saw a saving state that never resolved. The request now has a 7 minute timeout that fails it as a transient network error, which hands it to the retry path instead of blocking. The timeout is longer than the 5 minute stall threshold on purpose: the stalled report must fire before the retry, not instead of it. The leave-page warning also had gaps. It now tracks unsaved edits directly, through sync-unload-guard, which holds :persistence in st/ongoing-tasks while edits sit in the buffer or the queue. Edits kept in the queue after a failed save keep the guard on too, so closing the tab warns instead of dropping them. store.cljs guards its ongoing-tasks watcher with exists? js/window, because the new tests exercise it without a window. Four new tests: three for the guard across buffering, an in-flight save and a retained edit, and one for a hung request that must resend and save. Closes #12223 AI-assisted-by: space-bunny-free * 🐛 Skip the save when a missing font cannot be remapped fix-deleted-fonts emitted text and typography changes even when no replacement font existed, so opening a page that uses a deleted custom font rewrote identical content and cleared position-data on every visit. Every visit became a save with nothing to save. Both loops now compare the remapped value with the original and emit nothing when they match. The remapped path is unchanged. Two new tests cover the no-replacement case and the remapped case. Closes #12151 AI-assisted-by: space-bunny-free * ✨ Add a debug helper that simulates an intermediary response A save that never reaches the backend is hard to reproduce: you need a proxy, a captive portal or a CDN in the way. This wraps fetch and answers the named API method with a status and a body that carries no Penpot error code, leaving every other request alone. Without a duration the simulation is spent after answering one call. With one it answers every matching call for that long, which is how to watch the save retry, warn, keep retrying and finally recover. debug.simulateHttpError('update-file') debug.simulateHttpError('get-comment-threads', 403) debug.simulateHttpError('update-file', 524, 120) debug.clearHttpErrorSimulation() Dev-only and self-contained: no production path reads it. AI-assisted-by: space-bunny-free * 🌐 Add the save retry and connection error strings Three strings the new persistence and error paths need: - errors.save-retrying, shown once per retry episode while the queue resends a queued save after a connection loss. - workspace.header.retrying, the save indicator state for an episode in progress. - errors.connection-error, shown by the network and offline handlers, which no longer fall through to the generic message. Added to en.po only. The translations script also normalizes every other locale and drops 439 strings that were already stale before this work; that belongs in its own maintenance pass, not in a backport. AI-assisted-by: space-bunny-free * 🐛 Report save failures as unhandled when they are our defect flash-persistence reported every failure it did not delegate as handled-exception, including the ones that are plainly our fault. A validation failure means the client sent something the server refused, yet it landed in the audit log with a name that skips the internal reports and the alerts. Nobody was paged. The report type now follows the cause. :validation, :server-error and :internal report as unhandled-exception, so they alert. The rest are unchanged. Two exclusions are deliberate: - Environment failures stay handled. They reach this point after the retry budget is spent, and the user's connection is not our bug. - :assertion stays handled, because its own handler already reports it that way and the two paths should not disagree. The cause type is read from :cause-type first, since persistence-failed wraps the original under :type :persistence. Reading :type alone would match the wrapper every time and change nothing. The user still gets the same sticky toast and report link; only the audit event name moves. The test that asserted a validation failure is "still notified" now asserts which event name it gets, across the bare and wrapped shapes, the environment types that must stay handled, and an unclassified one. AI-assisted-by: space-bunny-free * 🐛 Unblock persistence when opening a file after a terminal failure A terminal save failure set the state to :error and left the queue alone. recover-persistence only resumed for :save-permission-denied, so for every other terminal code both of its branches were false and it emitted nothing. Leaving and reopening the same file therefore changed nothing: the editor stayed halted until a reload, which rebuilds the whole store and loses the queue anyway. The path had two jobs under one name. initialize-persistence fires when the workspace view mounts and tears down the previous subscription; recover-persistence fires when a file is opened or the role changes. They are now separate: initialization picks up a queue left behind by a previous session, and deliberately does not lift a halt, so undeliverable edits are released when the user acts rather than when the view mounts. Recovery lifts the halt by releasing what can never be saved. release-failed-file drops every queued commit of the file that failed, then resume-persistence clears the error and settles an empty queue to :saved. Retaining them until that point is deliberate: it is what keeps wait-persisted-or-error rejecting, which is what stops workspace/versions.cljs from building a snapshot out of unsaved edits. Releasing at failure time would make waiters resolve instead. Permission loss is treated as permanent too, which the previous exception got wrong. The backend raises :not-found for a missing edit permission, so a resend does not succeed — it routes the user to the exception page. The old branch made that reachable: it resumed the save whenever the cached can-edit flag said yes, and the server disagreed. The only case where the flag can return to true is a team switch, where the queue is gone anyway, so nothing real is lost by releasing. Because the queue is shared, dropping the failed file is also what lets the other files' pending edits drain. One file that cannot be saved no longer blocks everything else. Tests: the failure retains the queue and notifies once, recovery releases it without replaying, a failed file does not block a second one, permission loss is released rather than resumed, and initialization leaves a halt alone. AI-assisted-by: space-bunny-free * ✨ Offer a reload on the save failure toast A terminal save failure stops autosave for the whole session, but the toast asked the user to reload without offering the reload. It listed five implied actions, none of them reachable from the banner, and it never said that the changes could not be saved. flash now takes a :links vector of actions, appended after the download link that :report-link? produces, so a caller can add recovery actions without reaching into the report handling. flash-persistence passes a reload built from labels.reload-page and location.reload, the pattern already used elsewhere in the app for a full restart. The message now says what happened, what is lost, and what to do about it: autosave has stopped for this session, the changes since the last successful save cannot be saved, and either reloading or leaving the file and opening it again restarts it. Naming both matters, because the toast links are easy to miss. Saying the edits are lost is not optional: recovery releases them, so without it the user finds out when they come back to the file. Translated in en.po and es.po only. The translations script would also normalize all 40 locales and drop entries that were already stale, which is a maintenance pass of its own. The reload callback cannot be invoked under the test runner, since js/location does not exist there, so it is covered by its label and by being callable. Whether the reload works is a manual check. AI-assisted-by: space-bunny-free * 🐛 Classify the responses the backend never wrote A reply written by a proxy instead of the backend carries no Penpot error code, so handle-response raised it as :internal. A 504 on a save skipped the retry cycle and stopped autosave at once, and because :internal reports as unhandled, it raised an alert. In a standard deployment this is how a slow save ends: nginx answers 504 after 300s, before the client's 7 minute timeout fires. The status now decides the type: - 504 and the Cloudflare 520-524 family raise :gateway-error, which joins retryable-types. Saves retry with backoff like 502 and 503, GETs retry, and a save that spends its retries reports as handled, because the report rules derive from the same set. - 429 raises :rate-limit, and any other error status :unexpected-response. Neither retries; both show a toast instead of the error page. - :internal is kept for a response below 400 that does not decode. Bodies we cannot read are cut to 500 characters, since proxy error pages run to tens of kilobytes. Tests: classification and retries per status, the transport handlers keep the page, a 504 and a 524 built by the real handle-response go through the save path and land on a retry, and the new types stay handled once the retries are spent. AI-assisted-by: claude-opus-5-5 * ✨ Keep retrying failed saves on a slow cycle Once the three backoff retries were spent, a save that failed on the connection stopped autosave like any terminal failure, and opening a file again dropped its edits. A connection that comes back after a minute lost the work all the same. A transport failure that outlasts the backoff now arms a slow cycle: one silent resend every 30s, until a save lands or 24h have passed since the failure started. The cycle stops when persistence restarts, the queue resumes another way, or the workspace closes. A save that keeps failing slowly is not resent for a day. A gateway that answers 504 after its own timeout may leave the backend applying the save, so each resend can cost the whole save again, and the failing file holds back every other file's queued edits. An attempt that fails after 90s or more counts as a slow failure; at three, the slow cycle stops sending the commit and the save fails as before. Fast failures, such as being offline, never count. - The first failure shows a toast that says the changes are kept and keep being sent, with no reload link, since a reload loses them. A save that lands hides it. - Opening a file sends a queue still inside its retry window once more instead of releasing it. Failures outside the window, and every non-transport failure, are still released. - A freshly loaded file gets back the queued edits of its version, so reopening it shows the changes that are still waiting to be saved. An edit that no longer applies to the loaded data leaves the file as the server sent it, so the file still opens; the edits stay queued. - A commit whose earlier request is no longer in flight is simply sent again under the same commit id: changes are idempotent, so there is no unknown-outcome guard any more. A request still in flight still blocks a parallel send. flash takes a :tag so the save failure toast can be hidden by tag. Tests: the spent backoff arms the slow cycle, a 504 arms it, a terminal failure does not, the cycle stops when the window closes or at the slow failure limit, recovery resends instead of releasing, reopening restores only the loaded file's edits and survives one that no longer applies, and the retrying toast offers no reload. * 🐛 Turn off the leave-page warning when failed edits are released The leave-page warning only followed the queue on appends and discards. Opening a file again releases the edits of a file that failed for good, but that release changed the queue without either event, so the warning stayed on with nothing left to save. Leaving the page asked about unsaved changes until the next save went through. The warning now also follows the release. Tests: releasing the last failed edit turns the warning off. * 🐛 Stop retrying failed saves outside the workspace Leaving the workspace was meant to stop the slow retry cycle, but only a cycle already armed stopped. A backoff timer kept firing after the user left, and a backoff spent outside the workspace armed a slow cycle that kept sending for up to a day from the dashboard. Those sends check the edit permission of whatever team is open, so a switch to a team where the user only views failed the queued edits for good, and the next file they opened dropped them. Leaving the workspace now suspends retries. A pending backoff timer goes stale, and while suspended a transport failure schedules no backoff, a spent backoff arms no slow cycle and the online event sends nothing. The edits stay queued and a request in flight still lands. Opening a file lifts the suspension and resends a retrying save whose backoff was cut short; a role change does not lift it. Tests: leaving stops a pending backoff, a failure after leaving schedules no retry and ignores the online event, a backoff spent outside the workspace arms no slow cycle, and opening a file saves the held edit in each case. --------- Co-authored-by: alonso.torres <alonso.torres@kaleidos.net> |