mirror of
https://github.com/penpot/penpot.git
synced 2026-11-12 13:00:37 +00:00
34c702bc80
* ✨ Partition file_change by file_id Partition `file_change` by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition. The primary key becomes `(file_id, id)`, which a partitioned table needs, and every access filters by `file_id`. The new table carries a `kind` column (1 snapshot, 2 revision) as a reserve: the queries still tell them apart by the label, and `kind` takes over when they are rewritten. The payload of a snapshot stays in `file_data`; the `data` column is kept for reading old `legacy-db` payloads. The four snapshot commands require `file-id`, because it prunes the partition. The objects GC locks and deletes by the pair, in batches. The old table is kept as `file_change_legacy` as the way back. The automatic migration refuses to run above 10 GiB or 2M rows and points to the runbook; big instances partition online with that runbook, which copies by id, follows the `updated_at` watermark in delta passes and checks that no row of the source is missing from the target. Closes #12416 AI-assisted-by: deepseek-v4.1-flash * 🐛 Address the review of the file_change partition The production runbook promised a rollback it did not have: the section was a sketch, and its statements could write to the table about to be dropped. The rollback is removed, and so is every comment that described it. `file_change_legacy` stays as a frozen copy of the previous layout. The cutover check now aborts the transaction when a row of the old table is missing in the new one, instead of leaving the decision to the operator. The five secondary indexes of the new table are built after the last delta pass: the bulk and the delta passes only need the primary key, so the batches no longer maintain them. The copy of 0158 is now replayed in a test against the pre-0158 shape with a labelled and an unlabelled row, plus an empty instance that must leave no legacy table behind. The updated_at assertion of the snapshot mutations is bounded by the test clock, and the repair revision stops writing the `features` of the file, like the normal save. AI-assisted-by: deepseek-v4.1-flash * 📎 Fix linter issues * 🐛 Fix stale helper call in rpc_file_snapshot_test The suite does not load: `th/run-pending-tasks!` does not exist in backend-tests.helpers. The pending jobs runner is `run-pending-jobs` (no arguments), the name every other call site of the suite uses. AI-assisted-by: inkling-2-small-free * 📚 Align the partition change with the 2.20.0 milestone 2.19.0 is already shipped; this work targets 2.20.0. The operations file of the partition procedure carries the 2.20.0 prefix, the references that point at it (the migration, the runbook table and the memory) follow, and the RPC doc/changes tags of the snapshot commands record 2.20 as the release that makes `file-id` required. AI-assisted-by: inkling-2-small-free * 🐛 Fix review findings on the partition rebase The 0157 legacy test was missed by the rebase and inserted file_change rows without the new mandatory `kind` column, so the backend suite fails against the migrated schema. The inserts now carry `:kind` (snapshot-kind for the labelled rows, revision-kind for the rest). The post-cutover verification block of the runbook terminates the first query with a semicolon so the block pastes into psql without a syntax error. The guard message of the migration names the step (0158-partition-file-change) instead of "0158", ambiguous since two migrations share that number. Closes #12416 AI-assisted-by: inkling-2-small-free * 🐛 Fix the stale label in the snapshot restore test The rename block of snapshots-crud runs before the restore block, so the restored version is already named "label1-updated" when the restore publishes its message. The command names the version by the label the row has at that moment, not by the one it had when it was created, so the test was asserting a stale value. Assert the current label, and say why in the comment. AI-assisted-by: deepseek-v4.1-flash * ♻️ Split the version event tests into one test per command The workspace version events and the plugin proxy send the file-id of the open file on every snapshot command, because the command needs it to prune the partition of file_change. Both tests checked that with a doseq and t/testing blocks, so a failure named the whole group and not the command that broke. Give each command its own test. The workspace ones share a helper that runs the event and returns the command and params it sends; the plugin ones keep their own mock of rp/cmd!. AI-assisted-by: deepseek-v4.1-flash
294 lines
14 KiB
Clojure
294 lines
14 KiB
Clojure
;; This Source Code Form is subject to the terms of the Mozilla Public
|
|
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
;;
|
|
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
|
|
|
(ns frontend-tests.data.workspace-versions-test
|
|
(:require
|
|
[app.common.time :as ct]
|
|
[app.common.uuid :as uuid]
|
|
[app.main.data.changes :as dch]
|
|
[app.main.data.notifications :as ntf]
|
|
[app.main.data.persistence :as dps]
|
|
[app.main.data.workspace :as dw]
|
|
[app.main.data.workspace.pages :as dwpg]
|
|
[app.main.data.workspace.versions :as versions]
|
|
[app.main.refs :as refs]
|
|
[app.main.repo :as rp]
|
|
[app.main.ui.workspace.sidebar.versions :as versions-ui]
|
|
[app.util.timers :as tm]
|
|
[beicon.v2.core :as rx]
|
|
[cljs.test :as t :include-macros true]
|
|
[frontend-tests.helpers.mock :as mock]
|
|
[potok.v2.core :as ptk]))
|
|
|
|
(t/deftest version-creation-rejects-save-failure-and-timeout
|
|
(doseq [failure [:error :timeout]
|
|
operation [:create :plugin-create]]
|
|
(t/testing (str operation " after " failure)
|
|
(let [file-id (uuid/next)
|
|
resolved (atom [])
|
|
rejected (atom [])
|
|
emitted (atom [])
|
|
requests (atom [])
|
|
state {:current-file-id file-id}
|
|
event (case operation
|
|
:create (versions/create-version)
|
|
:plugin-create (versions/create-version-from-plugins
|
|
file-id "Version"
|
|
#(swap! resolved conj %) #(swap! rejected conj %)))]
|
|
(with-redefs [refs/persistence (atom {:status (if (= failure :error) :error :saving)
|
|
:error {:type :persistence :code :save-permission-denied}})
|
|
rp/cmd! (mock/stub (fn [cmd _]
|
|
(swap! requests conj cmd)
|
|
(rx/of nil)))
|
|
rx/timeout (mock/stub (fn [_ fallback source]
|
|
(if (= failure :timeout) fallback source)))]
|
|
(->> (ptk/watch event state (rx/empty))
|
|
(rx/subs! #(swap! emitted conj %) #(swap! rejected conj %)))
|
|
(t/is (empty? @requests) "Do not create a version without saving")
|
|
(t/is (empty? @resolved) "Do not resolve the plugin promise on failure")
|
|
(if (= operation :create)
|
|
;; The save failure is reported on its own; the user is told
|
|
;; why the version is missing instead
|
|
(let [notice (some #(when (= ::ntf/show (ptk/type %))
|
|
(:notification (ptk/update % {})))
|
|
@emitted)]
|
|
(t/is (empty? @rejected))
|
|
(t/is (= "workspace.versions.create-failed-unsaved" (:content notice)))
|
|
(t/is (= :warning (:level notice))))
|
|
(do
|
|
(t/is (= 1 (count @rejected)))
|
|
(t/is (= (if (= failure :error) :save-permission-denied :save-timeout)
|
|
(:code (ex-data (first @rejected))))))))))))
|
|
|
|
(t/deftest repair-snapshots-are-told-apart-from-autosaves
|
|
(t/is (versions/repair-snapshot? {:label "internal/repair/5" :created-by "system"}))
|
|
(t/is (not (versions/repair-snapshot? {:label "internal/snapshot/20" :created-by "system"})))
|
|
(t/is (not (versions/repair-snapshot? {:label "My version" :created-by "user"})))
|
|
(t/is (not (versions/repair-snapshot? {:created-by "user"}))))
|
|
|
|
;; Scenario: on one day the backend kept two autosaves and the copy from
|
|
;; before a repair. Proves: the history shows the repair copy as its own
|
|
;; entry, above the day's autosaves, so it can be found by its label.
|
|
(t/deftest the-history-lists-the-copy-before-a-repair-apart
|
|
(let [at (fn [hh] (ct/inst (str "2026-11-03T" hh ":00:00Z")))
|
|
data [{:id (uuid/next) :created-by "system" :label "internal/snapshot/0" :created-at (at "10")}
|
|
{:id (uuid/next) :created-by "system" :label "internal/repair/5" :created-at (at "11")}
|
|
{:id (uuid/next) :created-by "system" :label "internal/snapshot/20" :created-at (at "12")}]
|
|
entries (#'versions-ui/group-snapshots data)]
|
|
(t/is (= [:repair :autosave] (mapv :kind entries)))
|
|
(t/is (= [["internal/repair/5"] ["internal/snapshot/0" "internal/snapshot/20"]]
|
|
(mapv #(mapv :label (:snapshots %)) entries)))))
|
|
|
|
(t/deftest leaving-the-workspace-clears-the-preview-flags
|
|
(let [team-id (uuid/next)
|
|
file-id (uuid/next)
|
|
other-id (uuid/next)
|
|
output (atom [])
|
|
state (-> {:current-file-id file-id
|
|
:current-team-id team-id
|
|
:permissions {:can-edit true}
|
|
:files {file-id {:id file-id :revn 0 :vern 0}}
|
|
:workspace-global {:read-only? true :preview-id (uuid/next)}}
|
|
(as-> $ (ptk/update (dw/finalize-workspace team-id file-id) $))
|
|
(assoc :current-file-id other-id)
|
|
(assoc-in [:files other-id] {:id other-id :revn 0 :vern 0}))]
|
|
(t/is (nil? (get-in state [:workspace-global :preview-id])))
|
|
(when-let [result (ptk/watch (dch/commit-changes {:redo-changes [] :undo-changes []})
|
|
state (rx/empty))]
|
|
(->> result (rx/subs! #(swap! output conj %))))
|
|
(t/is (= 1 (count @output))
|
|
"Editing must work again on the next file opened in the tab")))
|
|
|
|
(t/deftest strict-persistence-wait-rejects-a-real-timeout
|
|
(t/async done
|
|
(mock/with-mocks
|
|
{refs/persistence (atom {:status :saving})}
|
|
(fn [finish]
|
|
(->> (dps/wait-persisted-or-error 0)
|
|
(rx/subs! (fn [_]
|
|
(t/is false "The stalled save must not succeed")
|
|
(finish))
|
|
(fn [cause]
|
|
(t/is (= :save-timeout (:code (ex-data cause))))
|
|
(finish)))))
|
|
done)))
|
|
|
|
(t/deftest strict-persistence-wait-observes-a-later-failure
|
|
(let [pstate (atom {:status :saving})
|
|
rejected (atom [])
|
|
resolved (atom [])]
|
|
(with-redefs [refs/persistence pstate]
|
|
(let [subscription (->> (dps/wait-persisted-or-error)
|
|
(rx/subs! #(swap! resolved conj %) #(swap! rejected conj %)))]
|
|
(try
|
|
(reset! pstate {:status :error :error {:code :missing-commit}})
|
|
(t/is (empty? @resolved))
|
|
(t/is (= :missing-commit (:code (ex-data (first @rejected)))))
|
|
(finally
|
|
(rx/dispose! subscription)))))))
|
|
|
|
(t/deftest failed-restore-returns-to-the-live-file-before-reporting
|
|
;; The plugin promise is rejected on a later tick, so every assertion about
|
|
;; the rejection runs once the event loop has drained.
|
|
(t/async done
|
|
(let [deferred (atom [])]
|
|
(doseq [operation [:ui :plugin]
|
|
failure [:saving :timeout :restore]
|
|
preview? [true false]]
|
|
(let [label (str operation " after " failure ", preview=" preview?)
|
|
file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
backup {:id file-id :revn 10}
|
|
snapshot {:id file-id :revn 5}
|
|
rejected (atom [])
|
|
resolved (atom [])
|
|
requests (atom [])
|
|
initialized (atom [])
|
|
at-reject (atom nil)
|
|
store (ptk/store {:state (cond-> {:current-file-id file-id
|
|
:files {file-id backup}}
|
|
preview?
|
|
(assoc :files {file-id snapshot}
|
|
:workspace-global {:read-only? true :preview-id version-id}
|
|
:workspace-versions {:backup backup}))
|
|
:on-error #(swap! rejected conj %)})
|
|
reject (fn [cause]
|
|
(reset! at-reject
|
|
{:file (get-in @store [:files file-id])
|
|
:preview-id (get-in @store [:workspace-global :preview-id])})
|
|
(swap! rejected conj cause))
|
|
event (if (= operation :ui)
|
|
(#'versions/restore-version version-id)
|
|
(versions/restore-version-from-plugin
|
|
file-id version-id #(swap! resolved conj %) reject))]
|
|
(t/testing label
|
|
(with-redefs [refs/persistence (atom {:status (case failure
|
|
:saving :error :timeout :saving :saved)})
|
|
rp/cmd! (mock/stub (fn [cmd _]
|
|
(swap! requests conj cmd)
|
|
(rx/throw (ex-info "Restore failed" {:type :internal}))))
|
|
rx/timeout (mock/stub (fn [_ fallback source]
|
|
(if (= failure :timeout) fallback source)))
|
|
dwpg/initialize-page (mock/stub
|
|
(fn [_ _]
|
|
(fn [state]
|
|
(swap! initialized conj (get-in state [:files file-id]))
|
|
state)))]
|
|
(ptk/emit! store event)
|
|
(t/is (empty? @resolved))
|
|
(t/is (= (if (= failure :restore) [:restore-file-snapshot] []) @requests)
|
|
"Do not restore a version without saving")
|
|
(t/is (= backup (get-in @store [:files file-id])))
|
|
(t/is (nil? (get-in @store [:workspace-global :read-only?])))
|
|
(t/is (nil? (get-in @store [:workspace-global :preview-id])))
|
|
(t/is (nil? (get-in @store [:workspace-versions :backup])))
|
|
(t/is (= (if preview? [backup] []) @initialized))))
|
|
(swap! deferred conj
|
|
(fn []
|
|
(t/testing label
|
|
(t/is (= 1 (count @rejected)))
|
|
(when (= operation :plugin)
|
|
(t/is (= {:file backup :preview-id nil} @at-reject)
|
|
"The live file is back before the plugin promise rejects")))
|
|
(rx/dispose! store)))))
|
|
(tm/schedule 50 (fn []
|
|
(doseq [check @deferred] (check))
|
|
(done))))))
|
|
|
|
(t/deftest successful-restore-clears-preview-only-after-loading
|
|
(let [file-id (uuid/next)
|
|
state {:current-file-id file-id
|
|
:current-team-id (uuid/next)
|
|
:current-page-id (uuid/next)
|
|
:workspace-global {:read-only? true :preview-id (uuid/next)}
|
|
:workspace-versions {:backup {:id file-id}}}
|
|
stream (rx/subject)
|
|
events (atom [])
|
|
cleanup #(filter (ptk/type? ::versions/clear-preview-state) @events)
|
|
sub (->> (ptk/watch (#'versions/initialize-version) state stream)
|
|
(rx/subs! #(swap! events conj %)))]
|
|
(try
|
|
(t/is (empty? (cleanup)))
|
|
(rx/push! stream (ptk/data-event ::dw/bundle-fetched nil))
|
|
(t/is (= 1 (count (cleanup))))
|
|
(let [result (ptk/update (first (cleanup)) state)]
|
|
(t/is (nil? (get-in result [:workspace-global :read-only?])))
|
|
(t/is (nil? (get-in result [:workspace-global :preview-id])))
|
|
(t/is (nil? (get-in result [:workspace-versions :backup]))))
|
|
(finally
|
|
(rx/dispose! sub)))))
|
|
|
|
;; The snapshot RPC commands require the `file-id` (they use it to prune the
|
|
;; partitions of `file_change`), so every workspace version event sends the
|
|
;; one of the file that is open.
|
|
|
|
(defn- version-event-request
|
|
"Run a workspace version event with `file-id` open and `version-id` in the
|
|
history, and return the `[command params]` of the RPC request it sends."
|
|
[file-id version-id event]
|
|
(let [requests (atom [])
|
|
state {:current-file-id file-id
|
|
:workspace-versions {:data [{:id version-id
|
|
:created-at (ct/now)
|
|
:created-by "system"}]}}]
|
|
(mock/with-mocks*
|
|
{rp/cmd! (mock/stub (fn [cmd params]
|
|
(swap! requests conj [cmd params])
|
|
(rx/of true)))}
|
|
(->> (ptk/watch event state (rx/empty))
|
|
(rx/subs! (fn [_])))
|
|
(first @requests))))
|
|
|
|
(t/deftest ^:async renaming-a-version-sends-the-current-file-id
|
|
(let [file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
[cmd params] (await (version-event-request
|
|
file-id version-id
|
|
(versions/rename-version version-id "name")))]
|
|
(t/is (= :update-file-snapshot cmd))
|
|
(t/is (= version-id (:id params)))
|
|
(t/is (= file-id (:file-id params)))))
|
|
|
|
(t/deftest ^:async deleting-a-version-sends-the-current-file-id
|
|
(let [file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
[cmd params] (await (version-event-request
|
|
file-id version-id
|
|
(versions/delete-version version-id)))]
|
|
(t/is (= :delete-file-snapshot cmd))
|
|
(t/is (= version-id (:id params)))
|
|
(t/is (= file-id (:file-id params)))))
|
|
|
|
(t/deftest ^:async locking-a-version-sends-the-current-file-id
|
|
(let [file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
[cmd params] (await (version-event-request
|
|
file-id version-id
|
|
(versions/lock-version version-id)))]
|
|
(t/is (= :lock-file-snapshot cmd))
|
|
(t/is (= version-id (:id params)))
|
|
(t/is (= file-id (:file-id params)))))
|
|
|
|
(t/deftest ^:async unlocking-a-version-sends-the-current-file-id
|
|
(let [file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
[cmd params] (await (version-event-request
|
|
file-id version-id
|
|
(versions/unlock-version version-id)))]
|
|
(t/is (= :unlock-file-snapshot cmd))
|
|
(t/is (= version-id (:id params)))
|
|
(t/is (= file-id (:file-id params)))))
|
|
|
|
(t/deftest ^:async pinning-a-version-sends-the-current-file-id
|
|
(let [file-id (uuid/next)
|
|
version-id (uuid/next)
|
|
[cmd params] (await (version-event-request
|
|
file-id version-id
|
|
(versions/pin-version version-id)))]
|
|
(t/is (= :update-file-snapshot cmd))
|
|
(t/is (= version-id (:id params)))
|
|
(t/is (= file-id (:file-id params)))))
|