✨ Prune the branch side of a diff to what its op log touched

The op log IS the delta between a branch and its merge base, so folding
validate/extract-affected-ids over it yields the exact set of pages the
branch can differ in. compute-merge takes :only-pages and bounds every
page pass to it, the presence pass included, which is where a whole-file
comparison spends its time.

The forward pass is bounded; the reverse one reports main's changes and
stays whole-file. The bound is dropped whenever the log holds a change
the reducer cannot scope to a page or a component (a deletion, page
order, a colour, a token), because for those the affected set is
incomplete and a bounded diff would hide work.
This commit is contained in:
Álvaro Tejero Cantero 2026-08-14 01:10:45 +02:00
parent 7e623c4526
commit d7241c6f9a
No known key found for this signature in database
5 changed files with 337 additions and 120 deletions

View File

@ -301,6 +301,14 @@
AND deleted_at IS NULL
ORDER BY revn ASC")
(defn get-branch-changes
"The branch's op log as decoded change batches, oldest first. This is
the complete delta between the branch and its merge base, which is what
lets a caller bound work to what the branch touched."
[{:keys [::db/conn] :as cfg} branch-file-id]
(->> (db/exec! conn [sql:get-branch-changes branch-file-id])
(mapv (fn [{:keys [changes]}] (blob/decode changes)))))
(defn- branch-file-data
"Derive the `:data` of a branch file: the merge-base snapshot plus
every op appended to the branch's log, replayed in revn order.

View File

@ -403,6 +403,47 @@
(-> (media-pairs cfg branch-file-id source-file-id)
(assoc branch-file-id source-file-id)))
(def ^:private scoped-change-types
"The change types `validate::extract-affected-ids` maps to a page or a
component id. Every other type (a page or component deletion, page
order, colors, typographies, tokens, file-level plugin data) is real in
a diff and invisible to that reducer, so a log containing one has no
complete affected set and cannot bound a comparison."
#{:add-obj :mod-obj :del-obj :fix-obj :mov-objects :reorder-children :reg-objects
:add-page :mod-page :add-component :mod-component :restore-component})
(defn- scoped-change?
[{:keys [type page-id component-id id]}]
(and (contains? scoped-change-types type)
(case type
(:add-page :mod-page :add-component :mod-component)
(some? id)
;; restores touch the component definition AND the page its main
;; instance lands on, so both ids have to be there for the set to
;; be complete
:restore-component
(and (some? id) (some? page-id))
;; a shape op carries either a page or a component; one without
;; both scopes to nothing the reducer can name
(or (some? page-id) (some? component-id)))))
(defn- branch-affected-pages
"The set of page ids a branch's op log touched, or nil when the log
holds a change that is not scoped to a page or a component.
It is exact rather than approximate because of the storage model: a
branch's `:data` IS the base snapshot plus this log, so a log that only
touches these pages cannot differ from base anywhere else. `nil` is the
honest answer for everything else, and it leaves the whole-file
comparison in place."
[cfg branch-file-id]
(let [batches (bfc/get-branch-changes cfg branch-file-id)
changes (into [] (mapcat identity) batches)]
(when (every? scoped-change? changes)
(:page-ids (cfv/extract-affected-ids changes)))))
(defn branch-diff-counts!
"Entity-level `[ahead behind conflicts]` change counts between a branch
file and its source (main) — the same numbers the compare dialog lists
@ -411,6 +452,11 @@
side hasn't moved. `main-data` may be pre-realized and shared across
branches of the same source; otherwise it is loaded on demand.
The forward pass is bounded to the pages the branch's op log touched
when that set is complete (`branch-affected-pages`); the reverse pass
reports MAIN's changes and stays whole-file, because nothing here knows
what main touched.
Raises on failure instead of degrading: the caller decides whether the
result is worth caching, and a value computed from an error must never
enter the summary cache."
@ -425,7 +471,9 @@
clean-count (fn [m] (let [s (:stats m)]
(+ (:added s) (:modified s) (:deleted s))))
fwd (when (pos? ahead-revn)
(bm/compute-merge base-data main-data branch-data :branch->main))
(bm/compute-merge base-data main-data branch-data :branch->main
(when-let [pages (branch-affected-pages cfg branch-file-id)]
{:only-pages pages})))
bwd (when (pos? behind-revn)
(bm/compute-merge base-data branch-data main-data :branch->main))]
[(if fwd (clean-count fwd) 0)
@ -554,19 +602,26 @@
(files/check-read-permissions! cfg profile-id (:source-file-id branch))
(let [main-file (bfc/get-file cfg (:source-file-id branch) :realize? true)
(let [dir (or direction :branch->main)
main-file (bfc/get-file cfg (:source-file-id branch) :realize? true)
branch-file (bfc/get-file cfg (:branch-file-id branch) :realize? true)
main-data (:data main-file)
branch-data (bm/remap-refs
(:data branch-file)
(branch-id-map cfg (:branch-file-id branch) (:source-file-id branch)))
base-data (get-base-data cfg branch)]
base-data (get-base-data cfg branch)
;; only the branch->main direction may be bounded to the pages
;; the branch touched: the other direction is main's changes and
;; nothing here knows which pages those are
opts (when (= dir :branch->main)
(when-let [pages (branch-affected-pages cfg (:branch-file-id branch))]
{:only-pages pages}))]
;; `:meta` carries the "when" of each side so the resolution UI can show
;; how recent main/branch are (base is pinned at branch creation), plus
;; `:main-revn` so the client can do optimistic concurrency on merge
;; (`expected-main-revn`). The last editor's identity is intentionally
;; omitted: files do not store a reliable "modified-by".
(-> (bm/compute-merge base-data main-data branch-data (or direction :branch->main))
(-> (bm/compute-merge base-data main-data branch-data dir opts)
(assoc :meta {:base-at (:created-at branch)
:main-at (:modified-at main-file)
:branch-at (:modified-at branch-file)

View File

@ -1006,3 +1006,132 @@
(t/is (nil? (:error out)))
(t/is (= :materialized (-> out :result :status)))
(t/is (false? (-> out :result :changed)))))))))
(defn- with-pruning-checked
"Run `f` with the diff engine wrapped so that every pruned comparison is
also computed whole-file and the two summaries must agree. Returns the
number of pruned comparisons the run performed."
[f]
(let [orig bm/compute-merge
pruned (atom 0)]
(with-redefs [bm/compute-merge
(fn [base main branch dir & [opts]]
(let [res (orig base main branch dir opts)]
(when (:only-pages opts)
(swap! pruned inc)
(t/is (= (orig base main branch dir nil) res)
"the pruned summary differs from the whole-file summary"))
res))]
(f))
@pruned))
(t/deftest pruned-diff-equals-the-whole-file-diff
(with-redefs [cf/flags (conj cf/flags :branching)]
(let [profile (th/create-profile* 1 {:is-active true})
proj-id (:default-project-id profile)
file (th/create-file* 1 {:profile-id (:id profile)
:project-id proj-id
:is-shared false})
page-id (-> (th/command! {::th/type :get-file
::rpc/profile-id (:id profile)
:id (:id file)})
:result :data :pages first)
create (:result (th/command! {::th/type :create-file-branch
::rpc/profile-id (:id profile)
:file-id (:id file)
:name "pruned"}))
branch-id (:id create)
branch-file-id (:branch-file-id create)
rect-id (uuid/random)]
(t/testing "a page-scoped edit on the branch and a page on main"
(let [bf (th/db-get :file {:id branch-file-id})
out (th/command! {::th/type :update-file
::rpc/profile-id (:id profile)
:id branch-file-id
:session-id (uuid/random)
:revn (:revn bf)
:vern (:vern bf)
:features cfeat/supported-features
:changes [{:type :add-obj
:page-id page-id
:id rect-id
:parent-id uuid/zero
:frame-id uuid/zero
:obj (cts/setup-shape
{:id rect-id :name "Pruned" :type :rect
:parent-id uuid/zero :frame-id uuid/zero})}]})]
(t/is (nil? (:error out))))
(let [mf (th/db-get :file {:id (:id file)})
out (th/command! {::th/type :update-file
::rpc/profile-id (:id profile)
:id (:id file)
:session-id (uuid/random)
:revn (:revn mf)
:vern (:vern mf)
:features cfeat/supported-features
:changes [{:type :add-page :id (uuid/random) :name "main-side"}]})]
(t/is (nil? (:error out)))))
(t/testing "the compare is pruned and reports the same summary"
(let [result (volatile! nil)
pruned (with-pruning-checked
(fn []
(vreset! result (th/command! {::th/type :get-branch-diff
::rpc/profile-id (:id profile)
:branch-id branch-id}))))]
(t/is (nil? (:error @result)))
(t/is (pos? pruned) "the compare was not pruned at all")
(t/is (some #(= rect-id (:id %)) (:changes (:result @result))))))
(t/testing "the listing is pruned and reports the same counts"
(let [result (volatile! nil)
pruned (with-pruning-checked
(fn []
(vreset! result (th/command! {::th/type :get-file-branches
::rpc/profile-id (:id profile)
:file-id (:id file)}))))
row (first (:result @result))]
(t/is (nil? (:error @result)))
(t/is (pos? pruned))
(t/is (= 1 (:ahead row)))
(t/is (= 1 (:behind row))))))))
(t/deftest an-unscoped-log-keeps-the-whole-file-diff
(with-redefs [cf/flags (conj cf/flags :branching)]
(let [profile (th/create-profile* 1 {:is-active true})
proj-id (:default-project-id profile)
file (th/create-file* 1 {:profile-id (:id profile)
:project-id proj-id
:is-shared false})
create (:result (th/command! {::th/type :create-file-branch
::rpc/profile-id (:id profile)
:file-id (:id file)
:name "unscoped"}))
branch-id (:id create)
branch-file-id (:branch-file-id create)
color-id (uuid/random)
color {:id color-id :name "Brand" :color "#ff0000" :opacity 1}]
(t/testing "a file-level change on the branch: the reducer cannot name it"
(let [bf (th/db-get :file {:id branch-file-id})
out (th/command! {::th/type :update-file
::rpc/profile-id (:id profile)
:id branch-file-id
:session-id (uuid/random)
:revn (:revn bf)
:vern (:vern bf)
:features cfeat/supported-features
:changes [{:type :add-color :color color}]})]
(t/is (nil? (:error out)))))
(t/testing "the compare falls back to the whole file and still sees it"
(let [result (volatile! nil)
pruned (with-pruning-checked
(fn []
(vreset! result (th/command! {::th/type :get-branch-diff
::rpc/profile-id (:id profile)
:branch-id branch-id}))))]
(t/is (nil? (:error @result)))
(t/is (zero? pruned) "an unscoped log must not be pruned")
(t/is (some #(= color-id (:id %)) (:changes (:result @result)))))))))

View File

@ -295,97 +295,106 @@
(or pd {}))))
(defn- diff-pages
[base theirs ours]
(let [bpi (or (:pages-index base) {}) tpi (or (:pages-index theirs) {}) opi (or (:pages-index ours) {})
bids (set (keys bpi)) tids (set (keys tpi)) oids (set (keys opi))
common (set/intersection bids tids oids)
([base theirs ours] (diff-pages base theirs ours nil))
([base theirs ours only-pages]
;; `only-pages` bounds every page pass, the presence one included,
;; because `page-content` strips the shapes of every page it looks at
;; and that is most of what a whole-file comparison costs. It is the
;; caller's job to establish that `ours` cannot differ from `base`
;; outside that set (see `compute-merge`).
(let [keep? (if (some? only-pages) #(contains? only-pages %) (constantly true))
bpi (or (:pages-index base) {}) tpi (or (:pages-index theirs) {}) opi (or (:pages-index ours) {})
bids (into #{} (filter keep?) (keys bpi))
tids (into #{} (filter keep?) (keys tpi))
oids (into #{} (filter keep?) (keys opi))
common (set/intersection bids tids oids)
;; page add/delete (mergeable: :page). Presence is diffed over the
;; page CONTENT so that deleting a page the other side edited raises
;; a delete conflict instead of silently dropping those edits;
;; content-only modifications are filtered out (granular passes own
;; them) and conflict payloads are slimmed (a full page is huge).
presence (-> (three-way-entities (page-content-map bpi bids)
(page-content-map tpi tids)
(page-content-map opi oids)
{:kind :page})
(presence-only)
(update :conflicts #(slim-conflict-sides slim-page %)))
;; page name/background/grid on common pages (mergeable: :page via mod-page)
meta-map (fn [pi] (into {} (map (fn [id] [id (page-meta (get pi id))])) common))
meta-diff (three-way-entities (meta-map bpi) (meta-map tpi) (meta-map opi) {:kind :page})
;; residual page attrs on common pages (NOT mergeable -> refused, never dropped)
extra-map (fn [pi] (into {} (map (fn [id] [id (page-extra (get pi id))])) common))
extra-diff (three-way-entities (extra-map bpi) (extra-map tpi) (extra-map opi) {:kind :page-attrs})
;; page order on common pages. The entity id is `:page-order` (NOT a
;; bare `:order`) so its conflict resolution cannot collide with the
;; token-set-order one and matches what `compute-changes` reads.
order-of (fn [data] (filterv common (or (:pages data) [])))
order (three-way-entities {:page-order (order-of base)} {:page-order (order-of theirs)} {:page-order (order-of ours)}
{:kind :page-order})
;; guides / flows per common page (mergeable: :page-guide / :page-flow)
sub-of (fn [data pid k] (get-in data [:pages-index pid k] {}))
guides-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :guides)
(sub-of theirs pid :guides)
(sub-of ours pid :guides)
{:kind :page-guide :page-id pid}))
common)
flows-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :flows)
(sub-of theirs pid :flows)
(sub-of ours pid :flows)
{:kind :page-flow :page-id pid}))
common)
;; default-grids per common page (mergeable: :page-grid -> :set-default-grid)
grids-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :default-grids)
(sub-of theirs pid :default-grids)
(sub-of ours pid :default-grids)
{:kind :page-grid :page-id pid}))
common)
;; page-level plugin-data per common page (mergeable: :page-plugin -> :set-plugin-data)
plugin-of (fn [data pid] (flatten-plugin-data (get-in data [:pages-index pid :plugin-data] {})))
plugins-diffs (map (fn [pid]
(three-way-entities (plugin-of base pid)
(plugin-of theirs pid)
(plugin-of ours pid)
{:kind :page-plugin :page-id pid}))
;; page add/delete (mergeable: :page). Presence is diffed over the
;; page CONTENT so that deleting a page the other side edited raises
;; a delete conflict instead of silently dropping those edits;
;; content-only modifications are filtered out (granular passes own
;; them) and conflict payloads are slimmed (a full page is huge).
presence (-> (three-way-entities (page-content-map bpi bids)
(page-content-map tpi tids)
(page-content-map opi oids)
{:kind :page})
(presence-only)
(update :conflicts #(slim-conflict-sides slim-page %)))
;; page name/background/grid on common pages (mergeable: :page via mod-page)
meta-map (fn [pi] (into {} (map (fn [id] [id (page-meta (get pi id))])) common))
meta-diff (three-way-entities (meta-map bpi) (meta-map tpi) (meta-map opi) {:kind :page})
;; residual page attrs on common pages (NOT mergeable -> refused, never dropped)
extra-map (fn [pi] (into {} (map (fn [id] [id (page-extra (get pi id))])) common))
extra-diff (three-way-entities (extra-map bpi) (extra-map tpi) (extra-map opi) {:kind :page-attrs})
;; page order on common pages. The entity id is `:page-order` (NOT a
;; bare `:order`) so its conflict resolution cannot collide with the
;; token-set-order one and matches what `compute-changes` reads.
order-of (fn [data] (filterv common (or (:pages data) [])))
order (three-way-entities {:page-order (order-of base)} {:page-order (order-of theirs)} {:page-order (order-of ours)}
{:kind :page-order})
;; guides / flows per common page (mergeable: :page-guide / :page-flow)
sub-of (fn [data pid k] (get-in data [:pages-index pid k] {}))
guides-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :guides)
(sub-of theirs pid :guides)
(sub-of ours pid :guides)
{:kind :page-guide :page-id pid}))
common)
;; objects (shapes) on common pages (mergeable: :shape). The page
;; root frame (uuid/zero) is skipped and the shapes are diffed
;; STRIPPED of derived attrs (`shape-ignored-attrs`) so that pure
;; `:shapes`/`:touched`/geometry-cache churn neither shows up as a
;; change nor manufactures false conflicts (their merge is driven
;; by add/del/move ops, not by these values).
obj-diffs (map (fn [pid]
(let [bo (get-in base [:pages-index pid :objects] {})
to (get-in theirs [:pages-index pid :objects] {})
oo (get-in ours [:pages-index pid :objects] {})
res (three-way-entities (strip-shapes bo) (strip-shapes to) (strip-shapes oo)
{:kind :shape :page-id pid
:ignore-ids #{uuid/zero}})
;; enrich each entry with the shape's type/component
;; nature, read from whichever side still has it
enrich (fn [e]
(merge e (shape-display-meta
(or (get oo (:id e))
(get to (:id e))
(get bo (:id e))))))
;; conflict payloads must carry the FULL shapes
;; (the diff classified stripped copies, but the
;; conflict UI renders real previews that need
;; :selrect/:points/:transform)
rehydrate (fn [e]
(assoc e
:base (get bo (:id e))
:main (get to (:id e))
:branch (get oo (:id e))))]
{:changes (mapv enrich (:changes res))
:conflicts (mapv (comp rehydrate enrich) (:conflicts res))}))
common)]
(merge-results (concat [presence meta-diff extra-diff order]
guides-diffs flows-diffs grids-diffs plugins-diffs obj-diffs))))
flows-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :flows)
(sub-of theirs pid :flows)
(sub-of ours pid :flows)
{:kind :page-flow :page-id pid}))
common)
;; default-grids per common page (mergeable: :page-grid -> :set-default-grid)
grids-diffs (map (fn [pid]
(three-way-entities (sub-of base pid :default-grids)
(sub-of theirs pid :default-grids)
(sub-of ours pid :default-grids)
{:kind :page-grid :page-id pid}))
common)
;; page-level plugin-data per common page (mergeable: :page-plugin -> :set-plugin-data)
plugin-of (fn [data pid] (flatten-plugin-data (get-in data [:pages-index pid :plugin-data] {})))
plugins-diffs (map (fn [pid]
(three-way-entities (plugin-of base pid)
(plugin-of theirs pid)
(plugin-of ours pid)
{:kind :page-plugin :page-id pid}))
common)
;; objects (shapes) on common pages (mergeable: :shape). The page
;; root frame (uuid/zero) is skipped and the shapes are diffed
;; STRIPPED of derived attrs (`shape-ignored-attrs`) so that pure
;; `:shapes`/`:touched`/geometry-cache churn neither shows up as a
;; change nor manufactures false conflicts (their merge is driven
;; by add/del/move ops, not by these values).
obj-diffs (map (fn [pid]
(let [bo (get-in base [:pages-index pid :objects] {})
to (get-in theirs [:pages-index pid :objects] {})
oo (get-in ours [:pages-index pid :objects] {})
res (three-way-entities (strip-shapes bo) (strip-shapes to) (strip-shapes oo)
{:kind :shape :page-id pid
:ignore-ids #{uuid/zero}})
;; enrich each entry with the shape's type/component
;; nature, read from whichever side still has it
enrich (fn [e]
(merge e (shape-display-meta
(or (get oo (:id e))
(get to (:id e))
(get bo (:id e))))))
;; conflict payloads must carry the FULL shapes
;; (the diff classified stripped copies, but the
;; conflict UI renders real previews that need
;; :selrect/:points/:transform)
rehydrate (fn [e]
(assoc e
:base (get bo (:id e))
:main (get to (:id e))
:branch (get oo (:id e))))]
{:changes (mapv enrich (:changes res))
:conflicts (mapv (comp rehydrate enrich) (:conflicts res))}))
common)]
(merge-results (concat [presence meta-diff extra-diff order]
guides-diffs flows-diffs grids-diffs plugins-diffs obj-diffs)))))
;; --- Tokens ---
;;
@ -631,33 +640,43 @@
:conflicts [<conflict descriptor> ...] ; need resolution
:stats {:added n :modified n :deleted n :conflicts n}}
`opts` may carry `:only-pages`, a set of page ids the page passes are
bounded to. It is sound exactly when `ours` cannot differ from `base`
outside that set, because then no page outside it can carry an
`ours`-side change and no page outside it can hold a conflict, which
needs one. A branch whose op log touches only those pages satisfies
that by construction, since its state IS the base plus that log. It is
NOT sound for the direction that reports main's changes, and passing it
there would hide them.
NOTE: token-lib diffing is not yet implemented (handled in a later
phase); `:tokens-lib` changes are not reported here."
[base main branch dir]
(let [[theirs ours] (if (= dir :main->branch) [branch main] [main branch])
results [(three-way-entities (strip-modified-at (:colors base))
(strip-modified-at (:colors theirs))
(strip-modified-at (:colors ours))
{:kind :color})
(three-way-entities (strip-modified-at (:typographies base))
(strip-modified-at (:typographies theirs))
(strip-modified-at (:typographies ours))
{:kind :typography})
(three-way-entities (strip-modified-at (:components base))
(strip-modified-at (:components theirs))
(strip-modified-at (:components ours))
{:kind :component})
(three-way-entities (:media base) (:media theirs) (:media ours)
{:kind :media})
(diff-pages base theirs ours)
(diff-tokens base theirs ours)]
{:keys [changes conflicts]} (merge-results results)]
{:changes changes
:conflicts conflicts
:stats {:added (count (filterv #(= :added (:status %)) changes))
:modified (count (filterv #(= :modified (:status %)) changes))
:deleted (count (filterv #(= :deleted (:status %)) changes))
:conflicts (count conflicts)}}))
([base main branch dir] (compute-merge base main branch dir nil))
([base main branch dir {:keys [only-pages]}]
(let [[theirs ours] (if (= dir :main->branch) [branch main] [main branch])
results [(three-way-entities (strip-modified-at (:colors base))
(strip-modified-at (:colors theirs))
(strip-modified-at (:colors ours))
{:kind :color})
(three-way-entities (strip-modified-at (:typographies base))
(strip-modified-at (:typographies theirs))
(strip-modified-at (:typographies ours))
{:kind :typography})
(three-way-entities (strip-modified-at (:components base))
(strip-modified-at (:components theirs))
(strip-modified-at (:components ours))
{:kind :component})
(three-way-entities (:media base) (:media theirs) (:media ours)
{:kind :media})
(diff-pages base theirs ours only-pages)
(diff-tokens base theirs ours)]
{:keys [changes conflicts]} (merge-results results)]
{:changes changes
:conflicts conflicts
:stats {:added (count (filterv #(= :added (:status %)) changes))
:modified (count (filterv #(= :modified (:status %)) changes))
:deleted (count (filterv #(= :deleted (:status %)) changes))
:conflicts (count conflicts)}})))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; MERGE -> CHANGES (Phase 3, no-conflict path)

View File

@ -885,14 +885,20 @@
(check-component component file)
(deref *errors*)))
(defn- extract-affected-ids
(defn extract-affected-ids
"Single reduce pass over a changes batch.
Returns {:page-ids #{uuid …} :component-ids #{uuid …}}.
Only entities that need re-validation after applying `changes` are
included. Deleted entities and pure file-level changes (colors,
tokens, typography…) produce no entries because there is nothing left
to check."
to check.
Public because bounding a diff wants the same set as bounding a
validation: `files-branch` folds it over a branch's op log to prune the
branch side of a comparison. A caller that prunes MUST first establish
that every change in the batch is scoped to a page or a component,
because the omissions above are safe for validation and not for a diff."
[changes]
(loop [changes (seq changes)