mirror of
https://github.com/penpot/penpot.git
synced 2026-10-29 22:10:34 +00:00
🐛 Name datascript's externs in the builds that rename properties
datascript.query labels a datom's slots with the string names "e", "a", "v" and "tx", and reads one back with a bracket access on the datom object. Closure renames the field and leaves the string, so the read succeeds and returns whichever field inherited the old name. The index reads the same slots as real fields, so storage keeps answering correctly and the corruption stays silent. Datascript ships datascript/externs.js to pin those five names and declares it in its deps.cljs. shadow-cljs collects that declaration into a :deps-externs entry it never populates, and reads it only behind an unset :cljsjs-externs flag, so the declaration is inert and the :worker release names the file itself. It costs 170 B on the probe build's 436 197 B. bench.debug becomes the gate rather than a reproducer: six readings, one per datom slot plus a join and an index control, each paired with the answer it must give, exiting non-zero on a wrong one. The new ds-probe build compiles it with the worker's release options. Without the externs entry that build reads nil for every transaction, and a vanilla shadow project loses the value slot as well, which is why the gate binds each slot on its own. The ab-bench pin stays at :simple, and its comment now names the reason that still holds: one compiler configuration across both engines, matching the 2026-08-21 V8 table. Answers mirror:ds-advanced-corruption. AI-assisted-by: opus-5
This commit is contained in:
parent
291d92f735
commit
c4ccbdb973
@ -122,6 +122,12 @@
|
||||
:source-map true
|
||||
:elide-asserts true
|
||||
:anon-fn-naming-policy :off
|
||||
;; datascript's query layer reads a datom's slots by string property
|
||||
;; name, so property renaming silently returns the wrong slot: see
|
||||
;; bench.debug, the gate that proves it, and mirror:ds-advanced-corruption.
|
||||
;; The library ships these externs and declares them in its deps.cljs,
|
||||
;; which shadow-cljs does not read, so the build names them here.
|
||||
:externs ["datascript/externs.js"]
|
||||
:source-map-detail-level :all}}}
|
||||
|
||||
;; FIXME: maybe rename to :components ? (there are nothing storybook
|
||||
@ -160,6 +166,10 @@
|
||||
{:test {:init-fn frontend-tests.runner/init
|
||||
:prepend-js ";if (typeof globalThis.navigator?.userAgent === 'undefined') { globalThis.navigator = {userAgent: ''}; };"}}}
|
||||
|
||||
;; The A/B bench pins :simple so the two engines are measured under one
|
||||
;; compiler configuration; the 2026-08-21 datahike report's V8 table was
|
||||
;; read that way. Lifting the pin needs :externs ["datascript/externs.js"]
|
||||
;; here, for the reason bench.debug states, and a fresh table.
|
||||
:ab-bench
|
||||
{:target :node-script
|
||||
:output-to "target/ab-bench.js"
|
||||
@ -171,6 +181,32 @@
|
||||
:output-wrapper false
|
||||
:warnings {:fn-deprecated false}}}
|
||||
|
||||
;; The gate for mirror:ds-advanced-corruption: bench.debug reads one
|
||||
;; database six ways and exits non-zero on a wrong answer. Its release
|
||||
;; options mirror :worker's, because the worker is the build the answer is
|
||||
;; about. To see the corruption, delete the :externs entry below and
|
||||
;; rebuild: `--config-merge` reaches a build's own keys and never a
|
||||
;; :release profile's, and externs from several places are concatenated, so
|
||||
;; no command line can take this one away.
|
||||
:ds-probe
|
||||
{:target :node-script
|
||||
:output-to "target/ds-probe.js"
|
||||
:output-dir "target/ds-probe/"
|
||||
:main bench.debug/-main
|
||||
|
||||
:compiler-options
|
||||
{:output-feature-set :es2020
|
||||
:output-wrapper false
|
||||
:warnings {:fn-deprecated false}}
|
||||
|
||||
:release
|
||||
{:compiler-options
|
||||
{:fn-invoke-direct true
|
||||
:optimizations #shadow/env ["PENPOT_BUILD_OPTIMIZATIONS" :as :keyword :default :advanced]
|
||||
:elide-asserts true
|
||||
:anon-fn-naming-policy :off
|
||||
:externs ["datascript/externs.js"]}}}
|
||||
|
||||
:bench
|
||||
{:target :node-script
|
||||
:output-to "target/bench.js"
|
||||
|
||||
@ -1,10 +1,70 @@
|
||||
(ns bench.debug
|
||||
"The `:advanced` gate for the engine that carries the worker overlay.
|
||||
|
||||
`datascript.query/lookup-pattern-db` labels a datom's slots with the
|
||||
string property names \"e\", \"a\", \"v\" and \"tx\", and the query
|
||||
engine reads them back with a raw property access
|
||||
(`datascript.arrays/aget`). Closure's `:advanced` property renaming
|
||||
moves those four fields and lets an unrelated one answer to the old
|
||||
name, so the read succeeds and returns the wrong slot: a full-scan
|
||||
query hands back a protocol-mask integer where a string belongs, and a
|
||||
bound-value query matches nothing and returns the empty set. The index
|
||||
reads the same datoms through real field access
|
||||
(`datascript.db/val-at-datom`) and stays correct, which is what makes
|
||||
the corruption silent.
|
||||
|
||||
Datascript ships the externs that pin those property names,
|
||||
`datascript/externs.js`, and declares them in its `deps.cljs`.
|
||||
Shadow-cljs collects that declaration into `:deps-externs` and never
|
||||
populates or reads it, so every build that ships the engine names the
|
||||
externs file in its own `:compiler-options`.
|
||||
|
||||
Run this against the release toolchain, never against dev, because dev
|
||||
renames nothing and answers correctly either way:
|
||||
|
||||
clojure -M:dev:shadow-cljs release ds-probe && node target/ds-probe.js
|
||||
|
||||
Exits non-zero and names every reading that came back wrong."
|
||||
(:require [datascript.core :as d]))
|
||||
|
||||
(defn- db
|
||||
"One entity, two datoms, `:x/name` indexed so the `:avet` read is legal."
|
||||
[]
|
||||
(d/db-with (d/empty-db {:x/name {:db/index true}})
|
||||
[{:db/id -1 :x/id 7 :x/name "a"}]))
|
||||
|
||||
(defn- readings
|
||||
"Six readings paired with the answer each must give. Four of them bind
|
||||
one datom slot apiece, because which slot a build corrupts is a fact
|
||||
about that build's whole property set and not about datascript. The
|
||||
fifth reads a slot across a join, which is the shape a real query has.
|
||||
The last one reads the same datoms through the index, so it stays
|
||||
correct in a corrupted build and tells a reader storage was never the
|
||||
problem; it is also the oracle for the transaction id, which no
|
||||
literal can name."
|
||||
[db]
|
||||
(let [datom (first (d/datoms db :eavt 1 :x/name))]
|
||||
[[:query-value (d/q '[:find ?v :where [?e :x/name ?v]] db) #{["a"]}]
|
||||
[:query-entity (d/q '[:find ?e :where [?e :x/name "a"]] db) #{[1]}]
|
||||
[:query-attribute (d/q '[:find ?a :where [1 ?a "a"]] db) #{[:x/name]}]
|
||||
[:query-tx (d/q '[:find ?tx :where [?e :x/name _ ?tx]] db) #{[(:tx datom)]}]
|
||||
[:query-join (d/q '[:find ?e :in $ ?n :where [?e :x/name ?n]] db "a") #{[1]}]
|
||||
[:index-control (mapv (juxt :e :a :v) (d/datoms db :avet :x/name "a"))
|
||||
[[1 :x/name "a"]]]]))
|
||||
|
||||
(defn -main [& _]
|
||||
(let [db (d/db-with (d/empty-db {:x/id {:db/index true}})
|
||||
[{:db/id -1 :x/id 7 :x/name "a"}])]
|
||||
(println :scan-q (pr-str (d/q '[:find ?e ?v :where [?e :x/name ?v]] db)))
|
||||
(println :bound-q (pr-str (d/q '[:find ?e :in $ ?n :where [?e :x/name ?n]] db "a")))
|
||||
(println :avet-part (pr-str (vec (d/datoms db :avet :x/name "a"))))
|
||||
(println :eavt-part (pr-str (vec (d/datoms db :eavt 1 :x/name))))
|
||||
(println :seek (pr-str (first (d/seek-datoms db :avet :x/name))))))
|
||||
(let [rs (readings (db))
|
||||
wrong (into [] (comp (remove (fn [[_ answer expected]] (= expected answer)))
|
||||
(map first))
|
||||
rs)]
|
||||
(doseq [[label answer expected] rs]
|
||||
(println label (pr-str answer)
|
||||
(if (= expected answer)
|
||||
"ok"
|
||||
(str "WRONG, expected " (pr-str expected)))))
|
||||
(if (seq wrong)
|
||||
(do (println "FAIL" (pr-str wrong)
|
||||
"— this build renamed the properties the query layer reads by name;"
|
||||
"add :externs [\"datascript/externs.js\"] to its :compiler-options")
|
||||
(js/process.exit 1))
|
||||
(println "PASS — the query layer survives this build's optimizations"))))
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user