From 909a29cf50d180b7d4fbe4bd73746cb8604c68d8 Mon Sep 17 00:00:00 2001 From: Andrey Antukh Date: Mon, 16 Nov 2026 12:41:05 +0100 Subject: [PATCH 1/3] :fire: Remove unused file --- tracebitmap.txt | 28 ---------------------------- 1 file changed, 28 deletions(-) delete mode 100644 tracebitmap.txt diff --git a/tracebitmap.txt b/tracebitmap.txt deleted file mode 100644 index a14ad92425..0000000000 --- a/tracebitmap.txt +++ /dev/null @@ -1,28 +0,0 @@ -How to trace a bitmap to svg in multiple colors: - - -Dependencies: imagemagick, netpbm, potrace - -Generate the PPM format from PNG: - - convert download.png download.ppm - -Generate a mask file for each color of the file: - - ppmcolormask "#184d47" download.ppm > download-mask1.pbm - ppmcolormask "#66462C" download.ppm > download-mask2.pbm - -Then trace the bitmap (bitmap -> svg): - - potrace --flat -b svg download-mask1.pbm -o download1.svg - potrace --flat -b svg download-mask2.pbm -o download2.svg - -Extract a list of colors from an image: - - convert download.png +dither -colors 10 -unique-colors txt: - - -Articles: - -- http://confluence.concord.org/display/CCTR/Removing%20Antialiasing.html -- http://confluence.concord.org/display/CCTR/Tracing%20Color%20Raster%20Images.html From 57b9b8463b62b35308b2ceb192121dd092cf4a19 Mon Sep 17 00:00:00 2001 From: Andrey Antukh Date: Mon, 16 Nov 2026 17:52:55 +0100 Subject: [PATCH 2/3] :bug: Keep app booting when SES bootstrap fails (#13085) * :bug: Keep app booting when SES bootstrap fails Wrap repairIntrinsics in a guarded bootstrap so a SES Temporal check failure no longer aborts bundle evaluation and blanks the whole app. When it fails, plugins and MCP stay unavailable for that session, a telemetry event reports the error, and trying to open a plugin shows an environment warning. Add vitest coverage for both bootstrap paths and a CLJS regression test for the runtime-unavailable toast. Closes #12924 AI-assisted-by: hy4-flash-free * :bug: Fix CI format check and plugin test entry point Wrap the long Prettier line in bootstrap.spec.ts so `pnpm run format:check` passes again in the plugins module, and document that the duplicated SES options are an intentional contract snapshot. Switch the runtime-unavailable CLJS test to the public open-plugin! entry point instead of the private load-plugin! function. The M2 test for signal-runtime-ready was dropped: loading app.plugins in the Node test runner needs packaging and DOM shim changes outside this fix. Closes #12924 AI-assisted-by: hy4-flash-free --- .../frontend/plugin-api-to-cljs-binding.md | 3 + .serena/memories/plugins/core.md | 1 + frontend/src/app/main/data/plugins.cljs | 3 +- frontend/src/app/plugins.cljs | 23 ++- .../plugins/runtime_unavailable_test.cljs | 57 ++++++++ frontend/test/frontend_tests/runner.cljs | 2 + frontend/translations/en.po | 7 + frontend/translations/es.po | 7 + plugins/CHANGELOG.md | 6 + plugins/libs/plugins-runtime/src/index.ts | 25 ++-- .../plugins-runtime/src/lib/bootstrap.spec.ts | 131 ++++++++++++++++++ .../libs/plugins-runtime/src/lib/bootstrap.ts | 27 ++++ 12 files changed, 279 insertions(+), 13 deletions(-) create mode 100644 frontend/test/frontend_tests/plugins/runtime_unavailable_test.cljs create mode 100644 plugins/libs/plugins-runtime/src/lib/bootstrap.spec.ts create mode 100644 plugins/libs/plugins-runtime/src/lib/bootstrap.ts diff --git a/.serena/memories/frontend/plugin-api-to-cljs-binding.md b/.serena/memories/frontend/plugin-api-to-cljs-binding.md index 8536ad245c..e6f3fef853 100644 --- a/.serena/memories/frontend/plugin-api-to-cljs-binding.md +++ b/.serena/memories/frontend/plugin-api-to-cljs-binding.md @@ -15,6 +15,9 @@ ## Runtime initialization and permissions - The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling. +- `app.plugins/init-plugins-runtime` returns whether `ɵloadPlugin` was installed. It calls `preg/signal-runtime-ready` even on failure: `workspace.cljs` waits on that promise before `mcp/init`, so skipping it hangs MCP initialization forever. +- When SES bootstrap fails, `app.plugins` emits one `plugins-runtime-unavailable` telemetry event (with the SES error message), and `app.main.data.plugins/load-plugin!` shows the lazy `plugins.runtime-unavailable` toast when the user opens a plugin. `start-plugin!` (MCP path) stays silent by design. +- Runtime availability is detected by the existing `(fn? (unchecked-get global "ɵloadPlugin"))` check; do not add a second mechanism. - Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id. - Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host. - The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code. diff --git a/.serena/memories/plugins/core.md b/.serena/memories/plugins/core.md index 64f4643860..16ec5165bf 100644 --- a/.serena/memories/plugins/core.md +++ b/.serena/memories/plugins/core.md @@ -18,6 +18,7 @@ ## Sandbox and global cleanup +- `src/index.ts` calls `bootstrapSES()` (in `src/lib/bootstrap.ts`) at module load. If `repairIntrinsics` throws (e.g. SES 2.2+ rejects a `Temporal` wrapped by a browser extension), the error is caught, `getPluginsRuntimeError()` exposes it, and `initPluginsRuntime` installs no `ɵ*` globals: the frontend app still boots, plugins and MCP are unavailable in that environment. On failure, `Function.prototype.toString` has already been replaced but no intrinsics are hardened and no compartment/sandbox is created — never run plugins on that path. - The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code. - Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json. - Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them. diff --git a/frontend/src/app/main/data/plugins.cljs b/frontend/src/app/main/data/plugins.cljs index 33d898cf69..61154aa4a3 100644 --- a/frontend/src/app/main/data/plugins.cljs +++ b/frontend/src/app/main/data/plugins.cljs @@ -109,7 +109,8 @@ (log/warn :hint "Plugin runtime not initialized yet" :plugin-id plugin-id :action "load-plugin!") - (st/emit! (remove-current-plugin plugin-id)))))) + (st/emit! (remove-current-plugin plugin-id) + (ntf/warn (tr "plugins.runtime-unavailable"))))))) (defn open-plugin! [{:keys [url] :as manifest} user-can-edit?] diff --git a/frontend/src/app/plugins.cljs b/frontend/src/app/plugins.cljs index 921cf740b9..f10e4213d4 100644 --- a/frontend/src/app/plugins.cljs +++ b/frontend/src/app/plugins.cljs @@ -8,6 +8,7 @@ "RPC for plugins runtime." (:require ["@penpot/plugins-runtime" :as runtime] + [app.main.data.event :as ev] [app.main.errors :as errors] [app.main.features :as features] [app.main.store :as st] @@ -20,14 +21,26 @@ [app.plugins.register :as preg] [app.plugins.ruler-guides :as rg] [app.plugins.shape :as shape] + [app.util.globals :as ug] [beicon.v2.core :as rx] [potok.v2.core :as ptk])) +(defn- runtime-available? + [] + (fn? (unchecked-get ug/global "ɵloadPlugin"))) + (defn init-plugins-runtime [] (runtime/initPluginsRuntime (fn [plugin-id] (api/create-context plugin-id))) - ;; Signal that runtime is ready - (preg/signal-runtime-ready)) + ;; Signal even when the runtime failed to start: MCP initialization waits + ;; for this promise and would otherwise hang forever. + (preg/signal-runtime-ready) + (runtime-available?)) + +(defn- runtime-unavailable-event + [] + (ev/event {::ev/name "plugins-runtime-unavailable" + :message (some-> (runtime/getPluginsRuntimeError) .-message)})) (defn initialize [] @@ -41,8 +54,10 @@ (rx/observe-on :async) (rx/filter #(features/active-feature? @st/state "plugins/runtime")) (rx/take 1) - (rx/tap init-plugins-runtime) - (rx/ignore))))) + (rx/map (fn [_] + (when-not (init-plugins-runtime) + (runtime-unavailable-event)))) + (rx/filter some?))))) ;; Prevent circular dependency (set! flex/shape-proxy? shape/shape-proxy?) diff --git a/frontend/test/frontend_tests/plugins/runtime_unavailable_test.cljs b/frontend/test/frontend_tests/plugins/runtime_unavailable_test.cljs new file mode 100644 index 0000000000..1c2d16bc56 --- /dev/null +++ b/frontend/test/frontend_tests/plugins/runtime_unavailable_test.cljs @@ -0,0 +1,57 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns frontend-tests.plugins.runtime-unavailable-test + (:require + [app.main.data.plugins :as dp] + [app.main.store :as st] + [app.util.globals :as ug] + [app.util.object :as obj] + [cljs.test :as t :include-macros true] + [frontend-tests.helpers.mock :as mock] + [potok.v2.core :as ptk] + [promesa.core :as p])) + +(def ^:private test-manifest + {:plugin-id "test-plugin" + :name "Test plugin" + :description "A plugin used by the runtime availability tests" + :host "https://example.com" + :code ""}) + +(defn- with-load-plugin + "Calls dp/open-plugin! with a mocked st/emit! and the given ɵloadPlugin + value on the global object, returning the emitted events." + [load-plugin-fn] + (let [global ug/global + had-plugin? (obj/contains? global "ɵloadPlugin") + previous (obj/get global "ɵloadPlugin") + emitted (atom [])] + (if (fn? load-plugin-fn) + (obj/set! global "ɵloadPlugin" load-plugin-fn) + (obj/unset! global "ɵloadPlugin")) + (-> (mock/with-mocks* + {st/emit! (fn [event & events] (swap! emitted into (cons event events)))} + (dp/open-plugin! test-manifest nil)) + (p/then (fn [_] @emitted)) + (p/finally (fn [] + (if had-plugin? + (obj/set! global "ɵloadPlugin" previous) + (obj/unset! global "ɵloadPlugin"))))))) + +(t/deftest ^:async load-plugin-notifies-when-runtime-is-unavailable + (let [emitted (await (with-load-plugin nil)) + types (mapv ptk/type emitted)] + (t/is (some #{:app.main.data.notifications/show} types) + "emits a notification when the plugin runtime is unavailable") + (t/is (some #{::dp/remove-current-plugin} types) + "still removes the plugin from the open list"))) + +(t/deftest ^:async load-plugin-does-not-notify-when-runtime-is-available + (let [emitted (await (with-load-plugin (fn [_ _] (p/resolved nil)))) + types (mapv ptk/type emitted)] + (t/is (not-any? #{:app.main.data.notifications/show} types) + "does not emit a notification when the plugin runtime is available"))) diff --git a/frontend/test/frontend_tests/runner.cljs b/frontend/test/frontend_tests/runner.cljs index 33f4b6f4f8..3bea068a6b 100644 --- a/frontend/test/frontend_tests/runner.cljs +++ b/frontend/test/frontend_tests/runner.cljs @@ -83,6 +83,7 @@ [frontend-tests.plugins.page-test] [frontend-tests.plugins.parser-test] [frontend-tests.plugins.register-test] + [frontend-tests.plugins.runtime-unavailable-test] [frontend-tests.plugins.shape-bugfixes-test] [frontend-tests.plugins.text-test] [frontend-tests.plugins.tokens-test] @@ -239,6 +240,7 @@ 'frontend-tests.plugins.page-test 'frontend-tests.plugins.parser-test 'frontend-tests.plugins.register-test + 'frontend-tests.plugins.runtime-unavailable-test 'frontend-tests.plugins.shape-bugfixes-test 'frontend-tests.plugins.text-test 'frontend-tests.plugins.tokens-test diff --git a/frontend/translations/en.po b/frontend/translations/en.po index 861862dcc3..10e0dbf6bd 100644 --- a/frontend/translations/en.po +++ b/frontend/translations/en.po @@ -4709,6 +4709,13 @@ msgstr "Start designing" msgid "options.deleted-token-with-name" msgstr "{%s} token does not exist or has been deleted." +#: src/app/main/data/plugins.cljs:113 +msgid "plugins.runtime-unavailable" +msgstr "" +"Plugins are unavailable in this browser. Something in your environment (for " +"example, an extension such as Tampermonkey) is preventing the plugin " +"runtime from starting. Disable extensions or reload the page to try again." + #: src/app/plugins/utils.cljs:555 msgid "plugins.validation.invalid-value" msgstr "expected %s, got %s (%s)" diff --git a/frontend/translations/es.po b/frontend/translations/es.po index 5ebc56901b..14a849f6af 100644 --- a/frontend/translations/es.po +++ b/frontend/translations/es.po @@ -4763,6 +4763,13 @@ msgstr "Empezar a diseñar" msgid "options.deleted-token-with-name" msgstr "El token {%s} no existe o ha sido borrado." +#: src/app/main/data/plugins.cljs:113 +msgid "plugins.runtime-unavailable" +msgstr "" +"Los plugins no están disponibles en este navegador. Algo en tu entorno (por " +"ejemplo, una extensión como Tampermonkey) impide que se inicie el entorno de " +"plugins. Desactiva las extensiones o vuelve a cargar la página." + #: src/app/plugins/utils.cljs:555 msgid "plugins.validation.invalid-value" msgstr "esperado %s, obtenido %s (%s)" diff --git a/plugins/CHANGELOG.md b/plugins/CHANGELOG.md index 8322c3a1f6..5303e424a5 100644 --- a/plugins/CHANGELOG.md +++ b/plugins/CHANGELOG.md @@ -1,3 +1,9 @@ +## 1.6.1 (Unreleased) + +### 🩹 Fixes + +- **plugins-runtime:** A SES `repairIntrinsics` failure at startup (for example, when a browser extension wraps the native `Temporal` global) no longer leaves the whole app blank. Plugins are skipped for that session, the error is logged, and a telemetry event reports the unavailable runtime; the editor, login and sign-up pages load as usual. Plugins work as before in every other environment. When startup fails, SES has already replaced `Function.prototype.toString` in the host realm before throwing, but no intrinsics are hardened and no sandbox is created. Closes #12924. + ## 1.6.0 (2026-11-10) ### 🚀 Features diff --git a/plugins/libs/plugins-runtime/src/index.ts b/plugins/libs/plugins-runtime/src/index.ts index a38203471d..ecb69143b9 100644 --- a/plugins/libs/plugins-runtime/src/index.ts +++ b/plugins/libs/plugins-runtime/src/index.ts @@ -1,6 +1,7 @@ import 'ses'; import './lib/modal/plugin-modal'; +import { bootstrapSES } from './lib/bootstrap.js'; import { ɵloadPlugin, setContextBuilder, @@ -15,19 +16,27 @@ import type { Context } from '@penpot/plugin-types'; console.log('%c[PLUGINS] Loading plugin system', 'color: #008d7c'); -repairIntrinsics({ - evalTaming: 'unsafeEval', - stackFiltering: 'verbose', - errorTaming: 'unsafe', - consoleTaming: 'unsafe', - errorTrapping: 'none', - unhandledRejectionTrapping: 'none', -}); +const sesError = bootstrapSES(); + +if (sesError) { + console.error( + '[PLUGINS] SES bootstrap failed; plugins disabled in this environment.', + sesError, + ); +} + +/** Returns the SES bootstrap error, or null when the runtime can be used. */ +export const getPluginsRuntimeError = (): Error | null => sesError; // eslint-disable-next-line @typescript-eslint/no-explicit-any const globalThisAny$ = globalThis as any; export const initPluginsRuntime = (contextBuilder: (id: string) => Context) => { + if (sesError) { + console.error('[PLUGINS] Skipping plugin runtime initialization.'); + return; + } + try { console.log('%c[PLUGINS] Initialize runtime', 'color: #008d7c'); setContextBuilder(contextBuilder); diff --git a/plugins/libs/plugins-runtime/src/lib/bootstrap.spec.ts b/plugins/libs/plugins-runtime/src/lib/bootstrap.spec.ts new file mode 100644 index 0000000000..d77b3e0803 --- /dev/null +++ b/plugins/libs/plugins-runtime/src/lib/bootstrap.spec.ts @@ -0,0 +1,131 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { bootstrapSES } from './bootstrap.js'; + +// Intentional snapshot of the production options in bootstrap.ts: if they +// change, this test must fail. Do not replace this literal with an import. +const REPAIR_OPTIONS = { + evalTaming: 'unsafeEval', + stackFiltering: 'verbose', + errorTaming: 'unsafe', + consoleTaming: 'unsafe', + errorTrapping: 'none', + unhandledRejectionTrapping: 'none', +}; + +const temporalError = () => + new TypeError('unexpected Temporal __proto__: [object Object]'); + +afterEach(() => { + vi.unstubAllGlobals(); + vi.restoreAllMocks(); + vi.doUnmock('ses'); + vi.doUnmock('./modal/plugin-modal'); + vi.resetModules(); +}); + +describe('bootstrapSES', () => { + it('repairs intrinsics with the production options and reports success', () => { + const repair = vi.fn(); + vi.stubGlobal('repairIntrinsics', repair); + + expect(bootstrapSES()).toBeNull(); + expect(repair).toHaveBeenCalledWith(REPAIR_OPTIONS); + }); + + it('returns the SES error instead of throwing it', () => { + const error = temporalError(); + vi.stubGlobal('repairIntrinsics', () => { + throw error; + }); + + expect(bootstrapSES()).toBe(error); + }); +}); + +describe('plugins runtime entry point', () => { + it('installs the runtime globals when SES bootstrap succeeds', async () => { + const globalAny = globalThis as unknown as Record; + const context = { theme: 'dark' }; + const runtimeKeys = [ + 'ɵcontext', + 'ɵloadPlugin', + 'ɵloadPluginByUrl', + 'ɵunloadPlugin', + ]; + const previous = runtimeKeys.map((key) => globalAny[key]); + + vi.stubGlobal('repairIntrinsics', vi.fn()); + vi.doMock('ses', () => ({})); + // The custom element is registered at module load; mock it so repeated + // entry-point imports do not redefine it in the same jsdom registry. + vi.doMock('./modal/plugin-modal', () => ({})); + vi.resetModules(); + + try { + const runtime = await import('../index.js'); + runtime.initPluginsRuntime(() => context as never); + + expect(runtime.getPluginsRuntimeError()).toBeNull(); + expect(globalAny['ɵcontext']).toBe(context); + expect(typeof globalAny['ɵloadPlugin']).toBe('function'); + expect(typeof globalAny['ɵloadPluginByUrl']).toBe('function'); + expect(typeof globalAny['ɵunloadPlugin']).toBe('function'); + } finally { + for (const [index, key] of runtimeKeys.entries()) { + if (previous[index] === undefined) { + delete globalAny[key]; + } else { + globalAny[key] = previous[index]; + } + } + } + }); + + it('loads with a broken SES bootstrap without installing runtime globals', async () => { + const error = temporalError(); + const globalAny = globalThis as unknown as Record; + const runtimeKeys = [ + 'ɵcontext', + 'ɵloadPlugin', + 'ɵloadPluginByUrl', + 'ɵunloadPlugin', + ]; + const previous = runtimeKeys.map((key) => globalAny[key]); + + vi.spyOn(console, 'error').mockImplementation(() => {}); + vi.stubGlobal('repairIntrinsics', () => { + throw error; + }); + // Keep the stub in place: `import 'ses'` in index.ts would otherwise + // install a new repairIntrinsics global before bootstrap runs. + vi.doMock('ses', () => ({})); + // The custom element is registered at module load; mock it so repeated + // entry-point imports do not redefine it in the same jsdom registry. + vi.doMock('./modal/plugin-modal', () => ({})); + vi.resetModules(); + + try { + for (const key of runtimeKeys) { + delete globalAny[key]; + } + + const runtime = await import('../index.js'); + + expect(runtime.getPluginsRuntimeError()).toBe(error); + expect(() => + runtime.initPluginsRuntime(() => ({}) as never), + ).not.toThrow(); + for (const key of runtimeKeys) { + expect(globalAny[key]).toBeUndefined(); + } + } finally { + for (const [index, key] of runtimeKeys.entries()) { + if (previous[index] === undefined) { + delete globalAny[key]; + } else { + globalAny[key] = previous[index]; + } + } + } + }); +}); diff --git a/plugins/libs/plugins-runtime/src/lib/bootstrap.ts b/plugins/libs/plugins-runtime/src/lib/bootstrap.ts new file mode 100644 index 0000000000..0057f0f413 --- /dev/null +++ b/plugins/libs/plugins-runtime/src/lib/bootstrap.ts @@ -0,0 +1,27 @@ +/** + * Repairs the JavaScript intrinsics needed by the SES sandbox. + * + * SES can throw during this step when the host page already contains a + * `Temporal` global it does not recognize (for example, one wrapped by a + * browser extension). The exception happens at module load time in + * `index.ts`, so an uncaught failure here takes down the whole app. + * + * Returns the caught error so callers can report it and disable plugins, + * instead of propagating it to the bundle loader. + */ +export function bootstrapSES(): Error | null { + try { + repairIntrinsics({ + evalTaming: 'unsafeEval', + stackFiltering: 'verbose', + errorTaming: 'unsafe', + consoleTaming: 'unsafe', + errorTrapping: 'none', + unhandledRejectionTrapping: 'none', + }); + + return null; + } catch (err) { + return err instanceof Error ? err : new Error(String(err)); + } +} From ea0ac02900c3997cc746307b3ceb144eabce3a47 Mon Sep 17 00:00:00 2001 From: Pablo Alba Date: Tue, 17 Nov 2026 09:36:40 +0100 Subject: [PATCH 3/3] :bug: Fix duplicate-file crash on nil keys in deleted component objects (#13045) * :bug: Fix duplicate-file crash on nil keys in deleted component objects * :recycle: Withdraw the legacy-36 migration in favor of 0036 legacy-36 and 0036-clear-nil-keys-in-objects perform the same cleanup of nil keys in `:objects`. Migrations only clean up invalid data here, so they carry no order dependency between them and running both is wasteful. Remove the legacy-36 defmethod and its id from `available-migrations`, the same way 0021-fix-shape-svg-attrs was withdrawn in favor of 0035. Files that recorded legacy-36 re-migrate through 0036; their persisted `:migrations` lists keep the old id, and the playwright fixtures recording it stay untouched. Add a migrate-file test proving a file that recorded legacy-36 is re-armed for 0036 and passes schema validation after. Document the withdrawal pattern in the migration subtleties memory. AI-assisted-by: inkling-2-small-free --------- Co-authored-by: Andrey Antukh --- ...-change-validation-migration-subtleties.md | 3 +- common/src/app/common/files/migrations.cljc | 41 +++--- common/src/app/common/types/file.cljc | 20 ++- .../files_migrations_0036_test.cljc | 130 ++++++++++++++++++ common/test/common_tests/runner.cljc | 4 + .../file_load_component_objects_test.cljc | 107 ++++++++++++++ 6 files changed, 284 insertions(+), 21 deletions(-) create mode 100644 common/test/common_tests/files_migrations_0036_test.cljc create mode 100644 common/test/common_tests/types/file_load_component_objects_test.cljc diff --git a/.serena/memories/common/file-change-validation-migration-subtleties.md b/.serena/memories/common/file-change-validation-migration-subtleties.md index 7333faadc3..a6b5121bf7 100644 --- a/.serena/memories/common/file-change-validation-migration-subtleties.md +++ b/.serena/memories/common/file-change-validation-migration-subtleties.md @@ -30,4 +30,5 @@ - Migrations are an ordered set mixing legacy version-derived ids and newer named ids. Keep append order stable; `migrate` applies the set difference between available migrations and file migrations. - `migrate-file` synthesizes legacy migration ids from old numeric versions when `:migrations` is absent, migrates legacy features, and records feature flags created through `cfeat/*new*`. - When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data. -- Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it. \ No newline at end of file +- Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it. +- Withdrawing a migration id: remove both the `defmethod` and the id from `available-migrations`; the replacement migration subsumes it and files that recorded the old id re-migrate through the replacement (persisted `:migrations` keep the old id via union — do not rewrite them, including playwright fixture JSONs). Idempotent/pure-cleanup migrations only; never withdraw one whose transform must not run again unless the replacement is safe for those files. Precedent: "legacy-36" -> "0036-clear-nil-keys-in-objects". diff --git a/common/src/app/common/files/migrations.cljc b/common/src/app/common/files/migrations.cljc index c2eba53139..6e24e1dc04 100644 --- a/common/src/app/common/files/migrations.cljc +++ b/common/src/app/common/files/migrations.cljc @@ -714,17 +714,6 @@ (update :pages-index d/update-vals update-container) (d/update-when :components d/update-vals update-container)))) -(defmethod migrate-data "legacy-36" - [data _] - (letfn [(update-container [container] - (d/update-when container :objects (fn [objects] - (if (contains? objects nil) - (dissoc objects nil) - objects))))] - (-> data - (update :pages-index d/update-vals update-container) - (d/update-when :components d/update-vals update-container)))) - (defmethod migrate-data "legacy-37" [data _] (d/without-nils data)) @@ -2354,13 +2343,13 @@ (let [shapes' (filterv #(contains? objects %) shapes)] (when (not= shapes shapes') (l/trc :hint "fix dangling children" - :container container-id - :shape-id (:id shape) + :container (str container-id) + :shape-id (str (:id shape)) :removed (remove #(contains? objects %) shapes))) shapes')))) (fix-objects [container-id objects] - (update-vals objects (partial fix-shape container-id objects))) + (d/update-vals objects (partial fix-shape container-id objects))) (update-container [container] (d/update-when container :objects #(fix-objects (:id container) %)))] @@ -2369,6 +2358,26 @@ (update :pages-index d/update-vals update-container) (d/update-when :components d/update-vals update-container)))) +(defmethod migrate-data "0036-clear-nil-keys-in-objects" + [data _] + ;; Soft-delete can leave a nil key in `:objects` (gsh/move on a missing + ;; main, then index-by :id). That fails the file schema on duplicate-file + ;; (#12336). "legacy-36" did the same cleanup but its id was withdrawn + ;; from `available-migrations` below: files that recorded it migrate + ;; again through this one. + (letfn [(update-container [container] + (d/update-when container :objects + (fn [objects] + (if (contains? objects nil) + (do + (l/trc :hint "clear nil key in objects" + :container-id (:id container)) + (dissoc objects nil)) + objects))))] + (-> data + (update :pages-index d/update-vals update-container) + (d/update-when :components d/update-vals update-container)))) + (def available-migrations (into (d/ordered-set) ["legacy-2" @@ -2396,7 +2405,6 @@ "legacy-32" "legacy-33" "legacy-34" - "legacy-36" "legacy-37" "legacy-38" "legacy-39" @@ -2461,4 +2469,5 @@ "0032-fix-children-not-in-parent" "0033-normalize-main-instance-names" "0034-fix-path-content-offset" - "0035-fix-dangling-children"])) + "0035-fix-dangling-children" + "0036-clear-nil-keys-in-objects"])) diff --git a/common/src/app/common/types/file.cljc b/common/src/app/common/types/file.cljc index 444484b68f..8206b66088 100644 --- a/common/src/app/common/types/file.cljc +++ b/common/src/app/common/types/file.cljc @@ -616,16 +616,28 @@ ([file-data component] (load-component-objects file-data component (gpt/point 0 0))) ([file-data component delta] - (if (and component (empty? (:objects component))) ;; This operation may be called twice, e.g. in an idempotent change + ;; nil? not empty?: a missing-main soft-delete stores {}, which must + ;; still count as already loaded so render/library callers do not re-enter. + (if (and component (nil? (:objects component))) ;; This operation may be called twice, e.g. in an idempotent change (let [component-page (get-component-page file-data component) page-objects (:objects component-page) - objects (->> (cons (:main-instance-id component) - (cfh/get-children-ids page-objects (:main-instance-id component))) - (map #(get page-objects %)) + ids (into [] + (cons (:main-instance-id component) + (cfh/get-children-ids page-objects (:main-instance-id component)))) + ;; Skip ids already gone from the page. Calling gsh/move on nil + ;; builds {:selrect nil :points []} and index-by :id stores it + ;; under nil, which later fails duplicate-file schema checks. + shapes (into [] (keep #(get page-objects %)) ids) + objects (->> shapes ;; when it is an undo of a cut-paste, we need to undo the movement ;; of the shapes so we need to move them delta (map #(gsh/move % delta)) (d/index-by :id))] + (when (not= (count shapes) (count ids)) + (l/wrn :hint "skipping missing shapes when loading component objects" + :component-id (str (:id component)) + :main-instance-id (str (:main-instance-id component)) + :file-id (str (:id file-data)))) (assoc component :objects objects)) component))) diff --git a/common/test/common_tests/files_migrations_0036_test.cljc b/common/test/common_tests/files_migrations_0036_test.cljc new file mode 100644 index 0000000000..d9789ea072 --- /dev/null +++ b/common/test/common_tests/files_migrations_0036_test.cljc @@ -0,0 +1,130 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns common-tests.files-migrations-0036-test + (:require + [app.common.files.migrations :as cfm] + [app.common.types.file :as ctf] + [app.common.types.shape :as cts] + [app.common.uuid :as uuid] + [clojure.test :as t])) + +;; ====================================================================== +;; 0036-clear-nil-keys-in-objects +;; Soft-delete can leave a nil key in a deleted component's `:objects` +;; (`{nil {:selrect nil :points []}}`). That fails the file schema on +;; duplicate-file (#12336). It replaces `legacy-36` (id withdrawn from +;; `available-migrations`): files that recorded it migrate again +;; through this one. +;; ====================================================================== + +(def ^:private migration "0036-clear-nil-keys-in-objects") + +(def ^:private nil-placeholder + {:selrect nil :points []}) + +(defn- root + [children] + (cts/setup-shape {:id uuid/zero + :type :frame + :name "Root" + :shapes children})) + +(t/deftest removes-nil-key-from-component-objects + (let [component-id (uuid/next) + data {:components + {component-id + {:id component-id + :name "Trashed" + :path "" + :deleted true + :main-instance-id (uuid/next) + :main-instance-page (uuid/next) + :objects {nil nil-placeholder}}}} + migrated (cfm/migrate-data data migration) + objects' (get-in migrated [:components component-id :objects])] + + (t/is (not (contains? objects' nil)) + "nil key is removed from the deleted component :objects") + (t/is (= {} objects') + "only the nil placeholder was present, so objects is empty"))) + +(t/deftest removes-nil-key-from-page-objects + (let [page-id (uuid/next) + frame-id (uuid/next) + frame (cts/setup-shape {:id frame-id :type :frame :name "Frame" :shapes []}) + data {:pages-index + {page-id + {:id page-id + :name "Page" + :objects {uuid/zero (root [frame-id]) + frame-id frame + nil nil-placeholder}}}} + migrated (cfm/migrate-data data migration) + objects' (get-in migrated [:pages-index page-id :objects])] + + (t/is (not (contains? objects' nil)) + "nil key is removed from the page :objects") + (t/is (some? (get objects' frame-id)) + "real shapes are kept"))) + +(t/deftest migrated-file-data-passes-schema + ;; Same failure mode duplicate-file hits: full schema check after migrate. + (let [file-id (uuid/next) + page-id (uuid/next) + component-id (uuid/next) + data (-> (ctf/make-file-data file-id page-id) + (assoc-in [:components component-id] + {:id component-id + :name "Trashed" + :path "" + :deleted true + :main-instance-id (uuid/next) + :main-instance-page page-id + :objects {nil nil-placeholder}}))] + + (t/is (thrown? #?(:clj Exception :cljs js/Error) + (ctf/check-file-data data)) + "corrupt trash objects fail the file schema") + + (let [data' (cfm/migrate-data data migration)] + (t/is (not (contains? (get-in data' [:components component-id :objects]) nil))) + (t/is (= data' (ctf/check-file-data data')) + "after clearing nil keys the file data passes the schema")))) + +(t/deftest migration-0036-runs-through-file-migration + ;; A file that recorded the withdrawn "legacy-36" (and everything else + ;; current) still needs migration: legacy-36 left + ;; `available-migrations`, which re-arms it for 0036. + (let [component-id (uuid/next) + file (ctf/make-file {:name "Legacy nil objects"}) + page-id (first (get-in file [:data :pages])) + file (-> file + (assoc :migrations (-> cfm/available-migrations + (disj migration) + (conj "legacy-36"))) + (assoc-in [:data :components component-id] + {:id component-id + :name "Trashed" + :path "" + :deleted true + :main-instance-id (uuid/next) + :main-instance-page page-id + :objects {nil nil-placeholder}})) + file' (cfm/migrate-file file {})] + + (t/is (cfm/need-migration? file) + "the withdrawn legacy-36 re-arms the cleanup for this file") + (t/is (not (cfm/need-migration? file')) + "new migration recorded") + (t/is (contains? (:migrations file') migration) + "0036 migration id persisted") + (t/is (contains? (:migrations file') "legacy-36") + "the old id stays in the recorded list") + (t/is (not (contains? cfm/available-migrations "legacy-36")) + "the withdrawn legacy-36 id stays out of available-migrations") + (t/is (not (contains? (get-in file' [:data :components component-id :objects]) nil)) + "nil key is cleared through the full migrate-file path"))) diff --git a/common/test/common_tests/runner.cljc b/common/test/common_tests/runner.cljc index c82551cfea..df2eadcd5f 100644 --- a/common/test/common_tests/runner.cljc +++ b/common/test/common_tests/runner.cljc @@ -25,6 +25,7 @@ [common-tests.files-migrations-0033-test] [common-tests.files-migrations-0034-test] [common-tests.files-migrations-0035-test] + [common-tests.files-migrations-0036-test] [common-tests.files-migrations-test] [common-tests.files.comp-processors-test] [common-tests.files.helpers-test] @@ -103,6 +104,7 @@ [common-tests.types.color-test] [common-tests.types.components-test] [common-tests.types.container-test] + [common-tests.types.file-load-component-objects-test] [common-tests.types.fill-test] [common-tests.types.font-test] [common-tests.types.modifiers-test] @@ -139,6 +141,7 @@ 'common-tests.files-migrations-0033-test 'common-tests.files-migrations-0034-test 'common-tests.files-migrations-0035-test + 'common-tests.files-migrations-0036-test 'common-tests.files-migrations-test 'common-tests.files.comp-processors-test 'common-tests.files.helpers-test @@ -217,6 +220,7 @@ 'common-tests.types.color-test 'common-tests.types.components-test 'common-tests.types.container-test + 'common-tests.types.file-load-component-objects-test 'common-tests.types.fill-test 'common-tests.types.font-test 'common-tests.types.modifiers-test diff --git a/common/test/common_tests/types/file_load_component_objects_test.cljc b/common/test/common_tests/types/file_load_component_objects_test.cljc new file mode 100644 index 0000000000..a1c8bfb0d7 --- /dev/null +++ b/common/test/common_tests/types/file_load_component_objects_test.cljc @@ -0,0 +1,107 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns common-tests.types.file-load-component-objects-test + (:require + [app.common.types.file :as ctf] + [app.common.types.shape :as cts] + [app.common.uuid :as uuid] + [clojure.test :as t])) + +;; Soft-delete snapshots the main instance into the component via +;; `load-component-objects`. When the main id is already missing from +;; the page, the old path called `gsh/move` on nil and indexed by `:id`, +;; storing `{nil {:selrect nil :points []}}` — which later breaks +;; duplicate-file schema validation (#12336). + +(defn- root + [children] + (cts/setup-shape {:id uuid/zero + :type :frame + :name "Root" + :shapes children})) + +(defn- file-data + [page-id component] + {:pages-index {page-id {:id page-id + :name "Page" + :objects {uuid/zero (root [])}}} + :components {(:id component) component}}) + +(t/deftest load-component-objects-skips-missing-main + (let [page-id (uuid/next) + component-id (uuid/next) + main-id (uuid/next) + component {:id component-id + :name "Orphan" + :path "" + :main-instance-id main-id + :main-instance-page page-id} + result (ctf/load-component-objects + (file-data page-id component) + component) + objects (:objects result)] + + (t/is (not (contains? objects nil)) + "missing main must not create a nil key in :objects") + (t/is (= {} objects) + "missing main yields an empty snapshot, not a placeholder shape"))) + +(t/deftest load-component-objects-keeps-present-main + (let [page-id (uuid/next) + component-id (uuid/next) + main-id (uuid/next) + main (cts/setup-shape {:id main-id + :type :frame + :name "Main" + :shapes []}) + component {:id component-id + :name "Present" + :path "" + :main-instance-id main-id + :main-instance-page page-id} + file-data {:pages-index {page-id {:id page-id + :name "Page" + :objects {uuid/zero (root [main-id]) + main-id main}}} + :components {component-id component}} + result (ctf/load-component-objects file-data component) + objects (:objects result)] + + (t/is (contains? objects main-id) + "present main is stored under its uuid") + (t/is (not (contains? objects nil)) + "present main never introduces a nil key") + (t/is (= main-id (get-in objects [main-id :id]))))) + +(t/deftest load-component-objects-treats-empty-objects-as-already-loaded + ;; A missing-main soft-delete stores `:objects {}`. Render/library paths + ;; call load-component-objects again on that component. The guard must + ;; treat `{}` as already loaded — otherwise it reloads from the page + ;; (and re-warns). Put the main on the page now so a mistaken re-entry + ;; would populate objects and fail this assertion. + (let [page-id (uuid/next) + component-id (uuid/next) + main-id (uuid/next) + main (cts/setup-shape {:id main-id + :type :frame + :name "Main" + :shapes []}) + component {:id component-id + :name "AlreadySnapshotted" + :path "" + :main-instance-id main-id + :main-instance-page page-id + :objects {}} + file-data {:pages-index {page-id {:id page-id + :name "Page" + :objects {uuid/zero (root [main-id]) + main-id main}}} + :components {component-id component}} + result (ctf/load-component-objects file-data component)] + + (t/is (= {} (:objects result)) + "empty :objects means already loaded; do not reload from the page")))