mirror of
https://github.com/penpot/penpot.git
synced 2026-11-16 15:11:17 +00:00
Merge remote-tracking branch 'origin/staging' into develop
This commit is contained in:
commit
5c45c8ddaf
@ -31,7 +31,7 @@ memory; it does not repeat it.
|
||||
layout, entry rules, the `## Breaking change note` comment contract, the
|
||||
advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
|
||||
2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
|
||||
`report`.
|
||||
`dev-only-bugs`, `report`.
|
||||
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
|
||||
|
||||
## Commands
|
||||
@ -39,6 +39,8 @@ memory; it does not repeat it.
|
||||
```bash
|
||||
python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"
|
||||
python3 scripts/gh.py advisories --patched-version "<VERSION>"
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
|
||||
python3 scripts/changelog.py check-merged <PR numbers...>
|
||||
python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
|
||||
python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
|
||||
@ -54,4 +56,5 @@ python3 scripts/changelog.py report "<VERSION>"
|
||||
- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
|
||||
- `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
|
||||
- Verify PR merge status before writing; only merged PRs are referenced.
|
||||
- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
|
||||
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.
|
||||
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@ -121,7 +121,7 @@ opencode.json
|
||||
/.opencode/reports
|
||||
/.opencode/prompts
|
||||
/.ci-logs
|
||||
/tmp/
|
||||
/tmp
|
||||
/.codex/
|
||||
/scripts/__pycache__/
|
||||
/tools/__pycache__
|
||||
|
||||
@ -67,7 +67,8 @@ Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` i
|
||||
- Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
|
||||
- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
|
||||
- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
|
||||
- The v3 import only materializes a library relation when both ids belong to the package and differ; a relation that points outside the package or a self link is ignored with a warning. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
|
||||
- The v3 and v1 imports only materialize a library relation when both ids belong to the package and the link would neither be a self link nor close a dependency cycle (the same `bfc/library-link-status` check that `link-file-to-library` uses to raise); relations are processed in sorted `(file_id, library_file_id)` order, which `get-files-rels` also returns, so the first edge wins deterministically, and every edge it ignores is logged with a warning. The v2 import stays out of that guard on purpose: it is only reachable from the developer REPL (`app.srepl.binfile`) and keeps inserting the relations as they come. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
|
||||
- A version 1 package lists its files in the metadata and the importer creates one file per entry. The v1 export (gone from the product; only tests and the developer REPL use it) unioned the requested files with their libraries without deduplicating, so a package exported from a file inside a library cycle lists that file twice. The importer reads the pair of objects of every entry to keep the stream in sync, imports the first occurrence only, warns about the repeats, and answers with one id per imported file.
|
||||
- The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
|
||||
- Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
|
||||
- The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.
|
||||
|
||||
@ -131,6 +131,21 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
|
||||
- Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`.
|
||||
- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/process-metadata`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
|
||||
|
||||
## File Change
|
||||
|
||||
- `file_change` is partitioned by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition (`enable_partitionwise_join` is set on every pooled connection).
|
||||
- The automatic migration (`0158-partition-file-change`) refuses to run when `file_change` is above 10 GiB total size or 2M rows (`pg_total_relation_size` + `pg_class.reltuples`, catalog reads). It analyzes the table first, because `reltuples` is -1 without statistics. It raises with the path of `operations/2.20.0-file-change-partition.md`, because the copy happens inside the migration transaction and a big instance would restart in a loop. Below the guard it raises `statement_timeout` to 30 minutes for the copy and the index builds, and analyzes the new partitions at the end, after the copy and the indexes, so the planner does not start with empty statistics.
|
||||
- Its primary key is `(file_id, id)`: a partitioned table needs the partition key in the primary key, so every access filters by `file_id`. From 2.20 the four snapshot commands (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot`, `unlock-file-snapshot`) require the `file-id` param, because they use it to prune the partition; without it they answer `:params-validation`.
|
||||
- It has a `kind` column: `1` is a snapshot (it has a label and its data lives in `file_data`) and `2` is a revision (it has no label and it holds the changes). The values are in `app.features.file-changes`.
|
||||
- `kind` is a deliberate reserve, not a leftover: today no query filters by it, the queries still tell a snapshot from a revision by its label (`label IS NOT NULL`), and `kind` takes over when they are rewritten. No constraint ties `kind` and `label` together on purpose: the day `kind` is the source of truth, that constraint would have to go.
|
||||
- Every write of the application to a row of `file_change` moves `updated_at` to the moment of the write (`lock-by!`, `unlock!`, the snapshot rename and delete, `delete-object`, the restore of a file, and the `legacy-db` snapshot write of `fdata`). The actions of the foreign keys (`ON DELETE SET NULL` on `locked_by` and `profile_id`) change the row without touching it; during the partition both tables carry the same keys, so both receive them. The runbook of the partition uses it as the watermark of its delta passes: a write of the application that leaves it behind is lost at the cutover.
|
||||
- `file_change_legacy` is a frozen copy of the previous layout of the partition: it keeps the rows and the indexes of the old table. A new instance (empty table) does not keep it. On an upgraded instance the table stays until the operator drops it by hand days later (`DROP TABLE file_change_legacy;`, see `operations/2.20.0-file-change-partition.md`); nothing drops it automatically.
|
||||
- A revision carries only the delta of a save (the changes). The features of the file are not repeated on it: nobody read them from there.
|
||||
- A snapshot carries the features and the migrations of the file.
|
||||
- It keeps the `data` column: the `legacy-db` backend is still supported (migration `0157-migrate-legacy-file-data` moved the existing blobs, but the backend keeps reading and writing that column). It has no `data_ref_id` or `data_backend` columns: nothing read them since the file storage refactor.
|
||||
- It has no index for the system snapshots. The old one filtered them by `data IS NOT NULL`, and with the payload living in `file_data` the column no longer tells snapshots apart. The new one could only filter them by `deleted_at IS NULL`, but the system snapshots always carry a deletion date in the future, so it would never match a row. The query that would use it filters that date with an "or", which no partial index can serve.
|
||||
- The `objects-gc` purge groups each chunk by `file_id` and locks it with `file_id = ?` and `id = ANY(?::uuid[])`: a tuple `IN ((file_id, id), ...)` gives the planner no constant partition key, so it scans all 16 partitions in every chunk. Each group is one `FOR UPDATE SKIP LOCKED` query inside the chunk transaction.
|
||||
|
||||
## Metrics
|
||||
|
||||
- `bucket` is always the Penpot logical bucket (object metadata), never an S3 bucket. Unknown/absent buckets are labeled `"unknown"`.
|
||||
|
||||
@ -3,6 +3,8 @@
|
||||
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
|
||||
|
||||
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
|
||||
- Test DB/Redis come from the `PENPOT_TEST_*` env vars already set in the shell (`PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`, ...); per instance they point at `penpot_test` / `penpot_test_wsN` and valkey DB `6+N` (`mem:devenv/core`). Do not override them with values copied from CI logs or other sources unless the user asks (e.g. "use a temporary database"); the CI env (`redis://redis/1`, `postgresql://postgres/penpot_test`) only works inside the CI docker network.
|
||||
- The test database may have migrations applied from newer versions (it is shared dev infra). Reinitializing it — `dropdb`/`createdb` of the `PENPOT_TEST_*` database only — is explicitly permitted when that blocks a test run; never drop the main `penpot` database or another instance's test database.
|
||||
- All CLI commands must be executed from the `backend/` subdirectory.
|
||||
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
|
||||
- **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand.
|
||||
|
||||
@ -11,4 +11,5 @@ Behaviors confirmed against the language/stdlib — do not re-derive from assump
|
||||
- Inside an `^:async` fn, a nested call that needs a temp binding (e.g. calling a local fn `(g (f x))`, whose arg gets a `G__` binding) compiles to `(await (async function(){…})())`: `await` unwraps a returned promise, so `(settled (upload (make-ctx)))` blocks until the upload promise settles instead of passing the promise on (a test that pushes the answer later hangs, idle). Bind each step in its own `let` binding.
|
||||
- `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
|
||||
- Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant).
|
||||
- The threading macros insert the threaded value at DIFFERENT positions: `->` and `some->` put it FIRST (`(-> x (f a))` → `(f x a)`), `->>` and `some->>` put it LAST (`(->> x (f a))` → `(f a x)`). `some->`/`some->>` also short-circuit on nil. With unary calls both nest the same way (`(-> x (f) (g))` and `(->> x (f) (g))` are both `(g (f x))`): what changes between them is the position inside each call, not the order of the steps. A pipeline that calls a fn whose first argument is a config/context (`app.features.fdata/resolve-file-data` is `(cfg, file)`) must use `->>`/`some->>`; with `->`/`some->` the arguments are swapped and the call silently returns the wrong value (it cost a broken `get-snapshot` in `app.features.file-snapshots`).
|
||||
- `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`).
|
||||
|
||||
7
.serena/memories/clojure/logging.md
Normal file
7
.serena/memories/clojure/logging.md
Normal file
@ -0,0 +1,7 @@
|
||||
# Clojure Logging Conventions
|
||||
|
||||
Structured logging through `app.common.logging` (`l/trc`/`l/dbg`/`l/info`/`l/wrn`/`l/error`/`l/fatal`): `:hint` carries the human-readable message, every other datum goes in its own keyword field (`:id`, `:container`, `:removed`, ...).
|
||||
|
||||
- `:hint` values start lowercase: `"fix dangling children"`, not `"Fix dangling children"`. Uppercase-first hints exist as legacy (~57 occurrences, the densest in `backend/src`; also `l/trc` in migration `legacy-28` and `log/error` in `app.common.files.repair`) — do not copy them, and do not mass-rewrite them unless the surrounding code needs the change.
|
||||
- A name that requires uppercase (an env var like `PENPOT_MEDIA_PROCESSING_SERVICE_URI`, an acronym like `MCP` or `LDAP`) stays uppercase inside an otherwise lowercase sentence.
|
||||
- Repair work done by a file migration logs at `l/trc`, only when it changes something, and with the ids it touched: the house style of `migrations.cljc` (`legacy-28` precedes; the file has no `warn`/`info` lines). The shipped `log4j2.xml` pins the `app` logger to `info` (static file, `monitorInterval="60"`), so those trace lines appear only after the operator raises the level; the durable record of a repaired file is the migration id in its `:migrations`.
|
||||
@ -52,3 +52,8 @@ its own drift. Comparing `:selrect.width` from shape A to
|
||||
`:selrect.width` from shape B is comparing two values that
|
||||
"semantically" should be equal but were computed via different
|
||||
operation chains — exact equality will often be false.
|
||||
|
||||
## Library sync comparison
|
||||
|
||||
- `update-attrs` (`app.common.logic.libraries`) compares main vs copy attrs exactly first, then with tolerance for: `drifting-geometry-attrs` (`:x :y :width :height :selrect :points`, via `gsh/close-attrs?`); path and bool `:content` (`segment/content-close?`; bool shapes keep their path in `:content`); and text `:position-data` (`position-data-close?` over `mth/close-map?`: every numeric field of every entry within `mth/close?`, other fields such as fills, fonts and text exact). All other attrs (`:opacity`, radii, `:transform`, `:rotation`, text `:content`) stay exact: tolerance there hides real changes.
|
||||
- `close-attrs?` pairs lists with `d/zip`, which truncates to the shorter list and treats nil vs vector as equal for `:points`/`:position-data`. Do not use it for variable-length data. That is why `:position-data` has its own helper, and why `attr-equal?` checks the `:points` count before calling it.
|
||||
|
||||
@ -6,6 +6,7 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
|
||||
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories.
|
||||
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
|
||||
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
|
||||
- Structured logging conventions (`:hint` casing and fields, log levels): `mem:clojure/logging` — read before adding or changing any log call.
|
||||
- Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
|
||||
- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
|
||||
- Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables.
|
||||
|
||||
@ -8,7 +8,8 @@
|
||||
- `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`.
|
||||
- `security <MILESTONE> [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix <description> (<advisory URL>) (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default.
|
||||
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design.
|
||||
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
|
||||
- `dev-only-bugs <MILESTONE> [--apply]` — list closed milestone bugs that never reached production, without `no changelog`. Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or this milestone (`only_unreleased_envs`; any production hint or other version keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup.
|
||||
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
@ -81,7 +81,7 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
|
||||
|
||||
Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts.
|
||||
|
||||
**Output**: JSON array to stdout; progress to stderr.
|
||||
**Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both.
|
||||
|
||||
### `comments`
|
||||
|
||||
|
||||
@ -56,6 +56,14 @@ that breaks them is wrong, even if the code is right.
|
||||
|
||||
`:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight
|
||||
|
||||
## Atomic Commits
|
||||
|
||||
A PR squash-merges into one commit by default, so the split inside the branch
|
||||
does not reach `main` and nothing depends on it. When the PR asks for a
|
||||
regular merge (`mem:workflow/creating-prs` → Merge Strategy), every commit
|
||||
must stand alone: it builds, passes the linter, and needs no later commit on
|
||||
the same branch to make sense.
|
||||
|
||||
## Referencing Issues
|
||||
|
||||
Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link.
|
||||
|
||||
@ -32,6 +32,19 @@ PR titles follow commit title conventions:
|
||||
|
||||
See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters.
|
||||
|
||||
## Merge Strategy
|
||||
|
||||
Policy: `CONTRIBUTING.md` → "Merging", under "Pull Requests".
|
||||
|
||||
- Squash merge is the default, so the PR's own commits never reach `main`
|
||||
and the PR title becomes the landed subject.
|
||||
- A regular merge is possible on request only. The request is the **first
|
||||
line** of the PR body, above the `**Note:**` line:
|
||||
`> **Merge:** do not squash; each commit is self-contained.`
|
||||
- Ask for it only when every commit on the branch is self-contained (builds
|
||||
and passes lint on its own). Default to squash unless the user asks for
|
||||
the opposite.
|
||||
|
||||
## Description Body
|
||||
|
||||
Include concise sections covering:
|
||||
@ -60,6 +73,8 @@ PR descriptions follow this structure:
|
||||
```
|
||||
|
||||
The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
|
||||
When the PR requests a regular merge instead of a squash merge, its
|
||||
`**Merge:**` line goes above the `**Note:**` line (see Merge Strategy).
|
||||
|
||||
## Writing Principles
|
||||
|
||||
|
||||
@ -24,11 +24,12 @@ Editing `CHANGES.md` happens only on explicit request or as part of the release
|
||||
| Is it a breaking change? | `breaking change` label |
|
||||
| Is it a highlight? | `release highlight` label |
|
||||
|
||||
- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
|
||||
- **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
|
||||
- **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug).
|
||||
- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry.
|
||||
- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge.
|
||||
- **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status.
|
||||
- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3).
|
||||
- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own.
|
||||
- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly.
|
||||
|
||||
@ -39,7 +40,7 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
|
||||
```markdown
|
||||
# CHANGELOG
|
||||
|
||||
## 2.19.0 (Unreleased)
|
||||
## 2.20.0 (Unreleased)
|
||||
|
||||
### :boom: Breaking changes & Deprecations
|
||||
|
||||
@ -57,21 +58,27 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
|
||||
|
||||
- Fix <user-facing description of the vulnerability> (<advisory URL>) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :bug: Bugs fixed
|
||||
|
||||
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :sparkles: New features & Enhancements
|
||||
### :tada: Features
|
||||
|
||||
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :sparkles: Enhancements
|
||||
|
||||
- Improve <user-facing description of the existing behaviour> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
|
||||
### :bug: Bugs
|
||||
|
||||
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
|
||||
```
|
||||
|
||||
- `:boom:` first, then `:rocket:`, `:lock:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
|
||||
- `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update.
|
||||
- **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it.
|
||||
- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
|
||||
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories.
|
||||
- One blank line between the last entry of a section and the next heading.
|
||||
- No manual line wrapping: one line per bullet or paragraph, however long.
|
||||
|
||||
## 4. Regular entries (`:bug:`, `:sparkles:`)
|
||||
## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`)
|
||||
|
||||
The entry shape is:
|
||||
|
||||
@ -153,15 +160,18 @@ Generated from the advisories that declare the version in `patched_versions`; ne
|
||||
|
||||
1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious.
|
||||
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
|
||||
3. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
|
||||
4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
|
||||
5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
|
||||
6. **Reconcile the existing text** (section 9).
|
||||
7. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
|
||||
8. **Generate `:rocket:`** from the `release highlight` label (section 6).
|
||||
9. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
|
||||
10. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
|
||||
11. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
|
||||
3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs "<VERSION>"` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration.
|
||||
- **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these.
|
||||
- **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or a version of this milestone. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10).
|
||||
4. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
|
||||
5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
|
||||
6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
|
||||
7. **Reconcile the existing text** (section 9).
|
||||
8. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
|
||||
9. **Generate `:rocket:`** from the `release highlight` label (section 6).
|
||||
10. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
|
||||
11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
|
||||
12. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
|
||||
|
||||
## 9. Pre-flight: reconcile before writing
|
||||
|
||||
@ -172,7 +182,7 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
|
||||
3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry.
|
||||
4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry.
|
||||
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently.
|
||||
6. **Issue Type changed to `Task`** → remove the entry.
|
||||
6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section.
|
||||
7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry.
|
||||
8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it.
|
||||
9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section.
|
||||
@ -189,6 +199,8 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
|
||||
- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights).
|
||||
- **D — `:rocket:` entry without issue AND PR references** (gap).
|
||||
|
||||
**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total.
|
||||
|
||||
Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run.
|
||||
|
||||
## 11. Pitfalls
|
||||
@ -203,6 +215,6 @@ Anything else is a rule violation, not a report item: fix it in the pre-flight a
|
||||
## 12. Where the code is
|
||||
|
||||
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`.
|
||||
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `report`. Details: `mem:scripts/changelog`.
|
||||
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`.
|
||||
- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`.
|
||||
- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**.
|
||||
|
||||
@ -20,6 +20,7 @@ Center](https://help.penpot.app/).
|
||||
- [Title format](#title)
|
||||
- [Description](#description)
|
||||
- [Review process](#review-process)
|
||||
- [Merging](#merging)
|
||||
- [What we won't accept](#what-we-wont-accept)
|
||||
- [Good first issues](#good-first-issues)
|
||||
- [Commit Guidelines](#commit-guidelines)
|
||||
@ -111,8 +112,8 @@ Branch names are not enforced, but we recommend the following:
|
||||
- Otherwise, use a short, descriptive name with words separated by hyphens
|
||||
and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`).
|
||||
|
||||
Since PRs are squash-merged, the branch name does not survive into the
|
||||
commit history — what matters is the [PR title](#title).
|
||||
Since PRs are squash-merged by default, the branch name does not survive
|
||||
into the commit history — what matters is the [PR title](#title).
|
||||
|
||||
### Format
|
||||
|
||||
@ -156,9 +157,27 @@ for more concrete information.
|
||||
- Address review feedback by **pushing new commits** — do not
|
||||
force-push during review, as it breaks comment threads.
|
||||
- PRs require at least **one approval** before merge.
|
||||
- We use **squash-merge** by default. The PR title becomes the final
|
||||
- We **squash-merge** by default; see [Merging](#merging) for the reason
|
||||
and for how to ask for a regular merge. The PR title becomes the final
|
||||
commit message, so follow the [title format](#title) above.
|
||||
|
||||
### Merging
|
||||
|
||||
PRs are **squash-merged** by default. One PR lands as one commit, which keeps
|
||||
the change atomic: the history stays readable, and `git bisect` never stops
|
||||
on an intermediate commit that fails the tests or the linter.
|
||||
|
||||
This is the default, not a hard rule. If you keep your commits atomic and
|
||||
self-contained — each one builds, passes the linter and makes sense on its
|
||||
own — you may ask for a regular merge instead. Put the request in the first
|
||||
line of the PR description, above everything else:
|
||||
|
||||
> **Merge:** do not squash; each commit is self-contained.
|
||||
|
||||
Maintainers honor the request when the commits hold up, and squash-merge
|
||||
otherwise. After a squash merge the PR title becomes the commit subject on
|
||||
the main branch; see [Title format](#title).
|
||||
|
||||
### What we won't accept
|
||||
|
||||
To save time on both sides, please avoid submitting PRs that:
|
||||
|
||||
@ -488,10 +488,13 @@
|
||||
ON fls.file_id = flr.file_id
|
||||
AND fls.library_file_id = flr.library_file_id
|
||||
WHERE flr.file_id = ANY(?)
|
||||
AND l.deleted_at IS NULL;")
|
||||
AND l.deleted_at IS NULL
|
||||
ORDER BY flr.file_id, flr.library_file_id;")
|
||||
|
||||
(defn get-files-rels
|
||||
"Given a set of file-id's, return all matching relations with the libraries"
|
||||
"Given a set of file-id's, return all matching relations with the libraries.
|
||||
Rows come back in stable `(file-id, library-file-id)` order so export
|
||||
and import resolve cycles deterministically."
|
||||
[cfg ids]
|
||||
|
||||
(assert (set? ids) "expected a set of uuids")
|
||||
@ -534,6 +537,20 @@
|
||||
(->> (db/exec! conn [sql:get-libraries ids'])
|
||||
(into #{} xf-map-id))))))
|
||||
|
||||
(defn library-link-status
|
||||
"Check whether linking `file-id` to `library-id` is safe. Returns
|
||||
`:ok` when the link can be created, `:self-link` when both ids are
|
||||
the same, and `:circular` when the link would close a dependency
|
||||
cycle (i.e. `file-id` is already among the transitive libraries of
|
||||
`library-id`). Shared by the UI command (`link-file-to-library`,
|
||||
which raises) and the v3 importer (`import-file-relations`, which
|
||||
skips and logs), so both stay consistent."
|
||||
[cfg file-id library-id]
|
||||
(cond
|
||||
(= file-id library-id) :self-link
|
||||
(contains? (get-libraries cfg [library-id]) file-id) :circular
|
||||
:else :ok))
|
||||
|
||||
(defn get-file-object-thumbnails
|
||||
"Return all file object thumbnails for a given file."
|
||||
[cfg file-id]
|
||||
|
||||
@ -482,9 +482,11 @@
|
||||
(bfm/apply-pending-migrations! cfg)
|
||||
|
||||
;; Knowing that the ids of the created files are in index,
|
||||
;; just lookup them and return them in the same shape as v3.
|
||||
;; just lookup them and return them in the same shape as v3. The
|
||||
;; metadata of an old package may repeat a file id, so the answer
|
||||
;; carries one id per imported file.
|
||||
(let [files (-> bfc/*state* deref :files)]
|
||||
{:file-ids (into [] (keep #(get-in @bfc/*state* [:index %])) files)})))))
|
||||
{:file-ids (into [] (comp (keep #(get-in @bfc/*state* [:index %])) (distinct)) files)})))))
|
||||
|
||||
(defmethod read-import :v1
|
||||
[options]
|
||||
@ -510,89 +512,114 @@
|
||||
|
||||
(defmethod read-section :v1/files
|
||||
[{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}]
|
||||
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
|
||||
(let [file (read-obj! input)
|
||||
media (read-obj! input)
|
||||
(let [imported (volatile! #{})]
|
||||
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
|
||||
(let [file (read-obj! input)
|
||||
media (read-obj! input)
|
||||
file-id (:id file)]
|
||||
|
||||
file-id (:id file)
|
||||
file-id' (bfc/lookup-index file-id)
|
||||
(when (not= file-id expected-file-id)
|
||||
(ex/raise :type :validation
|
||||
:code :inconsistent-penpot-file
|
||||
:found-id file-id
|
||||
:expected-id expected-file-id
|
||||
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
|
||||
|
||||
file (bfc/clean-file-features file)
|
||||
thumbnails (:thumbnails file)]
|
||||
(if (contains? @imported file-id)
|
||||
;; Old exports could repeat a file id in the metadata (the file was
|
||||
;; also one of its own libraries). The entry is read above to keep
|
||||
;; the stream in sync, but the file is imported only once.
|
||||
(l/warn :hint "ignoring repeated file entry"
|
||||
:file-id (str file-id)
|
||||
::l/sync? true)
|
||||
|
||||
(when (not= file-id expected-file-id)
|
||||
(ex/raise :type :validation
|
||||
:code :inconsistent-penpot-file
|
||||
:found-id file-id
|
||||
:expected-id expected-file-id
|
||||
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
|
||||
(let [file-id' (bfc/lookup-index file-id)
|
||||
file (bfc/clean-file-features file)
|
||||
thumbnails (:thumbnails file)]
|
||||
|
||||
(l/dbg :hint "processing file"
|
||||
:id (str file-id)
|
||||
:features (:features file)
|
||||
:version (-> file :data :version)
|
||||
:media (count media)
|
||||
:thumbnails (count thumbnails)
|
||||
::l/sync? true)
|
||||
(vswap! imported conj file-id)
|
||||
|
||||
(when (seq thumbnails)
|
||||
(let [thumbnails (remap-thumbnails thumbnails file-id')]
|
||||
(l/dbg :hint "updated index with thumbnails"
|
||||
:total (count thumbnails)
|
||||
::l/sync? true)
|
||||
(vswap! bfc/*state* update :thumbnails bfc/into-vec thumbnails)))
|
||||
(l/dbg :hint "processing file"
|
||||
:id (str file-id)
|
||||
:features (:features file)
|
||||
:version (-> file :data :version)
|
||||
:media (count media)
|
||||
:thumbnails (count thumbnails)
|
||||
::l/sync? true)
|
||||
|
||||
(when (seq media)
|
||||
;; Update index with media
|
||||
(l/dbg :hint "update index with media" :total (count media) ::l/sync? true)
|
||||
(vswap! bfc/*state* update :index bfc/update-index (map :id media))
|
||||
(when (seq thumbnails)
|
||||
(let [thumbnails (remap-thumbnails thumbnails file-id')]
|
||||
(l/dbg :hint "updated index with thumbnails"
|
||||
:total (count thumbnails)
|
||||
::l/sync? true)
|
||||
(vswap! bfc/*state* update :thumbnails bfc/into-vec thumbnails)))
|
||||
|
||||
;; Store file media for later insertion
|
||||
(l/dbg :hint "update media references" ::l/sync? true)
|
||||
(vswap! bfc/*state* update :media into (map #(update % :id bfc/lookup-index)) media))
|
||||
(when (seq media)
|
||||
;; Update index with media
|
||||
(l/dbg :hint "update index with media" :total (count media) ::l/sync? true)
|
||||
(vswap! bfc/*state* update :index bfc/update-index (map :id media))
|
||||
|
||||
(let [file (-> file
|
||||
(assoc :id file-id')
|
||||
(cond-> (and (= idx 0) (some? name))
|
||||
(assoc :name name))
|
||||
(assoc :project-id project-id)
|
||||
(dissoc :thumbnails))
|
||||
file (bfc/process-file system file)]
|
||||
;; Store file media for later insertion
|
||||
(l/dbg :hint "update media references" ::l/sync? true)
|
||||
(vswap! bfc/*state* update :media into (map #(update % :id bfc/lookup-index)) media))
|
||||
|
||||
;; All features that are enabled and requires explicit migration are
|
||||
;; added to the state for a posterior migration step.
|
||||
(doseq [feature (-> (::bfc/features system)
|
||||
(set/difference cfeat/no-migration-features)
|
||||
(set/difference (:features file)))]
|
||||
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature file-id']))
|
||||
(let [file (-> file
|
||||
(assoc :id file-id')
|
||||
(cond-> (and (= idx 0) (some? name))
|
||||
(assoc :name name))
|
||||
(assoc :project-id project-id)
|
||||
(dissoc :thumbnails))
|
||||
file (bfc/process-file system file)]
|
||||
|
||||
(l/dbg :hint "create file" :id (str file-id') ::l/sync? true)
|
||||
(bfc/save-file! system file ::db/return-keys false)
|
||||
;; All features that are enabled and requires explicit migration
|
||||
;; are added to the state for a posterior migration step.
|
||||
(doseq [feature (-> (::bfc/features system)
|
||||
(set/difference cfeat/no-migration-features)
|
||||
(set/difference (:features file)))]
|
||||
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature file-id']))
|
||||
|
||||
file-id'))))
|
||||
(l/dbg :hint "create file" :id (str file-id') ::l/sync? true)
|
||||
(bfc/save-file! system file ::db/return-keys false)
|
||||
|
||||
file-id')))))))
|
||||
|
||||
(defmethod read-section :v1/rels
|
||||
[{:keys [::db/conn ::bfc/input ::bfc/timestamp]}]
|
||||
(let [rels (read-obj! input)
|
||||
[{:keys [::db/conn ::bfc/input ::bfc/timestamp] :as cfg}]
|
||||
(let [rels (->> (read-obj! input)
|
||||
(sort-by (fn [{:keys [file-id library-file-id]}]
|
||||
[(str file-id) (str library-file-id)])))
|
||||
ids (into #{} (-> bfc/*state* deref :files))]
|
||||
;; Insert all file relations
|
||||
(doseq [{:keys [library-file-id] :as rel} rels]
|
||||
(let [rel (-> rel
|
||||
(update :file-id bfc/lookup-index)
|
||||
(update :library-file-id bfc/lookup-index))]
|
||||
|
||||
(if (contains? ids library-file-id)
|
||||
(do
|
||||
(l/dbg :hint "create file library link"
|
||||
:file-id (:file-id rel)
|
||||
:lib-id (:library-file-id rel)
|
||||
::l/sync? true)
|
||||
(let [rel-params (dissoc rel :synced-at)]
|
||||
(db/insert! conn :file-library-rel rel-params)
|
||||
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params)
|
||||
:library-file-id (:library-file-id rel-params)
|
||||
:synced-at (or (:synced-at rel)
|
||||
timestamp)})))
|
||||
(case (bfc/library-link-status cfg (:file-id rel) (:library-file-id rel))
|
||||
:self-link
|
||||
(l/warn :hint "ignoring self-referencing library link"
|
||||
:file-id (:file-id rel)
|
||||
:lib-id (:library-file-id rel)
|
||||
::l/sync? true)
|
||||
|
||||
:circular
|
||||
(l/warn :hint "ignoring circular library link"
|
||||
:file-id (:file-id rel)
|
||||
:lib-id (:library-file-id rel)
|
||||
::l/sync? true)
|
||||
|
||||
:ok
|
||||
(do
|
||||
(l/dbg :hint "create file library link"
|
||||
:file-id (:file-id rel)
|
||||
:lib-id (:library-file-id rel)
|
||||
::l/sync? true)
|
||||
(let [rel-params (dissoc rel :synced-at)]
|
||||
(db/insert! conn :file-library-rel rel-params)
|
||||
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params)
|
||||
:library-file-id (:library-file-id rel-params)
|
||||
:synced-at (or (:synced-at rel)
|
||||
timestamp)}))))
|
||||
|
||||
(l/warn :hint "ignoring file library link"
|
||||
:file-id (:file-id rel)
|
||||
|
||||
@ -319,6 +319,11 @@
|
||||
(db/insert! conn :file-tagged-object-thumbnail thumbnail
|
||||
{::db/return-keys false})))
|
||||
|
||||
;; NOTE: v2 packages are only produced and imported through the
|
||||
;; developer REPL (`app.srepl.binfile`), never through the product, so
|
||||
;; this path keeps inserting the relations as they come: unlike v1 and
|
||||
;; v3 it does not skip self-referencing or circular links (see
|
||||
;; `bfc/library-link-status`).
|
||||
(doseq [rel (read-obj cfg :file-rels file-id)]
|
||||
(let [rel (-> rel
|
||||
(update :file-id bfc/lookup-index)
|
||||
|
||||
@ -1038,26 +1038,49 @@
|
||||
file-id')))
|
||||
|
||||
(defn- import-file-relations
|
||||
"Import library relations from the manifest, ignoring relations whose
|
||||
endpoints are not both part of the package. Of the rest, skip edges that
|
||||
would close a dependency cycle or link a file to itself (same check as
|
||||
`link-file-to-library`, see `bfc/library-link-status`). Relations are
|
||||
processed in sorted manifest order and the first edge wins: an edge
|
||||
that would close a cycle with previously imported relations is skipped
|
||||
and logged."
|
||||
[{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}]
|
||||
(let [relations (:relations manifest)
|
||||
(let [relations (sort-by (fn [[file-id libr-id]] [(str file-id) (str libr-id)])
|
||||
(:relations manifest))
|
||||
file-ids (into #{} (map :id) (:files manifest))]
|
||||
(doseq [[index [file-id libr-id]] (d/enumerate relations)]
|
||||
(tap-progress cfg {:section :relations
|
||||
:current (inc index) :total (count relations)})
|
||||
:current (inc index)
|
||||
:total (count relations)})
|
||||
|
||||
(if (and (not= file-id libr-id)
|
||||
(contains? file-ids file-id)
|
||||
(if (and (contains? file-ids file-id)
|
||||
(contains? file-ids libr-id))
|
||||
(let [file-id (bfc/lookup-index file-id)
|
||||
libr-id (bfc/lookup-index libr-id)]
|
||||
(l/dbg :hint "create file library link"
|
||||
:file-id (str file-id)
|
||||
:lib-id (str libr-id)
|
||||
::l/sync? true)
|
||||
(let [rel-params {:file-id file-id
|
||||
:library-file-id libr-id}]
|
||||
(db/insert! conn :file-library-rel rel-params)
|
||||
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))
|
||||
(case (bfc/library-link-status cfg file-id libr-id)
|
||||
:self-link
|
||||
(l/warn :hint "ignoring self-referencing library link"
|
||||
:file-id (str file-id)
|
||||
:lib-id (str libr-id)
|
||||
::l/sync? true)
|
||||
|
||||
:circular
|
||||
(l/warn :hint "ignoring circular library link"
|
||||
:file-id (str file-id)
|
||||
:lib-id (str libr-id)
|
||||
::l/sync? true)
|
||||
|
||||
:ok
|
||||
(do
|
||||
(l/dbg :hint "create file library link"
|
||||
:file-id (str file-id)
|
||||
:lib-id (str libr-id)
|
||||
::l/sync? true)
|
||||
(let [rel-params {:file-id file-id
|
||||
:library-file-id libr-id}]
|
||||
(db/insert! conn :file-library-rel rel-params)
|
||||
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))
|
||||
(l/warn :hint "ignoring invalid file library relation"
|
||||
:file-id (str file-id)
|
||||
:lib-id (str libr-id))))))
|
||||
|
||||
@ -108,7 +108,10 @@
|
||||
|
||||
(def initsql
|
||||
(str "SET statement_timeout = 300000;\n"
|
||||
"SET idle_in_transaction_session_timeout = 300000;"))
|
||||
"SET idle_in_transaction_session_timeout = 300000;\n"
|
||||
;; lets the planner resolve a join between two partitioned tables
|
||||
;; pairwise, when both are partitioned with the same bounds
|
||||
"SET enable_partitionwise_join = on;"))
|
||||
|
||||
(defn- create-datasource-config
|
||||
[{:keys [::uri] :as cfg}]
|
||||
|
||||
@ -205,8 +205,11 @@
|
||||
(db/delete! cfg :file-data
|
||||
{:id id :file-id file-id :type "snapshot"}
|
||||
{::db/return-keys false})
|
||||
;; every mutation of `file_change` moves `updated_at`: the partition
|
||||
;; runbook uses it as the watermark of the delta passes
|
||||
(db/update! cfg :file-change
|
||||
{:data data}
|
||||
{:data data
|
||||
:updated-at (ct/now)}
|
||||
{:file-id file-id :id id}
|
||||
{::db/return-keys false}))
|
||||
|
||||
|
||||
21
backend/src/app/features/file_changes.clj
Normal file
21
backend/src/app/features/file_changes.clj
Normal file
@ -0,0 +1,21 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns app.features.file-changes
|
||||
"The kinds of the rows of the `file_change` table.
|
||||
|
||||
A snapshot is the state of a file at a point: it has a label and its
|
||||
data lives in `file_data` (old `legacy-db` payloads live in
|
||||
`file_change.data`). A revision is the delta of a save: it has
|
||||
no label and it holds the changes.")
|
||||
|
||||
(def snapshot-kind
|
||||
"The kind of a snapshot row."
|
||||
1)
|
||||
|
||||
(def revision-kind
|
||||
"The kind of a revision row."
|
||||
2)
|
||||
@ -19,6 +19,7 @@
|
||||
[app.db :as db]
|
||||
[app.db.sql :as-alias sql]
|
||||
[app.features.fdata :as fdata]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.jobs :as jobs]
|
||||
[app.storage :as sto]
|
||||
[app.util.blob :as blob]
|
||||
@ -74,57 +75,71 @@
|
||||
LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id)
|
||||
WHERE f.id = ?")
|
||||
|
||||
(def ^:private sql:snapshot-columns
|
||||
"c.id,
|
||||
c.label,
|
||||
c.revn,
|
||||
c.created_at,
|
||||
c.modified_at,
|
||||
c.deleted_at,
|
||||
c.profile_id,
|
||||
c.created_by,
|
||||
c.locked_by,
|
||||
c.features,
|
||||
c.metadata,
|
||||
c.migrations,
|
||||
c.version,
|
||||
c.file_id")
|
||||
|
||||
(def ^:private sql:snapshot-visible
|
||||
"AND CASE WHEN c.created_by = 'user'
|
||||
THEN c.deleted_at IS NULL
|
||||
WHEN c.created_by = 'system'
|
||||
THEN c.deleted_at IS NULL OR c.deleted_at >= ?::timestamptz
|
||||
END")
|
||||
|
||||
(def ^:private sql:get-snapshot-without-data
|
||||
(str "WITH snapshots AS (" sql:snapshots ")"
|
||||
"SELECT c.id,
|
||||
c.label,
|
||||
c.revn,
|
||||
c.created_at,
|
||||
c.modified_at,
|
||||
c.deleted_at,
|
||||
c.profile_id,
|
||||
c.created_by,
|
||||
c.locked_by,
|
||||
c.features,
|
||||
c.metadata,
|
||||
c.migrations,
|
||||
c.version,
|
||||
c.file_id
|
||||
FROM snapshots AS c
|
||||
WHERE c.id = ?
|
||||
AND CASE WHEN c.created_by = 'user'
|
||||
THEN c.deleted_at IS NULL
|
||||
WHEN c.created_by = 'system'
|
||||
THEN c.deleted_at IS NULL OR c.deleted_at >= ?::timestamptz
|
||||
END"))
|
||||
"SELECT " sql:snapshot-columns
|
||||
" FROM snapshots AS c
|
||||
WHERE c.file_id = ?
|
||||
AND c.id = ? " sql:snapshot-visible))
|
||||
|
||||
(defn get-minimal-snapshot
|
||||
[cfg snapshot-id]
|
||||
"Get the minimal snapshot info (without the data blob). The query filters
|
||||
by `file_id`, which lets the planner prune the partitions of `file_change`
|
||||
when it chooses an index scan. Raises `:snapshot-not-found` when there is
|
||||
no matching snapshot."
|
||||
[cfg file-id snapshot-id]
|
||||
(let [now (ct/now)]
|
||||
(-> (db/get-with-sql cfg [sql:get-snapshot-without-data snapshot-id now]
|
||||
{::db/remove-deleted false})
|
||||
(decode-snapshot))))
|
||||
(or (-> (db/get-with-sql cfg [sql:get-snapshot-without-data file-id snapshot-id now]
|
||||
{::db/remove-deleted false
|
||||
::db/throw-if-not-exists false})
|
||||
(decode-snapshot))
|
||||
(ex/raise :type :not-found
|
||||
:code :snapshot-not-found
|
||||
:hint "unable to find snapshot with the provided id"
|
||||
:snapshot-id snapshot-id
|
||||
:file-id file-id))))
|
||||
|
||||
(def ^:private sql:get-snapshot
|
||||
(str sql:snapshots
|
||||
" AND c.file_id = ?
|
||||
AND c.id = ?
|
||||
AND CASE WHEN c.created_by = 'user'
|
||||
THEN (c.deleted_at IS NULL)
|
||||
WHEN c.created_by = 'system'
|
||||
THEN (c.deleted_at IS NULL OR c.deleted_at >= ?::timestamptz)
|
||||
END"))
|
||||
AND c.id = ? "
|
||||
sql:snapshot-visible))
|
||||
|
||||
(defn get-snapshot
|
||||
"Get a fully decoded snapshot for read-only preview or restoration.
|
||||
Returns the snapshot map with decoded :data field."
|
||||
Returns the snapshot map with decoded :data field, or nil when the file has
|
||||
no snapshot with that id."
|
||||
[cfg file-id snapshot-id]
|
||||
(let [now (ct/now)]
|
||||
(->> (db/get-with-sql cfg [sql:get-snapshot file-id snapshot-id now]
|
||||
{::db/remove-deleted false})
|
||||
(decode-snapshot)
|
||||
(fdata/resolve-file-data cfg)
|
||||
(fdata/decode-file-data cfg))))
|
||||
(some->> (db/get-with-sql cfg [sql:get-snapshot file-id snapshot-id now]
|
||||
{::db/remove-deleted false
|
||||
::db/throw-if-not-exists false})
|
||||
(decode-snapshot)
|
||||
(fdata/resolve-file-data cfg)
|
||||
(fdata/decode-file-data cfg))))
|
||||
|
||||
(def ^:private sql:get-visible-snapshots
|
||||
(str "WITH "
|
||||
@ -267,6 +282,7 @@
|
||||
(-> snapshot
|
||||
(update :features into-array)
|
||||
(update :migrations into-array)
|
||||
(assoc :kind fchanges/snapshot-kind)
|
||||
(assoc :updated-at created-at)
|
||||
(assoc :profile-id profile-id)
|
||||
(assoc :session-id session-id)
|
||||
@ -313,7 +329,10 @@
|
||||
(-> (db/update! cfg :file-change
|
||||
{:label label
|
||||
:created-by "user"
|
||||
:updated-at modified-at
|
||||
;; the partition runbook uses updated_at as the
|
||||
;; watermark of its delta passes: a rename has to move
|
||||
;; it, unlike file_data.modified_at above
|
||||
:updated-at (ct/now)
|
||||
:deleted-at nil}
|
||||
{:file-id file-id
|
||||
:id id}
|
||||
@ -346,7 +365,7 @@
|
||||
(when-not snapshot
|
||||
(ex/raise :type :not-found
|
||||
:code :snapshot-not-found
|
||||
:hint "unable to find snapshot with the provided label"
|
||||
:hint "unable to find snapshot with the provided id"
|
||||
:snapshot-id snapshot-id
|
||||
:file-id file-id))
|
||||
|
||||
@ -426,7 +445,8 @@
|
||||
:file-id file-id
|
||||
:id id}})
|
||||
(db/update! cfg :file-change
|
||||
{:deleted-at deleted-at}
|
||||
{:deleted-at deleted-at
|
||||
:updated-at (ct/now)}
|
||||
{:id id :file-id file-id}
|
||||
{::db/return-keys false})
|
||||
true)
|
||||
@ -435,19 +455,25 @@
|
||||
(str sql:snapshots " AND c.file_id = ?"))
|
||||
|
||||
(defn lock-by!
|
||||
[conn id profile-id]
|
||||
[conn file-id id profile-id]
|
||||
(assert (uuid? file-id) "missing file-id")
|
||||
(assert (uuid? id) "missing id")
|
||||
(-> (db/update! conn :file-change
|
||||
{:locked-by profile-id}
|
||||
{:id id}
|
||||
{:locked-by profile-id
|
||||
:updated-at (ct/now)}
|
||||
{:id id :file-id file-id}
|
||||
{::db/return-keys false})
|
||||
(db/get-update-count)
|
||||
(pos?)))
|
||||
|
||||
(defn unlock!
|
||||
[conn id]
|
||||
[conn file-id id]
|
||||
(assert (uuid? file-id) "missing file-id")
|
||||
(assert (uuid? id) "missing id")
|
||||
(-> (db/update! conn :file-change
|
||||
{:locked-by nil}
|
||||
{:id id}
|
||||
{:locked-by nil
|
||||
:updated-at (ct/now)}
|
||||
{:id id :file-id file-id}
|
||||
{::db/return-keys false})
|
||||
(db/get-update-count)
|
||||
(pos?)))
|
||||
|
||||
@ -538,7 +538,10 @@
|
||||
:fn (mg/resource "app/migrations/sql/0157-migrate-legacy-file-data.sql")}
|
||||
|
||||
{:name "0158-del-forbidden-file-library-rels"
|
||||
:fn (mg/resource "app/migrations/sql/0158-del-forbidden-file-library-rels.sql")}])
|
||||
:fn (mg/resource "app/migrations/sql/0158-del-forbidden-file-library-rels.sql")}
|
||||
|
||||
{:name "0158-partition-file-change"
|
||||
:fn (mg/resource "app/migrations/sql/0158-partition-file-change.sql")}])
|
||||
|
||||
(defn apply-migrations!
|
||||
[pool name migrations]
|
||||
|
||||
251
backend/src/app/migrations/sql/0158-partition-file-change.sql
Normal file
251
backend/src/app/migrations/sql/0158-partition-file-change.sql
Normal file
@ -0,0 +1,251 @@
|
||||
-- Partition `file_change` by `file_id`.
|
||||
--
|
||||
-- The table grows with every file change and its TOAST holds the blobs of
|
||||
-- the changes, so it is the biggest relation of the database. Partitioning it
|
||||
-- by `file_id` lets the queries that filter by file prune to a single
|
||||
-- partition, and lets the join with `file_data` (already partitioned with the
|
||||
-- same modulus) be resolved partition by partition.
|
||||
--
|
||||
-- The primary key has to include the partition key, so it becomes
|
||||
-- `(file_id, id)`. Every access by id alone would lose its index, so the
|
||||
-- access paths of the application filter by `file_id` too.
|
||||
--
|
||||
-- The new table is created with the final schema: the columns that nobody
|
||||
-- used anymore are gone (`data_ref_id` and `data_backend`, dead since the
|
||||
-- file storage refactor), the `data` column is kept as-is (the `legacy-db`
|
||||
-- file data backend reads old payloads from it: migration
|
||||
-- 0157-migrate-legacy-file-data moved the existing blobs), and the new
|
||||
-- `kind` column tells the snapshots and the revisions apart.
|
||||
--
|
||||
-- The `kind` column is a reserve: the queries still tell a snapshot from a
|
||||
-- revision by its label (`label IS NOT NULL`), and `kind` takes over when
|
||||
-- they are rewritten. That is why no constraint ties the two together: the
|
||||
-- day `kind` is the source of truth, the constraint would have to go.
|
||||
--
|
||||
-- The old table is kept aside as `file_change_legacy`: a frozen copy of the
|
||||
-- previous layout. It is dropped by hand, days later (or right here, when it
|
||||
-- is empty: on a new instance there is nothing to keep).
|
||||
--
|
||||
-- This migration is not meant to run on a big instance: the copy of the
|
||||
-- rows of the table happens inside the transaction of the migration, and
|
||||
-- the pool sets a 5 minute statement timeout. On those instances mark the
|
||||
-- migration as applied and partition the table by hand, with the procedure
|
||||
-- of `operations/2.20.0-file-change-partition.md`.
|
||||
--
|
||||
-- It assumes the service is stopped (a stop-start upgrade): it renames the
|
||||
-- table, and a process of the previous version would keep writing to
|
||||
-- `file_change_legacy` through its server-side prepared statements, which
|
||||
-- hold the OID of the table and survive the rename. A rolling deployment is
|
||||
-- the scenario of the runbook, not of this path.
|
||||
--
|
||||
-- The GRANTs of the table are not carried over: this creates a new table,
|
||||
-- so an instance with custom grants on `file_change` has to re-apply them
|
||||
-- after the upgrade.
|
||||
|
||||
-- 0. The guard for the big instances, before anything is touched: a copy
|
||||
-- that does not fit keeps the backend down in a restart loop, and the
|
||||
-- operator has to learn it from a clear message instead of from a
|
||||
-- `statement timeout`. Both numbers are catalog reads, not table scans.
|
||||
--
|
||||
-- The row count comes from `pg_class.reltuples`, which is -1 on a
|
||||
-- relation that was never analyzed (a restore without `--analyze`, for
|
||||
-- instance). Analyze first so the number is real; on a table that fails
|
||||
-- the guard this costs seconds per boot attempt, not a table scan.
|
||||
--
|
||||
-- The copy, the index builds and the `ANALYZE` of an instance that
|
||||
-- passes the guard can take longer than the 5 minutes of the pool. The
|
||||
-- timeout is bounded, so a wrong estimate still fails instead of hanging
|
||||
-- the boot, and it covers the `ANALYZE`: on a cold, large table it can
|
||||
-- outlast the pool too, and the guard has to be the one that speaks.
|
||||
SET LOCAL statement_timeout = '30min';
|
||||
|
||||
ANALYZE file_change;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF pg_total_relation_size('file_change'::regclass) > 10::bigint * 1024 * 1024 * 1024
|
||||
OR (SELECT reltuples FROM pg_class WHERE oid = 'file_change'::regclass) > 2000000
|
||||
THEN
|
||||
RAISE EXCEPTION
|
||||
'file_change is too large for the automatic partition migration: mark 0158-partition-file-change as applied and follow operations/2.20.0-file-change-partition.md';
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
-- 1. The legacy blobs of the snapshots were already moved to `file_data`
|
||||
-- by migration 0157-migrate-legacy-file-data (on the big instances, by
|
||||
-- hand, with its runbook), so there is nothing to copy here. The `data`
|
||||
-- column travels with the row in step 4 below: the `legacy-db` backend
|
||||
-- still reads old payloads from it.
|
||||
|
||||
-- 2. The current table becomes the legacy one. The names of its indexes and
|
||||
-- constraints live in the schema, not in the table, so they are renamed
|
||||
-- with a `_legacy` suffix to free the canonical names.
|
||||
ALTER TABLE file_change RENAME TO file_change_legacy;
|
||||
|
||||
ALTER TABLE file_change_legacy RENAME CONSTRAINT file_change_pkey
|
||||
TO file_change_pkey_legacy;
|
||||
ALTER TABLE file_change_legacy RENAME CONSTRAINT file_change_file_id_fkey
|
||||
TO file_change_file_id_fkey_legacy;
|
||||
ALTER TABLE file_change_legacy RENAME CONSTRAINT file_change_locked_by_fkey
|
||||
TO file_change_locked_by_fkey_legacy;
|
||||
ALTER TABLE file_change_legacy RENAME CONSTRAINT file_change_profile_id_fkey
|
||||
TO file_change_profile_id_fkey_legacy;
|
||||
|
||||
ALTER INDEX file_change__data_ref_id__idx
|
||||
RENAME TO file_change__data_ref_id__idx_legacy;
|
||||
ALTER INDEX file_change__deleted_at__idx
|
||||
RENAME TO file_change__deleted_at__idx_legacy;
|
||||
ALTER INDEX file_change__file_id__revn__idx
|
||||
RENAME TO file_change__file_id__revn__idx_legacy;
|
||||
ALTER INDEX file_change__locked_by__idx
|
||||
RENAME TO file_change__locked_by__idx_legacy;
|
||||
ALTER INDEX file_change__profile_id__idx
|
||||
RENAME TO file_change__profile_id__idx_legacy;
|
||||
ALTER INDEX file_change__system_snapshots__idx
|
||||
RENAME TO file_change__system_snapshots__idx_legacy;
|
||||
|
||||
-- This one comes from another branch, so an instance may not have it.
|
||||
ALTER INDEX IF EXISTS file_change__user_snapshots__idx
|
||||
RENAME TO file_change__user_snapshots__idx_legacy;
|
||||
|
||||
-- 3. The new table, partitioned with the same bounds as `file_data`, so both
|
||||
-- relations can be joined partition by partition.
|
||||
CREATE TABLE file_change (
|
||||
file_id uuid NOT NULL,
|
||||
id uuid NOT NULL DEFAULT gen_random_uuid(),
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
session_id uuid,
|
||||
revn bigint NOT NULL DEFAULT 0,
|
||||
data bytea,
|
||||
changes bytea,
|
||||
profile_id uuid,
|
||||
features text[],
|
||||
label text,
|
||||
version integer,
|
||||
created_by text NOT NULL DEFAULT 'system',
|
||||
updated_at timestamptz DEFAULT now(),
|
||||
deleted_at timestamptz,
|
||||
migrations text[],
|
||||
locked_by uuid,
|
||||
kind smallint NOT NULL,
|
||||
CONSTRAINT file_change_kind_check CHECK (kind IN (1, 2))
|
||||
) PARTITION BY HASH (file_id);
|
||||
|
||||
-- the columns keep the storage of PostgreSQL by default (migration
|
||||
-- 0156-reset-pg-storage-defaults dropped the explicit EXTERNAL overrides)
|
||||
|
||||
COMMENT ON COLUMN file_change.kind IS
|
||||
'The kind of the row: 1 is a snapshot (it has a label and its data lives in file_data) and 2 is a revision (it has no label and it holds the changes).';
|
||||
COMMENT ON COLUMN file_change.locked_by IS
|
||||
'Profile ID of user who has locked this version. Only the creator can lock/unlock their own versions. Locked versions cannot be deleted by others.';
|
||||
|
||||
CREATE TABLE file_change_00 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 0);
|
||||
CREATE TABLE file_change_01 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 1);
|
||||
CREATE TABLE file_change_02 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 2);
|
||||
CREATE TABLE file_change_03 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 3);
|
||||
CREATE TABLE file_change_04 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 4);
|
||||
CREATE TABLE file_change_05 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 5);
|
||||
CREATE TABLE file_change_06 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 6);
|
||||
CREATE TABLE file_change_07 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 7);
|
||||
CREATE TABLE file_change_08 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 8);
|
||||
CREATE TABLE file_change_09 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 9);
|
||||
CREATE TABLE file_change_10 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 10);
|
||||
CREATE TABLE file_change_11 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 11);
|
||||
CREATE TABLE file_change_12 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 12);
|
||||
CREATE TABLE file_change_13 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 13);
|
||||
CREATE TABLE file_change_14 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 14);
|
||||
CREATE TABLE file_change_15 PARTITION OF file_change
|
||||
FOR VALUES WITH (MODULUS 16, REMAINDER 15);
|
||||
|
||||
-- 4. The rows move to the partitioned table. The column list is explicit
|
||||
-- because the legacy table has columns that the new one does not have
|
||||
-- (`data_ref_id` and `data_backend`).
|
||||
-- The kind comes from the label: a row with a label is a snapshot, a row
|
||||
-- without one is a revision.
|
||||
INSERT INTO file_change (id, file_id, created_at, session_id, revn, data, changes,
|
||||
profile_id, features, label, version, created_by,
|
||||
updated_at, deleted_at, migrations, locked_by, kind)
|
||||
SELECT id, file_id, created_at, session_id, revn, data, changes,
|
||||
profile_id, features, label, version, created_by,
|
||||
updated_at, deleted_at, migrations, locked_by,
|
||||
CASE WHEN label IS NOT NULL THEN 1 ELSE 2 END
|
||||
FROM file_change_legacy;
|
||||
|
||||
-- 5. The primary key includes the partition key now.
|
||||
ALTER TABLE file_change ADD PRIMARY KEY (file_id, id);
|
||||
|
||||
-- 6. The foreign keys.
|
||||
ALTER TABLE file_change ADD FOREIGN KEY (file_id)
|
||||
REFERENCES file(id) DEFERRABLE;
|
||||
ALTER TABLE file_change ADD FOREIGN KEY (locked_by)
|
||||
REFERENCES profile(id) ON DELETE SET NULL DEFERRABLE;
|
||||
ALTER TABLE file_change ADD FOREIGN KEY (profile_id)
|
||||
REFERENCES profile(id) ON DELETE SET NULL DEFERRABLE;
|
||||
|
||||
-- 7. The indexes, with the canonical names. The one of the deleted rows now
|
||||
-- carries the `file_id`, so the garbage collector can read the rows it has
|
||||
-- to delete without touching the table.
|
||||
--
|
||||
-- There is no index for the system snapshots: the old one filtered them by
|
||||
-- `data IS NOT NULL`, and with the payload living in `file_data` the
|
||||
-- column no longer tells snapshots apart. The new one could only filter
|
||||
-- them by `deleted_at IS NULL`, but the system snapshots always carry a
|
||||
-- deletion date in the future, so it would never match a row (and
|
||||
-- the query that would use it filters that date with an "or", which no
|
||||
-- partial index can serve).
|
||||
CREATE INDEX file_change__deleted_at__idx
|
||||
ON file_change (deleted_at, file_id, id) WHERE deleted_at IS NOT NULL;
|
||||
CREATE INDEX file_change__file_id__revn__idx
|
||||
ON file_change (file_id, revn);
|
||||
CREATE INDEX file_change__locked_by__idx
|
||||
ON file_change (locked_by) WHERE locked_by IS NOT NULL;
|
||||
CREATE INDEX file_change__profile_id__idx
|
||||
ON file_change (profile_id) WHERE profile_id IS NOT NULL;
|
||||
CREATE INDEX file_change__user_snapshots__idx
|
||||
ON file_change (file_id)
|
||||
WHERE label IS NOT NULL
|
||||
AND created_by = 'user'
|
||||
AND deleted_at IS NULL;
|
||||
|
||||
-- 8. Statistics for the new partitions. The `ANALYZE` of step 0 ran against
|
||||
-- the old table, before the copy: without this one the planner starts
|
||||
-- with empty statistics on the partitions and picks plans by guesswork
|
||||
-- until autovacuum gets to them. The backend starts serving right after
|
||||
-- this migration, so it is the worst moment to leave them unanalyzed.
|
||||
ANALYZE file_change;
|
||||
|
||||
-- 9. The legacy table is kept as a frozen copy of the previous layout. It is
|
||||
-- dropped by hand, days after the cutover:
|
||||
-- DROP TABLE file_change_legacy;
|
||||
--
|
||||
-- Its foreign key on `file_id` goes away with the copy: it is `NO
|
||||
-- ACTION`, so while the legacy table has rows, the objects GC cannot
|
||||
-- delete a file (it deletes the rows of the file first, and the legacy
|
||||
-- ones are not its business). The legacy table is frozen: nothing writes
|
||||
-- to it again.
|
||||
ALTER TABLE file_change_legacy
|
||||
DROP CONSTRAINT file_change_file_id_fkey_legacy;
|
||||
|
||||
-- On a new instance there is nothing to go back to, so the empty table
|
||||
-- is dropped here.
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM file_change_legacy) THEN
|
||||
DROP TABLE file_change_legacy;
|
||||
END IF;
|
||||
END $$;
|
||||
@ -1240,11 +1240,10 @@
|
||||
(check-edition-permissions! conn profile-id library-id)
|
||||
(check-library-team-ownership! conn file-id library-id)
|
||||
|
||||
(let [transitive-deps (bfc/get-libraries cfg [library-id])]
|
||||
(when (contains? transitive-deps file-id)
|
||||
(ex/raise :type :validation
|
||||
:code :circular-library-reference
|
||||
:hint "linking this library would create a circular dependency")))
|
||||
(when (= :circular (bfc/library-link-status cfg file-id library-id))
|
||||
(ex/raise :type :validation
|
||||
:code :circular-library-reference
|
||||
:hint "linking this library would create a circular dependency"))
|
||||
|
||||
(link-file-to-library conn params)
|
||||
(bfc/get-libraries cfg [library-id]))
|
||||
@ -1401,7 +1400,9 @@
|
||||
;; for ever. Rows that still carry a legacy payload without a label are
|
||||
;; garbage that no read path reaches; 0157-migrate-legacy-file-data deletes
|
||||
;; them, so they must keep their date and go with the collector.
|
||||
"UPDATE file_change SET deleted_at = null
|
||||
;; Every write moves `updated_at`: the delta passes copy the rows whose
|
||||
;; `updated_at` changed, so a restore that left it stale could be missed.
|
||||
"UPDATE file_change SET deleted_at = null, updated_at = now()
|
||||
WHERE file_id = ANY(?::uuid[])
|
||||
AND label IS NOT NULL")
|
||||
|
||||
|
||||
@ -52,6 +52,7 @@
|
||||
"Retrieve a file bundle with data from a specific snapshot for
|
||||
read-only preview. Does not modify any database state."
|
||||
{::doc/added "2.16"
|
||||
::doc/changes [["2.20" "An unknown snapshot is reported with the :snapshot-not-found error code"]]
|
||||
::sm/params schema:get-file-snapshot
|
||||
::sm/result files/schema:file-with-permissions
|
||||
::db/transaction true}
|
||||
@ -196,7 +197,7 @@
|
||||
(ex/raise :type :conflict
|
||||
:code :file-modified
|
||||
:hint "the file was modified during the restore process, please retry")))
|
||||
(let [snapshot (fsnap/get-minimal-snapshot cfg id)
|
||||
(let [snapshot (fsnap/get-minimal-snapshot cfg file-id id)
|
||||
vern (fsnap/restore! cfg file-id id)]
|
||||
;; Peers name the restored version by its label when a
|
||||
;; user named it, else by its date
|
||||
@ -214,28 +215,32 @@
|
||||
|
||||
(def ^:private schema:update-file-snapshot
|
||||
[:map {:title "update-file-snapshot"}
|
||||
[:file-id ::sm/uuid]
|
||||
[:id ::sm/uuid]
|
||||
[:label ::sm/text]])
|
||||
|
||||
(sv/defmethod ::update-file-snapshot
|
||||
{::doc/added "1.20"
|
||||
::doc/changes [["2.20" "The :file-id param is now required and an unknown snapshot answers :snapshot-not-found"]]
|
||||
::sm/params schema:update-file-snapshot
|
||||
::db/transaction true}
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id id label]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn id)]
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id file-id id label]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn file-id id)]
|
||||
(files/check-edition-permissions! conn profile-id (:file-id snapshot))
|
||||
(fsnap/update! conn (assoc snapshot :label label))))
|
||||
|
||||
(def ^:private schema:remove-file-snapshot
|
||||
[:map {:title "remove-file-snapshot"}
|
||||
[:file-id ::sm/uuid]
|
||||
[:id ::sm/uuid]])
|
||||
|
||||
(sv/defmethod ::delete-file-snapshot
|
||||
{::doc/added "1.20"
|
||||
::doc/changes [["2.20" "The :file-id param is now required and an unknown snapshot answers :snapshot-not-found"]]
|
||||
::sm/params schema:remove-file-snapshot
|
||||
::db/transaction true}
|
||||
[{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn id)]
|
||||
[{:keys [::db/conn] :as cfg} {:keys [::rpc/profile-id file-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn file-id id)]
|
||||
(files/check-edition-permissions! conn profile-id (:file-id snapshot))
|
||||
|
||||
(when (not= (:created-by snapshot) "user")
|
||||
@ -263,14 +268,16 @@
|
||||
|
||||
(def ^:private schema:lock-file-snapshot
|
||||
[:map {:title "lock-file-snapshot"}
|
||||
[:file-id ::sm/uuid]
|
||||
[:id ::sm/uuid]])
|
||||
|
||||
(sv/defmethod ::lock-file-snapshot
|
||||
{::doc/added "1.20"
|
||||
::doc/changes [["2.20" "The :file-id param is now required and an unknown snapshot answers :snapshot-not-found"]]
|
||||
::sm/params schema:lock-file-snapshot
|
||||
::db/transaction true}
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn id)]
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id file-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn file-id id)]
|
||||
(files/check-edition-permissions! conn profile-id (:file-id snapshot))
|
||||
|
||||
(when (not= (:created-by snapshot) "user")
|
||||
@ -298,18 +305,20 @@
|
||||
:profile-id profile-id
|
||||
:locked-by (:locked-by snapshot)))
|
||||
|
||||
(fsnap/lock-by! conn id profile-id)))
|
||||
(fsnap/lock-by! conn (:file-id snapshot) id profile-id)))
|
||||
|
||||
(def ^:private schema:unlock-file-snapshot
|
||||
[:map {:title "unlock-file-snapshot"}
|
||||
[:file-id ::sm/uuid]
|
||||
[:id ::sm/uuid]])
|
||||
|
||||
(sv/defmethod ::unlock-file-snapshot
|
||||
{::doc/added "1.20"
|
||||
::doc/changes [["2.20" "The :file-id param is now required and an unknown snapshot answers :snapshot-not-found"]]
|
||||
::sm/params schema:unlock-file-snapshot
|
||||
::db/transaction true}
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn id)]
|
||||
[{:keys [::db/conn]} {:keys [::rpc/profile-id file-id id]}]
|
||||
(let [snapshot (fsnap/get-minimal-snapshot conn file-id id)]
|
||||
(files/check-edition-permissions! conn profile-id (:file-id snapshot))
|
||||
|
||||
(when (not= (:created-by snapshot) "user")
|
||||
@ -336,4 +345,4 @@
|
||||
:snapshot-id id
|
||||
:profile-id profile-id))
|
||||
|
||||
(fsnap/unlock! conn id)))
|
||||
(fsnap/unlock! conn (:file-id snapshot) id)))
|
||||
|
||||
@ -21,6 +21,7 @@
|
||||
[app.config :as cf]
|
||||
[app.db :as db]
|
||||
[app.features.fdata :as fdata]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.features.file-snapshots :as fsnap]
|
||||
[app.features.logical-deletion :as ldel]
|
||||
[app.http.errors :as errors]
|
||||
@ -371,6 +372,7 @@
|
||||
(when repair
|
||||
(db/insert! conn :file-change
|
||||
{:id (uuid/next)
|
||||
:kind fchanges/revision-kind
|
||||
:session-id repair-session-id
|
||||
:profile-id profile-id
|
||||
:created-at timestamp
|
||||
@ -379,7 +381,6 @@
|
||||
:file-id (:id file)
|
||||
:revn (:revn repair)
|
||||
:version (:version file)
|
||||
:features (into-array (:features file))
|
||||
:changes (blob/encode (:changes repair))}
|
||||
{::db/return-keys false})
|
||||
(l/wrn :hint "file auto-repaired"
|
||||
@ -391,8 +392,13 @@
|
||||
|
||||
;; Insert change (xlog) with deleted_at in a future data for
|
||||
;; make them automatically eleggible for GC once they expires
|
||||
;;
|
||||
;; The revision carries only the delta of the save: the features of the
|
||||
;; file are not repeated here (nobody reads them from a revision, and
|
||||
;; they are duplicated on every save).
|
||||
(db/insert! conn :file-change
|
||||
{:id (uuid/next)
|
||||
:kind fchanges/revision-kind
|
||||
:session-id session-id
|
||||
:profile-id profile-id
|
||||
:created-at change-created-at
|
||||
@ -401,7 +407,6 @@
|
||||
:file-id (:id file)
|
||||
:revn (:revn file)
|
||||
:version (:version file)
|
||||
:features (into-array (:features file))
|
||||
:changes (blob/encode changes)}
|
||||
{::db/return-keys false})
|
||||
|
||||
|
||||
@ -112,6 +112,7 @@
|
||||
[:font-family types.font/schema:font-family]
|
||||
[:font-weight [::sm/one-of {:format "number"} valid-weight]]
|
||||
[:font-style [::sm/one-of {:format "string"} valid-style]]
|
||||
[:variant-name {:optional true} [:maybe [::sm/text {:max 250}]]]
|
||||
[:uploads [:map-of ::sm/text ::sm/uuid]]])
|
||||
|
||||
(defn- prepare-font-data-from-uploads
|
||||
@ -139,7 +140,8 @@
|
||||
chunked-upload API)."
|
||||
{::doc/added "1.18"
|
||||
::doc/changes [["2.16" "Add :uploads param for chunked upload support"]
|
||||
["2.18" "Remove :data param, use :uploads exclusively"]]
|
||||
["2.18" "Remove :data param, use :uploads exclusively"]
|
||||
["2.20" "Add optional :variant-name param"]]
|
||||
::climit/id [[:process-font/by-profile ::rpc/profile-id]
|
||||
[:process-font/global]]
|
||||
::webhooks/event? true
|
||||
|
||||
@ -423,6 +423,8 @@
|
||||
(sm/lazy-validator schema:snapshots-per-file))
|
||||
|
||||
(def ^:private sql:get-snapshots-per-file
|
||||
;; A version is told apart by its label, and the partial index with the same
|
||||
;; predicate answers the count without touching the table.
|
||||
"SELECT count(*) AS total
|
||||
FROM file_change AS fc
|
||||
WHERE fc.file_id = ?
|
||||
@ -454,11 +456,16 @@
|
||||
(sm/lazy-validator schema:snapshots-per-team))
|
||||
|
||||
(def ^:private sql:get-snapshots-per-team
|
||||
;; The files of the team are resolved first, so the count filters by
|
||||
;; `file_id`: the planner can prune the partitions of `file_change` when it
|
||||
;; chooses an index scan (with a sequential scan it reads all of them). A
|
||||
;; version is told apart by its label.
|
||||
"SELECT count(*) AS total
|
||||
FROM file_change AS fc
|
||||
JOIN file AS f ON (f.id = fc.file_id)
|
||||
JOIN project AS p ON (p.id = f.project_id)
|
||||
WHERE p.team_id = ?
|
||||
WHERE fc.file_id = ANY (SELECT f.id
|
||||
FROM file AS f
|
||||
JOIN project AS p ON (p.id = f.project_id)
|
||||
WHERE p.team_id = ?)
|
||||
AND fc.created_by = 'user'
|
||||
AND fc.deleted_at IS NULL
|
||||
AND fc.label IS NOT NULL")
|
||||
|
||||
@ -39,6 +39,7 @@
|
||||
`[.data[\"pages-index\"][] | {id, name}]`, etc."
|
||||
(:require
|
||||
[app.binfile.common :as bfc]
|
||||
[app.common.exceptions :as ex]
|
||||
[app.common.json :as json]
|
||||
[app.db :as db]
|
||||
[app.features.file-snapshots :as fsnap]
|
||||
@ -183,16 +184,23 @@
|
||||
(defn get-snapshot-json
|
||||
"Return a snapshot as JSON (realized). For nREPL.
|
||||
|
||||
Takes the same `format` and `output` as `get-file-json`."
|
||||
Takes the same `format` and `output` as `get-file-json`. It raises when the
|
||||
file has no snapshot with that id."
|
||||
[system file-id snapshot-id & {:keys [format output] :or {format :full}}]
|
||||
(let [snapshot (db/run! system
|
||||
(fn [cfg]
|
||||
(fsnap/get-snapshot cfg file-id snapshot-id)))
|
||||
data (case format
|
||||
:full snapshot
|
||||
:data (:data snapshot)
|
||||
:summary (summarize-data (:data snapshot))
|
||||
:pages (:pages (summarize-data (:data snapshot))))]
|
||||
data (if snapshot
|
||||
(case format
|
||||
:full snapshot
|
||||
:data (:data snapshot)
|
||||
:summary (summarize-data (:data snapshot))
|
||||
:pages (:pages (summarize-data (:data snapshot))))
|
||||
(ex/raise :type :not-found
|
||||
:code :snapshot-not-found
|
||||
:hint "unable to find snapshot with the provided id"
|
||||
:snapshot-id snapshot-id
|
||||
:file-id file-id))]
|
||||
(write-output (data->json data) output)))
|
||||
|
||||
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
||||
|
||||
@ -53,7 +53,8 @@
|
||||
:deleted-at (ct/format-inst deleted-at))
|
||||
|
||||
(db/update! conn :file-change
|
||||
{:deleted-at deleted-at}
|
||||
{:deleted-at deleted-at
|
||||
:updated-at (ct/now)}
|
||||
{:id id :file-id file-id}
|
||||
{::db/return-keys false})
|
||||
|
||||
@ -93,7 +94,8 @@
|
||||
;; lifetime; the collector would still take it, just much later than it
|
||||
;; should.
|
||||
(db/update! conn :file-change
|
||||
{:deleted-at deleted-at}
|
||||
{:deleted-at deleted-at
|
||||
:updated-at (ct/now)}
|
||||
{:file-id id :deleted-at nil}
|
||||
{::db/return-keys false})
|
||||
|
||||
|
||||
@ -673,41 +673,48 @@
|
||||
{:fetch-size 5}))))))
|
||||
|
||||
(def ^:private sql:get-file-change
|
||||
"SELECT id FROM file_change
|
||||
"SELECT id, file_id FROM file_change
|
||||
WHERE deleted_at IS NOT NULL
|
||||
AND deleted_at <= ?
|
||||
ORDER BY deleted_at ASC")
|
||||
|
||||
(def ^:private sql:lock-file-change
|
||||
"SELECT id, file_id, deleted_at FROM file_change
|
||||
WHERE id = ANY(?::uuid[])
|
||||
"SELECT file_id, id, deleted_at FROM file_change
|
||||
WHERE file_id = ?
|
||||
AND id = ANY(?::uuid[])
|
||||
AND deleted_at IS NOT NULL
|
||||
AND deleted_at <= ?
|
||||
FOR UPDATE
|
||||
SKIP LOCKED")
|
||||
|
||||
(defn- delete-file-changes-chunk
|
||||
[cfg ids]
|
||||
[cfg rows]
|
||||
(try
|
||||
(db/tx-run! cfg
|
||||
(fn [{:keys [::db/conn ::threshold]}]
|
||||
(db/exec-one! conn ["SET LOCAL rules.deletion_protection TO off"])
|
||||
(let [ids (db/create-array conn "uuid" ids)]
|
||||
(->> (db/exec! conn [sql:lock-file-change ids threshold])
|
||||
(reduce (fn [total {:keys [id file-id deleted-at]}]
|
||||
(l/trc :obj "file-change"
|
||||
:id (str id)
|
||||
:file-id (str file-id)
|
||||
:deleted-at (ct/format-inst deleted-at))
|
||||
;; The rows are locked per file, with a constant `file_id`,
|
||||
;; so the planner prunes to the partition that holds each
|
||||
;; one instead of scanning all of them for a tuple `IN`.
|
||||
(->> (group-by :file-id rows)
|
||||
(mapcat (fn [[file-id file-rows]]
|
||||
(let [ids (db/create-array conn "uuid" (map :id file-rows))]
|
||||
(db/exec! conn [sql:lock-file-change file-id ids threshold]))))
|
||||
(reduce (fn [total {:keys [id file-id deleted-at]}]
|
||||
(l/trc :obj "file-change"
|
||||
:id (str id)
|
||||
:file-id (str file-id)
|
||||
:deleted-at (ct/format-inst deleted-at))
|
||||
|
||||
(let [affected (-> (db/delete! conn :file-change {:id id})
|
||||
(db/get-update-count))]
|
||||
(+ total affected)))
|
||||
0)))))
|
||||
(let [affected (-> (db/delete! conn :file-change
|
||||
{:id id :file-id file-id})
|
||||
(db/get-update-count))]
|
||||
(+ total affected)))
|
||||
0))))
|
||||
(catch Throwable cause
|
||||
(l/err :obj "file-change"
|
||||
:hint "error on chunk deletion, skipping it"
|
||||
:chunk ids
|
||||
:chunk rows
|
||||
:cause cause)
|
||||
0)))
|
||||
|
||||
@ -720,7 +727,10 @@
|
||||
(+ total deleted))))]
|
||||
(db/tx-run! (assoc cfg ::db/read-only true)
|
||||
(fn [{:keys [::db/conn]}]
|
||||
(transduce (comp (map :id) (partition-all chunk-size)) rf 0
|
||||
(transduce (comp (map (fn [row] {:id (:id row)
|
||||
:file-id (:file-id row)}))
|
||||
(partition-all chunk-size))
|
||||
rf 0
|
||||
(db/plan conn [sql:get-file-change threshold]
|
||||
{:fetch-size 5}))))))
|
||||
|
||||
|
||||
@ -3154,7 +3154,8 @@
|
||||
[[(str (:id file)) (str (:id file))]])
|
||||
[file-id :as result] (import-bundle profile evil)]
|
||||
(t/is (= 1 (count result)))
|
||||
(t/is (empty? (th/db-query :file-library-rel {:file-id file-id})))))
|
||||
(t/is (empty? (th/db-query :file-library-rel {:file-id file-id})))
|
||||
(t/is (empty? (th/db-query :file-library-sync {:file-id file-id})))))
|
||||
|
||||
(t/deftest import-v3-keeps-valid-library-relation
|
||||
(let [profile (th/create-profile* 1)
|
||||
@ -3173,6 +3174,251 @@
|
||||
(t/is (= 1 (count (th/db-query :file-library-rel
|
||||
{:file-id (:id new-file)})))))))
|
||||
|
||||
(t/deftest import-skips-circular-library-relations
|
||||
;; https://github.com/penpot/penpot/issues/12052: two libraries that use
|
||||
;; each other (possible before the 2.17.1 guard) export both relations in
|
||||
;; the manifest; the import must skip the one closing the cycle instead
|
||||
;; of recreating it. The cycle is built with direct inserts because the
|
||||
;; UI command `link-file-to-library` already rejects it. Relations are
|
||||
;; processed in sorted manifest order and the first edge wins.
|
||||
(let [profile (th/create-profile* 1)
|
||||
lib-a (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library A"})
|
||||
lib-b (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library B"})]
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-a)
|
||||
:library-file-id (:id lib-b)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-b)
|
||||
:library-file-id (:id lib-a)})
|
||||
|
||||
(let [output (tmp/tempfile :suffix ".zip")]
|
||||
(v3/export-files!
|
||||
(-> th/*system*
|
||||
(assoc ::bfc/file-ids #{(:id lib-a) (:id lib-b)})
|
||||
(assoc ::bfc/export-type :include-libraries)
|
||||
(assoc ::bfc/output (io/output-stream output))))
|
||||
|
||||
(let [result (import-bundle profile output)
|
||||
rels (mapcat #(db/query th/*system* :file-library-rel {:file-id %})
|
||||
result)
|
||||
syncs (mapcat #(db/query th/*system* :file-library-sync {:file-id %})
|
||||
result)]
|
||||
(t/is (= 2 (count result)))
|
||||
;; only one of the two cycle edges survives the import
|
||||
(t/is (= 1 (count rels)))
|
||||
;; the skipped edge leaves no sync row behind
|
||||
(t/is (= 1 (count syncs)))
|
||||
(t/is (= (set (map (juxt :file-id :library-file-id) syncs))
|
||||
(set (map (juxt :file-id :library-file-id) rels))))
|
||||
;; no imported file lists itself among its transitive libraries
|
||||
(doseq [file-id result]
|
||||
(t/is (not (contains? (bfc/get-libraries th/*system* [file-id])
|
||||
file-id))))))))
|
||||
|
||||
(t/deftest import-skips-transitive-cycle
|
||||
;; A three-node cycle (A -> B -> C -> A) only breaks when the check uses
|
||||
;; the transitive closure: a direct-edge-only check would keep all three
|
||||
;; edges. Built with direct inserts because the UI command rejects it.
|
||||
(let [profile (th/create-profile* 1)
|
||||
lib-a (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library A"})
|
||||
lib-b (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library B"})
|
||||
lib-c (th/create-file* 3 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library C"})]
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-a)
|
||||
:library-file-id (:id lib-b)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-b)
|
||||
:library-file-id (:id lib-c)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-c)
|
||||
:library-file-id (:id lib-a)})
|
||||
|
||||
(let [output (tmp/tempfile :suffix ".zip")]
|
||||
(v3/export-files!
|
||||
(-> th/*system*
|
||||
(assoc ::bfc/file-ids #{(:id lib-a) (:id lib-b) (:id lib-c)})
|
||||
(assoc ::bfc/export-type :include-libraries)
|
||||
(assoc ::bfc/output (io/output-stream output))))
|
||||
|
||||
(let [result (import-bundle profile output)
|
||||
rels (mapcat #(db/query th/*system* :file-library-rel {:file-id %})
|
||||
result)
|
||||
syncs (mapcat #(db/query th/*system* :file-library-sync {:file-id %})
|
||||
result)]
|
||||
(t/is (= 3 (count result)))
|
||||
;; only two of the three cycle edges survive the import
|
||||
(t/is (= 2 (count rels)))
|
||||
;; the skipped edge leaves no sync row behind
|
||||
(t/is (= 2 (count syncs)))
|
||||
(t/is (= (set (map (juxt :file-id :library-file-id) syncs))
|
||||
(set (map (juxt :file-id :library-file-id) rels))))
|
||||
;; no imported file lists itself among its transitive libraries
|
||||
(doseq [file-id result]
|
||||
(t/is (not (contains? (bfc/get-libraries th/*system* [file-id])
|
||||
file-id))))))))
|
||||
|
||||
(t/deftest import-keeps-acyclic-library-relations
|
||||
;; Control test: a chain without cycles must be imported untouched, so
|
||||
;; the new guard only skips edges that would close a cycle.
|
||||
(let [profile (th/create-profile* 1)
|
||||
lib-a (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library A"})
|
||||
lib-b (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library B"})
|
||||
lib-c (th/create-file* 3 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library C"})]
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-a)
|
||||
:library-file-id (:id lib-b)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-b)
|
||||
:library-file-id (:id lib-c)})
|
||||
|
||||
(let [output (tmp/tempfile :suffix ".zip")]
|
||||
(v3/export-files!
|
||||
(-> th/*system*
|
||||
(assoc ::bfc/file-ids #{(:id lib-a) (:id lib-b) (:id lib-c)})
|
||||
(assoc ::bfc/export-type :include-libraries)
|
||||
(assoc ::bfc/output (io/output-stream output))))
|
||||
|
||||
(let [result (import-bundle profile output)
|
||||
rels (mapcat #(db/query th/*system* :file-library-rel {:file-id %})
|
||||
result)
|
||||
syncs (mapcat #(db/query th/*system* :file-library-sync {:file-id %})
|
||||
result)]
|
||||
(t/is (= 3 (count result)))
|
||||
(t/is (= 2 (count rels)))
|
||||
;; every kept relation has its sync row
|
||||
(t/is (= 2 (count syncs)))
|
||||
(t/is (= (set (map (juxt :file-id :library-file-id) syncs))
|
||||
(set (map (juxt :file-id :library-file-id) rels))))))))
|
||||
|
||||
(t/deftest import-v1-skips-circular-library-relations
|
||||
;; Issue #12052 also names the version 1 importer: importing a package
|
||||
;; whose libraries use each other must not recreate the cycle. The cycle
|
||||
;; is built with direct inserts because `link-file-to-library` already
|
||||
;; rejects it. The exported file only *uses* one of the libraries, so the
|
||||
;; package carries the cycle the same way a real export does.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (prepare-simple-file profile)
|
||||
lib-a (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library A"})
|
||||
lib-b (th/create-file* 3 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library B"})]
|
||||
(th/link-file-to-library* {:file-id (:id file)
|
||||
:library-id (:id lib-a)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-a)
|
||||
:library-file-id (:id lib-b)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-b)
|
||||
:library-file-id (:id lib-a)})
|
||||
|
||||
(let [output (tmp/tempfile :suffix ".penpot")]
|
||||
(v1/export-files!
|
||||
(-> th/*system*
|
||||
(assoc ::bfc/file-ids #{(:id file)})
|
||||
(assoc ::bfc/include-libraries true))
|
||||
(io/output-stream output))
|
||||
|
||||
(let [result (:file-ids (-> th/*system*
|
||||
(assoc ::bfc/project-id (:default-project-id profile))
|
||||
(assoc ::bfc/profile-id (:id profile))
|
||||
(assoc ::bfc/input output)
|
||||
(v1/import-files!)))
|
||||
rels (mapcat #(db/query th/*system* :file-library-rel {:file-id %})
|
||||
result)
|
||||
syncs (mapcat #(db/query th/*system* :file-library-sync {:file-id %})
|
||||
result)]
|
||||
(t/is (= 3 (count result)))
|
||||
;; the relation of the file to its library, plus one of the two
|
||||
;; cycle edges; the edge that would close the cycle is skipped
|
||||
(t/is (= 2 (count rels)))
|
||||
;; the skipped edge leaves no sync row behind
|
||||
(t/is (= 2 (count syncs)))
|
||||
(t/is (= (set (map (juxt :file-id :library-file-id) syncs))
|
||||
(set (map (juxt :file-id :library-file-id) rels))))
|
||||
;; no imported file lists itself among its transitive libraries
|
||||
(doseq [file-id result]
|
||||
(t/is (not (contains? (bfc/get-libraries th/*system* [file-id])
|
||||
file-id))))))))
|
||||
|
||||
(t/deftest import-v1-tolerates-repeated-file-entries
|
||||
;; Old v1 exports could list the same file twice in the package metadata:
|
||||
;; the exporter unioned the requested files with their libraries without
|
||||
;; deduplicating, and a file inside a library cycle is one of its own
|
||||
;; libraries. The import must read every entry and import each file once.
|
||||
(let [profile (th/create-profile* 1)
|
||||
lib-a (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library A"})
|
||||
lib-b (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared true
|
||||
:name "Library B"})]
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-a)
|
||||
:library-file-id (:id lib-b)})
|
||||
(db/insert! th/*system* :file-library-rel
|
||||
{:file-id (:id lib-b)
|
||||
:library-file-id (:id lib-a)})
|
||||
|
||||
(let [output (tmp/tempfile :suffix ".penpot")]
|
||||
;; exporting only lib-a repeats its id in the metadata, because it is
|
||||
;; among its own transitive libraries
|
||||
(v1/export-files!
|
||||
(-> th/*system*
|
||||
(assoc ::bfc/file-ids #{(:id lib-a)})
|
||||
(assoc ::bfc/include-libraries true))
|
||||
(io/output-stream output))
|
||||
|
||||
(let [result (:file-ids (-> th/*system*
|
||||
(assoc ::bfc/project-id (:default-project-id profile))
|
||||
(assoc ::bfc/profile-id (:id profile))
|
||||
(assoc ::bfc/input output)
|
||||
(v1/import-files!)))
|
||||
rels (mapcat #(db/query th/*system* :file-library-rel {:file-id %})
|
||||
result)
|
||||
syncs (mapcat #(db/query th/*system* :file-library-sync {:file-id %})
|
||||
result)]
|
||||
;; two files: the repeated entry is imported once
|
||||
(t/is (= 2 (count result)))
|
||||
;; one of the two cycle edges survives the import
|
||||
(t/is (= 1 (count rels)))
|
||||
(t/is (= 1 (count syncs)))
|
||||
(t/is (= (set (map (juxt :file-id :library-file-id) syncs))
|
||||
(set (map (juxt :file-id :library-file-id) rels))))
|
||||
;; no imported file lists itself among its transitive libraries
|
||||
(doseq [file-id result]
|
||||
(t/is (not (contains? (bfc/get-libraries th/*system* [file-id])
|
||||
file-id))))))))
|
||||
|
||||
(t/deftest export-multi-file-includes-package-relations
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (prepare-simple-file profile)
|
||||
|
||||
@ -445,10 +445,9 @@
|
||||
(t/is (thrown? clojure.lang.ExceptionInfo
|
||||
(invoke! cfg (fn [_ _] :never) {}))))
|
||||
|
||||
;; the count came back down instead of staying up for good. Only the
|
||||
;; last publication is asserted, not the exact list: the sampler
|
||||
;; republishes the same count when its first tick lands inside the
|
||||
;; window.
|
||||
;; the count came back down instead of staying up for good. The
|
||||
;; republisher daemon thread can add publications inside the window,
|
||||
;; so what is asserted is the last value, not the whole sequence.
|
||||
(t/is (= 0 (waiting-count climit name)))
|
||||
(t/is (= 0 (waiting-last name)))
|
||||
|
||||
|
||||
@ -80,3 +80,11 @@
|
||||
|
||||
(t/deftest uuid->hash-code-stable-for-zero-uuid
|
||||
(t/is (= 3659997967308761462 (db/uuid->hash-code uuid/zero))))
|
||||
|
||||
(t/deftest pool-enables-the-partitionwise-join
|
||||
;; The pool sets the GUC on every connection, so the planner can resolve a
|
||||
;; join between two partitioned tables pairwise, partition by partition.
|
||||
(let [row (db/exec-one! th/*pool*
|
||||
["SELECT current_setting('enable_partitionwise_join') AS partitionwise_join"])]
|
||||
|
||||
(t/is (= "on" (:partitionwise-join row)))))
|
||||
|
||||
42
backend/test/backend_tests/fdata_resolve_test.clj
Normal file
42
backend/test/backend_tests/fdata_resolve_test.clj
Normal file
@ -0,0 +1,42 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS INC Sucursal en España SL
|
||||
|
||||
(ns backend-tests.fdata-resolve-test
|
||||
"Resolution of file data by backend (`app.features.fdata/resolve-file-data`).
|
||||
|
||||
A row without a `file_data` twin carries no backend: the read queries mark
|
||||
it with `coalesce(fd.backend, 'legacy-db')` and hand its blob over as
|
||||
`:legacy-data`. The `legacy-db` backend is still supported, so that case
|
||||
has to resolve to the blob instead of nil."
|
||||
(:require
|
||||
[app.features.fdata :as fdata]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/deftest legacy-db-resolves-the-legacy-blob
|
||||
(let [blob (vec (.getBytes "legacy-blob" "UTF-8"))
|
||||
file {:backend "legacy-db"
|
||||
:legacy-data (byte-array blob)}
|
||||
out (fdata/resolve-file-data nil file)]
|
||||
(t/is (= blob (vec (:data out))))
|
||||
(t/is (not (contains? out :legacy-data)))))
|
||||
|
||||
(t/deftest missing-backend-defaults-to-legacy-db
|
||||
;; The queries leave `:backend` nil when the `LEFT JOIN` with `file_data`
|
||||
;; finds no twin; the dispatch default keeps that row on `legacy-db`.
|
||||
(let [blob (vec (.getBytes "legacy-blob" "UTF-8"))
|
||||
file {:legacy-data (byte-array blob)}
|
||||
out (fdata/resolve-file-data nil file)]
|
||||
(t/is (= blob (vec (:data out))))
|
||||
(t/is (not (contains? out :legacy-data)))))
|
||||
|
||||
(t/deftest db-backend-drops-the-legacy-key
|
||||
(let [data (vec (.getBytes "db-blob" "UTF-8"))
|
||||
file {:backend "db"
|
||||
:data (byte-array data)
|
||||
:legacy-data (byte-array (vec (.getBytes "stale" "UTF-8")))}
|
||||
out (fdata/resolve-file-data nil file)]
|
||||
(t/is (= data (vec (:data out))))
|
||||
(t/is (not (contains? out :legacy-data)))))
|
||||
55
backend/test/backend_tests/fdata_upsert_test.clj
Normal file
55
backend/test/backend_tests/fdata_upsert_test.clj
Normal file
@ -0,0 +1,55 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns backend-tests.fdata-upsert-test
|
||||
"Covers the invariants of `fdata/upsert!` that the partition runbook relies
|
||||
on."
|
||||
|
||||
(:require
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.db :as db]
|
||||
[app.features.fdata :as fdata]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[backend-tests.helpers :as th]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :once th/state-init)
|
||||
(t/use-fixtures :each th/database-reset)
|
||||
|
||||
(t/deftest legacy-db-snapshot-moves-the-updated-at-watermark
|
||||
;; The runbook of the partition picks the rows changed since the bulk copy
|
||||
;; with `updated_at`, so every mutation of `file_change` has to move it. The
|
||||
;; `legacy-db` snapshot path writes the payload into the row, and it used to
|
||||
;; leave the watermark behind.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})
|
||||
id (uuid/next)
|
||||
past (ct/in-past "2h")
|
||||
payload (byte-array [1 2 3])]
|
||||
|
||||
(th/db-insert! :file-change
|
||||
{:id id
|
||||
:file-id (:id file)
|
||||
:revn 1
|
||||
:label "v1"
|
||||
:kind fchanges/snapshot-kind
|
||||
:updated-at past})
|
||||
|
||||
(t/is (true? (fdata/upsert! th/*system*
|
||||
{:id id
|
||||
:file-id (:id file)
|
||||
:type "snapshot"
|
||||
:backend "legacy-db"
|
||||
:data payload})))
|
||||
|
||||
(let [row (th/db-get :file-change {:id id} {::db/remove-deleted false})]
|
||||
(t/testing "the payload lands in the legacy column"
|
||||
(t/is (= (seq payload) (seq ^bytes (:data row)))))
|
||||
|
||||
(t/testing "the watermark moves to the moment of the mutation"
|
||||
(t/is (pos? (compare (inst-ms (:updated-at row)) (inst-ms past))))))))
|
||||
@ -10,7 +10,7 @@
|
||||
A file has two kinds of row. Durable history: the snapshots, which a restore
|
||||
brings back. And ephemeral revisions: the xlog of an edit, born already
|
||||
scheduled for deletion one hour later, which nobody needs once the sessions
|
||||
that were going to read them are gone.
|
||||
that were going to read them are gone. The `kind` column tells them apart.
|
||||
|
||||
The invariant under test is that a deletion date is assigned once and never
|
||||
moved: restoring a file must not unmark a revision, and deleting a file must
|
||||
@ -21,6 +21,7 @@
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.db :as db]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.rpc :as-alias rpc]
|
||||
[app.rpc.commands.files :as files]
|
||||
[app.util.blob :as blob]
|
||||
@ -64,6 +65,7 @@
|
||||
:version 0
|
||||
:created-by (or created-by "system")
|
||||
:label label
|
||||
:kind (if label fchanges/snapshot-kind fchanges/revision-kind)
|
||||
:data data
|
||||
:changes changes}
|
||||
{::db/return-keys false})
|
||||
@ -170,6 +172,7 @@
|
||||
|
||||
out (th/command! {::th/type :update-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id file-id
|
||||
:id revision-id
|
||||
:label "renamed"})]
|
||||
|
||||
|
||||
110
backend/test/backend_tests/file_change_schema_test.clj
Normal file
110
backend/test/backend_tests/file_change_schema_test.clj
Normal file
@ -0,0 +1,110 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS INC Sucursal en España SL
|
||||
|
||||
(ns backend-tests.file-change-schema-test
|
||||
(:require
|
||||
[app.db :as db]
|
||||
[backend-tests.helpers :as th]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :once th/state-init)
|
||||
(t/use-fixtures :each th/database-reset)
|
||||
|
||||
(defn- query
|
||||
"Runs a query that returns a single value and returns it."
|
||||
[sql & params]
|
||||
(-> (db/exec! th/*pool* (into [sql] params))
|
||||
first
|
||||
(vals)
|
||||
first))
|
||||
|
||||
(defn- partition-names
|
||||
[]
|
||||
(->> (db/exec! th/*pool* ["SELECT c.relname AS name
|
||||
FROM pg_class c
|
||||
JOIN pg_partition_tree('file_change') t ON t.relid = c.oid
|
||||
WHERE t.isleaf
|
||||
ORDER BY c.relname"])
|
||||
(mapv :name)))
|
||||
|
||||
(t/deftest file-change-is-partitioned-in-sixteen-named-partitions
|
||||
(t/is (= (mapv #(format "file_change_%02d" %) (range 16))
|
||||
(partition-names))))
|
||||
|
||||
(t/deftest file-change-primary-key-is-file-id-and-id
|
||||
(t/is (= "PRIMARY KEY (file_id, id)"
|
||||
(query "SELECT pg_get_constraintdef(oid) AS def
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'file_change'::regclass AND contype = 'p'"))))
|
||||
|
||||
(t/deftest file-change-has-three-foreign-keys-and-one-check
|
||||
(t/is (= 3 (query "SELECT count(*) AS total
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'file_change'::regclass AND contype = 'f'")))
|
||||
(t/is (= 1 (query "SELECT count(*) AS total
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'file_change'::regclass AND contype = 'c'")))
|
||||
(t/is (re-find #"kind"
|
||||
(query "SELECT pg_get_constraintdef(oid) AS def
|
||||
FROM pg_constraint
|
||||
WHERE conrelid = 'file_change'::regclass AND contype = 'c'"))))
|
||||
|
||||
(t/deftest file-change-has-the-six-canonical-indexes
|
||||
(t/is (= ["file_change__deleted_at__idx"
|
||||
"file_change__file_id__revn__idx"
|
||||
"file_change__locked_by__idx"
|
||||
"file_change__profile_id__idx"
|
||||
"file_change__user_snapshots__idx"
|
||||
"file_change_pkey"]
|
||||
(->> (db/exec! th/*pool* ["SELECT indexname AS name
|
||||
FROM pg_indexes
|
||||
WHERE tablename = 'file_change'
|
||||
AND schemaname = current_schema()
|
||||
ORDER BY indexname"])
|
||||
(mapv :name)))))
|
||||
|
||||
(t/deftest file-change-keeps-the-data-column-and-a-kind-column
|
||||
;; The `data` column is kept: the `legacy-db` backend reads old payloads
|
||||
;; from it (migration 0157 moved the existing blobs). The dead
|
||||
;; `data_ref_id`/`data_backend` columns stay gone: no code read them since
|
||||
;; the file storage refactor.
|
||||
(t/is (= []
|
||||
(->> (db/exec! th/*pool* ["SELECT attname AS name
|
||||
FROM pg_attribute
|
||||
WHERE attrelid = 'file_change'::regclass
|
||||
AND attnum > 0 AND NOT attisdropped
|
||||
AND attname IN ('data_ref_id', 'data_backend')"])
|
||||
(mapv :name))))
|
||||
(t/is (= ["bytea"]
|
||||
(->> (db/exec! th/*pool* ["SELECT pg_catalog.format_type(atttypid, atttypmod) AS type
|
||||
FROM pg_attribute
|
||||
WHERE attrelid = 'file_change'::regclass
|
||||
AND attname = 'data'"])
|
||||
(mapv :type))))
|
||||
(t/is (true? (query "SELECT attnotnull AS notnull
|
||||
FROM pg_attribute
|
||||
WHERE attrelid = 'file_change'::regclass
|
||||
AND attname = 'kind'"))))
|
||||
|
||||
(t/deftest inserting-a-row-without-a-kind-fails
|
||||
;; The column has no default on purpose: an insert that forgets it has to
|
||||
;; fail instead of storing the wrong kind silently.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})]
|
||||
(t/is (thrown? java.sql.SQLException
|
||||
(db/exec! th/*pool* ["INSERT INTO file_change (file_id, id)
|
||||
VALUES (?, gen_random_uuid())"
|
||||
(:id file)])))))
|
||||
|
||||
(t/deftest inserting-a-row-with-an-unknown-kind-fails
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})]
|
||||
(t/is (thrown? java.sql.SQLException
|
||||
(db/exec! th/*pool* ["INSERT INTO file_change (file_id, id, kind)
|
||||
VALUES (?, gen_random_uuid(), 3)"
|
||||
(:id file)])))))
|
||||
@ -13,6 +13,7 @@
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.db :as db]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.util.blob :as blob]
|
||||
[backend-tests.helpers :as th]
|
||||
[clojure.java.io :as io]
|
||||
@ -58,6 +59,7 @@
|
||||
:revn 1
|
||||
:data snap-data
|
||||
:label "v1"
|
||||
:kind fchanges/snapshot-kind
|
||||
:created-at created-at
|
||||
:updated-at updated-at
|
||||
:deleted-at deleted-at})
|
||||
@ -69,7 +71,8 @@
|
||||
{:id rev-id
|
||||
:file-id fid
|
||||
:revn 2
|
||||
:data rev-data})
|
||||
:data rev-data
|
||||
:kind fchanges/revision-kind})
|
||||
|
||||
;; Live revision (changes, no data, no label): untouched.
|
||||
live-id (uuid/next)
|
||||
@ -77,7 +80,8 @@
|
||||
{:id live-id
|
||||
:file-id fid
|
||||
:revn 3
|
||||
:changes (blob/encode [{:type :dummy}])})]
|
||||
:changes (blob/encode [{:type :dummy}])
|
||||
:kind fchanges/revision-kind})]
|
||||
|
||||
(run-migration)
|
||||
|
||||
@ -168,7 +172,8 @@
|
||||
:file-id fid
|
||||
:revn 1
|
||||
:label "v1"
|
||||
:data stale})]
|
||||
:data stale
|
||||
:kind fchanges/snapshot-kind})]
|
||||
|
||||
(run-migration)
|
||||
|
||||
|
||||
@ -0,0 +1,210 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns backend-tests.migrations-partition-file-change-test
|
||||
"Covers the data copy of 0158-partition-file-change: the rows of the old
|
||||
table move to the partitioned one with their columns in place, the `kind`
|
||||
comes from the label (1 snapshot, 2 revision), and the legacy table keeps
|
||||
the rows."
|
||||
|
||||
(:require
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.db :as db]
|
||||
[app.util.blob :as blob]
|
||||
[backend-tests.helpers :as th]
|
||||
[clojure.java.io :as io]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :once th/state-init)
|
||||
(t/use-fixtures :each th/database-reset)
|
||||
|
||||
(def ^:private sql:old-schema
|
||||
"CREATE TABLE file (id uuid PRIMARY KEY);
|
||||
CREATE TABLE profile (id uuid PRIMARY KEY);
|
||||
|
||||
CREATE TABLE file_change (
|
||||
file_id uuid NOT NULL,
|
||||
id uuid NOT NULL DEFAULT gen_random_uuid(),
|
||||
created_at timestamptz NOT NULL DEFAULT now(),
|
||||
session_id uuid,
|
||||
revn bigint NOT NULL DEFAULT 0,
|
||||
data bytea,
|
||||
changes bytea,
|
||||
profile_id uuid,
|
||||
features text[],
|
||||
label text,
|
||||
version integer,
|
||||
created_by text NOT NULL DEFAULT 'system',
|
||||
updated_at timestamptz DEFAULT now(),
|
||||
deleted_at timestamptz,
|
||||
migrations text[],
|
||||
locked_by uuid,
|
||||
data_ref_id uuid,
|
||||
data_backend text,
|
||||
CONSTRAINT file_change_pkey PRIMARY KEY (id),
|
||||
CONSTRAINT file_change_file_id_fkey FOREIGN KEY (file_id)
|
||||
REFERENCES file(id) DEFERRABLE,
|
||||
CONSTRAINT file_change_locked_by_fkey FOREIGN KEY (locked_by)
|
||||
REFERENCES profile(id) ON DELETE SET NULL DEFERRABLE,
|
||||
CONSTRAINT file_change_profile_id_fkey FOREIGN KEY (profile_id)
|
||||
REFERENCES profile(id) ON DELETE SET NULL DEFERRABLE);
|
||||
|
||||
CREATE INDEX file_change__data_ref_id__idx
|
||||
ON file_change (data_ref_id);
|
||||
CREATE INDEX file_change__deleted_at__idx
|
||||
ON file_change (deleted_at) WHERE deleted_at IS NOT NULL;
|
||||
CREATE INDEX file_change__file_id__revn__idx
|
||||
ON file_change (file_id, revn);
|
||||
CREATE INDEX file_change__locked_by__idx
|
||||
ON file_change (locked_by) WHERE locked_by IS NOT NULL;
|
||||
CREATE INDEX file_change__profile_id__idx
|
||||
ON file_change (profile_id) WHERE profile_id IS NOT NULL;
|
||||
CREATE INDEX file_change__system_snapshots__idx
|
||||
ON file_change (file_id, created_at)
|
||||
WHERE created_by = 'system' AND deleted_at IS NULL;
|
||||
CREATE INDEX file_change__user_snapshots__idx
|
||||
ON file_change (file_id)
|
||||
WHERE label IS NOT NULL AND created_by = 'user' AND deleted_at IS NULL;")
|
||||
|
||||
(defn- run-migration!
|
||||
[conn]
|
||||
(let [sql (slurp (io/resource "app/migrations/sql/0158-partition-file-change.sql"))]
|
||||
(db/exec! conn [sql])))
|
||||
|
||||
(t/deftest copies-the-rows-into-the-partitioned-table
|
||||
;; The migration runs in its own schema, with the shape of the table before
|
||||
;; it: the run below is the only place where its INSERT..SELECT runs with
|
||||
;; rows, so a swapped column or a wrong `kind` fails here and not on a real
|
||||
;; instance.
|
||||
(let [schema (str "test_0158_" (subs (str (uuid/next)) 0 8))
|
||||
fid (uuid/next)
|
||||
snap-id (uuid/next)
|
||||
rev-id (uuid/next)
|
||||
created-at (ct/in-past "2h")
|
||||
updated-at (ct/in-past "1h")
|
||||
deleted-at (ct/in-past "30m")
|
||||
snap-data (blob/encode {:snapshot 1})
|
||||
rev-changes (blob/encode [{:type :add-obj :id (uuid/next)}])]
|
||||
|
||||
(db/tx-run! th/*pool*
|
||||
(fn [{:keys [::db/conn]}]
|
||||
;; the schema is dropped before the commit, and a failure
|
||||
;; rolls the whole transaction back: nothing is left behind
|
||||
(db/exec! conn [(str "CREATE SCHEMA " schema)])
|
||||
(db/exec! conn [(str "SET LOCAL search_path TO " schema)])
|
||||
(db/exec! conn [sql:old-schema])
|
||||
|
||||
(db/insert! conn :file {:id fid})
|
||||
|
||||
(db/insert! conn :file-change
|
||||
{:id snap-id
|
||||
:file-id fid
|
||||
:revn 1
|
||||
:data snap-data
|
||||
:label "v1"
|
||||
:version 1
|
||||
:created-by "user"
|
||||
:created-at created-at
|
||||
:updated-at updated-at
|
||||
:deleted-at deleted-at
|
||||
;; a row that still carries the dead columns
|
||||
:data-ref-id (uuid/next)
|
||||
:data-backend "legacy-db"})
|
||||
|
||||
(db/insert! conn :file-change
|
||||
{:id rev-id
|
||||
:file-id fid
|
||||
:revn 2
|
||||
:changes rev-changes
|
||||
:created-at created-at
|
||||
:updated-at updated-at})
|
||||
|
||||
(run-migration! conn)
|
||||
|
||||
(let [rows (db/exec! conn ["SELECT id, revn, kind, label, data, changes,
|
||||
created_at, updated_at, deleted_at, created_by
|
||||
FROM file_change
|
||||
ORDER BY revn"])
|
||||
snapshot (first rows)
|
||||
revision (second rows)
|
||||
legacy (db/exec! conn ["SELECT id, revn, data, changes
|
||||
FROM file_change_legacy
|
||||
ORDER BY revn"])]
|
||||
|
||||
(t/testing "both rows are in the partitioned table"
|
||||
(t/is (= 2 (count rows)))
|
||||
(t/is (= [1 2] (mapv :revn rows))))
|
||||
|
||||
(t/testing "the labelled row is a snapshot with its payload"
|
||||
(t/is (= 1 (:kind snapshot)))
|
||||
(t/is (= "v1" (:label snapshot)))
|
||||
(t/is (= "user" (:created-by snapshot)))
|
||||
(t/is (= (seq snap-data) (seq ^bytes (:data snapshot))))
|
||||
(t/is (nil? (:changes snapshot)))
|
||||
(t/is (= (inst-ms created-at) (inst-ms (:created-at snapshot))))
|
||||
(t/is (= (inst-ms updated-at) (inst-ms (:updated-at snapshot))))
|
||||
(t/is (= (inst-ms deleted-at) (inst-ms (:deleted-at snapshot)))))
|
||||
|
||||
(t/testing "the unlabelled row is a revision with its changes"
|
||||
(t/is (= 2 (:kind revision)))
|
||||
(t/is (nil? (:label revision)))
|
||||
(t/is (= (seq rev-changes) (seq ^bytes (:changes revision))))
|
||||
(t/is (nil? (:data revision)))
|
||||
(t/is (= (inst-ms created-at) (inst-ms (:created-at revision)))))
|
||||
|
||||
(t/testing "the legacy table keeps the rows for the rollback"
|
||||
(t/is (= 2 (count legacy)))
|
||||
(t/is (= [snap-id rev-id] (mapv :id legacy)))
|
||||
(t/is (= (seq snap-data)
|
||||
(seq ^bytes (:data (first legacy)))))
|
||||
(t/is (= (seq rev-changes)
|
||||
(seq ^bytes (:changes (second legacy)))))))
|
||||
|
||||
(t/testing "the legacy table no longer points at file"
|
||||
;; Its foreign key on `file_id` is `NO ACTION`: while the table has
|
||||
;; rows, the objects GC cannot delete the files they point to. The
|
||||
;; migration drops that one and keeps the two `ON DELETE SET NULL` ones.
|
||||
(t/is (= ["file_change_locked_by_fkey_legacy"
|
||||
"file_change_profile_id_fkey_legacy"]
|
||||
(->> (db/exec! conn ["SELECT con.conname AS name
|
||||
FROM pg_constraint con
|
||||
WHERE con.conrelid = 'file_change_legacy'::regclass
|
||||
AND con.contype = 'f'
|
||||
ORDER BY con.conname"])
|
||||
(mapv :name)))))
|
||||
|
||||
(t/testing "the dead columns are not carried over"
|
||||
;; The old row carried `data_ref_id` and `data_backend`;
|
||||
;; the new table has no such columns.
|
||||
(t/is (= []
|
||||
(->> (db/exec! conn ["SELECT attname AS name
|
||||
FROM pg_attribute
|
||||
WHERE attrelid = 'file_change'::regclass
|
||||
AND attnum > 0 AND NOT attisdropped
|
||||
AND attname IN ('data_ref_id', 'data_backend')"])
|
||||
(mapv :name)))))
|
||||
|
||||
(db/exec! conn [(str "DROP SCHEMA " schema " CASCADE")])))))
|
||||
|
||||
(t/deftest drops-the-legacy-table-when-it-is-empty
|
||||
;; On a new instance there is nothing to keep: the migration leaves no
|
||||
;; legacy table behind. The relation is asserted on its own, so a missing
|
||||
;; table can never pass for a clean one.
|
||||
(let [schema (str "test_0158_empty_" (subs (str (uuid/next)) 0 8))]
|
||||
(db/tx-run! th/*pool*
|
||||
(fn [{:keys [::db/conn]}]
|
||||
(db/exec! conn [(str "CREATE SCHEMA " schema)])
|
||||
(db/exec! conn [(str "SET LOCAL search_path TO " schema)])
|
||||
(db/exec! conn [sql:old-schema])
|
||||
|
||||
(run-migration! conn)
|
||||
|
||||
(t/is (-> (db/exec-one! conn ["SELECT to_regclass('file_change_legacy') IS NULL AS empty"])
|
||||
:empty)
|
||||
"the empty legacy table is dropped")
|
||||
|
||||
(db/exec! conn [(str "DROP SCHEMA " schema " CASCADE")])))))
|
||||
@ -8,8 +8,10 @@
|
||||
(:require
|
||||
[app.common.logging :as l]
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.config :as cf]
|
||||
[app.db :as db]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.rpc :as-alias rpc]
|
||||
[app.storage :as sto]
|
||||
[app.tasks.objects-gc :as ogc]
|
||||
@ -73,6 +75,69 @@
|
||||
(t/is (pos? (:processed result)))
|
||||
(t/is (nil? (find-file (:id file))))))
|
||||
|
||||
(t/deftest deletes-the-pending-file-changes
|
||||
;; The pending file changes are locked per file, with a constant `file_id`,
|
||||
;; so the rows of a file that is pending deletion are gone after the run.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})
|
||||
|
||||
_ (th/db-insert! :file-change
|
||||
{:file-id (:id file)
|
||||
:kind fchanges/revision-kind
|
||||
:deleted-at (ct/in-past "10m")})
|
||||
|
||||
result (th/run-task! :objects-gc {})]
|
||||
|
||||
(t/is (pos? (:processed result)))
|
||||
(t/is (nil? (th/db-get :file-change {:file-id (:id file)}
|
||||
{::db/remove-deleted false})))))
|
||||
|
||||
(t/deftest deletes-the-pending-file-changes-of-several-chunks
|
||||
;; More pending rows than the chunk size: the plan walks the cursor and each
|
||||
;; chunk locks its own rows, one query per file, with a constant `file_id`
|
||||
;; so the planner prunes the partition.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})
|
||||
ids (repeatedly 5 uuid/next)
|
||||
_ (doseq [id ids]
|
||||
(th/db-insert! :file-change
|
||||
{:id id
|
||||
:file-id (:id file)
|
||||
:kind fchanges/revision-kind
|
||||
:deleted-at (ct/in-past "10m")}))
|
||||
result (th/run-task! :objects-gc {:chunk-size 2})]
|
||||
|
||||
(t/is (pos? (:processed result)))
|
||||
(t/is (empty? (th/db-query :file-change {:file-id (:id file)}
|
||||
{::db/remove-deleted false})))))
|
||||
|
||||
(t/deftest a-chunk-spanning-several-files-deletes-all-of-them
|
||||
;; The rows of one chunk can belong to different files. Each group is locked
|
||||
;; with its own `file_id`, and every row is still deleted; buffering the
|
||||
;; groups instead of locking them would leave them behind.
|
||||
(let [profile (th/create-profile* 1)
|
||||
opts {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)}
|
||||
files [(th/create-file* 1 opts)
|
||||
(th/create-file* 2 opts)]
|
||||
|
||||
_ (doseq [file files]
|
||||
(dotimes [_ 2]
|
||||
(th/db-insert! :file-change
|
||||
{:file-id (:id file)
|
||||
:kind fchanges/revision-kind
|
||||
:deleted-at (ct/in-past "10m")})))
|
||||
|
||||
;; one chunk holds the rows of both files
|
||||
result (th/run-task! :objects-gc {:chunk-size 4})]
|
||||
|
||||
(t/is (pos? (:processed result)))
|
||||
(doseq [file files]
|
||||
(t/is (empty? (th/db-query :file-change {:file-id (:id file)}
|
||||
{::db/remove-deleted false}))))))
|
||||
|
||||
(t/deftest profile-cascade-keeps-profile-deletion-date
|
||||
;; The cascade marks the owned rows with the deletion date of the profile,
|
||||
;; like the RPC delete-profile flow does, not with the timestamp of the run.
|
||||
|
||||
@ -9,6 +9,7 @@
|
||||
[app.common.time :as ct]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.db :as db]
|
||||
[app.features.file-changes :as fchanges]
|
||||
[app.rpc.commands.files :as files]
|
||||
[app.srepl.main :as srepl]
|
||||
[app.storage :as sto]
|
||||
@ -130,3 +131,29 @@
|
||||
(srepl/restore-team! (:id team)))
|
||||
|
||||
(t/is (nil? (:deleted-at (find-object sobject))))))
|
||||
|
||||
(t/deftest restoring-a-file-updates-its-file-changes
|
||||
;; Restoring a file brings its file changes back to life, and that mutation
|
||||
;; has to be observable through `updated_at`, the watermark used by the
|
||||
;; online partitioning copy of `file_change`.
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})
|
||||
|
||||
stale-at #inst "2020-01-01T00:00:00.000Z"
|
||||
|
||||
_ (th/db-insert! :file-change
|
||||
{:file-id (:id file)
|
||||
:label "v1"
|
||||
:kind fchanges/snapshot-kind
|
||||
:deleted-at (ct/in-past {:days 1})
|
||||
:updated-at stale-at})]
|
||||
|
||||
(restore-file! (:id file))
|
||||
|
||||
(let [change (th/db-get :file-change {:file-id (:id file)}
|
||||
{::db/remove-deleted false})]
|
||||
(t/is (nil? (:deleted-at change)))
|
||||
(t/is (pos? (- (inst-ms (:updated-at change))
|
||||
(inst-ms stale-at)))
|
||||
"updated_at must be bumped by the restore"))))
|
||||
|
||||
@ -88,6 +88,19 @@
|
||||
(t/is (= 0 (:revn row)))
|
||||
(t/is (= (:id profile) (:profile-id row))))))
|
||||
|
||||
(t/testing "update snapshot with file-id"
|
||||
(let [params {::th/type :update-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:id @snapshot-id
|
||||
:label "label1-updated"}
|
||||
out (th/command! params)]
|
||||
;; (th/print-result! out)
|
||||
(t/is (nil? (:error out)))
|
||||
(t/is (true? (:result out)))
|
||||
(t/is (= "label1-updated"
|
||||
(:label (th/db-get :file-change {:id @snapshot-id}))))))
|
||||
|
||||
(t/testing "restore snapshot"
|
||||
(let [params {::th/type :restore-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
@ -102,9 +115,10 @@
|
||||
;; (th/print-result! out)
|
||||
(t/is (nil? (:error out)))
|
||||
(t/is (nil? (:result out)))
|
||||
;; Peers learn who restored which version
|
||||
;; Peers learn who restored which version, named by the label the
|
||||
;; snapshot has now (the previous block renamed it)
|
||||
(t/is (= (:id profile) (:profile-id sent)))
|
||||
(t/is (= "label1" (:label sent)))
|
||||
(t/is (= "label1-updated" (:label sent)))
|
||||
(t/is (ct/inst? (:created-at sent))))
|
||||
|
||||
(let [[row1 row2 :as rows]
|
||||
@ -748,3 +762,231 @@
|
||||
(let [stored (fsnap/get-snapshot th/*system* file-id snap-id)]
|
||||
(t/is (nil? (:migrations stored)))
|
||||
(t/is (= 60 (:version stored)))))))
|
||||
(t/deftest snapshots-require-file-id
|
||||
;; Since version 2.20 the snapshot commands require the `file-id`
|
||||
;; parameter, so partition pruning can always be applied on the
|
||||
;; `file_change` table.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
team-id (:default-team-id profile)
|
||||
proj-id (:default-project-id profile)
|
||||
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id proj-id
|
||||
:is-shared false})
|
||||
|
||||
snapshot
|
||||
(let [params {::th/type :create-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:label "label1"}
|
||||
out (th/command! params)]
|
||||
;; (th/print-result! out)
|
||||
|
||||
(t/is (nil? (:error out)))
|
||||
(:result out))]
|
||||
|
||||
(doseq [type [:update-file-snapshot :lock-file-snapshot
|
||||
:unlock-file-snapshot :delete-file-snapshot]]
|
||||
(let [params (merge {::th/type type
|
||||
::rpc/profile-id (:id profile)
|
||||
:id (:id snapshot)}
|
||||
(when (= type :update-file-snapshot)
|
||||
{:label "label2"}))
|
||||
out (th/command! params)]
|
||||
|
||||
(t/testing (str (name type) " without file-id")
|
||||
(t/is (th/ex-of-code? (:error out) :params-validation)))))))
|
||||
|
||||
(t/deftest snapshots-reject-a-file-id-that-does-not-match
|
||||
;; The snapshot commands filter by `file_id` so that, once `file_change` is
|
||||
;; hash-partitioned by it, the lookup prunes to a single partition. Sending
|
||||
;; a `file-id` of another file must not find nor touch the snapshot.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
file-a (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
file-b (th/create-file* 2 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
|
||||
snapshot (-> (th/command! {::th/type :create-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file-a)
|
||||
:label "label1"})
|
||||
:result)]
|
||||
|
||||
(doseq [type [:update-file-snapshot :lock-file-snapshot
|
||||
:unlock-file-snapshot :delete-file-snapshot]]
|
||||
(let [params (merge {::th/type type
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file-b)
|
||||
:id (:id snapshot)}
|
||||
(when (= type :update-file-snapshot)
|
||||
{:label "label2"}))
|
||||
out (th/command! params)]
|
||||
|
||||
(t/testing (name type)
|
||||
(t/is (th/ex-info? (:error out)))
|
||||
(t/is (= :not-found (:type (ex-data (:error out)))))
|
||||
(t/is (= :snapshot-not-found (:code (ex-data (:error out)))))
|
||||
(t/is (= "label1"
|
||||
(:label (th/db-get :file-change {:id (:id snapshot)}
|
||||
{::db/remove-deleted false})))
|
||||
"the snapshot of file-a stays intact"))))))
|
||||
|
||||
(t/deftest file-change-mutations-maintain-updated-at
|
||||
;; `updated_at` is the per-row watermark used by the online partitioning
|
||||
;; copy of `file_change` (hash partitioning by file_id): every mutation
|
||||
;; must bump it, otherwise the copy misses the row. Lock, unlock and the
|
||||
;; soft-delete used to leave it stale.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
|
||||
snapshot
|
||||
(let [params {::th/type :create-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:label "label1"}
|
||||
out (th/command! params)]
|
||||
;; (th/print-result! out)
|
||||
|
||||
(t/is (nil? (:error out)))
|
||||
(:result out))
|
||||
|
||||
stale-at #inst "2020-01-01T00:00:00.000Z"]
|
||||
|
||||
(doseq [type [:lock-file-snapshot :unlock-file-snapshot
|
||||
:update-file-snapshot :delete-file-snapshot]]
|
||||
(t/testing (name type)
|
||||
;; make the row look old, so the mutation has to bump it
|
||||
(th/db-update! :file-change {:updated-at stale-at}
|
||||
{:id (:id snapshot)}
|
||||
{::db/remove-deleted false})
|
||||
|
||||
(let [params (cond-> {::th/type type
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:id (:id snapshot)}
|
||||
(= :update-file-snapshot type)
|
||||
(assoc :label "renamed"))
|
||||
out (th/command! params)
|
||||
row (th/db-get :file-change {:id (:id snapshot)}
|
||||
{::db/remove-deleted false})]
|
||||
|
||||
(t/is (nil? (:error out)))
|
||||
(t/is (true? (:result out)))
|
||||
(t/is (pos? (- (inst-ms (:updated-at row)) (inst-ms stale-at)))
|
||||
"updated_at must be bumped by the mutation")
|
||||
(t/is (<= (inst-ms (:updated-at row)) (inst-ms (ct/now)))
|
||||
"updated_at must not be in the future")
|
||||
(t/is (<= (- (inst-ms (ct/now)) (inst-ms (:updated-at row))) 60000)
|
||||
"updated_at must be the moment of the mutation")
|
||||
(when (= type :delete-file-snapshot)
|
||||
(t/is (< (inst-ms (:updated-at row)) (inst-ms (:deleted-at row)))
|
||||
"the soft delete must not stamp updated_at with the future deleted_at")))))
|
||||
|
||||
(t/testing "the deleted snapshot is marked as deleted"
|
||||
(t/is (some? (:deleted-at (th/db-get :file-change {:id (:id snapshot)}
|
||||
{::db/remove-deleted false})))))
|
||||
|
||||
;; process the delete-object job enqueued by the snapshot delete
|
||||
(th/run-pending-jobs)))
|
||||
|
||||
(t/deftest unknown-snapshot-is-reported-as-not-found
|
||||
;; The error points at the id that was not found: the snapshots are looked
|
||||
;; up by id, not by label. Both commands that resolve a snapshot report it
|
||||
;; the same way.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
|
||||
params {::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:id (uuid/random)}]
|
||||
|
||||
(doseq [type [:restore-file-snapshot :get-file-snapshot]]
|
||||
(t/testing (name type)
|
||||
(let [out (th/command! (assoc params ::th/type type))]
|
||||
|
||||
(t/is (th/ex-of-code? (:error out) :snapshot-not-found))
|
||||
(t/is (= "unable to find snapshot with the provided id"
|
||||
(:hint (ex-data (:error out))))))))))
|
||||
|
||||
(t/deftest revision-and-snapshot-carry-different-content
|
||||
;; A revision is the delta of a save: it carries the changes and nothing
|
||||
;; else. A snapshot is the state of the file at a point: it carries the
|
||||
;; features and the migrations.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
page-id (uuid/random)]
|
||||
|
||||
(update-file!
|
||||
:file-id (:id file)
|
||||
:profile-id (:id profile)
|
||||
:revn 0
|
||||
:changes
|
||||
[{:type :add-page
|
||||
:name "test"
|
||||
:id page-id}])
|
||||
|
||||
(t/testing "the revision carries the changes and not the features"
|
||||
(let [rows (th/db-query :file-change {:file-id (:id file)})
|
||||
revision (first (filter #(nil? (:label %)) rows))]
|
||||
|
||||
(t/is (some? revision))
|
||||
(t/is (some? (:changes revision)))
|
||||
(t/is (nil? (:features revision)))))
|
||||
|
||||
(t/testing "the snapshot carries the migrations and the features"
|
||||
(let [out (th/command! {::th/type :create-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:label "label1"})]
|
||||
|
||||
(t/is (nil? (:error out)))
|
||||
|
||||
(let [snapshot (th/db-get :file-change {:id (get-in out [:result :id])})]
|
||||
(t/is (= "label1" (:label snapshot)))
|
||||
(t/is (some? (:migrations snapshot)))
|
||||
(t/is (some? (:features snapshot))))))))
|
||||
|
||||
(t/deftest file-change-rows-have-a-kind
|
||||
;; The kind tells the two kinds of rows of `file_change` apart: a snapshot
|
||||
;; (1) and a revision (2). The values are the contract with the database.
|
||||
(let [profile (th/create-profile* 1 {:is-active true})
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)
|
||||
:is-shared false})
|
||||
page-id (uuid/random)]
|
||||
|
||||
(update-file!
|
||||
:file-id (:id file)
|
||||
:profile-id (:id profile)
|
||||
:revn 0
|
||||
:changes
|
||||
[{:type :add-page
|
||||
:name "test"
|
||||
:id page-id}])
|
||||
|
||||
(t/testing "the revision is a revision"
|
||||
(let [rows (th/db-query :file-change {:file-id (:id file)})
|
||||
revision (first (filter #(nil? (:label %)) rows))]
|
||||
|
||||
(t/is (some? revision))
|
||||
(t/is (= 2 (:kind revision)))))
|
||||
|
||||
(t/testing "the snapshot is a snapshot"
|
||||
(let [out (th/command! {::th/type :create-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:label "label1"})]
|
||||
|
||||
(t/is (nil? (:error out)))
|
||||
|
||||
(let [snapshot (th/db-get :file-change {:id (get-in out [:result :id])})]
|
||||
(t/is (= 1 (:kind snapshot))))))))
|
||||
|
||||
@ -2749,6 +2749,43 @@
|
||||
(let [edata (-> out :error ex-data)]
|
||||
(t/is (= :circular-library-reference (:code edata)))))))
|
||||
|
||||
(t/deftest link-file-to-library-self-reference
|
||||
(let [profile (th/create-profile* 1)
|
||||
file (th/create-file* 1 {:profile-id (:id profile)
|
||||
:project-id (:default-project-id profile)})]
|
||||
(let [out (th/command! {::th/type :link-file-to-library
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:library-id (:id file)})]
|
||||
(t/is (th/ex-info? (:error out)))
|
||||
(t/is (th/ex-of-type? (:error out) :validation))
|
||||
(t/is (= :invalid-library (:code (ex-data (:error out))))))))
|
||||
|
||||
(t/deftest permissions-check-precedes-circular-library-check
|
||||
;; A caller without edition permission must get `:not-found`, not the
|
||||
;; circular error: the permission checks run before the cycle check, so
|
||||
;; the dependency structure of files the caller cannot read never leaks.
|
||||
(let [profile1 (th/create-profile* 1)
|
||||
profile2 (th/create-profile* 2)
|
||||
file1 (th/create-file* 1 {:profile-id (:id profile1)
|
||||
:project-id (:default-project-id profile1)
|
||||
:is-shared true})
|
||||
file2 (th/create-file* 2 {:profile-id (:id profile1)
|
||||
:project-id (:default-project-id profile1)
|
||||
:is-shared true})
|
||||
file3 (th/create-file* 3 {:profile-id (:id profile1)
|
||||
:project-id (:default-project-id profile1)
|
||||
:is-shared true})]
|
||||
;; file3 -> file2 -> file1 makes linking file1 -> file3 circular
|
||||
(th/link-file-to-library* {:file-id (:id file3) :library-id (:id file2)})
|
||||
(th/link-file-to-library* {:file-id (:id file2) :library-id (:id file1)})
|
||||
(let [out (th/command! {::th/type :link-file-to-library
|
||||
::rpc/profile-id (:id profile2)
|
||||
:file-id (:id file1)
|
||||
:library-id (:id file3)})]
|
||||
(t/is (th/ex-info? (:error out)))
|
||||
(t/is (th/ex-of-type? (:error out) :not-found)))))
|
||||
|
||||
(t/deftest get-file-etag-includes-deleted-at
|
||||
(let [profile-id (uuid/random)
|
||||
file1 {:modified-at (ct/now)
|
||||
|
||||
@ -346,6 +346,10 @@
|
||||
(t/is (= 2 (count new)))
|
||||
(t/is (= (inc n) (:revn (first new))))
|
||||
(t/is (= files-update/repair-session-id (:session-id (first new))))
|
||||
;; the repair is a revision like the save: no features, a changes blob
|
||||
(t/is (= 2 (:kind (first new))))
|
||||
(t/is (nil? (:features (first new))))
|
||||
(t/is (some? (:changes (first new))))
|
||||
(t/is (some? (:deleted-at (first new))))
|
||||
(t/is (= (:created-at (first new)) (:updated-at (first new))))
|
||||
(t/is (= (+ n 2) (:revn (second new))))
|
||||
|
||||
@ -403,6 +403,51 @@
|
||||
(t/is (nil? (:error out)))
|
||||
(assert-font-variant-result params (:result out)))))
|
||||
|
||||
(t/deftest create-font-variant-persists-variant-name
|
||||
"The optional :variant-name is stored and returned by get-font-variants."
|
||||
(with-mocks [_mock {:target 'app.rpc.quotes/check! :return nil}]
|
||||
(let [prof (th/create-profile* 1 {:is-active true})
|
||||
team-id (:default-team-id prof)
|
||||
font-id (uuid/custom 10 34)
|
||||
font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
|
||||
session-id (upload-font-chunked! prof font-bytes "font/ttf" (* 4 1024 1024))
|
||||
out (th/command! {::th/type :create-font-variant
|
||||
::rpc/profile-id (:id prof)
|
||||
:team-id team-id
|
||||
:font-id font-id
|
||||
:font-family "variant-name-font"
|
||||
:font-weight 600
|
||||
:font-style "normal"
|
||||
:variant-name "SemiBold"
|
||||
:uploads {"font/ttf" session-id}})]
|
||||
(t/is (th/success? out))
|
||||
(t/is (= "SemiBold" (-> out :result :variant-name)))
|
||||
|
||||
(let [variants (-> (th/command! {::th/type :get-font-variants
|
||||
::rpc/profile-id (:id prof)
|
||||
:team-id team-id})
|
||||
:result)
|
||||
variant (first (filter #(= font-id (:font-id %)) variants))]
|
||||
(t/is (= "SemiBold" (:variant-name variant)))))
|
||||
|
||||
;; An explicit nil must be accepted (the param is optional).
|
||||
(let [prof (th/create-profile* 2 {:is-active true})
|
||||
team-id (:default-team-id prof)
|
||||
font-id (uuid/custom 10 35)
|
||||
font-bytes (-> (io/resource "backend_tests/test_files/font-1.ttf") (io/read*))
|
||||
session-id (upload-font-chunked! prof font-bytes "font/ttf" (* 4 1024 1024))
|
||||
out (th/command! {::th/type :create-font-variant
|
||||
::rpc/profile-id (:id prof)
|
||||
:team-id team-id
|
||||
:font-id font-id
|
||||
:font-family "variant-name-nil"
|
||||
:font-weight 400
|
||||
:font-style "normal"
|
||||
:variant-name nil
|
||||
:uploads {"font/ttf" session-id}})]
|
||||
(t/is (th/success? out))
|
||||
(t/is (nil? (-> out :result :variant-name))))))
|
||||
|
||||
;; -----------------------------------------------------------------------
|
||||
;; Error cases
|
||||
;; -----------------------------------------------------------------------
|
||||
|
||||
@ -563,6 +563,7 @@
|
||||
;; deleting a version takes it out of the count right away
|
||||
(let [out (th/command! {::th/type :delete-file-snapshot
|
||||
::rpc/profile-id (:id profile)
|
||||
:file-id (:id file)
|
||||
:id (:id snapshot)})]
|
||||
(t/is (nil? (:error out)))
|
||||
(t/is (true? (:result out)))
|
||||
|
||||
@ -10,11 +10,14 @@
|
||||
[app.common.uuid :as uuid]
|
||||
[app.scripts.decode-file-data :as dfd]
|
||||
[app.util.blob :as blob]
|
||||
[backend-tests.helpers :as th]
|
||||
[clojure.java.io :as io]
|
||||
[clojure.test :as t])
|
||||
(:import
|
||||
java.util.Base64))
|
||||
|
||||
(t/use-fixtures :once th/state-init)
|
||||
|
||||
(defn- sample-file-data
|
||||
[]
|
||||
(let [p1 (uuid/next)
|
||||
@ -80,3 +83,11 @@
|
||||
parsed (json/decode (slurp tmp-out))]
|
||||
(t/is (== 2 (get-in parsed ["stats" "pages"])))
|
||||
(t/is (= ["Accueil1" "Contact"] (mapv #(get % "name") (get parsed "pages"))))))
|
||||
|
||||
(t/deftest get-snapshot-json-fails-when-the-snapshot-does-not-exist
|
||||
;; The snapshot lookup returns nil when there is no match, and a nil
|
||||
;; snapshot used to be printed as an empty JSON instead of failing.
|
||||
(let [file-id (uuid/next)
|
||||
snapshot-id (uuid/next)]
|
||||
(t/is (thrown? clojure.lang.ExceptionInfo
|
||||
(dfd/get-snapshot-json th/*system* file-id snapshot-id)))))
|
||||
|
||||
@ -2233,44 +2233,49 @@
|
||||
;; been added to enforce this from now on.
|
||||
;; Main instances in variants are a special case because they need to match the variant name.
|
||||
(letfn [(update-shape [container shape-id]
|
||||
(let [shape (ctn/get-shape container shape-id)
|
||||
(let [shape (ctn/get-shape container shape-id)]
|
||||
;; A shape may reference a child that does not exist in the
|
||||
;; container (a dangling reference). There is nothing to update
|
||||
;; on it, and storing it would add a nil entry to the objects
|
||||
;; map, making the whole file invalid.
|
||||
(if (nil? shape)
|
||||
container
|
||||
(let [shape (cond
|
||||
;; Variant container must have a normalized name
|
||||
(ctk/is-variant-container? shape)
|
||||
(let [name (:name shape)
|
||||
clean-name (cpn/clean-path name)
|
||||
clean-name (if (str/blank? clean-name)
|
||||
"(empty)" ;; this may occur if the name contains only separators
|
||||
clean-name)]
|
||||
(if (= name clean-name)
|
||||
shape
|
||||
(assoc shape :name clean-name)))
|
||||
|
||||
shape (cond
|
||||
;; Variant container must have a normalized name
|
||||
(ctk/is-variant-container? shape)
|
||||
(let [name (:name shape)
|
||||
clean-name (cpn/clean-path name)
|
||||
clean-name (if (str/blank? clean-name)
|
||||
"(empty)" ;; this may occur if the name contains only separators
|
||||
clean-name)]
|
||||
(if (= name clean-name)
|
||||
shape
|
||||
(assoc shape :name clean-name)))
|
||||
;; Main instance inside a variant must have the same name as the container
|
||||
(ctk/main-instance? shape)
|
||||
(if (ctk/is-variant? shape)
|
||||
(let [variant-container (ctn/get-shape container (:parent-id shape))]
|
||||
(if (or (= (:name shape) (:name variant-container))
|
||||
(nil? variant-container))
|
||||
shape
|
||||
(assoc shape :name (:name variant-container))))
|
||||
|
||||
;; Main instance inside a variant must have the same name as the container
|
||||
(ctk/main-instance? shape)
|
||||
(if (ctk/is-variant? shape)
|
||||
(let [variant-container (ctn/get-shape container (:parent-id shape))]
|
||||
(if (or (= (:name shape) (:name variant-container))
|
||||
(nil? variant-container))
|
||||
shape
|
||||
(assoc shape :name (:name variant-container))))
|
||||
;; Other main instances must have the same name as the path+name of the component
|
||||
(let [component (ctkl/get-component data (:component-id shape) true)
|
||||
expected-name (when component
|
||||
(cpn/merge-path-item (:path component) (:name component)))]
|
||||
(if (or (= expected-name (:name shape))
|
||||
(nil? expected-name))
|
||||
shape
|
||||
(assoc shape :name expected-name))))
|
||||
|
||||
;; Other main instances must have the same name as the path+name of the component
|
||||
(let [component (ctkl/get-component data (:component-id shape) true)
|
||||
expected-name (when component
|
||||
(cpn/merge-path-item (:path component) (:name component)))]
|
||||
(if (or (= expected-name (:name shape))
|
||||
(nil? expected-name))
|
||||
shape
|
||||
(assoc shape :name expected-name))))
|
||||
;; Any other shape does not need processing
|
||||
:else shape)]
|
||||
|
||||
;; Any other shape does not need processing
|
||||
:else shape)]
|
||||
|
||||
(reduce update-shape
|
||||
(ctt/set-shape container shape)
|
||||
(:shapes shape))))
|
||||
(reduce update-shape
|
||||
(ctt/set-shape container shape)
|
||||
(:shapes shape))))))
|
||||
|
||||
(update-container
|
||||
[container]
|
||||
@ -2322,6 +2327,35 @@
|
||||
(update :pages-index d/update-vals update-container)
|
||||
(d/update-when :components d/update-vals update-container)))))
|
||||
|
||||
(defmethod migrate-data "0035-fix-dangling-children"
|
||||
[data _]
|
||||
;; A shape may list a child id that does not exist in the objects map of
|
||||
;; the page or component that owns it (a dangling reference). Such a
|
||||
;; reference is invalid: any operation that walks the shape tree through
|
||||
;; the children lists trips on it, and the file cannot be migrated nor
|
||||
;; opened. Here we remove the dangling ids from the children lists,
|
||||
;; keeping every shape that really exists.
|
||||
(letfn [(fix-shape [container-id objects shape]
|
||||
(d/update-when shape :shapes
|
||||
(fn [shapes]
|
||||
(let [shapes' (filterv #(contains? objects %) shapes)]
|
||||
(when (not= shapes shapes')
|
||||
(l/trc :hint "fix dangling children"
|
||||
:container container-id
|
||||
:shape-id (:id shape)
|
||||
:removed (remove #(contains? objects %) shapes)))
|
||||
shapes'))))
|
||||
|
||||
(fix-objects [container-id objects]
|
||||
(update-vals objects (partial fix-shape container-id objects)))
|
||||
|
||||
(update-container [container]
|
||||
(d/update-when container :objects #(fix-objects (:id container) %)))]
|
||||
|
||||
(-> data
|
||||
(update :pages-index d/update-vals update-container)
|
||||
(d/update-when :components d/update-vals update-container))))
|
||||
|
||||
(defmethod migrate-data "0035-fix-shape-svg-attrs-and-defs"
|
||||
;; Stored svg attribute keys are kebab-case (issue #11947): the v3 reader
|
||||
;; already outputs kebab and birth normalizes any outside spelling, so
|
||||
@ -2459,4 +2493,5 @@
|
||||
"0032-fix-children-not-in-parent"
|
||||
"0033-normalize-main-instance-names"
|
||||
"0034-fix-path-content-offset"
|
||||
"0035-fix-dangling-children"
|
||||
"0035-fix-shape-svg-attrs-and-defs"]))
|
||||
|
||||
@ -28,6 +28,25 @@
|
||||
(map #(ctcl/get-component data % true))
|
||||
reverse)))
|
||||
|
||||
(defn collect-variant-properties
|
||||
"Map of component-id → variant-properties for clipboard copy.
|
||||
Variant container: every related component. Variant main: that
|
||||
component only. Anything else: empty map."
|
||||
[data objects shape]
|
||||
(cond
|
||||
(ctc/is-variant-container? shape)
|
||||
(into {}
|
||||
(map (juxt :id :variant-properties))
|
||||
(find-variant-components data objects (:id shape)))
|
||||
|
||||
(ctc/is-variant-main? shape)
|
||||
(if-let [component (ctcl/get-component data (:component-id shape) true)]
|
||||
{(:id component) (:variant-properties component)}
|
||||
{})
|
||||
|
||||
:else
|
||||
{}))
|
||||
|
||||
(defn extract-properties-names
|
||||
[shape data]
|
||||
(->> shape
|
||||
|
||||
@ -58,6 +58,44 @@
|
||||
;; We don't do automatic update of the `layout-grid-cells` property.
|
||||
(remove #(= :layout-grid-cells %))))
|
||||
|
||||
;; Attrs whose value drifts by float noise when a shape is moved; they
|
||||
;; are compared with tolerance during sync. The rest compare exactly.
|
||||
(def ^:private drifting-geometry-attrs
|
||||
#{:x :y :width :height :selrect :points})
|
||||
|
||||
(defn- position-data-close?
|
||||
"Compares text position-data entry by entry: numbers with tolerance,
|
||||
every other field (fills, fonts, text) exactly."
|
||||
[a b]
|
||||
(let [a (vec a)
|
||||
b (vec b)]
|
||||
(and (= (count a) (count b))
|
||||
(every? true? (map mth/close-map? a b)))))
|
||||
|
||||
(defn- attr-equal?
|
||||
[attr origin-shape dest-shape]
|
||||
(let [origin-val (get origin-shape attr)
|
||||
dest-val (get dest-shape attr)]
|
||||
(or (= origin-val dest-val)
|
||||
(cond
|
||||
;; close-attrs? zips :points and would ignore a length mismatch
|
||||
(= :points attr)
|
||||
(and (= (count origin-val) (count dest-val))
|
||||
(gsh/close-attrs? attr origin-val dest-val))
|
||||
|
||||
(contains? drifting-geometry-attrs attr)
|
||||
(gsh/close-attrs? attr origin-val dest-val)
|
||||
|
||||
(and (= :content attr)
|
||||
(or (cfh/path-shape? origin-shape) (cfh/bool-shape? origin-shape)))
|
||||
(segment/content-close? origin-val dest-val)
|
||||
|
||||
(= :position-data attr)
|
||||
(position-data-close? origin-val dest-val)
|
||||
|
||||
:else
|
||||
false))))
|
||||
|
||||
;; Flex-child layout attrs synced between main and copy (interactions are only kept on swap)
|
||||
(def ^:private flex-child-attrs
|
||||
(disj ctk/swap-keep-attrs :interactions))
|
||||
@ -1993,7 +2031,7 @@
|
||||
reset-pos-data?
|
||||
(and (cfh/text-shape? origin-shape)
|
||||
(= attr :position-data)
|
||||
(not= (:position-data origin-shape) (:position-data dest-shape))
|
||||
(not (attr-equal? :position-data origin-shape dest-shape))
|
||||
(touched :geometry-group))
|
||||
|
||||
;; On texts, when we want to omit the touched attrs, both text (the actual letters)
|
||||
@ -2007,7 +2045,7 @@
|
||||
(touched sync-group))
|
||||
|
||||
skip-operations?
|
||||
(or (= (get origin-shape attr) (get dest-shape attr))
|
||||
(or (attr-equal? attr origin-shape dest-shape)
|
||||
(and (touched sync-group)
|
||||
omit-touched?
|
||||
;; When it is a text-partial-change, we should generate operations
|
||||
@ -2728,7 +2766,8 @@
|
||||
page-id
|
||||
nil
|
||||
variant-id
|
||||
props))]
|
||||
props
|
||||
{:apply-changes-local-library? true}))]
|
||||
[root-shape changes]))
|
||||
|
||||
(defn generate-add-component
|
||||
@ -3165,6 +3204,7 @@
|
||||
is-component-main? (ctk/main-instance? obj)
|
||||
subinstance-head? (ctk/subinstance-head? obj)
|
||||
instance-root? (ctk/instance-root? obj)
|
||||
parent-not-in-copy (not (ctk/in-component-copy? parent))
|
||||
|
||||
into-component? (and duplicating-component?
|
||||
(ctn/in-any-component? objects parent))
|
||||
@ -3174,6 +3214,8 @@
|
||||
(ctn/get-nesting-level-delta objects obj parent))
|
||||
new-shape-ref (ctf/advance-shape-ref nil page libraries obj level-delta {:include-deleted? true})
|
||||
|
||||
|
||||
|
||||
regenerate-component
|
||||
(fn [changes shape]
|
||||
(let [[_ changes] (generate-add-component-changes changes shape objects file-id (:id page) variant-props)]
|
||||
@ -3188,7 +3230,8 @@
|
||||
|
||||
(cond-> (and (not instance-root?)
|
||||
subinstance-head?
|
||||
remove-swap-slot?)
|
||||
remove-swap-slot?
|
||||
parent-not-in-copy)
|
||||
(ctk/remove-swap-slot))
|
||||
|
||||
(dissoc :shapes
|
||||
@ -3240,6 +3283,25 @@
|
||||
(and is-component-root? is-component-main?)
|
||||
(regenerate-component new-obj))
|
||||
|
||||
;; Unknown-component paste into a variant set: align props/names
|
||||
;; after regenerate. Reload the shape so we use the new component-id.
|
||||
;; Prefit the component name/path to the container so
|
||||
;; create-new-properties-from-variant does not treat the component
|
||||
;; full name as a former set name and inflate properties (add-name?).
|
||||
changes (cond-> changes
|
||||
(and is-component-root?
|
||||
is-component-main?
|
||||
(ctk/is-variant-container? parent))
|
||||
(as-> $
|
||||
(let [shape (get (pcb/get-objects $) (:id new-obj))
|
||||
[cpath cname] (cpn/split-group-name (:name parent))
|
||||
changes (pcb/update-component
|
||||
$
|
||||
(:component-id shape)
|
||||
#(assoc % :name cname :path cpath)
|
||||
{:apply-changes-local-library? true})]
|
||||
(clvp/generate-make-shapes-variant changes [shape] parent))))
|
||||
|
||||
;; This is needed for the recursive call to find the new object as parent
|
||||
page' (ctst/add-shape (:id new-obj)
|
||||
new-obj
|
||||
|
||||
@ -266,10 +266,20 @@
|
||||
(first
|
||||
(reduce
|
||||
(fn [[changes used-values] shape]
|
||||
(let [component (ctcl/get-component data (:component-id shape) true)]
|
||||
(if (or (zero? num-shapes) ;; do nothing if there are no shapes
|
||||
(and (= variant-id (:variant-id shape)) ;; or we are only moving the shape inside its parent (it is
|
||||
(not (:deleted component)))) ;; the same parent and the component isn't deleted)
|
||||
(let [component (ctcl/get-component data (:component-id shape) true)
|
||||
already-here? (= variant-id (:variant-id shape))
|
||||
prop-names (cfv/extract-properties-names shape data)
|
||||
base-names (map :name base-props)
|
||||
props-ok? (= prop-names base-names)
|
||||
names-ok? (= (cpn/merge-path-item (:path component) (:name component))
|
||||
container-name)
|
||||
variant-ok? (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties component)))
|
||||
aligned? (and props-ok? names-ok? variant-ok?)]
|
||||
(if (or (zero? num-shapes) ;; do nothing if there are no shapes
|
||||
(and already-here? ;; or already a fully aligned child of this container
|
||||
aligned?
|
||||
(not (:deleted component))))
|
||||
[changes used-values]
|
||||
(let [props (-> (create-new-properties shape total-props)
|
||||
(fill-blank-properties used-values))
|
||||
|
||||
@ -208,6 +208,19 @@
|
||||
([num1 num2 precision]
|
||||
(<= (abs (- num1 num2)) precision)))
|
||||
|
||||
(defn close-map?
|
||||
"Equality for maps: same keys, numeric values within `close?`, any other
|
||||
value compared exactly."
|
||||
[a b]
|
||||
(and (= (count a) (count b))
|
||||
(every? (fn [[k v]]
|
||||
(and (contains? b k)
|
||||
(let [w (get b k)]
|
||||
(if (number? v)
|
||||
(and (number? w) (close? v w))
|
||||
(= v w)))))
|
||||
a)))
|
||||
|
||||
(defn lerp
|
||||
"Calculates a the linear interpolation between two values and a given percent"
|
||||
[v0 v1 t]
|
||||
|
||||
@ -98,36 +98,90 @@
|
||||
(let [image-extensions-re #"(\.png)|(\.jpg)|(\.jpeg)|(\.webp)|(\.gif)|(\.svg)$"]
|
||||
(str/replace filename image-extensions-re "")))
|
||||
|
||||
(def ^:private font-variant-tokens
|
||||
"Weight and style tokens stripped from a font name to derive its family.
|
||||
Keep the alternatives in sync with `parse-font-weight` and
|
||||
`parse-font-style`."
|
||||
["extra\\s*black" "ultra\\s*black" "extra\\s*bold" "ultra\\s*bold"
|
||||
"semi\\s*bold" "demi\\s*bold" "extra\\s*light" "ultra\\s*light"
|
||||
"hairline" "thin" "light" "normal" "regular" "medium" "bold" "black"
|
||||
"heavy" "solid" "italic" "oblique"])
|
||||
|
||||
(def ^:private font-variant-tokens-re
|
||||
;; The trailing boundary is a lookahead, so two tokens separated by a single
|
||||
;; space are both stripped in one pass (the separator is not consumed).
|
||||
(re-pattern (str "(?i)(^|[-_\\s])(" (str/join "|" font-variant-tokens) ")(?=[-_\\s]|$)")))
|
||||
|
||||
(def ^:private font-concatenated-style-re
|
||||
;; Separate a style token glued to a weight token, e.g. "BoldItalic" ->
|
||||
;; "Bold Italic". Only a known weight token followed by a style token is
|
||||
;; split, so families such as "OpenSans" are left untouched. The trailing
|
||||
;; boundary is a lookahead, so a glued style token followed by a technical
|
||||
;; suffix (e.g. "Italic_v2") is split too.
|
||||
#"(?i)(black|bold|heavy|solid|medium|light|thin|hairline|regular|normal)(italic|oblique)(?=[-_\s]|$)")
|
||||
|
||||
(def ^:private font-technical-suffix-re
|
||||
;; Trailing foundry/build suffixes, e.g. "Roboto-Bold-VF" or "Inter_v2".
|
||||
;; Requires a separator; stripped before token detection so the parsers
|
||||
;; interpret the name the same way.
|
||||
#"(?i)[-_\s]+(?:vf|v[0-9]+)$")
|
||||
|
||||
(defn- normalize-font-name
|
||||
"Normalize a font name or variant string before token detection:
|
||||
split a style token glued to a weight token, then strip a trailing
|
||||
technical suffix. Applied by `parse-font-family`, `parse-font-weight`
|
||||
and `parse-font-style` so all three interpret the name the same way."
|
||||
[s]
|
||||
(-> s
|
||||
(str/replace font-concatenated-style-re "$1 $2")
|
||||
(str/replace font-technical-suffix-re "")))
|
||||
|
||||
(defn parse-font-family
|
||||
"Derive a font family name from a base name (a filename without its
|
||||
extension), stripping the known weight/style tokens."
|
||||
[base-name]
|
||||
(let [normalized (normalize-font-name base-name)
|
||||
stripped (-> normalized
|
||||
(str/replace font-variant-tokens-re "$1")
|
||||
(str/replace #"[-_\s]+" " ")
|
||||
(str/trim))]
|
||||
(if (str/blank? stripped) base-name stripped)))
|
||||
|
||||
(defn parse-font-weight
|
||||
"Parses a font variant/subfamily string into a numeric weight.
|
||||
A nil or non-string variant means the font carries no subfamily
|
||||
metadata, so it defaults to 400 (regular), the same as a blank string."
|
||||
[variant]
|
||||
(let [variant (if (string? variant) variant "")]
|
||||
(let [variant (normalize-font-name (if (string? variant) variant ""))]
|
||||
(cond
|
||||
(re-seq #"(?i)(?:^|[-_\s])(hairline|thin)(?=(?:[-_\s]|$|italic\b))" variant) 100
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*light|ultra\s*light)(?=(?:[-_\s]|$|italic\b))" variant) 200
|
||||
(re-seq #"(?i)(?:^|[-_\s])(light)(?=(?:[-_\s]|$|italic\b))" variant) 300
|
||||
(re-seq #"(?i)(?:^|[-_\s])(normal|regular)(?=(?:[-_\s]|$|italic\b))" variant) 400
|
||||
(re-seq #"(?i)(?:^|[-_\s])(medium)(?=(?:[-_\s]|$|italic\b))" variant) 500
|
||||
(re-seq #"(?i)(?:^|[-_\s])(semi\s*bold|demi\s*bold)(?=(?:[-_\s]|$|italic\b))" variant) 600
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*bold|ultra\s*bold)(?=(?:[-_\s]|$|italic\b))" variant) 800
|
||||
(re-seq #"(?i)(?:^|[-_\s])(bold)(?=(?:[-_\s]|$|italic\b))" variant) 700
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*black|ultra\s*black)(?=(?:[-_\s]|$|italic\b))" variant) 950
|
||||
(re-seq #"(?i)(?:^|[-_\s])(black|heavy|solid)(?=(?:[-_\s]|$|italic\b))" variant) 900
|
||||
:else 400)))
|
||||
(re-seq #"(?i)(?:^|[-_\s])(hairline|thin)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 100
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*light|ultra\s*light)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 200
|
||||
(re-seq #"(?i)(?:^|[-_\s])(light)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 300
|
||||
(re-seq #"(?i)(?:^|[-_\s])(normal|regular)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 400
|
||||
(re-seq #"(?i)(?:^|[-_\s])(medium)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 500
|
||||
(re-seq #"(?i)(?:^|[-_\s])(semi\s*bold|demi\s*bold)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 600
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*bold|ultra\s*bold)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 800
|
||||
(re-seq #"(?i)(?:^|[-_\s])(bold)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 700
|
||||
(re-seq #"(?i)(?:^|[-_\s])(extra\s*black|ultra\s*black)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 950
|
||||
(re-seq #"(?i)(?:^|[-_\s])(black|heavy|solid)(?=(?:[-_\s]|$|italic\b|oblique\b))" variant) 900
|
||||
:else 400)))
|
||||
|
||||
(defn parse-font-style
|
||||
"Parses a font variant/subfamily string into \"italic\" or \"normal\".
|
||||
A nil or non-string variant means the font carries no subfamily
|
||||
metadata, so it defaults to \"normal\"."
|
||||
[variant]
|
||||
(let [variant (if (string? variant) variant "")]
|
||||
(if (or (re-seq #"(?i)(?:^|[-_\s])(italic)(?:[-_\s]|$)" variant)
|
||||
(re-seq #"(?i)italic$" variant))
|
||||
(let [variant (normalize-font-name (if (string? variant) variant ""))]
|
||||
(if (or (re-seq #"(?i)(?:^|[-_\s])(italic|oblique)(?:[-_\s]|$)" variant)
|
||||
(re-seq #"(?i)(?:italic|oblique)$" variant))
|
||||
"italic"
|
||||
"normal")))
|
||||
|
||||
(defn first-nonblank-string
|
||||
"Return the first nonblank string from the candidates, or nil."
|
||||
[& candidates]
|
||||
(first (filter #(and (string? %) (not (str/blank? %))) candidates)))
|
||||
|
||||
(defn font-weight->name
|
||||
[weight]
|
||||
(case (long weight)
|
||||
|
||||
@ -196,7 +196,7 @@
|
||||
;; Calculate how many parent heads share in common the original
|
||||
;; shape and the new parent.
|
||||
pairs (map vector orig-heads dest-heads)
|
||||
common-count (count (take-while (fn [a b] (= a b)) pairs))]
|
||||
common-count (count (take-while (fn [[a b]] (= a b)) pairs))]
|
||||
|
||||
(- (count orig-heads) common-count)))
|
||||
|
||||
|
||||
@ -1355,6 +1355,20 @@
|
||||
(grc/points->rect extremities)
|
||||
(grc/make-rect))))
|
||||
|
||||
(defn- segment-close?
|
||||
[a b]
|
||||
(and (= (:command a) (:command b))
|
||||
(mth/close-map? (:params a) (:params b))))
|
||||
|
||||
(defn content-close?
|
||||
"True when both path contents have the same segments and every param
|
||||
is within `mth/close?` of its counterpart."
|
||||
[content-a content-b]
|
||||
(let [a (vec content-a)
|
||||
b (vec content-b)]
|
||||
(and (= (count a) (count b))
|
||||
(every? true? (map segment-close? a b)))))
|
||||
|
||||
(defn content-center
|
||||
[content]
|
||||
(-> content
|
||||
|
||||
49
common/test/common_tests/files/variant_test.cljc
Normal file
49
common/test/common_tests/files/variant_test.cljc
Normal file
@ -0,0 +1,49 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.files.variant-test
|
||||
(:require
|
||||
[app.common.files.variant :as cfv]
|
||||
[app.common.test-helpers.components :as thc]
|
||||
[app.common.test-helpers.files :as thf]
|
||||
[app.common.test-helpers.ids-map :as thi]
|
||||
[app.common.test-helpers.shapes :as ths]
|
||||
[app.common.test-helpers.variants :as thv]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :each thi/test-fixture)
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-main
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
main (ths/get-shape file :m01)
|
||||
comp (thc/get-component file :c01)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) main)]
|
||||
|
||||
(t/is (= {(:id comp) (:variant-properties comp)} collected))))
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-container
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
container (ths/get-shape file :v01)
|
||||
c01 (thc/get-component file :c01)
|
||||
c02 (thc/get-component file :c02)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) container)]
|
||||
|
||||
(t/is (= {(:id c01) (:variant-properties c01)
|
||||
(:id c02) (:variant-properties c02)}
|
||||
collected))))
|
||||
|
||||
(t/deftest test-collect-variant-properties-for-plain-shape
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(ths/add-sample-shape :rect1 :type :rect))
|
||||
page (thf/current-page file)
|
||||
shape (ths/get-shape file :rect1)
|
||||
collected (cfv/collect-variant-properties (:data file) (:objects page) shape)]
|
||||
|
||||
(t/is (= {} collected))))
|
||||
138
common/test/common_tests/files_migrations_0033_test.cljc
Normal file
138
common/test/common_tests/files_migrations_0033_test.cljc
Normal file
@ -0,0 +1,138 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.files-migrations-0033-test
|
||||
"Regression coverage for the shape tree traversal of the
|
||||
`0033-normalize-main-instance-names` migration.
|
||||
|
||||
The migration walks a container from its root down, following the
|
||||
`:shapes` vectors. A file can hold a `:shapes` entry pointing to a
|
||||
shape that is not in the `:objects` map (a dangling reference). The
|
||||
traversal must skip it instead of storing a nil shape into the
|
||||
`:objects` map, and must keep normalizing the names of the shapes
|
||||
that do exist."
|
||||
(:require
|
||||
[app.common.files.migrations :as cfm]
|
||||
[app.common.types.shape :as cts]
|
||||
[app.common.uuid :as uuid]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(def ^:private migration "0033-normalize-main-instance-names")
|
||||
|
||||
(defn- make-root
|
||||
[children]
|
||||
(cts/setup-shape {:id uuid/zero
|
||||
:type :frame
|
||||
:name "Root"
|
||||
:shapes children}))
|
||||
|
||||
(defn- make-main-instance
|
||||
"A frame declared as the main instance of a component."
|
||||
[main-id file-id component-id name]
|
||||
(-> (cts/setup-shape {:id main-id :type :frame :name name})
|
||||
(assoc :main-instance true
|
||||
:component-id component-id
|
||||
:component-file file-id)))
|
||||
|
||||
(defn- page-data
|
||||
"File data with a single page owning `objects`."
|
||||
[page-id objects]
|
||||
{:pages-index {page-id {:id page-id :objects objects}}})
|
||||
|
||||
(defn- component-data
|
||||
"File data with a single component owning `objects`."
|
||||
[component-id objects]
|
||||
{:components {component-id {:id component-id :objects objects}}})
|
||||
|
||||
;; --- The guard: a missing child must not become a nil shape
|
||||
|
||||
(t/deftest missing-child-of-a-page-does-not-create-a-nil-shape
|
||||
(let [page-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
root (make-root [child-id])
|
||||
data (page-data page-id {uuid/zero root})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= 1 (count objects'))
|
||||
"the objects map keeps only the shapes that exist")
|
||||
(t/is (= root (get objects' uuid/zero))
|
||||
"the root is left untouched")))
|
||||
|
||||
(t/deftest missing-child-of-a-component-does-not-create-a-nil-shape
|
||||
(let [component-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
root (make-root [child-id])
|
||||
data (component-data component-id {uuid/zero root})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= 1 (count objects'))
|
||||
"the objects map keeps only the shapes that exist")
|
||||
(t/is (= root (get objects' uuid/zero))
|
||||
"the component root is left untouched")))
|
||||
|
||||
(t/deftest missing-child-deep-in-the-tree-does-not-create-a-nil-shape
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
frame (-> (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
(assoc :shapes [missing-id]))
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= frame (get objects' frame-id))
|
||||
"the parent of the missing child is left untouched")))
|
||||
|
||||
;; --- The behavior: names keep being normalized next to a missing child
|
||||
|
||||
(t/deftest main-instance-of-a-page-is-renamed-next-to-a-missing-child
|
||||
(let [file-id (uuid/next)
|
||||
page-id (uuid/next)
|
||||
component-id (uuid/next)
|
||||
main-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
main-shape (make-main-instance main-id file-id component-id "Group/Name")
|
||||
data (assoc (page-data page-id {uuid/zero (make-root [missing-id main-id])
|
||||
main-id main-shape})
|
||||
:components {component-id {:id component-id
|
||||
:name "Name"
|
||||
:path "Group"
|
||||
:main-instance-id main-id}})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= "Group / Name" (get-in objects' [main-id :name]))
|
||||
"the main instance takes the path and name of its component")))
|
||||
|
||||
(t/deftest main-instance-of-a-component-is-renamed-next-to-a-missing-child
|
||||
(let [file-id (uuid/next)
|
||||
component-id (uuid/next)
|
||||
main-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
main-shape (make-main-instance main-id file-id component-id "Group/Name")
|
||||
data (component-data component-id {uuid/zero (make-root [missing-id main-id])
|
||||
main-id main-shape})
|
||||
data (assoc-in data [:components component-id :name] "Name")
|
||||
data (assoc-in data [:components component-id :path] "Group")
|
||||
data (assoc-in data [:components component-id :main-instance-id] main-id)
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (not (contains? objects' nil))
|
||||
"no nil key is added to the objects map")
|
||||
(t/is (= "Group / Name" (get-in objects' [main-id :name]))
|
||||
"the main instance takes the path and name of its component")))
|
||||
@ -10,10 +10,186 @@
|
||||
[app.common.geom.matrix :as gmt]
|
||||
[app.common.geom.rect :as grc]
|
||||
[app.common.types.file :as ctf]
|
||||
[app.common.types.objects-map :as omap]
|
||||
[app.common.types.shape :as cts]
|
||||
[app.common.uuid :as uuid]
|
||||
[clojure.test :as t]))
|
||||
|
||||
;; ======================================================================
|
||||
;; 0035-fix-dangling-children
|
||||
;; A shape can list a child id that is not present in the `:objects` map
|
||||
;; of the page or component owning it (a dangling reference). Any
|
||||
;; traversal of the shape tree trips on it, and the file becomes
|
||||
;; unopenable. The migration drops those ids and keeps every shape that
|
||||
;; really exists.
|
||||
;; ======================================================================
|
||||
|
||||
(def ^:private migration "0035-fix-dangling-children")
|
||||
|
||||
(defn- make-root
|
||||
[children]
|
||||
(cts/setup-shape {:id uuid/zero
|
||||
:type :frame
|
||||
:name "Root"
|
||||
:shapes children}))
|
||||
|
||||
(defn- make-frame
|
||||
[frame-id child-ids]
|
||||
(-> (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
(assoc :shapes child-ids)))
|
||||
|
||||
(defn- page-data
|
||||
[page-id objects]
|
||||
{:pages-index {page-id {:id page-id :objects objects}}})
|
||||
|
||||
(defn- component-data
|
||||
[component-id objects]
|
||||
{:components {component-id {:id component-id :objects objects}}})
|
||||
|
||||
(t/deftest removes-the-dangling-children-of-a-page
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (make-frame frame-id [child-id])
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed from the root children list")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the dangling id is removed from the frame children list")
|
||||
(t/is (= 2 (count objects'))
|
||||
"every shape that exists is kept")
|
||||
(t/is (some? (get objects' frame-id))
|
||||
"the frame itself is kept")))
|
||||
|
||||
(t/deftest removes-the-dangling-children-of-a-component
|
||||
(let [component-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
data (component-data component-id {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:components component-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed from the children list")
|
||||
(t/is (= 2 (count objects'))
|
||||
"every shape that exists is kept")
|
||||
(t/is (= frame (get objects' frame-id))
|
||||
"the existing shape is left untouched")))
|
||||
|
||||
(t/deftest migration-works-with-an-objects-map
|
||||
;; The backend stores the page :objects as an ObjectsMap (the lazy
|
||||
;; representation enabled by the fdata/objects-map feature). It is an
|
||||
;; IPersistentMap that does not implement IFn on the JVM: a migration
|
||||
;; must consult it with contains?/get, never by calling it as a
|
||||
;; function.
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
frame (make-frame frame-id [child-id])
|
||||
objects (omap/wrap {uuid/zero (make-root [frame-id child-id])
|
||||
frame-id frame})
|
||||
data (page-data page-id objects)
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling id is removed with the ObjectsMap representation")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the frame children list is pruned the same way")))
|
||||
|
||||
(t/deftest keeps-the-order-and-the-shapes-of-a-valid-tree
|
||||
(let [page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
rect-id (uuid/next)
|
||||
frame (cts/setup-shape {:id frame-id :type :frame :name "Frame"})
|
||||
rect (cts/setup-shape {:id rect-id :type :rect :name "Rect"})
|
||||
data (page-data page-id {uuid/zero (make-root [frame-id rect-id])
|
||||
frame-id frame
|
||||
rect-id rect})
|
||||
migrated (cfm/migrate-data data migration)
|
||||
objects' (get-in migrated [:pages-index page-id :objects])]
|
||||
|
||||
(t/is (= [frame-id rect-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the children list is kept as it was")
|
||||
(t/is (= data migrated)
|
||||
"a tree without dangling references is not modified")))
|
||||
|
||||
(t/deftest leaves-shapes-with-empty-children-alone
|
||||
(let [page-id (uuid/next)
|
||||
frame (cts/setup-shape {:id (uuid/next) :type :frame :name "Frame"})
|
||||
data (page-data page-id {uuid/zero (make-root [])
|
||||
(:id frame) frame})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= data migrated)
|
||||
"empty children vectors are left alone")))
|
||||
|
||||
(t/deftest leaves-shapes-without-children-key-alone
|
||||
(let [page-id (uuid/next)
|
||||
rect-id (uuid/next)
|
||||
rect (cts/setup-shape {:id rect-id :type :rect :name "Rect"})
|
||||
data (page-data page-id {uuid/zero (make-root [])
|
||||
rect-id rect})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= data migrated)
|
||||
"shapes without a children key are left alone")
|
||||
(t/is (not (contains? (get-in migrated [:pages-index page-id :objects rect-id]) :shapes))
|
||||
"no children key is created on the shape")))
|
||||
|
||||
(t/deftest migration-is-idempotent
|
||||
(let [page-id (uuid/next)
|
||||
child-id (uuid/next)
|
||||
data (page-data page-id {uuid/zero (make-root [child-id])})
|
||||
migrated (cfm/migrate-data data migration)]
|
||||
|
||||
(t/is (= migrated (cfm/migrate-data migrated migration))
|
||||
"applying the migration twice does not change the result")))
|
||||
|
||||
(t/deftest file-with-dangling-children-passes-the-whole-migration
|
||||
(let [file-id (uuid/next)
|
||||
page-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
missing-id (uuid/next)
|
||||
frame (make-frame frame-id [missing-id])
|
||||
base (ctf/make-file-data file-id page-id)
|
||||
data (-> base
|
||||
(assoc-in [:pages-index page-id :objects uuid/zero]
|
||||
(make-root [frame-id missing-id]))
|
||||
(assoc-in [:pages-index page-id :objects frame-id] frame))
|
||||
;; The file reached the state before 0033 ran, so 0033, 0034 and
|
||||
;; 0035 are all pending, exactly as in the reported failure.
|
||||
file {:id file-id
|
||||
:data data
|
||||
:migrations (disj cfm/available-migrations
|
||||
"0033-normalize-main-instance-names"
|
||||
"0034-fix-path-content-offset"
|
||||
migration)}]
|
||||
|
||||
(t/is (cfm/need-migration? file)
|
||||
"the file is scheduled for migration")
|
||||
|
||||
(let [migrated (cfm/migrate-file file {})
|
||||
objects' (get-in migrated [:data :pages-index page-id :objects])]
|
||||
|
||||
(t/is (contains? (:migrations migrated) migration)
|
||||
"the migration is recorded as applied")
|
||||
(t/is (not (contains? objects' nil))
|
||||
"the migrated data has no nil shape entry")
|
||||
(t/is (= [frame-id] (get-in objects' [uuid/zero :shapes]))
|
||||
"the dangling reference of the root is removed")
|
||||
(t/is (= [] (get-in objects' [frame-id :shapes]))
|
||||
"the dangling reference of the frame is removed")
|
||||
(t/is (some? (get objects' frame-id))
|
||||
"the frame that exists is kept"))))
|
||||
|
||||
;; ======================================================================
|
||||
;; 0035-fix-shape-svg-attrs-and-defs
|
||||
;; Stored svg attribute keys are kebab-case (issue #11947). Old birth and
|
||||
;; the cleaner era stored camelCase React prop names, which the v3 reader
|
||||
@ -22,6 +198,7 @@
|
||||
;; through the whitelist+kebab transform, svg-raw `:content` attrs are
|
||||
;; kebab-ized spelling-only. It replaces "0021-fix-shape-svg-attrs" (id
|
||||
;; withdrawn): files that recorded it migrate again through this one.
|
||||
;; ======================================================================
|
||||
|
||||
(def ^:private migration-id
|
||||
"0035-fix-shape-svg-attrs-and-defs")
|
||||
|
||||
@ -567,3 +567,62 @@
|
||||
;; blue2' has not swap-id after move
|
||||
(t/is (some? blue2'))
|
||||
(t/is (nil? (ctk/get-swap-slot blue2')))))
|
||||
|
||||
(defn- setup-nested-swap-file
|
||||
[]
|
||||
;; {:frame-icon-a} [:name Frame1] # [Component :icon-a]
|
||||
;; {:frame-icon-b} [:name Frame1] # [Component :icon-b]
|
||||
;; {:frame-card} [:name Frame1] # [Component :card]
|
||||
;; :frame-inner [:name Frame1]
|
||||
;; :icon-copy [:name Frame1] @--> :frame-icon-a
|
||||
;; :card-copy [:name Frame1] @--> :frame-card
|
||||
;; :inner-copy [:name Frame1] @--> :frame-inner
|
||||
;; :icon-in-card-copy [:name Frame1] @--> :frame-icon-a
|
||||
;; swapped to :icon-b, it becomes
|
||||
;; :icon-b-copy [:name Frame1, swap-slot] @--> :frame-icon-b
|
||||
(-> (thf/sample-file :file1)
|
||||
(tho/add-frame :frame-icon-a)
|
||||
(thc/make-component :icon-a :frame-icon-a)
|
||||
(tho/add-frame :frame-icon-b)
|
||||
(thc/make-component :icon-b :frame-icon-b)
|
||||
(tho/add-frame :frame-card)
|
||||
(thc/make-component :card :frame-card)
|
||||
(tho/add-frame :frame-inner :parent-label :frame-card)
|
||||
(thc/instantiate-component :icon-a :icon-copy :parent-label :frame-inner)
|
||||
(thc/instantiate-component :card :card-copy
|
||||
:children-labels [:inner-copy :icon-in-card-copy])
|
||||
(thc/component-swap :icon-in-card-copy :icon-b :icon-b-copy)))
|
||||
|
||||
(t/deftest test-keep-swap-slot-duplicating-swapped-nested-copy-inside-copy
|
||||
(let [;; ==== Setup
|
||||
file (setup-nested-swap-file)
|
||||
|
||||
page (thf/current-page file)
|
||||
icon-b-copy (ths/get-shape file :icon-b-copy)
|
||||
|
||||
;; ==== Action
|
||||
changes (-> (pcb/empty-changes nil)
|
||||
(cll/generate-duplicate-changes (:objects page) ;; objects
|
||||
page ;; page
|
||||
#{(:id icon-b-copy)} ;; ids
|
||||
(gpt/point 0 0) ;; delta
|
||||
{(:id file) file} ;; libraries
|
||||
(:data file) ;; library-data
|
||||
(:id file)) ;; file-id
|
||||
(cll/generate-duplicate-changes-update-indices (:objects page) ;; objects
|
||||
#{(:id icon-b-copy)})) ;; ids
|
||||
|
||||
file' (thf/apply-changes file changes)
|
||||
|
||||
;; ==== Get
|
||||
page' (thf/current-page file')
|
||||
duplicated-icon-b' (find-duplicated-shape icon-b-copy page')]
|
||||
|
||||
;; ==== Check
|
||||
|
||||
;; icon-b-copy has swap-slot
|
||||
(t/is (some? (ctk/get-swap-slot icon-b-copy)))
|
||||
|
||||
;; duplicated icon-b copy keeps its swap-slot
|
||||
(t/is (some? duplicated-icon-b'))
|
||||
(t/is (some? (ctk/get-swap-slot duplicated-icon-b')))))
|
||||
|
||||
@ -19,6 +19,7 @@
|
||||
[app.common.test-helpers.shapes :as ths]
|
||||
[app.common.types.component :as ctk]
|
||||
[app.common.types.container :as ctn]
|
||||
[app.common.types.modifiers :as ctm]
|
||||
[app.common.types.shape-tree :as ctst]
|
||||
[clojure.test :as t]))
|
||||
|
||||
@ -552,6 +553,330 @@
|
||||
{(:id file) file}
|
||||
(:id file)))
|
||||
|
||||
(defn- changes-attrs
|
||||
"Set of attrs touched by the :mod-obj changes of a changes map."
|
||||
[changes]
|
||||
(->> (:redo-changes changes)
|
||||
(filter #(= :mod-obj (:type %)))
|
||||
(mapcat :operations)
|
||||
(map :attr)
|
||||
(into #{})))
|
||||
|
||||
(defn- file-with-path-component
|
||||
[]
|
||||
(let [content [{:command :move-to :params {:x 10.3 :y 10.7}}
|
||||
{:command :line-to :params {:x 24.3 :y 10.7}}
|
||||
{:command :line-to :params {:x 17.3 :y 24.7}}
|
||||
{:command :close-path}]]
|
||||
(-> (thf/sample-file :file1)
|
||||
(tho/add-frame :main-root {:width 107.3 :height 41.7})
|
||||
(ths/add-sample-shape :main-path :type :path :parent-label :main-root :content content)
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-path]}))))
|
||||
|
||||
(t/deftest test-no-sync-changes-when-position-changes-with-float-drift
|
||||
;; Regression (#12719): geometry in production carries sub-pixel drift
|
||||
;; and non-integer sizes, so move(+delta) followed by reposition(-delta)
|
||||
;; does not cancel exactly. A position-only move of the main must still
|
||||
;; produce no sync changes, and so no "library updated" notification.
|
||||
(let [;; ==== Setup
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-simple-component-with-copy :component1
|
||||
:main-root
|
||||
:main-child
|
||||
:copy-root
|
||||
:main-root-params {:width 107.3 :height 41.7}
|
||||
:main-child-params {:width 33.3 :height 17.7}
|
||||
:copy-root-params {:children-labels [:copy-child]}))
|
||||
page (thf/current-page file)
|
||||
main-root (ths/get-shape file :main-root)
|
||||
main-child (ths/get-shape file :main-child)
|
||||
copy-child (ths/get-shape file :copy-child)
|
||||
|
||||
;; Override on the copy, so it carries touched state
|
||||
changes0 (cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id copy-child)}
|
||||
(fn [shape]
|
||||
(assoc shape :fills (ths/sample-fills-color :fill-color "#fabada")))
|
||||
(:objects page)
|
||||
{})
|
||||
file (thf/apply-changes file changes0)
|
||||
page (thf/current-page file)
|
||||
|
||||
;; ==== Action
|
||||
;; Move the whole main by 1 px plus sub-pixel noise
|
||||
delta (gpt/point 1.0000123 0.0000071)
|
||||
changes1 (cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id main-root) (:id main-child)}
|
||||
(fn [shape] (gsh/move shape delta))
|
||||
(:objects page)
|
||||
{})
|
||||
|
||||
updated-file (thf/apply-changes file changes1)
|
||||
|
||||
sync-changes (cll/generate-sync-file-changes (pcb/empty-changes)
|
||||
nil
|
||||
:components
|
||||
(:id updated-file)
|
||||
(thi/id :component1)
|
||||
(:id updated-file)
|
||||
{(:id updated-file) updated-file}
|
||||
(:id updated-file))]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (empty? (:redo-changes sync-changes)))))
|
||||
|
||||
(t/deftest test-sync-size-change-of-main-child
|
||||
;; The drift tolerance must not swallow real size edits (>> 0.001).
|
||||
(let [;; ==== Setup
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-simple-component-with-copy :component1
|
||||
:main-root
|
||||
:main-child
|
||||
:copy-root
|
||||
:main-root-params {:width 107.3 :height 41.7}
|
||||
:main-child-params {:width 33.3 :height 17.7}
|
||||
:copy-root-params {:children-labels [:copy-child]}))
|
||||
page (thf/current-page file)
|
||||
|
||||
;; ==== Action
|
||||
updated-file (thf/apply-changes file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(thi/id :main-child)}
|
||||
#(gsh/transform-shape % (ctm/change-size % (+ (:width %) 1) nil))
|
||||
(:objects page)
|
||||
{}))
|
||||
attrs (changes-attrs (sync-file-changes updated-file))]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? attrs :width))
|
||||
(t/is (contains? attrs :selrect))))
|
||||
|
||||
(t/deftest test-sync-text-position-data-change-with-same-xy
|
||||
;; A change in a text run (here its fill) that keeps x/y must still be
|
||||
;; synced: position-data is not a drifting geometry attr.
|
||||
(let [;; ==== Setup
|
||||
run (fn [color]
|
||||
[{:x 0 :y 0 :width 50 :height 20 :text "hello"
|
||||
:fills [{:fill-color color :fill-opacity 1}]}])
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-frame-with-text :main-root :main-child "hello")
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-child]}))
|
||||
set-pd (fn [file color]
|
||||
(let [page (thf/current-page file)
|
||||
main-child (ths/get-shape file :main-child)]
|
||||
(thf/apply-changes
|
||||
file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id main-child)}
|
||||
#(assoc % :position-data (run color))
|
||||
(:objects page)
|
||||
{}))))
|
||||
|
||||
file1 (set-pd file "#000000")
|
||||
file2 (thf/apply-changes file1 (sync-file-changes file1))
|
||||
file3 (set-pd file2 "#ff0000")
|
||||
|
||||
;; ==== Action
|
||||
sync-changes (sync-file-changes file3)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs sync-changes) :position-data))))
|
||||
|
||||
(t/deftest test-sync-small-opacity-change
|
||||
;; The drift tolerance must not swallow real non-geometry changes.
|
||||
(let [;; ==== Setup
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-simple-component-with-copy :component1
|
||||
:main-root
|
||||
:main-child
|
||||
:copy-root
|
||||
:copy-root-params {:children-labels [:copy-child]}))
|
||||
set-opacity (fn [file opacity]
|
||||
(let [page (thf/current-page file)
|
||||
main-child (ths/get-shape file :main-child)]
|
||||
(thf/apply-changes
|
||||
file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id main-child)}
|
||||
#(assoc % :opacity opacity)
|
||||
(:objects page)
|
||||
{}))))
|
||||
|
||||
file1 (set-opacity file 1)
|
||||
file2 (thf/apply-changes file1 (sync-file-changes file1))
|
||||
file3 (set-opacity file2 0.9995)
|
||||
|
||||
;; ==== Action
|
||||
sync-changes (sync-file-changes file3)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs sync-changes) :opacity))))
|
||||
|
||||
(t/deftest test-no-sync-changes-when-path-moves-with-float-drift
|
||||
;; Path content stores absolute coordinates, so it drifts like the other
|
||||
;; geometry when the main is moved.
|
||||
(let [;; ==== Setup
|
||||
file (file-with-path-component)
|
||||
page (thf/current-page file)
|
||||
main-root (ths/get-shape file :main-root)
|
||||
main-path (ths/get-shape file :main-path)
|
||||
|
||||
;; ==== Action
|
||||
changes1 (cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id main-root) (:id main-path)}
|
||||
#(gsh/move % (gpt/point 1.0000123 0.0000071))
|
||||
(:objects page)
|
||||
{})
|
||||
updated-file (thf/apply-changes file changes1)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (empty? (:redo-changes (sync-file-changes updated-file))))))
|
||||
|
||||
(t/deftest test-sync-path-moved-inside-main
|
||||
;; Moving only the path inside its main is a real change and must sync.
|
||||
(let [;; ==== Setup
|
||||
file (file-with-path-component)
|
||||
page (thf/current-page file)
|
||||
main-path (ths/get-shape file :main-path)
|
||||
|
||||
;; ==== Action
|
||||
changes1 (cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(:id main-path)}
|
||||
#(gsh/move % (gpt/point 1 0))
|
||||
(:objects page)
|
||||
{})
|
||||
updated-file (thf/apply-changes file changes1)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs (sync-file-changes updated-file)) :content))))
|
||||
|
||||
(defn- file-with-bool-component
|
||||
[]
|
||||
(let [content [{:command :move-to :params {:x 10.3 :y 10.7}}
|
||||
{:command :line-to :params {:x 24.3 :y 10.7}}
|
||||
{:command :line-to :params {:x 17.3 :y 24.7}}
|
||||
{:command :close-path}]]
|
||||
(-> (thf/sample-file :file1)
|
||||
(tho/add-frame :main-root {:width 107.3 :height 41.7})
|
||||
(ths/add-sample-shape :main-bool :type :bool :bool-type :union :parent-label :main-root :content content)
|
||||
(ths/add-sample-shape :main-p1 :type :path :parent-label :main-bool :content content)
|
||||
(ths/add-sample-shape :main-p2 :type :path :parent-label :main-bool :content content)
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-bool :copy-p1 :copy-p2]}))))
|
||||
|
||||
(defn- move-shapes
|
||||
[file labels delta]
|
||||
(let [page (thf/current-page file)]
|
||||
(thf/apply-changes file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
(into #{} (map #(thi/id %)) labels)
|
||||
#(gsh/move % delta)
|
||||
(:objects page)
|
||||
{}))))
|
||||
|
||||
(t/deftest test-no-sync-changes-when-text-moves-with-float-drift
|
||||
;; Text position-data moves with the shape and drifts like geometry, so
|
||||
;; a position-only move of the main must produce no sync changes.
|
||||
(let [;; ==== Setup
|
||||
position-data [{:x 10.3 :y 10.7 :width 33.3 :height 17.7 :text "hello"
|
||||
:fills [{:fill-color "#000000" :fill-opacity 1}]}]
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-frame-with-text :main-root :main-child "hello")
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-child]}))
|
||||
page (thf/current-page file)
|
||||
file (thf/apply-changes file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(thi/id :main-child)}
|
||||
#(assoc % :position-data position-data)
|
||||
(:objects page)
|
||||
{}))
|
||||
file (thf/apply-changes file (sync-file-changes file))
|
||||
|
||||
;; ==== Action
|
||||
updated-file (move-shapes file [:main-root :main-child] (gpt/point 1.0000123 0.0000071))
|
||||
sync-changes (sync-file-changes updated-file)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (= #{} (changes-attrs sync-changes)))
|
||||
(t/is (empty? (:redo-changes sync-changes)))))
|
||||
|
||||
(t/deftest test-sync-text-moved-inside-main
|
||||
;; Moving only the text inside its main is a real change beyond the
|
||||
;; tolerance and must sync its position-data.
|
||||
(let [;; ==== Setup
|
||||
position-data [{:x 10.3 :y 10.7 :width 33.3 :height 17.7 :text "hello"
|
||||
:fills [{:fill-color "#000000" :fill-opacity 1}]}]
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-frame-with-text :main-root :main-child "hello")
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-child]}))
|
||||
page (thf/current-page file)
|
||||
file (thf/apply-changes file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(thi/id :main-child)}
|
||||
#(assoc % :position-data position-data)
|
||||
(:objects page)
|
||||
{}))
|
||||
file (thf/apply-changes file (sync-file-changes file))
|
||||
|
||||
;; ==== Action
|
||||
updated-file (move-shapes file [:main-child] (gpt/point 5 0))]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs (sync-file-changes updated-file)) :position-data))))
|
||||
|
||||
(t/deftest test-sync-text-position-data-entry-count-change
|
||||
;; A text that gains a line keeps its first entry untouched but has one
|
||||
;; more entry: a length mismatch is a real change and must sync.
|
||||
(let [;; ==== Setup
|
||||
entry {:x 10.3 :y 10.7 :width 33.3 :height 17.7 :text "hello"
|
||||
:fills [{:fill-color "#000000" :fill-opacity 1}]}
|
||||
set-pd (fn [file position-data]
|
||||
(let [page (thf/current-page file)]
|
||||
(thf/apply-changes file
|
||||
(cls/generate-update-shapes (pcb/empty-changes nil (:id page))
|
||||
#{(thi/id :main-child)}
|
||||
#(assoc % :position-data position-data)
|
||||
(:objects page)
|
||||
{}))))
|
||||
file (-> (thf/sample-file :file1)
|
||||
(tho/add-frame-with-text :main-root :main-child "hello")
|
||||
(thc/make-component :component1 :main-root)
|
||||
(thc/instantiate-component :component1 :copy-root {:children-labels [:copy-child]}))
|
||||
file (set-pd file [entry])
|
||||
file (thf/apply-changes file (sync-file-changes file))
|
||||
|
||||
;; ==== Action
|
||||
updated-file (set-pd file [entry (assoc entry :y 28.7 :text "world")])]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs (sync-file-changes updated-file)) :position-data))))
|
||||
|
||||
(t/deftest test-no-sync-changes-when-bool-moves-with-float-drift
|
||||
;; Bool content stores absolute coordinates, so it drifts like path content.
|
||||
(let [;; ==== Action
|
||||
file (file-with-bool-component)
|
||||
updated-file (move-shapes file
|
||||
[:main-root :main-bool :main-p1 :main-p2]
|
||||
(gpt/point 1.0000123 0.0000071))
|
||||
sync-changes (sync-file-changes updated-file)]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (= #{} (changes-attrs sync-changes)))
|
||||
(t/is (empty? (:redo-changes sync-changes)))))
|
||||
|
||||
(t/deftest test-sync-bool-content-change-inside-main
|
||||
;; Moving only the bool inside its main is a real change and must sync.
|
||||
(let [;; ==== Action
|
||||
file (file-with-bool-component)
|
||||
updated-file (move-shapes file [:main-bool :main-p1 :main-p2] (gpt/point 1 0))]
|
||||
|
||||
;; ==== Check
|
||||
(t/is (contains? (changes-attrs (sync-file-changes updated-file)) :content))))
|
||||
|
||||
(t/deftest test-sync-many-updated-shapes
|
||||
;; The undo changes of a sync used to grow one lazy level per updated
|
||||
;; shape (#12576). Overflowing the stack needs thousands of shapes, which
|
||||
|
||||
202
common/test/common_tests/logic/variants_paste_test.cljc
Normal file
202
common/test/common_tests/logic/variants_paste_test.cljc
Normal file
@ -0,0 +1,202 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns common-tests.logic.variants-paste-test
|
||||
(:require
|
||||
[app.common.data :as d]
|
||||
[app.common.files.changes-builder :as pcb]
|
||||
[app.common.files.helpers :as cfh]
|
||||
[app.common.files.validate :as cfv]
|
||||
[app.common.geom.point :as gpt]
|
||||
[app.common.logic.libraries :as cll]
|
||||
[app.common.logic.variant-properties :as clvp]
|
||||
[app.common.test-helpers.components :as thc]
|
||||
[app.common.test-helpers.compositions :as tho]
|
||||
[app.common.test-helpers.files :as thf]
|
||||
[app.common.test-helpers.ids-map :as thi]
|
||||
[app.common.test-helpers.shapes :as ths]
|
||||
[app.common.test-helpers.variants :as thv]
|
||||
[app.common.types.component :as ctk]
|
||||
[app.common.types.variant :as ctv]
|
||||
[clojure.test :as t]))
|
||||
|
||||
(t/use-fixtures :each thi/test-fixture)
|
||||
|
||||
(defn- paste-variants-into-container
|
||||
"Paste individual variants from file-a into file-b's variant container.
|
||||
Omit file-a from `libraries` to hit the unknown-component path; include
|
||||
it to hit `duplicate-variant`. Optional `variant-props` mirrors clipboard
|
||||
collection (Task 2)."
|
||||
[file-a file-b source-labels target-container-label
|
||||
& {:keys [libraries variant-props]
|
||||
:or {libraries {(:id file-b) file-b}}}]
|
||||
(let [page-a (thf/current-page file-a)
|
||||
page-b (thf/current-page file-b)
|
||||
container-b (ths/get-shape file-b target-container-label)
|
||||
parent-id (:id container-b)
|
||||
|
||||
source-ids (mapv thi/id source-labels)
|
||||
tree-ids (into []
|
||||
(mapcat #(cfh/get-children-ids-with-self (:objects page-a) %))
|
||||
source-ids)
|
||||
|
||||
pasted (->> tree-ids
|
||||
(map (d/getf (:objects page-a)))
|
||||
(map (fn [shape]
|
||||
(cond-> shape
|
||||
(contains? (set source-ids) (:id shape))
|
||||
(assoc :parent-id parent-id
|
||||
:frame-id parent-id))))
|
||||
(d/index-by :id))
|
||||
|
||||
all-objects (merge (:objects page-b) pasted)]
|
||||
(-> (pcb/empty-changes nil)
|
||||
(pcb/with-page-id (:id page-b))
|
||||
(pcb/with-library-data (:data file-b))
|
||||
(pcb/with-objects all-objects)
|
||||
(cll/generate-duplicate-changes all-objects
|
||||
page-b
|
||||
(set source-ids)
|
||||
(gpt/point 0 0)
|
||||
libraries
|
||||
(:data file-b)
|
||||
(:id file-b)
|
||||
{:variant-props variant-props}))))
|
||||
|
||||
(defn- property-names
|
||||
[component]
|
||||
(mapv :name (:variant-properties component)))
|
||||
|
||||
(defn- variant-mains-in
|
||||
[file container-label]
|
||||
(let [container (ths/get-shape file container-label)]
|
||||
(->> (vals (:objects (thf/current-page file)))
|
||||
(filter ctk/main-instance?)
|
||||
(filter #(= (:id container) (:variant-id %))))))
|
||||
|
||||
(defn- other-set-file
|
||||
[]
|
||||
(-> (thf/sample-file :file-b)
|
||||
(thv/add-variant-two-properties :vb :cb1 :mb1 :cb2 :mb2)
|
||||
(ths/update-shape :vb :name "Other")
|
||||
(ths/update-shape :mb1 :name "Other")
|
||||
(ths/update-shape :mb2 :name "Other")
|
||||
(thc/update-component :cb1 {:name "Other"})
|
||||
(thc/update-component :cb2 {:name "Other"})))
|
||||
|
||||
(t/deftest test-paste-unknown-variant-into-container-validates
|
||||
;; #10704: paste individual variants from file A into file B's set
|
||||
;; when A is not in libraries must leave a valid file (desired behavior).
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
errors (cfv/validate-file file' {})
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %))))]
|
||||
|
||||
(t/is (nil? errors))
|
||||
(doseq [shape pasted]
|
||||
(let [comp (thc/get-component-by-id file' (:component-id shape))]
|
||||
(t/is (= ["Property 1" "Property 2"] (property-names comp)))
|
||||
(t/is (= "Other" (:name shape)))
|
||||
(t/is (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties comp))))))))
|
||||
|
||||
(t/deftest test-duplicate-variant-in-own-container-bumps-last-value
|
||||
;; Same-container duplicate via duplicate-variant: originals keep their
|
||||
;; props; the new main gets a bumped last-property value. Does not cover
|
||||
;; generate-make-shapes-variant's already-aligned early-exit (new mains
|
||||
;; enter with a nil variant-id).
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
c01-props (:variant-properties (thc/get-component file :c01))
|
||||
c02-props (:variant-properties (thc/get-component file :c02))
|
||||
file' (tho/duplicate-shape file :m01)
|
||||
mains (variant-mains-in file' :v01)
|
||||
new-main (->> mains
|
||||
(remove #(#{(thi/id :m01) (thi/id :m02)} (:id %)))
|
||||
first)
|
||||
new-comp (thc/get-component-by-id file' (:component-id new-main))]
|
||||
|
||||
(thf/validate-file! file')
|
||||
(t/is (= 3 (count mains)))
|
||||
(t/is (= c01-props (:variant-properties (thc/get-component file' :c01))))
|
||||
(t/is (= c02-props (:variant-properties (thc/get-component file' :c02))))
|
||||
(t/is (= ["Property 1"] (property-names new-comp)))
|
||||
(t/is (= (str ctv/value-prefix 3)
|
||||
(-> new-comp :variant-properties last :value)))))
|
||||
|
||||
(t/deftest test-make-shapes-variant-skips-when-already-aligned
|
||||
;; Locks the narrowed early-exit: a fully aligned child must not have its
|
||||
;; props or variant-name rewritten.
|
||||
(let [file (-> (thf/sample-file :file1)
|
||||
(thv/add-variant :v01 :c01 :m01 :c02 :m02))
|
||||
page (thf/current-page file)
|
||||
container (ths/get-shape file :v01)
|
||||
shape (ths/get-shape file :m01)
|
||||
before (:variant-properties (thc/get-component file :c01))
|
||||
before-vn (:variant-name shape)
|
||||
changes (-> (pcb/empty-changes nil)
|
||||
(pcb/with-page page)
|
||||
(pcb/with-objects (:objects page))
|
||||
(pcb/with-library-data (:data file))
|
||||
(clvp/generate-make-shapes-variant [shape] container))
|
||||
file' (thf/apply-changes file changes)
|
||||
shape' (ths/get-shape-by-id file' (:id shape))]
|
||||
|
||||
(t/is (= before (:variant-properties (thc/get-component file' :c01))))
|
||||
(t/is (= before-vn (:variant-name shape')))))
|
||||
|
||||
(t/deftest test-paste-known-variant-into-other-container-aligns
|
||||
;; file A present in libraries → duplicate-variant path; props must match
|
||||
;; the target set and the file must validate.
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
libraries {(:id file-a) file-a
|
||||
(:id file-b) file-b}
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb
|
||||
:libraries libraries)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
errors (cfv/validate-file file' {})
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %))))]
|
||||
|
||||
(t/is (nil? errors))
|
||||
(t/is (= 2 (count pasted)))
|
||||
(doseq [shape pasted]
|
||||
(let [comp (thc/get-component-by-id file' (:component-id shape))]
|
||||
(t/is (= ["Property 1" "Property 2"] (property-names comp)))
|
||||
(t/is (= "Other" (:name shape)))
|
||||
(t/is (= (:variant-name shape)
|
||||
(ctv/properties-to-name (:variant-properties comp))))))))
|
||||
|
||||
(t/deftest test-paste-unknown-preserves-matching-property-values
|
||||
;; Clipboard props (Task 2) whose names exist on the target survive regenerate.
|
||||
(let [file-a (-> (thf/sample-file :file-a)
|
||||
(thv/add-variant :va :ca1 :ma1 :ca2 :ma2))
|
||||
file-b (other-set-file)
|
||||
props {(:id (thc/get-component file-a :ca1))
|
||||
(:variant-properties (thc/get-component file-a :ca1))
|
||||
(:id (thc/get-component file-a :ca2))
|
||||
(:variant-properties (thc/get-component file-a :ca2))}
|
||||
changes (paste-variants-into-container file-a file-b [:ma1 :ma2] :vb
|
||||
:variant-props props)
|
||||
file' (thf/apply-changes file-b changes :validate? false)
|
||||
pasted (->> (variant-mains-in file' :vb)
|
||||
(remove #(#{(thi/id :mb1) (thi/id :mb2)} (:id %)))
|
||||
(map #(thc/get-component-by-id file' (:component-id %))))
|
||||
values (->> pasted
|
||||
(mapcat :variant-properties)
|
||||
(filter #(= "Property 1" (:name %)))
|
||||
(map :value)
|
||||
set)]
|
||||
|
||||
(thf/validate-file! file')
|
||||
(t/is (contains? values "Value1"))
|
||||
(t/is (contains? values "Value2"))))
|
||||
@ -57,3 +57,21 @@
|
||||
:clj
|
||||
(t/testing "finite? returns false for Double/NaN (CLJ)"
|
||||
(t/is (false? (mth/finite? Double/NaN))))))
|
||||
|
||||
(t/deftest close-map?-test
|
||||
(t/testing "numbers within tolerance are close"
|
||||
(t/is (mth/close-map? {:x 1.0 :y 2.0} {:x 1.0004 :y 2.0})))
|
||||
|
||||
(t/testing "numbers beyond tolerance are not close"
|
||||
(t/is (not (mth/close-map? {:x 1.0} {:x 1.01}))))
|
||||
|
||||
(t/testing "non-numeric values compare exactly"
|
||||
(t/is (mth/close-map? {:text "a"} {:text "a"}))
|
||||
(t/is (not (mth/close-map? {:text "a"} {:text "b"}))))
|
||||
|
||||
(t/testing "different key sets are not close"
|
||||
(t/is (not (mth/close-map? {:x 1.0} {:x 1.0 :y 1.0})))
|
||||
(t/is (not (mth/close-map? {:x 1.0 :y nil} {:x 1.0 :z nil}))))
|
||||
|
||||
(t/testing "a number is not close to a non-number"
|
||||
(t/is (not (mth/close-map? {:x 1.0} {:x "1.0"})))))
|
||||
|
||||
@ -37,6 +37,11 @@
|
||||
(t/is (= 400 (media/parse-font-weight "Thinker")))
|
||||
(t/is (= 400 (media/parse-font-weight "Mediaeval"))))
|
||||
|
||||
(t/testing "matches a weight glued to a style token"
|
||||
(t/is (= 700 (media/parse-font-weight "Roboto-BoldItalic")))
|
||||
(t/is (= 700 (media/parse-font-weight "Roboto-BoldOblique")))
|
||||
(t/is (= 600 (media/parse-font-weight "Roboto-SemiBoldItalic"))))
|
||||
|
||||
(t/testing "defaults to regular when the variant metadata is missing"
|
||||
(t/is (= 400 (media/parse-font-weight nil)))
|
||||
(t/is (= 400 (media/parse-font-weight "")))
|
||||
@ -54,11 +59,79 @@
|
||||
(t/is (= "normal" (media/parse-font-style "Italica")))
|
||||
(t/is (= "normal" (media/parse-font-style "Roboto-Regular"))))
|
||||
|
||||
(t/testing "treats oblique as italic"
|
||||
(t/is (= "italic" (media/parse-font-style "Roboto-Oblique")))
|
||||
(t/is (= "italic" (media/parse-font-style "Roboto-BoldOblique")))
|
||||
(t/is (= "italic" (media/parse-font-style "RobotoOblique"))))
|
||||
|
||||
(t/testing "defaults to normal when the variant metadata is missing"
|
||||
(t/is (= "normal" (media/parse-font-style nil)))
|
||||
(t/is (= "normal" (media/parse-font-style "")))
|
||||
(t/is (= "normal" (media/parse-font-style " ")))))
|
||||
|
||||
(t/deftest test-parse-font-family
|
||||
(t/testing "strips weight and style tokens"
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-Bold")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto_Bold")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto Bold")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-Oblique")))
|
||||
(t/is (= "OpenSans" (media/parse-font-family "OpenSans-SemiBold")))
|
||||
(t/is (= "Lato" (media/parse-font-family "Lato-Light"))))
|
||||
|
||||
(t/testing "separates a style token glued to a weight token"
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-BoldItalic")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-SemiBoldItalic")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-LightOblique")))
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-BlackOblique"))))
|
||||
|
||||
(t/testing "does not strip tokens embedded in words"
|
||||
(t/is (= "Boldini" (media/parse-font-family "Boldini-Bold")))
|
||||
(t/is (= "Lighthaus" (media/parse-font-family "Lighthaus")))
|
||||
(t/is (= "Blackwood" (media/parse-font-family "Blackwood")))
|
||||
(t/is (= "Mediaeval" (media/parse-font-family "Mediaeval"))))
|
||||
|
||||
(t/testing "keeps unknown tokens in the family"
|
||||
(t/is (= "Satoshi Variable" (media/parse-font-family "Satoshi-Variable"))))
|
||||
|
||||
(t/testing "falls back to the base name when everything is stripped"
|
||||
(t/is (= "BoldItalic" (media/parse-font-family "BoldItalic")))))
|
||||
|
||||
(t/deftest test-font-glued-tokens
|
||||
(t/testing "a glued style token is recognized consistently by all three parsers"
|
||||
;; From EvaMarco's review on #11951: without a shared interpretation of
|
||||
;; the name, the bold-italic face collided with the bold-only one and
|
||||
;; was silently overwritten on upload.
|
||||
(t/is (= "opensans webfont" (media/parse-font-family "opensans-bolditalic-webfont")))
|
||||
(t/is (= 700 (media/parse-font-weight "opensans-bolditalic-webfont")))
|
||||
(t/is (= "italic" (media/parse-font-style "opensans-bolditalic-webfont"))))
|
||||
|
||||
(t/testing "a glued style token followed by a technical suffix"
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-BoldItalic-VF")))
|
||||
(t/is (= 700 (media/parse-font-weight "Roboto-BoldItalic-VF")))
|
||||
(t/is (= "italic" (media/parse-font-style "Roboto-BoldItalic-VF"))))
|
||||
|
||||
(t/testing "numeric build suffix after a glued style token"
|
||||
(t/is (= "Roboto" (media/parse-font-family "Roboto-LightItalic_v2")))
|
||||
(t/is (= 300 (media/parse-font-weight "Roboto-LightItalic_v2")))
|
||||
(t/is (= "italic" (media/parse-font-style "Roboto-LightItalic_v2"))))
|
||||
|
||||
(t/testing "the bold-only control case is unchanged"
|
||||
(t/is (= "opensans webfont" (media/parse-font-family "opensans-bold-webfont")))
|
||||
(t/is (= 700 (media/parse-font-weight "opensans-bold-webfont")))
|
||||
(t/is (= "normal" (media/parse-font-style "opensans-bold-webfont")))))
|
||||
|
||||
(t/deftest test-first-nonblank-string
|
||||
(t/testing "returns the first nonblank string"
|
||||
(t/is (= "Bold" (media/first-nonblank-string " " "Bold")))
|
||||
(t/is (= "Bold" (media/first-nonblank-string "Bold" "Light")))
|
||||
(t/is (= "Bold" (media/first-nonblank-string nil "Bold")))
|
||||
(t/is (= "Inter" (media/first-nonblank-string "" "Inter"))))
|
||||
|
||||
(t/testing "returns nil when every candidate is blank or not a string"
|
||||
(t/is (nil? (media/first-nonblank-string nil " ")))
|
||||
(t/is (nil? (media/first-nonblank-string)))
|
||||
(t/is (nil? (media/first-nonblank-string 42 " ")))))
|
||||
|
||||
(t/deftest test-strip-image-extension
|
||||
(t/testing "removes extension from supported image files"
|
||||
(t/is (= (media/strip-image-extension "foo.png") "foo"))
|
||||
|
||||
@ -22,6 +22,7 @@
|
||||
[common-tests.files-changes-test]
|
||||
[common-tests.files-migrations-0025-test]
|
||||
[common-tests.files-migrations-0026-test]
|
||||
[common-tests.files-migrations-0033-test]
|
||||
[common-tests.files-migrations-0034-test]
|
||||
[common-tests.files-migrations-0035-test]
|
||||
[common-tests.files-migrations-test]
|
||||
@ -32,6 +33,7 @@
|
||||
[common-tests.files.shapes-builder-test]
|
||||
[common-tests.files.tokens-test]
|
||||
[common-tests.files.validate-test]
|
||||
[common-tests.files.variant-test]
|
||||
[common-tests.geom-align-test]
|
||||
[common-tests.geom-bounds-layout-nil-test]
|
||||
[common-tests.geom-bounds-map-test]
|
||||
@ -81,6 +83,7 @@
|
||||
[common-tests.logic.text-touched-test]
|
||||
[common-tests.logic.token-apply-test]
|
||||
[common-tests.logic.token-test]
|
||||
[common-tests.logic.variants-paste-test]
|
||||
[common-tests.logic.variants-reparent-test]
|
||||
[common-tests.logic.variants-switch-test]
|
||||
[common-tests.logic.variants-test]
|
||||
@ -136,6 +139,7 @@
|
||||
'common-tests.files-changes-test
|
||||
'common-tests.files-migrations-0025-test
|
||||
'common-tests.files-migrations-0026-test
|
||||
'common-tests.files-migrations-0033-test
|
||||
'common-tests.files-migrations-0034-test
|
||||
'common-tests.files-migrations-0035-test
|
||||
'common-tests.files-migrations-test
|
||||
@ -146,6 +150,7 @@
|
||||
'common-tests.files.shapes-builder-test
|
||||
'common-tests.files.tokens-test
|
||||
'common-tests.files.validate-test
|
||||
'common-tests.files.variant-test
|
||||
'common-tests.geom-align-test
|
||||
'common-tests.geom-bounds-layout-nil-test
|
||||
'common-tests.geom-bounds-map-test
|
||||
@ -195,6 +200,7 @@
|
||||
'common-tests.logic.text-touched-test
|
||||
'common-tests.logic.token-apply-test
|
||||
'common-tests.logic.token-test
|
||||
'common-tests.logic.variants-paste-test
|
||||
'common-tests.logic.variants-reparent-test
|
||||
'common-tests.logic.variants-switch-test
|
||||
'common-tests.logic.variants-test
|
||||
|
||||
@ -291,3 +291,42 @@
|
||||
(t/testing "two ordinary outside shapes are both kept"
|
||||
(t/is (= [outside-id outside-2-id]
|
||||
(vec (ctc/ids-for-structure-op objects [outside-id outside-2-id])))))))
|
||||
|
||||
(t/deftest get-nesting-level-delta
|
||||
(let [page-id uuid/zero
|
||||
head-id (uuid/next)
|
||||
frame-id (uuid/next)
|
||||
sub-id (uuid/next)
|
||||
head-2-id (uuid/next)
|
||||
plain-id (uuid/next)
|
||||
;; copy head with a plain frame and a nested copy head inside
|
||||
head (make-shape head-id
|
||||
:parent-id page-id
|
||||
:type :frame
|
||||
:component-id (uuid/next)
|
||||
:component-root true
|
||||
:shape-ref (uuid/next))
|
||||
frame (make-shape frame-id :parent-id head-id :type :frame :shape-ref (uuid/next))
|
||||
sub (make-shape sub-id
|
||||
:parent-id frame-id
|
||||
:type :frame
|
||||
:component-id (uuid/next)
|
||||
:shape-ref (uuid/next))
|
||||
head-2 (make-shape head-2-id
|
||||
:parent-id page-id
|
||||
:type :frame
|
||||
:component-id (uuid/next)
|
||||
:component-root true
|
||||
:shape-ref (uuid/next))
|
||||
plain (make-shape plain-id :parent-id page-id)
|
||||
root (root-shape)
|
||||
objects (objects-map root head frame sub head-2 plain)]
|
||||
(t/testing "shape stays under the same copy head"
|
||||
(t/is (= 0 (ctc/get-nesting-level-delta objects sub frame))))
|
||||
(t/testing "shape leaves its copy head to the page"
|
||||
(t/is (= 1 (ctc/get-nesting-level-delta objects sub root))))
|
||||
(t/testing "shape moves to a different copy head"
|
||||
(t/is (= 1 (ctc/get-nesting-level-delta objects sub head-2))))
|
||||
(t/testing "shape outside any copy head goes inside one"
|
||||
(t/is (= 0 (ctc/get-nesting-level-delta objects plain frame))))))
|
||||
|
||||
|
||||
@ -2864,3 +2864,30 @@
|
||||
result (path/splice-content content sub)]
|
||||
(t/is (path.impl/path-data? result))
|
||||
(t/is (= (into (vec content) sub) (vec result)))))
|
||||
|
||||
(t/deftest content-close?
|
||||
(let [content [{:command :move-to :params {:x 10.3 :y 10.7}}
|
||||
{:command :line-to :params {:x 24.3 :y 10.7}}
|
||||
{:command :close-path}]
|
||||
shifted (fn [f] (update-in content [1 :params :x] f))]
|
||||
(t/testing "params within tolerance are close"
|
||||
(t/is (path.segment/content-close? content (shifted #(+ % 0.0004)))))
|
||||
|
||||
(t/testing "a param beyond tolerance is not close"
|
||||
(t/is (not (path.segment/content-close? content (shifted #(+ % 0.01))))))
|
||||
|
||||
(t/testing "a different command is not close"
|
||||
(t/is (not (path.segment/content-close? content (assoc-in content [1 :command] :move-to)))))
|
||||
|
||||
(t/testing "a different segment count is not close"
|
||||
(t/is (not (path.segment/content-close? content (butlast content)))))
|
||||
|
||||
(t/testing "a different param count is not close"
|
||||
(t/is (not (path.segment/content-close? content (assoc-in content [1 :params] {:x 24.3})))))
|
||||
|
||||
(t/testing "plain vector and PathData are comparable"
|
||||
(t/is (path.segment/content-close? content (path.impl/path-data content))))
|
||||
|
||||
(t/testing "nil equals an empty content, but not a non-empty one"
|
||||
(t/is (path.segment/content-close? nil []))
|
||||
(t/is (not (path.segment/content-close? nil content))))))
|
||||
|
||||
@ -65,7 +65,7 @@ There are three key pieces:
|
||||
|
||||
* **MCP server**: a service that exposes tools to your AI client. It receives requests from the client and forwards them to Penpot.
|
||||
* **MCP plugin in Penpot**: a plugin that runs inside Penpot and connects your open file to the MCP server. It allows the server to access that file and its pages.
|
||||
* **MCP client**: the tool where you write prompts (Cursor, Claude Code, Copilot-style tools, etc.). It connects to the MCP server using a server URL and an MCP key (or your active Penpot session in the current local setup).
|
||||
* **MCP client**: the tool where you write prompts (Cursor, Claude Code, Copilot-style tools, etc.). It connects to the MCP server using a server URL and an MCP key. In remote MCP, the client can get the key by asking you to sign in to Penpot, or you can add it to the URL yourself. The local setup uses your active Penpot session instead.
|
||||
|
||||

|
||||
|
||||
@ -73,7 +73,7 @@ There are three key pieces:
|
||||
|
||||
Some important concepts for users:
|
||||
* **Integrations page**: MCP is configured under **Your account → Integrations → MCP Server**. Here you enable or disable MCP, get the server URL and manage the MCP key.
|
||||
* **MCP key**: a personal, non-recoverable token that authenticates your AI client with the MCP server. Only one key can exist per user at a time. This is used by the remote MCP setup.
|
||||
* **MCP key**: a personal, non-recoverable token that authenticates your AI client with the MCP server. Only one key can exist per user at a time. This is used by the remote MCP setup. Clients connected through Penpot sign-in use this same key.
|
||||
* **MCP session**: a connection from a Penpot tab to the MCP server, identified by a short session ID derived from the tab's user session. Several tabs can connect independently, including tabs showing the same file.
|
||||
* **Current page**: the page open in the selected session. Switching browser focus to another tab does not change which session the agent uses.
|
||||
|
||||
@ -130,7 +130,29 @@ Because **remote MCP** does not expose local file-system access:
|
||||
|
||||
If you just want to try Penpot AI workflows quickly through the MCP, follow this path for the **hosted (remote) MCP server**.
|
||||
|
||||
### Remote MCP in 5 steps
|
||||
### Remote MCP in 3 steps
|
||||
|
||||
Use this path if your MCP client can sign in to remote MCP servers (OAuth), for example Claude Code, Codex, or Claude.ai custom connectors. It works in the official SaaS and in self-hosted instances that [enable it](/technical-guide/configuration/#mcp-sign-in).
|
||||
|
||||
1. #### Add the server to your MCP client
|
||||
Use the server URL without a key:
|
||||
|
||||
```
|
||||
https://<your-penpot-domain>/mcp/stream
|
||||
```
|
||||
|
||||
In the official SaaS the domain is `design.penpot.app`. See [Connect your MCP client](#connect-your-mcp-client) for client-specific commands.
|
||||
|
||||
2. #### Sign in and allow access
|
||||
Your client opens Penpot in the browser. Sign in if needed, check the client name and your account, and choose **Allow access**. Penpot enables MCP and creates your MCP key if you do not have one yet. If you already have a key, Penpot keeps it. Your client stores the key; you never need to copy it.
|
||||
|
||||
3. #### Open a Penpot file and connect MCP
|
||||
In Penpot, open a design file and choose **MCP → Connect** in the toolbar. Allowing access does not connect a file automatically.
|
||||
|
||||
<a id="remote-mcp-in-5-steps"></a>
|
||||
### Remote MCP with an MCP key
|
||||
|
||||
Use this path if your client cannot sign in to remote MCP servers, or if your Penpot instance does not offer sign-in.
|
||||
|
||||
1. #### Enable MCP in Penpot
|
||||
Go to **Your account → Integrations → MCP Server** and enable the feature.
|
||||
@ -226,7 +248,7 @@ You can use Penpot MCP server in two main ways:
|
||||
* Hosted for you (no need to run anything on your machine).
|
||||
* Best option for most users, simpler installation and fewer moving parts.
|
||||
* Does **not** have privileged access to your local file system, it can only work with what Penpot exposes (design files, libraries, tokens, etc.).
|
||||
* The **server URL** is provided in **Your account → Integrations → MCP Server** and looks like:
|
||||
* The **server URL** is `https://<your-penpot-domain>/mcp/stream`. Clients that sign in to Penpot use it as it is. Other clients use the URL shown in **Your account → Integrations → MCP Server**, which adds your key:
|
||||
* `https://<your-penpot-domain>/mcp/stream?userToken=YOUR_MCP_KEY`
|
||||
* The domain depends on the Penpot installation. In the official SaaS it will be `design.penpot.app`.
|
||||
* **Local MCP server**
|
||||
@ -239,10 +261,17 @@ You can use Penpot MCP server in two main ways:
|
||||
|
||||
## Connect your MCP client
|
||||
|
||||
* **Remote MCP**
|
||||
* **Remote MCP, signing in to Penpot**
|
||||
* URL: `https://<your-penpot-domain>/mcp/stream`, with no key.
|
||||
* Add the server to your client, then start the sign-in. Penpot opens in the browser and asks you to allow access. For example:
|
||||
* Claude Code: run `claude mcp add --transport http penpot https://<your-penpot-domain>/mcp/stream`, then run `/mcp`, select `penpot` and choose **Authenticate**.
|
||||
* Codex: run `codex mcp add penpot --url https://<your-penpot-domain>/mcp/stream`, then `codex mcp login penpot`.
|
||||
* Claude.ai and Claude Desktop: add a custom [connector](http://claude.ai/customize/connectors) with the URL above and choose **Connect**.
|
||||
* Other clients: follow your client's instructions for remote MCP servers that use OAuth.
|
||||
* **Remote MCP, with the MCP key in the URL**
|
||||
* URL (copy from your Penpot account overview): `https://<your-penpot-domain>/mcp/stream?userToken=YOUR_MCP_KEY`
|
||||
* Configure your MCP client with `npx -y add-mcp -g -n penpot <URL>` or by following your client's instructions for connecting to a remote MCP server.
|
||||
For Claude Desktop, the server can be added as a custom [connector](http://claude.ai/customize/connectors).
|
||||
* Use this option for clients that cannot sign in, or if your Penpot instance does not offer sign-in.
|
||||
* **Local MCP**
|
||||
* Configure your MCP client with `npx -y add-mcp -g -n penpot http://localhost:4401/mcp` (adjust the port if you changed `PENPOT_MCP_SERVER_PORT`) or by following your client's instructions for connecting to an HTTP MCP server.
|
||||
|
||||
@ -273,6 +302,8 @@ Remote MCP is the easiest way to start using AI agents with Penpot. It's hosted
|
||||
<a id="install-and-activate-remote"></a>
|
||||
### Install and activate
|
||||
|
||||
If your client signs in to Penpot, you can skip this section: allowing access from the client enables MCP and creates your key. See [Remote MCP in 3 steps](#remote-mcp-in-3-steps).
|
||||
|
||||
1. Open **Your account → Integrations**.
|
||||
2. In the **MCP Server** section, read the short description to confirm that feature is available for your account.
|
||||
3. Use the **Status** toggle to enable MCP Server. Penpot remembers this state per user across sessions. You still need to connect each tab explicitly.
|
||||
@ -286,8 +317,16 @@ Remote MCP is the easiest way to start using AI agents with Penpot. It's hosted
|
||||
<a id="connect-remote"></a>
|
||||
### Connect
|
||||
|
||||
For remote mode, use the URL shown in **Your account → Integrations → MCP Server**, which includes your `userToken`.
|
||||
See the section **Connect your MCP client** for details on how to connect.
|
||||
There are two ways to connect a client in remote mode:
|
||||
|
||||
* **Sign in to Penpot.** Add `https://<your-penpot-domain>/mcp/stream` to your client without a key. When the client starts the sign-in, Penpot opens in the browser. If you are not logged in, Penpot asks you to log in first. The approval page shows the client that is asking for access and the account you are signed in with. Choose **Allow access** to connect, or **Cancel** to stop without changing anything.
|
||||
* **Use the MCP key.** Use the URL shown in **Your account → Integrations → MCP Server**, which includes your `userToken`.
|
||||
|
||||
Allowing access enables MCP for your account and gives the client your MCP key. Penpot creates a key only if you do not have a valid one; an existing key stays the same, so clients that already use it keep working. The client receives the key directly from Penpot. It never appears in the browser address bar.
|
||||
|
||||
If the approval page says the request expired or was already used, start the connection again from your MCP client.
|
||||
|
||||
See the section [Connect your MCP client](#connect-your-mcp-client) for details on how to connect.
|
||||
|
||||
### Setup videos
|
||||
|
||||
@ -355,16 +394,16 @@ If no key exists and you enable MCP, a modal guides you through creating one. Th
|
||||
|
||||
**Regenerate MCP key**
|
||||
|
||||
The **Regenerate MCP key** action immediately revokes the current key. A warning explains that any client using the old key will stop working until you update its configuration. After regenerating, Penpot shows the new key and an updated configuration snippet with the new `userToken` that you need to copy into your MCP client configuration.
|
||||
The **Regenerate MCP key** action immediately revokes the current key. A warning explains that any client using the old key will stop working until you update its configuration. This includes clients connected by signing in to Penpot: sign in again from those clients to give them the new key. After regenerating, Penpot shows the new key and an updated configuration snippet with the new `userToken` that you need to copy into your MCP client configuration.
|
||||
|
||||
**Expired key state**
|
||||
|
||||
If your key is expired, an error block explains that the connection cannot be established until you regenerate the key. Regenerating the key clears the error state.
|
||||
If your key is expired, an error block explains that the connection cannot be established until you regenerate the key. Regenerating the key clears the error state. Signing in again from a client also replaces an expired key with a new one.
|
||||
|
||||
Security recommendations to highlight in the Help Center:
|
||||
|
||||
* Treat your MCP key like a password or access token, do not share it in screenshots or code samples.
|
||||
* Regenerate the key if you suspect it may have leaked.
|
||||
* Regenerate the key if you suspect it may have leaked. Clients connected by signing in to Penpot hold the same key, so regenerating it disconnects them too.
|
||||
* Disabling MCP Server disconnects your integrated sessions. Disconnecting one tab stops access through that connection; other connected sessions remain available.
|
||||
|
||||
***
|
||||
|
||||
@ -983,6 +983,33 @@ set, and sends it through the [email configuration](#email-configuration). Each
|
||||
email carries the name and email address of the user who sent it, and uses that
|
||||
address as reply-to.
|
||||
|
||||
#### MCP sign-in
|
||||
|
||||
MCP clients with OAuth support can connect with the URL
|
||||
`https://<your-penpot-domain>/mcp/stream`, without a key. The client opens Penpot
|
||||
in the browser, the user allows access, and Penpot gives the client the user's MCP
|
||||
key. This is off by default. Turn it on by setting the public URL of the
|
||||
authorization service on the MCP container:
|
||||
|
||||
```bash
|
||||
# MCP
|
||||
PENPOT_MCP_AUTH_URI: https://penpot.example.com/api/mcp/oauth
|
||||
```
|
||||
|
||||
- `PENPOT_MCP_AUTH_URI`: Your Penpot public URI followed by `/api/mcp/oauth`. It
|
||||
must use HTTPS and have no credentials, query or fragment. When unset, clients
|
||||
must add the key to the URL as `userToken`.
|
||||
|
||||
The MCP server must run in multi-user mode, which the official `penpotapp/mcp`
|
||||
image does by default. The server does not start if the value is invalid, and it
|
||||
logs at startup whether sign-in is on.
|
||||
|
||||
The frontend container forwards the sign-in requests to the backend. If your proxy
|
||||
sends `/api/` straight to the backend instead, send
|
||||
`/.well-known/oauth-authorization-server/api/mcp/oauth` to the backend too.
|
||||
|
||||
Clients that already add the key to the URL keep working with sign-in on.
|
||||
|
||||
### Internal resolver
|
||||
|
||||
The frontend container resolves the backend, exporter and MCP service names with the
|
||||
|
||||
@ -86,6 +86,13 @@ desc: Streamline your design workflow with Penpot's Components guide! Learn to c
|
||||
<img src="/img/variants/04-variants-properties-edit.webp" alt="Edit variant property" />
|
||||
</figure>
|
||||
|
||||
<h5>Order properties and values</h5>
|
||||
<ul>
|
||||
<li><strong>Properties:</strong> In the Design tab, drag a property by its handle to move it up or down.</li>
|
||||
<li><strong>Values:</strong> The values dropdown follows the order of the variants in the Layers panel, from top to bottom. Each value appears where the first variant that uses it sits. To change the order, reorder the variants in the Layers panel. Empty values always go last.</li>
|
||||
</ul>
|
||||
<p class="advice">The variant order applies to all properties at once, so you may not be able to set the value order you want for two properties at the same time.</p>
|
||||
|
||||
<h5>Delete properties</h5>
|
||||
<ul>
|
||||
<li><strong>From the Design tab:</strong> Select the main component (not an individual variant) and press the minus button next to the property.</li>
|
||||
|
||||
@ -1 +1,3 @@
|
||||
{ "~:authorized": true }
|
||||
{
|
||||
"~:authorized": true
|
||||
}
|
||||
|
||||
@ -0,0 +1,91 @@
|
||||
[
|
||||
{
|
||||
"~:features": {
|
||||
"~#set": [
|
||||
"layout/grid",
|
||||
"styles/v2",
|
||||
"fdata/pointer-map",
|
||||
"fdata/objects-map",
|
||||
"components/v2",
|
||||
"fdata/shape-data-type"
|
||||
]
|
||||
},
|
||||
"~:permissions": {
|
||||
"~:type": "~:membership",
|
||||
"~:is-owner": true,
|
||||
"~:is-admin": true,
|
||||
"~:can-edit": true
|
||||
},
|
||||
"~:name": "Brand Studio",
|
||||
"~:modified-at": "~m1713533116375",
|
||||
"~:id": "~uc7ce0794-0992-8105-8004-38e630f40f6d",
|
||||
"~:created-at": "~m1713533116375",
|
||||
"~:is-default": true,
|
||||
"~:organization": {
|
||||
"~:id": "~u11111111-1111-4111-8111-111111111111",
|
||||
"~:name": "Brand Studio",
|
||||
"~:slug": "brand-studio",
|
||||
"~:owner-id": "~u22222222-2222-4222-8222-222222222222",
|
||||
"~:avatar-bg-url": null,
|
||||
"~:permissions": {
|
||||
"~:create-teams": "any"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"~:features": {
|
||||
"~#set": [
|
||||
"layout/grid",
|
||||
"styles/v2",
|
||||
"fdata/pointer-map",
|
||||
"fdata/objects-map",
|
||||
"components/v2",
|
||||
"fdata/shape-data-type"
|
||||
]
|
||||
},
|
||||
"~:permissions": {
|
||||
"~:type": "~:membership",
|
||||
"~:is-owner": false,
|
||||
"~:is-admin": false,
|
||||
"~:can-edit": true
|
||||
},
|
||||
"~:name": "Product",
|
||||
"~:modified-at": "~m1713533116375",
|
||||
"~:id": "~udd33ff88-f4e5-8033-8003-8096cc07bdf3",
|
||||
"~:created-at": "~m1713533116375",
|
||||
"~:is-default": true,
|
||||
"~:organization": {
|
||||
"~:id": "~u33333333-3333-4333-8333-333333333333",
|
||||
"~:name": "Product Studio With a Long Organization Name That Must Remain Available",
|
||||
"~:slug": "product-studio",
|
||||
"~:owner-id": "~u44444444-4444-4444-8444-444444444444",
|
||||
"~:avatar-bg-url": null,
|
||||
"~:permissions": {
|
||||
"~:create-teams": "any"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"~:features": {
|
||||
"~#set": [
|
||||
"layout/grid",
|
||||
"styles/v2",
|
||||
"fdata/pointer-map",
|
||||
"fdata/objects-map",
|
||||
"components/v2",
|
||||
"fdata/shape-data-type"
|
||||
]
|
||||
},
|
||||
"~:permissions": {
|
||||
"~:type": "~:membership",
|
||||
"~:is-owner": true,
|
||||
"~:is-admin": true,
|
||||
"~:can-edit": true
|
||||
},
|
||||
"~:name": "Personal projects",
|
||||
"~:modified-at": "~m1713533116375",
|
||||
"~:id": "~ue3c2fd6d-2d3e-47b3-87f6-41b467dff123",
|
||||
"~:created-at": "~m1713533116375",
|
||||
"~:is-default": true
|
||||
}
|
||||
]
|
||||
@ -24,3 +24,66 @@ test("BUG 12359 - Selected invitations count is not pluralized", async ({
|
||||
await page.getByText("test2@mail.com").check();
|
||||
await expect(page.getByText("2 invitations selected")).toBeVisible();
|
||||
});
|
||||
|
||||
async function setupOrganizationTeamSwitcher(page) {
|
||||
await DashboardPage.mockConfigFlags(page, [
|
||||
"disable-onboarding",
|
||||
"enable-admin-console",
|
||||
]);
|
||||
await DashboardPage.mockRPC(
|
||||
page,
|
||||
"get-teams",
|
||||
"dashboard/get-teams-organization-switcher.json",
|
||||
);
|
||||
await DashboardPage.mockRPC(
|
||||
page,
|
||||
"check-nitrate-sso",
|
||||
"dashboard/check-nitrate-sso-authorized.json",
|
||||
);
|
||||
|
||||
const dashboardPage = new DashboardPage(page);
|
||||
await dashboardPage.goToDashboard();
|
||||
return dashboardPage;
|
||||
}
|
||||
|
||||
test("User sees the destination organization when creating a team", async ({
|
||||
page,
|
||||
}) => {
|
||||
const organizationName =
|
||||
"Product Studio With a Long Organization Name That Must Remain Available";
|
||||
const dashboardPage = await setupOrganizationTeamSwitcher(page);
|
||||
|
||||
await dashboardPage.teamDropdown.click();
|
||||
await page.getByRole("menuitem", { name: organizationName }).click();
|
||||
await page.getByRole("menuitem", { name: "Create new team" }).click();
|
||||
|
||||
const title = page.getByRole("heading", {
|
||||
name: `Create new team in ${organizationName}`,
|
||||
exact: true,
|
||||
});
|
||||
await expect(title).toBeVisible();
|
||||
await expect(title).toHaveText(`Create new team in ${organizationName}`);
|
||||
|
||||
const titleFits = await title.evaluate(
|
||||
(element) => element.scrollWidth <= element.clientWidth,
|
||||
);
|
||||
expect(titleFits).toBe(true);
|
||||
|
||||
const modal = title.locator("xpath=../../..");
|
||||
const modalBounds = await modal.boundingBox();
|
||||
expect(modalBounds.width).toBeLessThanOrEqual(512);
|
||||
});
|
||||
|
||||
test("Creating a team under Other teams does not name an organization", async ({
|
||||
page,
|
||||
}) => {
|
||||
const dashboardPage = await setupOrganizationTeamSwitcher(page);
|
||||
|
||||
await dashboardPage.teamDropdown.click();
|
||||
await page.getByRole("menuitem", { name: "Other teams" }).click();
|
||||
await page.getByRole("menuitem", { name: "Create new team" }).click();
|
||||
|
||||
await expect(
|
||||
page.getByRole("heading", { name: "Create new team", exact: true }),
|
||||
).toBeVisible();
|
||||
});
|
||||
|
||||
3
frontend/resources/images/icons/pencil.svg
Normal file
3
frontend/resources/images/icons/pencil.svg
Normal file
@ -0,0 +1,3 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 16">
|
||||
<path d="M10.000089645385742,6.791984558105469L5.012907028198242,0.00038909720024093986L0.00008960580453276634,6.818962097167969L0.00008960580453276634,15.989189147949219L10.000089645385742,16.000358581542969L10.000089645385742,6.791984558105469M5.901090621948242,3.9651718139648438L7.837614059448242,6.597831726074219L2.187589645385742,6.597831726074219L4.104093551635742,3.993461608886719L5.901090621948242,3.9651718139648438M1.6542644500732422,7.930595397949219L4.333707809448242,7.930595397949219L4.333707809448242,15.983268737792969L1.6542644500732422,15.995780944824219L1.6542644500732422,7.930595397949219M5.708829879760742,7.930595397949219L8.374113082885742,7.930595397949219L8.374113082885742,15.996421813964844L5.708829879760742,15.991172790527344" fill="currentColor" />
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 853 B |
@ -373,7 +373,7 @@
|
||||
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(let [params {:id id :name name}]
|
||||
(let [params (with-meta {:id id :name name} (meta params))]
|
||||
(->> (rp/cmd! :rename-project params)
|
||||
(rx/ignore))))))
|
||||
|
||||
@ -435,7 +435,7 @@
|
||||
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(let [params (select-keys params [:id :name])]
|
||||
(let [params (with-meta (select-keys params [:id :name]) (meta params))]
|
||||
(->> (rp/cmd! :rename-file params)
|
||||
(rx/ignore))))))
|
||||
|
||||
@ -460,15 +460,18 @@
|
||||
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(let [params {:id id :is-shared is-shared}]
|
||||
(let [mdata (meta params)
|
||||
origin (::ev/origin mdata "dashboard")]
|
||||
(rx/concat
|
||||
(->> (rp/cmd! :set-file-shared params)
|
||||
(->> (rp/cmd! :set-file-shared (with-meta {:id id :is-shared is-shared} mdata))
|
||||
(rx/ignore))
|
||||
(when is-shared
|
||||
(->> (rp/cmd! :get-file-summary {:id id})
|
||||
(rx/map (fn [summary]
|
||||
(when (-> summary :variants :count pos?)
|
||||
(ev/event {::ev/name "set-file-variants-shared" ::ev/origin "dashboard"})))))))))))
|
||||
(ev/event (merge {::ev/name "set-file-variants-shared"
|
||||
::ev/origin origin}
|
||||
(select-keys mdata [::ev/context])))))))))))))
|
||||
|
||||
(defn set-file-thumbnail
|
||||
[file-id thumbnail-id]
|
||||
|
||||
@ -94,7 +94,7 @@
|
||||
can assemble the chunks and materialise the final font-variant record.
|
||||
|
||||
Returns an observable that emits the created font-variant."
|
||||
[{:keys [data team-id font-id font-family font-weight font-style] :as _item}]
|
||||
[{:keys [data team-id font-id font-family font-weight font-style variant-name] :as _item}]
|
||||
;; Upload each mtype as a separate chunked session in parallel, collect
|
||||
;; all [mtype session-id] pairs, then call create-font-variant with :uploads.
|
||||
(->> (rx/from (seq data))
|
||||
@ -108,12 +108,100 @@
|
||||
{})
|
||||
(rx/mapcat (fn [uploads]
|
||||
(rp/cmd! :create-font-variant
|
||||
{:team-id team-id
|
||||
:font-id font-id
|
||||
:font-family font-family
|
||||
:font-weight font-weight
|
||||
:font-style font-style
|
||||
:uploads uploads})))))
|
||||
{:team-id team-id
|
||||
:font-id font-id
|
||||
:font-family font-family
|
||||
:font-weight font-weight
|
||||
:font-style font-style
|
||||
:variant-name variant-name
|
||||
:uploads uploads})))))
|
||||
|
||||
(defn resolve-font-metadata
|
||||
"Resolve the upload metadata for a font file.
|
||||
|
||||
`names` is a map carrying the `:family` and `:variant` values read by
|
||||
opentype.js; each may be nil when the font has no usable name records.
|
||||
`filename` is the original file name. The opentype.js values win per
|
||||
field; any missing value is derived from the filename instead."
|
||||
[names filename]
|
||||
(let [{:keys [family variant]} names
|
||||
base-name (str/replace filename #"\.[^.]+$" "")
|
||||
variant (not-empty (str/trim (or variant "")))
|
||||
family (not-empty (str/trim (or family "")))
|
||||
variant-source (or variant base-name)]
|
||||
{:font-family (or family (cm/parse-font-family base-name))
|
||||
:font-weight (cm/parse-font-weight variant-source)
|
||||
:font-style (cm/parse-font-style variant-source)
|
||||
:variant-name variant}))
|
||||
|
||||
(defn prepare-font-entry
|
||||
"Prepare a single read font entry for upload.
|
||||
|
||||
`:font` is the object parsed by opentype.js (nil for woff2, which cannot
|
||||
be parsed); `:name`, `:type` and `:data` come from reading the file.
|
||||
Returns the entry metadata, the `:content` to upload and, for parsed
|
||||
fonts, the `:height-warning?` flag. A font with missing vertical-metric
|
||||
tables uploads with `:height-warning? false` instead of failing."
|
||||
[{:keys [font type name data] :as _params}]
|
||||
(if font
|
||||
;; Font was parsed with opentype.js (ttf, otf, woff)
|
||||
(let [;; Select the first nonblank name candidate: a blank
|
||||
;; preferredFamily/preferredSubfamily must not shadow a valid
|
||||
;; fontFamily/fontSubfamily.
|
||||
family (cm/first-nonblank-string
|
||||
(.getEnglishName ^js font "preferredFamily")
|
||||
(.getEnglishName ^js font "fontFamily"))
|
||||
variant (cm/first-nonblank-string
|
||||
(.getEnglishName ^js font "preferredSubfamily")
|
||||
(.getEnglishName ^js font "fontSubfamily"))
|
||||
|
||||
;; Vertical metrics determine the baseline in a text and the space between lines of
|
||||
;; text. For historical reasons, there are three pairs of ascender/descender
|
||||
;; values, known as hhea, OS/2 and uSWin metrics. Depending on the font, operating
|
||||
;; system and application a different set will be used to render text on the
|
||||
;; screen. On Mac, Safari and Chrome use the hhea values to render text. Firefox
|
||||
;; will respect the useTypoMetrics setting and will use the OS/2 if it is set. If
|
||||
;; the useTypoMetrics is not set, Firefox will also use metrics from the hhea
|
||||
;; table. On Windows, all browsers use the usWin metrics, but respect the
|
||||
;; useTypoMetrics setting and if set will use the OS/2 values.
|
||||
|
||||
hhea (some-> ^js font .-tables .-hhea)
|
||||
os2 (some-> ^js font .-tables .-os2)
|
||||
|
||||
hhea-ascender (some-> ^js hhea .-ascender abs)
|
||||
hhea-descender (some-> ^js hhea .-descender abs)
|
||||
|
||||
win-ascent (some-> ^js os2 .-usWinAscent abs)
|
||||
win-descent (some-> ^js os2 .-usWinDescent abs)
|
||||
|
||||
os2-ascent (some-> ^js os2 .-sTypoAscender abs)
|
||||
os2-descent (some-> ^js os2 .-sTypoDescender abs)
|
||||
|
||||
;; useTypoMetrics can be read from the 7th bit
|
||||
f-selection (some-> ^js os2 .-fsSelection (bit-test 7))
|
||||
|
||||
height-warning? (boolean
|
||||
(and hhea-ascender hhea-descender win-ascent win-descent
|
||||
(or (not= hhea-ascender win-ascent)
|
||||
(not= hhea-descender win-descent)
|
||||
(and f-selection os2-ascent os2-descent
|
||||
(or (not= hhea-ascender os2-ascent)
|
||||
(not= hhea-descender os2-descent))))))
|
||||
data (js/Uint8Array. data)
|
||||
metadata (resolve-font-metadata {:family family :variant variant} name)]
|
||||
(assoc metadata
|
||||
:content {:data data
|
||||
:name name
|
||||
:type type}
|
||||
:height-warning? height-warning?))
|
||||
;; Font could not be parsed (woff2), extract metadata from filename
|
||||
(let [data (js/Uint8Array. data)
|
||||
metadata (resolve-font-metadata {} name)]
|
||||
(assoc metadata
|
||||
:content {:data data
|
||||
:name name
|
||||
:type type}
|
||||
:height-warning? false))))
|
||||
|
||||
(defn- parse-font-mtype
|
||||
"Detects a font mime-type from the first four bytes of an ArrayBuffer.
|
||||
@ -159,84 +247,6 @@
|
||||
(log/warn :msg (str/fmt "skipping file %s, unsupported format" name))
|
||||
{:error name}))))
|
||||
|
||||
(defn- prepare-font
|
||||
"Builds a ready-to-upload font entry from a parsed blob.
|
||||
|
||||
Never throws for missing font metadata: a nil subfamily (opentype.js
|
||||
returns nil when the font carries no English preferred/subfamily
|
||||
names, as seen with some Fontshare families) defaults to regular
|
||||
weight/style, and missing vertical-metric tables default to no
|
||||
height warning."
|
||||
[{:keys [font type name data] :as _params}]
|
||||
(let [name (or name "")]
|
||||
(if (some? font)
|
||||
;; Font was parsed with opentype.js (ttf, otf, woff)
|
||||
(let [family (or (.getEnglishName ^js font "preferredFamily")
|
||||
(.getEnglishName ^js font "fontFamily")
|
||||
"")
|
||||
variant (or (.getEnglishName ^js font "preferredSubfamily")
|
||||
(.getEnglishName ^js font "fontSubfamily")
|
||||
"")
|
||||
|
||||
;; Vertical metrics determine the baseline in a text and the space between lines of
|
||||
;; text. For historical reasons, there are three pairs of ascender/descender
|
||||
;; values, known as hhea, OS/2 and uSWin metrics. Depending on the font, operating
|
||||
;; system and application a different set will be used to render text on the
|
||||
;; screen. On Mac, Safari and Chrome use the hhea values to render text. Firefox
|
||||
;; will respect the useTypoMetrics setting and will use the OS/2 if it is set. If
|
||||
;; the useTypoMetrics is not set, Firefox will also use metrics from the hhea
|
||||
;; table. On Windows, all browsers use the usWin metrics, but respect the
|
||||
;; useTypoMetrics setting and if set will use the OS/2 values.
|
||||
|
||||
hhea (some-> ^js font .-tables .-hhea)
|
||||
os2 (some-> ^js font .-tables .-os2)
|
||||
|
||||
hhea-ascender (some-> ^js hhea .-ascender abs)
|
||||
hhea-descender (some-> ^js hhea .-descender abs)
|
||||
|
||||
win-ascent (some-> ^js os2 .-usWinAscent abs)
|
||||
win-descent (some-> ^js os2 .-usWinDescent abs)
|
||||
|
||||
os2-ascent (some-> ^js os2 .-sTypoAscender abs)
|
||||
os2-descent (some-> ^js os2 .-sTypoDescender abs)
|
||||
|
||||
;; useTypoMetrics can be read from the 7th bit
|
||||
f-selection (some-> ^js os2 .-fsSelection (bit-test 7))
|
||||
|
||||
height-warning? (boolean
|
||||
(and hhea-ascender hhea-descender win-ascent win-descent
|
||||
(or (not= hhea-ascender win-ascent)
|
||||
(not= hhea-descender win-descent)
|
||||
(and f-selection os2-ascent os2-descent
|
||||
(or (not= hhea-ascender os2-ascent)
|
||||
(not= hhea-descender os2-descent))))))
|
||||
data (js/Uint8Array. data)]
|
||||
{:content {:data data
|
||||
:name name
|
||||
:type type}
|
||||
:font-family (or family "")
|
||||
:font-weight (cm/parse-font-weight variant)
|
||||
:font-style (cm/parse-font-style variant)
|
||||
:variant-name variant
|
||||
:height-warning? height-warning?})
|
||||
;; Font could not be parsed (woff2), extract metadata from filename
|
||||
(let [base-name (str/replace name #"\.[^.]+$" "")
|
||||
;; Strip known weight/style tokens and separators to derive family name
|
||||
;; Use word boundaries to avoid matching substrings (e.g. "Boldini" should not match "bold")
|
||||
raw-family-name (-> base-name
|
||||
(str/replace #"(?i)(^|[-_\s])(extra\s*black|ultra\s*black|extra\s*bold|ultra\s*bold|semi\s*bold|demi\s*bold|extra\s*light|ultra\s*light|hairline|thin|light|normal|regular|medium|bold|black|heavy|solid|italic)([-_\s]|$)" "$1$3")
|
||||
(str/replace #"[-_\s]+" " ")
|
||||
(str/trim))
|
||||
family-name (if (str/blank? raw-family-name) base-name raw-family-name)
|
||||
data (js/Uint8Array. data)]
|
||||
{:content {:data data
|
||||
:name name
|
||||
:type type}
|
||||
:font-family family-name
|
||||
:font-weight (cm/parse-font-weight base-name)
|
||||
:font-style (cm/parse-font-style base-name)
|
||||
:height-warning? false}))))
|
||||
|
||||
(defn process-upload
|
||||
"Given a seq of blobs and the team id, creates a ready-to-use fonts
|
||||
map with temporal ID's associated to each font entry.
|
||||
@ -296,7 +306,7 @@
|
||||
(if (:error item)
|
||||
(update acc :errors conj (str "'" (:error item) "'"))
|
||||
(try
|
||||
(update acc :fonts #(join % (prepare-font item)))
|
||||
(update acc :fonts #(join % (prepare-font-entry item)))
|
||||
(catch :default cause
|
||||
(log/warn :msg "skipping font file" :cause cause :name (:name item))
|
||||
(update acc :errors conj (str "'" (:name item) "'"))))))
|
||||
|
||||
@ -57,61 +57,75 @@
|
||||
(update [_ state]
|
||||
(update-in state [:workspace-local :open-plugins] (fnil disj #{}) id))))
|
||||
|
||||
(defn- global-blocked?
|
||||
"True, after warning the user, when the plugin asks for global scope
|
||||
or management permissions it is not allowed to have."
|
||||
[plugin]
|
||||
(when (and (preg/global-requested? plugin)
|
||||
(not (preg/global-allowed? plugin)))
|
||||
(log/warn :hint "Global plugins can only be loaded from localhost"
|
||||
:plugin-id (:plugin-id plugin)
|
||||
:host (:host plugin))
|
||||
(st/emit! (ntf/warn (tr "workspace.plugins.error.global-not-allowed")))
|
||||
true))
|
||||
|
||||
(defn start-plugin!
|
||||
[{:keys [plugin-id name version description host code permissions allow-background scope]} ^js extensions]
|
||||
(let [load-plugin (unchecked-get ug/global "ɵloadPlugin")]
|
||||
(if (fn? load-plugin)
|
||||
(-> (load-plugin
|
||||
#js {:pluginId plugin-id
|
||||
:name name
|
||||
:version version
|
||||
:description description
|
||||
:host host
|
||||
:code code
|
||||
:allowBackground (boolean allow-background)
|
||||
:scope scope
|
||||
:permissions (apply array permissions)}
|
||||
nil
|
||||
extensions)
|
||||
[{:keys [plugin-id name version description host code permissions allow-background scope] :as plugin} ^js extensions]
|
||||
(when-not (global-blocked? plugin)
|
||||
(let [load-plugin (unchecked-get ug/global "ɵloadPlugin")]
|
||||
(if (fn? load-plugin)
|
||||
(-> (load-plugin
|
||||
#js {:pluginId plugin-id
|
||||
:name name
|
||||
:version version
|
||||
:description description
|
||||
:host host
|
||||
:code code
|
||||
:allowBackground (boolean allow-background)
|
||||
:scope scope
|
||||
:permissions (apply array permissions)}
|
||||
nil
|
||||
extensions)
|
||||
|
||||
(p/catch (fn [cause]
|
||||
(ex/print-throwable cause :prefix "Plugin Error")
|
||||
(errors/flash :cause cause :type :handled))))
|
||||
(p/catch (fn [cause]
|
||||
(ex/print-throwable cause :prefix "Plugin Error")
|
||||
(errors/flash :cause cause :type :handled))))
|
||||
|
||||
(log/warn :hint "Plugin runtime not initialized yet"
|
||||
:plugin-id plugin-id
|
||||
:action "start-plugin!"))))
|
||||
|
||||
(defn- load-plugin!
|
||||
[{:keys [plugin-id name version description host code icon permissions scope]}]
|
||||
(st/emit! (pflag/initialize plugin-id version)
|
||||
(save-current-plugin plugin-id))
|
||||
|
||||
(let [load-plugin (unchecked-get ug/global "ɵloadPlugin")]
|
||||
(if (fn? load-plugin)
|
||||
(-> (load-plugin
|
||||
#js {:pluginId plugin-id
|
||||
:name name
|
||||
:description description
|
||||
:version version
|
||||
:host host
|
||||
:code code
|
||||
:icon icon
|
||||
:scope scope
|
||||
:permissions (apply array permissions)}
|
||||
(fn []
|
||||
(st/emit! (remove-current-plugin plugin-id))))
|
||||
|
||||
(p/catch (fn [cause]
|
||||
(st/emit! (remove-current-plugin plugin-id))
|
||||
(ex/print-throwable cause :prefix "Plugin Error")
|
||||
(errors/flash :cause cause :type :handled))))
|
||||
|
||||
(do
|
||||
(log/warn :hint "Plugin runtime not initialized yet"
|
||||
:plugin-id plugin-id
|
||||
:action "load-plugin!")
|
||||
(st/emit! (remove-current-plugin plugin-id))))))
|
||||
:action "start-plugin!")))))
|
||||
|
||||
(defn- load-plugin!
|
||||
[{:keys [plugin-id name version description host code icon permissions scope] :as plugin}]
|
||||
(when-not (global-blocked? plugin)
|
||||
(st/emit! (pflag/initialize plugin-id version)
|
||||
(save-current-plugin plugin-id))
|
||||
|
||||
(let [load-plugin (unchecked-get ug/global "ɵloadPlugin")]
|
||||
(if (fn? load-plugin)
|
||||
(-> (load-plugin
|
||||
#js {:pluginId plugin-id
|
||||
:name name
|
||||
:description description
|
||||
:version version
|
||||
:host host
|
||||
:code code
|
||||
:icon icon
|
||||
:scope scope
|
||||
:permissions (apply array permissions)}
|
||||
(fn []
|
||||
(st/emit! (remove-current-plugin plugin-id))))
|
||||
|
||||
(p/catch (fn [cause]
|
||||
(st/emit! (remove-current-plugin plugin-id))
|
||||
(ex/print-throwable cause :prefix "Plugin Error")
|
||||
(errors/flash :cause cause :type :handled))))
|
||||
|
||||
(do
|
||||
(log/warn :hint "Plugin runtime not initialized yet"
|
||||
:plugin-id plugin-id
|
||||
:action "load-plugin!")
|
||||
(st/emit! (remove-current-plugin plugin-id)))))))
|
||||
|
||||
(defn open-plugin!
|
||||
[{:keys [url] :as manifest} user-can-edit?]
|
||||
@ -154,10 +168,12 @@
|
||||
;; Manifests match exactly - safe to load
|
||||
:else
|
||||
(load-plugin! manifest))))
|
||||
(fn [_err]
|
||||
(fn [err]
|
||||
;; Error fetching the manifest - can't verify integrity
|
||||
;; Show error instead of loading potentially tampered code
|
||||
(st/emit! (ntf/warn (tr "workspace.plugins.error.unreachable"))))))
|
||||
(st/emit! (ntf/warn (if (= ::preg/global-not-allowed (:type (ex-data err)))
|
||||
(tr "workspace.plugins.error.global-not-allowed")
|
||||
(tr "workspace.plugins.error.unreachable")))))))
|
||||
;; Bundled plugins (no URL) - trusted, load directly
|
||||
(load-plugin! manifest)))
|
||||
|
||||
|
||||
@ -13,6 +13,7 @@
|
||||
[app.common.types.organization :as cto]
|
||||
[app.common.types.team :as ctt]
|
||||
[app.common.uri :as u]
|
||||
[app.common.uuid :as uuid]
|
||||
[app.config :as cf]
|
||||
[app.main.data.event :as ev]
|
||||
[app.main.data.helpers :as dsh]
|
||||
@ -24,7 +25,9 @@
|
||||
[app.main.repo :as rp]
|
||||
[app.main.router :as rt]
|
||||
[app.util.clipboard :as clipboard]
|
||||
[app.util.dom :as dom]
|
||||
[app.util.i18n :refer [tr]]
|
||||
[app.util.session-state :as ss]
|
||||
[app.util.storage :as storage]
|
||||
[beicon.v2.core :as rx]
|
||||
[clojure.string :as str]
|
||||
@ -111,8 +114,8 @@
|
||||
(f team)))))))
|
||||
|
||||
(defn check-and-create-team
|
||||
"Fetches fresh team data from the server to ensure up-to-date organization
|
||||
permissions, then shows the team-form modal or a no-permission modal."
|
||||
"Fetches fresh team data, checks organization permissions and SSO, then
|
||||
shows the team form or the appropriate access error."
|
||||
[team-id]
|
||||
(ptk/reify ::check-and-create-team
|
||||
ptk/WatchEvent
|
||||
@ -127,13 +130,44 @@
|
||||
{:organization-perms organization
|
||||
:profile-id profile-id
|
||||
:team-perms (:permissions team)})
|
||||
true)]
|
||||
(rx/of (if can-create?
|
||||
(modal/show :team-form (if in-organization?
|
||||
{:organization-id (:id organization)
|
||||
:organization-name (:name organization)}
|
||||
{}))
|
||||
(modal/show :no-permission-modal {:type :create-team}))))))))))
|
||||
true)
|
||||
team-form-modal (modal/show :team-form (if in-organization?
|
||||
{:organization-id (:id organization)
|
||||
:organization-name (:name organization)}
|
||||
{}))]
|
||||
(cond
|
||||
(not can-create?)
|
||||
(rx/of (modal/show :no-permission-modal {:type :create-team}))
|
||||
|
||||
(not in-organization?)
|
||||
(rx/of team-form-modal)
|
||||
|
||||
:else
|
||||
(let [pending-id (str (uuid/next))
|
||||
callback-url (dom/append-query-param (rt/get-current-href)
|
||||
:pending-action-id pending-id)]
|
||||
(->> (rp/cmd! :check-nitrate-sso {:team-id team-id
|
||||
:url callback-url})
|
||||
(rx/mapcat
|
||||
(fn [{:keys [authorized redirect-uri]}]
|
||||
(cond
|
||||
authorized
|
||||
(rx/of team-form-modal)
|
||||
|
||||
redirect-uri
|
||||
(do
|
||||
(ss/save-pending-action! pending-id
|
||||
{:type :create-team-in-organization
|
||||
:team-id team-id})
|
||||
(rx/of (rt/nav-raw :uri (str redirect-uri))))
|
||||
|
||||
:else
|
||||
(rx/of (rt/assign-exception
|
||||
{:type :sso-error
|
||||
:organization-id (:id organization)
|
||||
:organization-name (:name organization)
|
||||
:team-id team-id
|
||||
:is-dashboard true})))))))))))))))
|
||||
|
||||
(defn check-and-delete-team
|
||||
"Fetches fresh team data from the server to ensure up-to-date organization
|
||||
|
||||
@ -160,10 +160,8 @@
|
||||
(let [page-id (:current-page-id state)
|
||||
data (dsh/lookup-file-data state)
|
||||
objects (-> (dsh/get-page data page-id)
|
||||
(get :objects))
|
||||
|
||||
components (cfv/find-variant-components data objects (:id shape))]
|
||||
(into {} (map (juxt :id :variant-properties) components))))
|
||||
(get :objects))]
|
||||
(cfv/collect-variant-properties data objects shape)))
|
||||
|
||||
|
||||
;; Collects all the items together and split images into a
|
||||
@ -179,7 +177,8 @@
|
||||
(some? images)
|
||||
(update :images into images)
|
||||
|
||||
(ctk/is-variant-container? item)
|
||||
(or (ctk/is-variant-container? item)
|
||||
(ctk/is-variant-main? item))
|
||||
(update :variant-properties merge (collect-variants state item))))
|
||||
|
||||
(maybe-translate [shape objects parent-frame-id]
|
||||
|
||||
@ -1849,6 +1849,7 @@
|
||||
(let [libraries (:shared-files state)
|
||||
library (get libraries library-id)
|
||||
variants-count (-> library :library-summary :variants count)
|
||||
tokens-count (or (-> library :library-summary :tokens-count) 0)
|
||||
|
||||
loaded-libraries (->> (dsh/lookup-libraries state)
|
||||
(remove (fn [[_ lib]]
|
||||
@ -1869,14 +1870,25 @@
|
||||
(rx/of (ntf/error (tr "errors.circular-library-reference")))
|
||||
(rx/throw cause))))))
|
||||
(rx/of (ptk/reify ::link-file-to-library-finished))
|
||||
(when (pos? variants-count)
|
||||
(when (or (pos? variants-count) (pos? tokens-count))
|
||||
(->> (rp/cmd! :get-library-usage {:file-id library-id})
|
||||
(rx/map (fn [library-usage]
|
||||
(ev/event {::ev/name "link-file-to-library-variants"
|
||||
:file-id file-id
|
||||
:library-id library-id
|
||||
:variants-count variants-count
|
||||
:library-used-in (:used-in library-usage)}))))))))))
|
||||
(rx/mapcat (fn [library-usage]
|
||||
(let [used-in (:used-in library-usage)]
|
||||
(rx/from
|
||||
(cond-> []
|
||||
(pos? variants-count)
|
||||
(conj (ev/event {::ev/name "link-file-to-library-variants"
|
||||
:file-id file-id
|
||||
:library-id library-id
|
||||
:variants-count variants-count
|
||||
:library-used-in used-in}))
|
||||
|
||||
(pos? tokens-count)
|
||||
(conj (ev/event {::ev/name "link-file-to-library-tokens"
|
||||
:file-id file-id
|
||||
:library-id library-id
|
||||
:tokens-count tokens-count
|
||||
:library-used-in used-in}))))))))))))))
|
||||
|
||||
(defn cleanup-unlinked-libraries
|
||||
"Remove libraries from state that are no longer linked to the given file.
|
||||
|
||||
@ -436,6 +436,14 @@
|
||||
(ptk/data-event :layout/update {:ids ids})
|
||||
(dwu/commit-undo-transaction undo-id))))))
|
||||
|
||||
(defn grid-in-copy?
|
||||
"Check if the grid is a component copy or lives inside one. Copies
|
||||
must keep the structure of their main, so tracks that add shapes are
|
||||
not allowed there."
|
||||
[objects grid]
|
||||
(or (ctc/in-component-copy? grid)
|
||||
(ctn/has-any-copy-parent? objects grid)))
|
||||
|
||||
(defn duplicate-layout-track
|
||||
[ids type index]
|
||||
(assert (#{:row :column} type))
|
||||
@ -443,53 +451,53 @@
|
||||
(ptk/reify ::duplicate-layout-track
|
||||
ptk/WatchEvent
|
||||
(watch [it state _]
|
||||
(let [file-id (:current-file-id state)
|
||||
page (dsh/lookup-page state)
|
||||
objects (:objects page)
|
||||
libraries (dsh/lookup-libraries state)
|
||||
library-data (dsh/lookup-file state file-id)
|
||||
shape-id (first ids)
|
||||
base-shape (get objects shape-id)
|
||||
(let [objects (dsh/lookup-page-objects state)
|
||||
base-shape (get objects (first ids))]
|
||||
(when-not (grid-in-copy? objects base-shape)
|
||||
(let [file-id (:current-file-id state)
|
||||
page (dsh/lookup-page state)
|
||||
libraries (dsh/lookup-libraries state)
|
||||
library-data (dsh/lookup-file state file-id)
|
||||
|
||||
shapes-by-track
|
||||
(if (= type :column)
|
||||
(ctl/shapes-by-column base-shape index false)
|
||||
(ctl/shapes-by-row base-shape index false))
|
||||
shapes-by-track
|
||||
(if (= type :column)
|
||||
(ctl/shapes-by-column base-shape index false)
|
||||
(ctl/shapes-by-row base-shape index false))
|
||||
|
||||
;; Change to set in order to use auxiliary functions
|
||||
selected (set shapes-by-track)
|
||||
;; Change to set in order to use auxiliary functions
|
||||
selected (set shapes-by-track)
|
||||
|
||||
changes
|
||||
(-> (pcb/empty-changes it)
|
||||
(cll/generate-duplicate-changes objects page selected (gpt/point 0 0) libraries library-data file-id)
|
||||
(cll/generate-duplicate-changes-update-indices objects selected))
|
||||
changes
|
||||
(-> (pcb/empty-changes it)
|
||||
(cll/generate-duplicate-changes objects page selected (gpt/point 0 0) libraries library-data file-id)
|
||||
(cll/generate-duplicate-changes-update-indices objects selected))
|
||||
|
||||
;; Creates a map with shape-id => duplicated-shape-id
|
||||
ids-map
|
||||
(->> changes
|
||||
:redo-changes
|
||||
(filter #(= (:type %) :add-obj))
|
||||
(filter #(selected (:old-id %)))
|
||||
(map #(vector (:old-id %) (get-in % [:obj :id])))
|
||||
(into {}))
|
||||
;; Creates a map with shape-id => duplicated-shape-id
|
||||
ids-map
|
||||
(->> changes
|
||||
:redo-changes
|
||||
(filter #(= (:type %) :add-obj))
|
||||
(filter #(selected (:old-id %)))
|
||||
(map #(vector (:old-id %) (get-in % [:obj :id])))
|
||||
(into {}))
|
||||
|
||||
changes
|
||||
(-> changes
|
||||
(pcb/update-shapes
|
||||
ids
|
||||
(fn [shape objects]
|
||||
;; The duplication could have altered the grid so we restore the values, we'll calculate the good ones now
|
||||
(let [shape (merge shape (select-keys base-shape [:layout-grid-cells :layout-grid-columns :layout-grid-rows]))]
|
||||
(case type
|
||||
:row (ctl/duplicate-row shape objects index ids-map)
|
||||
:column (ctl/duplicate-column shape objects index ids-map))))
|
||||
{:with-objects? true}))
|
||||
changes
|
||||
(-> changes
|
||||
(pcb/update-shapes
|
||||
ids
|
||||
(fn [shape objects]
|
||||
;; The duplication could have altered the grid so we restore the values, we'll calculate the good ones now
|
||||
(let [shape (merge shape (select-keys base-shape [:layout-grid-cells :layout-grid-columns :layout-grid-rows]))]
|
||||
(case type
|
||||
:row (ctl/duplicate-row shape objects index ids-map)
|
||||
:column (ctl/duplicate-column shape objects index ids-map))))
|
||||
{:with-objects? true}))
|
||||
|
||||
undo-id (js/Symbol)]
|
||||
(rx/of (dwu/start-undo-transaction undo-id)
|
||||
(dch/commit-changes changes)
|
||||
(ptk/data-event :layout/update {:ids ids})
|
||||
(dwu/commit-undo-transaction undo-id))))))
|
||||
undo-id (js/Symbol)]
|
||||
(rx/of (dwu/start-undo-transaction undo-id)
|
||||
(dch/commit-changes changes)
|
||||
(ptk/data-event :layout/update {:ids ids})
|
||||
(dwu/commit-undo-transaction undo-id))))))))
|
||||
|
||||
(defn reorder-layout-track
|
||||
[ids type from-index to-index move-content?]
|
||||
@ -918,7 +926,9 @@
|
||||
type (:type copied)
|
||||
undo-id (js/Symbol)]
|
||||
|
||||
(when (and (seq track-indices) (some? type))
|
||||
(when (and (seq track-indices)
|
||||
(some? type)
|
||||
(not (grid-in-copy? objects grid)))
|
||||
(let [shapes-by-track-fn
|
||||
(if (= type :row)
|
||||
ctl/shapes-by-row
|
||||
|
||||
@ -733,6 +733,8 @@
|
||||
(all-attrs-appliable-for-token? attributes (:type token)))))))
|
||||
shape-ids (d/nilv (keys shapes) [])
|
||||
any-variant? (->> shapes vals (some ctk/is-variant?) boolean)
|
||||
file-data (dsh/lookup-file-data state)
|
||||
from-library (not (cfo/effective-tokens-source? file-data (:id file-data)))
|
||||
|
||||
resolved-value (get-in resolved-tokens [(:name token) :resolved-value])
|
||||
tokenized-attributes (cfo/attributes-map attributes token)
|
||||
@ -743,7 +745,9 @@
|
||||
(-> {::ev/name "apply-tokens"
|
||||
:type type
|
||||
:applied-to attributes
|
||||
:applied-to-variant any-variant?}
|
||||
:applied-to-variant any-variant?
|
||||
:shape-count (count shape-ids)
|
||||
:from-library from-library}
|
||||
(merge (meta it)))))
|
||||
(dwu/start-undo-transaction undo-id)
|
||||
(dwsh/update-shapes shape-ids (fn [shape]
|
||||
|
||||
@ -106,7 +106,7 @@
|
||||
(rx/of (update-versions-state {:editing nil})
|
||||
(ev/event {::ev/name "rename-version"
|
||||
:file-id file-id}))
|
||||
(->> (rp/cmd! :update-file-snapshot {:id id :label label})
|
||||
(->> (rp/cmd! :update-file-snapshot {:id id :file-id file-id :label label})
|
||||
(rx/map fetch-versions)))))))
|
||||
|
||||
(defn- clear-preview-state
|
||||
@ -149,9 +149,10 @@
|
||||
|
||||
(ptk/reify ::delete-version
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(->> (rp/cmd! :delete-file-snapshot {:id id})
|
||||
(rx/map fetch-versions)))))
|
||||
(watch [_ state _]
|
||||
(let [file-id (:current-file-id state)]
|
||||
(->> (rp/cmd! :delete-file-snapshot {:id id :file-id file-id})
|
||||
(rx/map fetch-versions))))))
|
||||
|
||||
(defn pin-version
|
||||
[id]
|
||||
@ -161,8 +162,10 @@
|
||||
(watch [_ state _]
|
||||
(when-let [version (->> (dm/get-in state [:workspace-versions :data])
|
||||
(d/seek #(= (:id %) id)))]
|
||||
(let [params {:id id
|
||||
:label (ct/format-inst (:created-at version) :localized-date)}]
|
||||
(let [file-id (:current-file-id state)
|
||||
params {:id id
|
||||
:file-id file-id
|
||||
:label (ct/format-inst (:created-at version) :localized-date)}]
|
||||
|
||||
(->> (rp/cmd! :update-file-snapshot params)
|
||||
(rx/mapcat (fn [_]
|
||||
@ -175,18 +178,20 @@
|
||||
(assert (uuid? id) "expected valid uuid for `id`")
|
||||
(ptk/reify ::lock-version
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(->> (rp/cmd! :lock-file-snapshot {:id id})
|
||||
(rx/map fetch-versions)))))
|
||||
(watch [_ state _]
|
||||
(let [file-id (:current-file-id state)]
|
||||
(->> (rp/cmd! :lock-file-snapshot {:id id :file-id file-id})
|
||||
(rx/map fetch-versions))))))
|
||||
|
||||
(defn unlock-version
|
||||
[id]
|
||||
(assert (uuid? id) "expected valid uuid for `id`")
|
||||
(ptk/reify ::unlock-version
|
||||
ptk/WatchEvent
|
||||
(watch [_ _ _]
|
||||
(->> (rp/cmd! :unlock-file-snapshot {:id id})
|
||||
(rx/map fetch-versions)))))
|
||||
(watch [_ state _]
|
||||
(let [file-id (:current-file-id state)]
|
||||
(->> (rp/cmd! :unlock-file-snapshot {:id id :file-id file-id})
|
||||
(rx/map fetch-versions))))))
|
||||
|
||||
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
|
||||
;; RESTORE VERSION EVENTS
|
||||
|
||||
@ -18,6 +18,7 @@
|
||||
[app.main.data.plugins :as dp]
|
||||
[app.main.data.profile :as dprof]
|
||||
[app.main.data.project :as dpj]
|
||||
[app.main.data.team :as dtm]
|
||||
[app.main.refs :as refs]
|
||||
[app.main.router :as rt]
|
||||
[app.main.store :as st]
|
||||
@ -224,8 +225,10 @@
|
||||
(st/emit! (ev/event {::ev/name "install-plugin" :name (:name plugin) :url plugin-url}))
|
||||
(open-permissions-dialog plugin))
|
||||
(st/emit! (notif/error (tr "dashboard.plugins.parse-error")))))
|
||||
(fn [_]
|
||||
(st/emit! (notif/error (tr "dashboard.plugins.bad-url"))))))
|
||||
(fn [err]
|
||||
(st/emit! (notif/error (if (= ::preg/global-not-allowed (:type (ex-data err)))
|
||||
(tr "workspace.plugins.error.global-not-allowed")
|
||||
(tr "dashboard.plugins.bad-url")))))))
|
||||
(binding [storage/*sync* true]
|
||||
(swap! storage/session dissoc :plugin-url))))))
|
||||
|
||||
@ -296,6 +299,8 @@
|
||||
(st/emit! (dnt/add-team-to-organization {:team-id (:team-id action)
|
||||
:organization-id (:organization-id action)
|
||||
:skip-audit? true}))
|
||||
:create-team-in-organization
|
||||
(st/emit! (dtm/check-and-create-team (:team-id action)))
|
||||
nil)))))
|
||||
|
||||
(mf/defc dashboard*
|
||||
|
||||
@ -656,7 +656,7 @@
|
||||
[:*
|
||||
(if (contains? cf/flags :admin-console)
|
||||
[:*
|
||||
[:> nitrate-sidebar* {:profile profile :teams teams}]
|
||||
[:> nitrate-sidebar* {:profile profile :teams teams :team team}]
|
||||
[:> nitrate-current-plan* {:profile profile}]]
|
||||
(when (contains? cf/flags :subscriptions)
|
||||
(if (show-subscription-dashboard-banner? profile)
|
||||
|
||||
@ -28,6 +28,18 @@
|
||||
type
|
||||
"professional"))
|
||||
|
||||
(defn get-team-plan-type
|
||||
[team]
|
||||
(if (some? (dm/get-in team [:organization :id]))
|
||||
"enterprise"
|
||||
(get-subscription-type (:subscription team))))
|
||||
|
||||
(defn show-team-plan?
|
||||
[is-owner team]
|
||||
(not (and is-owner
|
||||
(nil? (dm/get-in team [:organization :id]))
|
||||
(= "enterprise" (get-subscription-type (:subscription team))))))
|
||||
|
||||
(mf/defc cta-power-up*
|
||||
[{:keys [top-title top-description bottom-description bottom-button bottom-button-href has-dropdown is-highlighted]}]
|
||||
(let [show-data* (mf/use-state false)
|
||||
@ -118,8 +130,46 @@
|
||||
:has-dropdown false
|
||||
:is-highlighted false}]))))
|
||||
|
||||
(mf/defc nitrate-enterprise-banner*
|
||||
[{:keys [on-click button-label]}]
|
||||
(let [collapsed* (mf/use-state false)
|
||||
collapsed? (deref collapsed*)
|
||||
toggle-collapse
|
||||
(mf/use-fn #(swap! collapsed* not))]
|
||||
|
||||
[:div {:class (stl/css-case :nitrate-banner true
|
||||
:nitrate-enterprise-banner true
|
||||
:collapsed collapsed?)}
|
||||
[:button {:class (stl/css :nitrate-enterprise-toggle)
|
||||
:type "button"
|
||||
:aria-controls "nitrate-enterprise-content"
|
||||
:aria-expanded (not collapsed?)
|
||||
:on-click toggle-collapse}
|
||||
[:> icon* {:icon-id i/pencil
|
||||
:size "m"
|
||||
:class (stl/css :nitrate-enterprise-pencil)
|
||||
:aria-hidden true}]
|
||||
[:span {:class (stl/css :nitrate-enterprise-title)}
|
||||
(tr "subscription.dashboard.banner.unlock-enterprise")]
|
||||
[:> icon* {:icon-id i/arrow-down
|
||||
:size "m"
|
||||
:class (stl/css :nitrate-enterprise-chevron)
|
||||
:aria-hidden true}]]
|
||||
[:div {:id "nitrate-enterprise-content"
|
||||
:class (stl/css :nitrate-enterprise-panel)
|
||||
:aria-hidden collapsed?}
|
||||
[:div {:class (stl/css :nitrate-enterprise-panel-inner)}
|
||||
[:span {:class (stl/css :nitrate-enterprise-description)}
|
||||
(tr "subscription.dashboard.banner.unlock-features-description")]
|
||||
[:> button* {:variant "primary"
|
||||
:type "button"
|
||||
:class (stl/css :nitrate-bottom-button)
|
||||
:on-click on-click
|
||||
:disabled collapsed?}
|
||||
button-label]]]]))
|
||||
|
||||
(mf/defc nitrate-sidebar*
|
||||
[{:keys [profile teams]}]
|
||||
[{:keys [profile teams team]}]
|
||||
(let [nitrate? (dnt/is-valid-license? profile)
|
||||
nitrate-license (:subscription profile)
|
||||
manual-license? (:manual nitrate-license)
|
||||
@ -151,6 +201,12 @@
|
||||
subscription-type
|
||||
(if nitrate? (:type nitrate-license) (get-subscription-type (-> profile :props :subscription)))
|
||||
|
||||
show-unlock-features?
|
||||
(and (contains? cf/flags :admin-console)
|
||||
(not nitrate?)
|
||||
(contains? #{"professional" "unlimited"} subscription-type)
|
||||
(nil? (dm/get-in team [:organization :id])))
|
||||
|
||||
team-count (count teams)
|
||||
|
||||
account-age-days (dnt/account-age-days profile)
|
||||
@ -210,19 +266,13 @@
|
||||
:class (stl/css :nitrate-bottom-button)
|
||||
:on-click handle-go-to-cc} (tr "nitrate.activation-success.create-organization")]]]
|
||||
|
||||
;; Banner for users without nitrate license
|
||||
(when (not nitrate?)
|
||||
[:div {:class (stl/css :nitrate-banner :highlighted)}
|
||||
[:div {:class (stl/css :nitrate-content)}
|
||||
[:span {:class (stl/css :nitrate-title)} (tr "subscription.dashboard.banner.unlock-features")]]
|
||||
[:div {:class (stl/css :nitrate-content)}
|
||||
[:span {:class (stl/css :nitrate-info)} (tr "subscription.dashboard.banner.unlock-features-description-text")]
|
||||
[:> button* {:variant "primary"
|
||||
:type "button"
|
||||
:class (stl/css :nitrate-bottom-button)
|
||||
:on-click handle-click} (if (:subscription profile)
|
||||
(tr "subscription.dashboard.banner.upgrade-nitrate")
|
||||
(tr "nitrate.form.try-free"))]]]))
|
||||
;; Banner for Professional/Unlimited users outside an organization
|
||||
(when show-unlock-features?
|
||||
[:> nitrate-enterprise-banner*
|
||||
{:on-click handle-click
|
||||
:button-label (if (:subscription profile)
|
||||
(tr "subscription.dashboard.banner.upgrade-nitrate")
|
||||
(tr "nitrate.form.try-free"))}]))
|
||||
|
||||
;; Banner for users with nitrate license almost expired or expired
|
||||
(when show-subscription-warning?
|
||||
@ -272,6 +322,7 @@
|
||||
[{:keys [is-owner team]}]
|
||||
(let [subscription (:subscription team)
|
||||
subscription-type (get-subscription-type subscription)
|
||||
team-plan-type (get-team-plan-type team)
|
||||
subscription-is-trial (= "trialing" (:status subscription))
|
||||
|
||||
go-to-manage-subscription
|
||||
@ -286,22 +337,23 @@
|
||||
href (str "payments/subscriptions/show?returnUrl=" href)]
|
||||
(st/emit! (rt/nav-raw :href href)))))]
|
||||
|
||||
[:div {:class (stl/css :team)}
|
||||
[:div {:class (stl/css :team-label)}
|
||||
(tr "subscription.dashboard.team-plan")]
|
||||
[:span {:class (stl/css :team-text)}
|
||||
(case subscription-type
|
||||
"professional" (tr "subscription.settings.professional")
|
||||
"unlimited" (if subscription-is-trial
|
||||
(tr "subscription.settings.unlimited-trial")
|
||||
(tr "subscription.settings.unlimited"))
|
||||
(when (show-team-plan? is-owner team)
|
||||
[:div {:class (stl/css :team)}
|
||||
[:div {:class (stl/css :team-label)}
|
||||
(tr "subscription.dashboard.team-plan")]
|
||||
[:span {:class (stl/css :team-text)}
|
||||
(case team-plan-type
|
||||
"professional" (tr "subscription.settings.professional")
|
||||
"unlimited" (if subscription-is-trial
|
||||
(tr "subscription.settings.unlimited-trial")
|
||||
(tr "subscription.settings.unlimited"))
|
||||
|
||||
"enterprise" (tr "subscription.settings.enterprise"))]
|
||||
(when (and is-owner (not= subscription-type "professional"))
|
||||
[:button {:class (stl/css :manage-subscription-link)
|
||||
:on-click go-to-manage-subscription
|
||||
:data-testid "manage-subscription-link"}
|
||||
(tr "subscription.settings.manage-your-subscription")])]))
|
||||
"enterprise" (tr "subscription.settings.enterprise"))]
|
||||
(when (and is-owner (not= subscription-type "professional"))
|
||||
[:button {:class (stl/css :manage-subscription-link)
|
||||
:on-click go-to-manage-subscription
|
||||
:data-testid "manage-subscription-link"}
|
||||
(tr "subscription.settings.manage-your-subscription")])])))
|
||||
|
||||
(mf/defc menu-team-icon*
|
||||
[{:keys [subscription-type]}]
|
||||
|
||||
@ -208,25 +208,132 @@
|
||||
}
|
||||
|
||||
.nitrate-banner {
|
||||
--nitrate-enterprise-gradient-start: var(--color-accent-success);
|
||||
--nitrate-enterprise-gradient-end: var(--color-token-border);
|
||||
|
||||
:global(.light) & {
|
||||
--nitrate-enterprise-gradient-start: var(--color-background-info);
|
||||
--nitrate-enterprise-gradient-end: var(--color-accent-primary-muted);
|
||||
}
|
||||
|
||||
display: flex;
|
||||
border-radius: var(--sp-s);
|
||||
flex-direction: column;
|
||||
margin: var(--sp-m) var(--sp-m) 0;
|
||||
background: var(--color-accent-select);
|
||||
background: linear-gradient(
|
||||
110deg,
|
||||
var(--nitrate-enterprise-gradient-start),
|
||||
var(--nitrate-enterprise-gradient-end)
|
||||
);
|
||||
border: $b-1 solid var(--color-accent-primary-muted);
|
||||
padding: var(--sp-l);
|
||||
}
|
||||
|
||||
.nitrate-enterprise-banner {
|
||||
display: grid;
|
||||
grid-template-rows: auto minmax(0, 1fr);
|
||||
background: linear-gradient(
|
||||
110deg,
|
||||
var(--nitrate-enterprise-gradient-start),
|
||||
var(--nitrate-enterprise-gradient-end)
|
||||
);
|
||||
transition: grid-template-rows 240ms ease;
|
||||
}
|
||||
|
||||
.nitrate-enterprise-banner.collapsed {
|
||||
grid-template-rows: auto minmax(0, 0fr);
|
||||
}
|
||||
|
||||
.nitrate-enterprise-toggle {
|
||||
appearance: none;
|
||||
background: transparent;
|
||||
border: 0;
|
||||
color: var(--color-foreground-primary);
|
||||
cursor: pointer;
|
||||
display: grid;
|
||||
grid-template-columns: auto minmax(0, 1fr) auto;
|
||||
align-items: center;
|
||||
gap: var(--sp-s);
|
||||
inline-size: 100%;
|
||||
padding: 0;
|
||||
text-align: start;
|
||||
font: inherit;
|
||||
|
||||
&:focus-visible {
|
||||
outline: var(--color-accent-primary) solid $b-2;
|
||||
outline-offset: var(--sp-xxs);
|
||||
border-radius: $br-4;
|
||||
}
|
||||
}
|
||||
|
||||
.nitrate-enterprise-title {
|
||||
@include use-typography("body-large");
|
||||
|
||||
color: var(--color-foreground-primary);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.nitrate-enterprise-pencil {
|
||||
--icon-fill-color: currentcolor;
|
||||
--icon-stroke-color: none;
|
||||
|
||||
color: var(--color-foreground-primary);
|
||||
}
|
||||
|
||||
.nitrate-enterprise-chevron {
|
||||
color: var(--color-foreground-secondary);
|
||||
transition: transform 240ms ease;
|
||||
}
|
||||
|
||||
.collapsed .nitrate-enterprise-chevron {
|
||||
transform: rotate(180deg);
|
||||
}
|
||||
|
||||
.nitrate-enterprise-panel {
|
||||
min-block-size: 0;
|
||||
overflow: hidden;
|
||||
opacity: 1;
|
||||
transition: opacity 180ms ease;
|
||||
}
|
||||
|
||||
.collapsed .nitrate-enterprise-panel {
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.nitrate-enterprise-panel-inner {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-block-size: 0;
|
||||
padding-block-start: var(--sp-s);
|
||||
}
|
||||
|
||||
.nitrate-enterprise-description {
|
||||
@include use-typography("body-medium");
|
||||
|
||||
color: var(--color-foreground-primary);
|
||||
margin-block-end: var(--sp-l);
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
.nitrate-enterprise-banner,
|
||||
.nitrate-enterprise-chevron,
|
||||
.nitrate-enterprise-panel {
|
||||
transition: none;
|
||||
}
|
||||
}
|
||||
|
||||
.nitrate-title {
|
||||
@include use-typography("body-large");
|
||||
|
||||
color: var(--color-foreground-primary);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.nitrate-info {
|
||||
@include use-typography("body-medium");
|
||||
|
||||
color: var(--color-foreground-secondary);
|
||||
color: var(--color-foreground-primary);
|
||||
margin-block: var(--sp-s) var(--sp-xxl);
|
||||
}
|
||||
|
||||
|
||||
@ -83,7 +83,7 @@
|
||||
(mf/defc team-form-modal
|
||||
{::mf/register modal/components
|
||||
::mf/register-as :team-form}
|
||||
[{:keys [team organization-id] :as props}]
|
||||
[{:keys [team organization-id organization-name]}]
|
||||
(let [initial (mf/use-memo
|
||||
(mf/deps team organization-id)
|
||||
(fn []
|
||||
@ -117,7 +117,9 @@
|
||||
[:h2 {:class (stl/css :modal-title)}
|
||||
(tr "labels.rename-team")]
|
||||
[:h2 {:class (stl/css :modal-title)}
|
||||
(tr "labels.create-team")])
|
||||
(if organization-name
|
||||
(tr "dashboard.create-team-in-organization" organization-name)
|
||||
(tr "labels.create-team"))])
|
||||
|
||||
[:> icon-button* {:variant "ghost"
|
||||
:tooltip-class (stl/css :modal-close-btn)
|
||||
|
||||
@ -26,6 +26,9 @@
|
||||
@include use-typography("headline-large");
|
||||
|
||||
color: var(--color-foreground-primary);
|
||||
max-inline-size: 100%;
|
||||
padding-inline-end: var(--sp-xxl);
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
// Tooltip wrapper of the close button: it places it in the corner.
|
||||
|
||||
@ -223,6 +223,7 @@
|
||||
(def ^:icon-id padding-top "padding-top")
|
||||
(def ^:icon-id padding-top-bottom "padding-top-bottom")
|
||||
(def ^:icon-id path "path")
|
||||
(def ^:icon-id pencil "pencil")
|
||||
(def ^:icon-id pentool "pentool")
|
||||
(def ^:icon-id percentage "percentage")
|
||||
(def ^:icon-id picker "picker")
|
||||
|
||||
@ -873,6 +873,10 @@
|
||||
{::mf/private true}
|
||||
[{:keys [mdata]}]
|
||||
(let [{:keys [type index grid-id]} mdata
|
||||
|
||||
objects (deref refs/workspace-page-objects)
|
||||
in-copy? (dwsl/grid-in-copy? objects (get objects grid-id))
|
||||
|
||||
do-delete-track
|
||||
(mf/use-fn
|
||||
(mf/deps grid-id type index)
|
||||
@ -905,14 +909,14 @@
|
||||
|
||||
(if (= type :column)
|
||||
[:*
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.duplicate") :on-click do-duplicate-track}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.duplicate") :on-click do-duplicate-track :disabled in-copy?}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.add-before") :on-click do-add-track-before}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.add-after") :on-click do-add-track-after}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.delete") :on-click do-delete-track}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.column.delete-shapes") :on-click do-delete-track-shapes}]]
|
||||
|
||||
[:*
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.row.duplicate") :on-click do-duplicate-track}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.row.duplicate") :on-click do-duplicate-track :disabled in-copy?}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.row.add-before") :on-click do-add-track-before}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.row.add-after") :on-click do-add-track-after}]
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-track.row.delete") :on-click do-delete-track}]
|
||||
@ -949,6 +953,9 @@
|
||||
grid-edition (mf/deref grid-edition-ref)
|
||||
has-copied-tracks? (some? (:copied-tracks grid-edition))
|
||||
|
||||
objects (deref refs/workspace-page-objects)
|
||||
in-copy? (dwsl/grid-in-copy? objects (get objects grid-id))
|
||||
|
||||
do-merge-cells
|
||||
(mf/use-fn
|
||||
(mf/deps grid-id cells)
|
||||
@ -1002,7 +1009,7 @@
|
||||
|
||||
[:> menu-entry* {:title (tr "workspace.context-menu.grid-cells.paste-tracks")
|
||||
:on-click do-paste-tracks
|
||||
:disabled (not has-copied-tracks?)}]]))
|
||||
:disabled (or (not has-copied-tracks?) in-copy?)}]]))
|
||||
|
||||
|
||||
(def guide-color-presets
|
||||
|
||||
@ -111,12 +111,13 @@
|
||||
plugin-url* (mf/use-state "")
|
||||
plugin-url (deref plugin-url*)
|
||||
|
||||
input-status* (mf/use-state nil) ;; :error-url :error-manifest :success
|
||||
input-status* (mf/use-state nil) ;; :error-url :error-manifest :error-global :success
|
||||
input-status (deref input-status*)
|
||||
|
||||
error-url? (= :error-url input-status)
|
||||
error-manifest? (= :error-manifest input-status)
|
||||
error? (or error-url? error-manifest?)
|
||||
error-global? (= :error-global input-status)
|
||||
error? (or error-url? error-manifest? error-global?)
|
||||
|
||||
permissions (mf/deref refs/permissions)
|
||||
user-can-edit? (get permissions :can-edit)
|
||||
@ -159,7 +160,10 @@
|
||||
(fn [err]
|
||||
(.error js/console err)
|
||||
(reset! fetching-manifest? false)
|
||||
(reset! input-status* :error-url))))))
|
||||
(reset! input-status*
|
||||
(if (= ::preg/global-not-allowed (:type (ex-data err)))
|
||||
:error-global
|
||||
:error-url)))))))
|
||||
|
||||
on-open-plugin
|
||||
(mf/use-fn
|
||||
@ -236,6 +240,10 @@
|
||||
[:div {:class (stl/css-case :info true :error error?)}
|
||||
(tr "workspace.plugins.error.manifest")])
|
||||
|
||||
(when error-global?
|
||||
[:div {:class (stl/css-case :info true :error error?)}
|
||||
(tr "workspace.plugins.error.global-not-allowed")])
|
||||
|
||||
(when-not (empty? plugins-state)
|
||||
[:> i18n/tr-html*
|
||||
{:class (stl/css :discover)
|
||||
|
||||
@ -54,7 +54,7 @@
|
||||
|
||||
:else
|
||||
(do (swap! data assoc :label value :created-by "user")
|
||||
(->> (rp/cmd! :update-file-snapshot {:id (:id @data) :label value})
|
||||
(->> (rp/cmd! :update-file-snapshot {:id (:id @data) :file-id file-id :label value})
|
||||
(rx/take 1)
|
||||
(rx/subs! #(st/emit! (se/event plugin-id "rename-version" :file-id file-id)))))))}
|
||||
|
||||
@ -95,7 +95,7 @@
|
||||
|
||||
:else
|
||||
(let [version-id (:id @data)]
|
||||
(->> (rp/cmd! :delete-file-snapshot {:id version-id})
|
||||
(->> (rp/cmd! :delete-file-snapshot {:id version-id :file-id file-id})
|
||||
(rx/map (constantly nil))
|
||||
(rx/subs! resolve reject)))))))
|
||||
|
||||
@ -111,15 +111,22 @@
|
||||
(u/reject-not-valid reject :pin "Only auto-saved versions can be pinned")
|
||||
|
||||
:else
|
||||
(let [params {:id (:id @data)
|
||||
:label (ct/format-inst (:created-at @data) :localized-date)}]
|
||||
(let [params {:id (:id @data)
|
||||
:file-id file-id
|
||||
:label (ct/format-inst (:created-at @data) :localized-date)}]
|
||||
(->> (rx/zip (rp/cmd! :get-team-users {:file-id file-id})
|
||||
(rp/cmd! :update-file-snapshot params))
|
||||
(rx/subs!
|
||||
(fn [[users data]]
|
||||
(fn [[users _]]
|
||||
(let [users (d/index-by :id users)]
|
||||
(st/emit! (se/event plugin-id "pin-version" :file-id file-id))
|
||||
(resolve (file-version-proxy plugin-id file-id users @data))))
|
||||
;; the command answers `true`: the pinned row is the
|
||||
;; current one with the new label, as a user version
|
||||
(resolve (file-version-proxy plugin-id file-id users
|
||||
(assoc @data
|
||||
:label (:label params)
|
||||
:created-by "user"
|
||||
:deleted-at nil)))))
|
||||
reject))))))))))
|
||||
|
||||
(defn file-proxy? [p]
|
||||
|
||||
@ -1022,7 +1022,8 @@
|
||||
(let [file (u/locate-file file-id)
|
||||
component (u/locate-library-component file-id id)
|
||||
root (ctf/get-component-root (:data file) component)]
|
||||
(when (some? root)
|
||||
;; A deleted component keeps its root only in its own objects, not on a page
|
||||
(when (and (some? root) (not (:deleted component)))
|
||||
(shape/shape-proxy plugin-id file-id (:main-instance-page component) (:id root)))))
|
||||
|
||||
:isVariant
|
||||
|
||||
@ -46,6 +46,14 @@
|
||||
(update-fn state)
|
||||
state))))
|
||||
|
||||
;; Dashboard events are logged by the backend, which reads the origin from
|
||||
;; the `x-event-origin` header that `rp/cmd!` builds from the params meta.
|
||||
(defn- with-origin
|
||||
"Tags RPC `params` with the plugin or MCP origin, as `se/add-event` does
|
||||
for frontend events."
|
||||
[plugin-id params]
|
||||
(se/add-event params plugin-id))
|
||||
|
||||
(defn- refresh-restored-project
|
||||
[team-id project-id]
|
||||
(if (dashboard-of-team? @st/state team-id)
|
||||
@ -65,8 +73,9 @@
|
||||
|
||||
(defn- restore-deleted-files
|
||||
"Uses the dashboard's Trash operation and waits for its final result."
|
||||
[team-id ids]
|
||||
(->> (rp/cmd! ::sse/restore-deleted-team-files {:team-id team-id :ids ids})
|
||||
[plugin-id team-id ids]
|
||||
(->> (rp/cmd! ::sse/restore-deleted-team-files
|
||||
(with-origin plugin-id {:team-id team-id :ids ids}))
|
||||
(rx/filter sse/end-of-stream?)
|
||||
(rx/take 1)
|
||||
(rx/map sse/get-payload)
|
||||
@ -252,7 +261,7 @@
|
||||
[plugin-id team-id options]
|
||||
(if-let [name (valid-name (obj/get options "name"))]
|
||||
(request plugin-id "manage:projects"
|
||||
#(->> (rp/cmd! :create-project {:team-id team-id :name name})
|
||||
#(->> (rp/cmd! :create-project (with-origin plugin-id {:team-id team-id :name name}))
|
||||
(rx/map (fn [project] (assoc project :count 0)))
|
||||
(rx/tap (fn [project]
|
||||
(update-dashboard team-id
|
||||
@ -309,7 +318,7 @@
|
||||
|
||||
:else
|
||||
(set-property plugin-id data :name name
|
||||
(dtm/update-team {:id (:id @data) :name name})))))}
|
||||
(dtm/update-team (with-origin plugin-id {:id (:id @data) :name name}))))))}
|
||||
|
||||
:listProjects
|
||||
(fn [] (list-projects plugin-id (:id @data)))
|
||||
@ -355,7 +364,7 @@
|
||||
|
||||
:else
|
||||
(set-property plugin-id data :name name
|
||||
(dd/rename-project {:id (:id @data) :name name})))))}
|
||||
(dd/rename-project (with-origin plugin-id {:id (:id @data) :name name}))))))}
|
||||
|
||||
:pinned
|
||||
{:get #(boolean (:is-pinned @data))
|
||||
@ -370,9 +379,10 @@
|
||||
|
||||
:else
|
||||
(set-property plugin-id data :is-pinned value
|
||||
(set-project-pin {:id (:id @data)
|
||||
:team-id (:team-id @data)
|
||||
:is-pinned value}))))}
|
||||
(set-project-pin (with-origin plugin-id
|
||||
{:id (:id @data)
|
||||
:team-id (:team-id @data)
|
||||
:is-pinned value})))))}
|
||||
|
||||
:listFiles
|
||||
(fn []
|
||||
@ -395,7 +405,7 @@
|
||||
(let [ids (into #{} (comp (filter (fn [file] (= id (:project-id file))))
|
||||
(map :id)) files)]
|
||||
(if (seq ids)
|
||||
(restore-deleted-files team-id ids)
|
||||
(restore-deleted-files plugin-id team-id ids)
|
||||
(rx/throw (js/Error. "Cannot restore a project without recoverable files"))))))
|
||||
(rx/tap (fn [_] (swap! data dissoc :deleted-at)))
|
||||
(rx/mapcat (fn [_] (refresh-restored-project team-id id)))))))
|
||||
@ -406,7 +416,7 @@
|
||||
(request plugin-id "content:write"
|
||||
#(let [{:keys [id team-id]} @data
|
||||
features (set/difference (:features @st/state #{}) cfeat/frontend-only-features)]
|
||||
(->> (rp/cmd! :create-file {:project-id id :name name :features features})
|
||||
(->> (rp/cmd! :create-file (with-origin plugin-id {:project-id id :name name :features features}))
|
||||
(rx/map (fn [file] (-> (dissoc file :data) (assoc :team-id team-id))))
|
||||
(rx/tap (fn [file]
|
||||
(update-dashboard team-id (fn [state] (ptk/update (dd/file-created file) state)))))
|
||||
@ -420,8 +430,9 @@
|
||||
(js/Promise.reject (js/Error. "Expected a name with 1 to 250 characters"))
|
||||
(request plugin-id "manage:projects"
|
||||
#(let [{:keys [id team-id]} @data]
|
||||
(->> (rp/cmd! :duplicate-project (cond-> {:project-id id}
|
||||
(some? name) (assoc :name (valid-name name))))
|
||||
(->> (rp/cmd! :duplicate-project (with-origin plugin-id
|
||||
(cond-> {:project-id id}
|
||||
(some? name) (assoc :name (valid-name name)))))
|
||||
(rx/mapcat dd/project-with-file-count)
|
||||
(rx/tap (fn [project]
|
||||
(update-dashboard team-id (fn [state] (ptk/update (dd/project-duplicated project) state)))))
|
||||
@ -432,7 +443,7 @@
|
||||
(if-let [team-id (uuid/parse* (obj/get team "id"))]
|
||||
(request plugin-id "manage:teams"
|
||||
#(let [{:keys [id] :as project} @data]
|
||||
(->> (rp/cmd! :move-project {:project-id id :team-id team-id})
|
||||
(->> (rp/cmd! :move-project (with-origin plugin-id {:project-id id :team-id team-id}))
|
||||
(rx/tap (fn [_]
|
||||
(swap! data assoc :team-id team-id)
|
||||
(update-dashboard (:team-id project) (fn [state] (update state :projects dissoc id)))))
|
||||
@ -443,7 +454,7 @@
|
||||
(fn []
|
||||
(request plugin-id "manage:delete"
|
||||
#(let [{:keys [id team-id]} @data]
|
||||
(->> (rp/cmd! :delete-project {:id id})
|
||||
(->> (rp/cmd! :delete-project (with-origin plugin-id {:id id}))
|
||||
(rx/tap (fn [_]
|
||||
(update-dashboard team-id (fn [state] (update state :projects dissoc id)))))
|
||||
(rx/map (constantly nil)))))))))
|
||||
@ -480,7 +491,7 @@
|
||||
|
||||
:else
|
||||
(set-property plugin-id data :name name
|
||||
(dd/rename-file {:id (:id @data) :name name})))))}
|
||||
(dd/rename-file (with-origin plugin-id {:id (:id @data) :name name}))))))}
|
||||
|
||||
:shared
|
||||
{:get #(boolean (:is-shared @data))
|
||||
@ -495,7 +506,7 @@
|
||||
|
||||
:else
|
||||
(set-property plugin-id data :is-shared value
|
||||
(dd/set-file-shared {:id (:id @data) :is-shared value}))))}
|
||||
(dd/set-file-shared (with-origin plugin-id {:id (:id @data) :is-shared value})))))}
|
||||
|
||||
:open
|
||||
(fn []
|
||||
@ -506,7 +517,7 @@
|
||||
:restore
|
||||
(fn []
|
||||
(request plugin-id "content:write"
|
||||
#(->> (restore-deleted-files (:team-id @data) #{(:id @data)})
|
||||
#(->> (restore-deleted-files plugin-id (:team-id @data) #{(:id @data)})
|
||||
(rx/tap (fn [_]
|
||||
(swap! data dissoc :deleted-at :will-be-deleted-at)))
|
||||
(rx/mapcat (fn [_] (refresh-restored-project (:team-id @data) (:project-id @data)))))))
|
||||
@ -519,7 +530,7 @@
|
||||
(->> (rp/cmd! :get-project {:id project-id})
|
||||
(rx/mapcat (fn [target]
|
||||
(if (= team-id (:team-id target))
|
||||
(rp/cmd! :move-files {:ids #{id} :project-id project-id})
|
||||
(rp/cmd! :move-files (with-origin plugin-id {:ids #{id} :project-id project-id}))
|
||||
(rx/throw (js/Error. "Cannot move a file to a project in another team")))))
|
||||
(rx/tap (fn [_]
|
||||
(update-dashboard team-id (fn [state] (ptk/update (dd/move-files {:ids #{id} :project-id project-id}) state)))
|
||||
@ -534,8 +545,9 @@
|
||||
(js/Promise.reject (js/Error. "Expected a name with 1 to 250 characters"))
|
||||
(request plugin-id "content:write"
|
||||
#(let [{:keys [id team-id]} @data]
|
||||
(->> (rp/cmd! :duplicate-file (cond-> {:file-id id}
|
||||
(some? name) (assoc :name (valid-name name))))
|
||||
(->> (rp/cmd! :duplicate-file (with-origin plugin-id
|
||||
(cond-> {:file-id id}
|
||||
(some? name) (assoc :name (valid-name name)))))
|
||||
(rx/map (fn [file] (-> (dissoc file :data) (assoc :team-id team-id))))
|
||||
(rx/tap (fn [file]
|
||||
(update-dashboard team-id (fn [state] (ptk/update (dd/file-created file) state)))))
|
||||
@ -545,7 +557,7 @@
|
||||
(fn []
|
||||
(request plugin-id "manage:delete"
|
||||
#(let [{:keys [id project-id team-id]} @data]
|
||||
(->> (rp/cmd! :delete-file {:id id})
|
||||
(->> (rp/cmd! :delete-file (with-origin plugin-id {:id id}))
|
||||
(rx/tap (fn [_]
|
||||
(update-dashboard team-id
|
||||
(fn [state]
|
||||
@ -576,7 +588,7 @@
|
||||
(fn [options]
|
||||
(if-let [name (valid-name (obj/get options "name"))]
|
||||
(request plugin-id "manage:teams"
|
||||
#(->> (rp/cmd! :create-team {:name name :features features/global-enabled-features})
|
||||
#(->> (rp/cmd! :create-team (with-origin plugin-id {:name name :features features/global-enabled-features}))
|
||||
(rx/tap (fn [_] (st/emit! (dtm/fetch-teams))))
|
||||
(rx/map (partial team-proxy plugin-id))))
|
||||
(js/Promise.reject (js/Error. "Expected a name with 1 to 250 characters"))))
|
||||
|
||||
@ -16,6 +16,7 @@
|
||||
[app.main.store :as st]
|
||||
[app.util.object :as obj]
|
||||
[beicon.v2.core :as rx]
|
||||
[cuerdas.core :as str]
|
||||
[promesa.core :as p]))
|
||||
|
||||
;; Needs to be here because moving it to `app.main.data.workspace.mcp` will
|
||||
@ -29,6 +30,33 @@
|
||||
;; Keeps the rollbacks not reported yet across reloads
|
||||
"allow:localstorage"})
|
||||
|
||||
;; Permissions only granted to plugins that pass `global-allowed?`
|
||||
(def global-permissions
|
||||
#{"allow:global" "manage:projects" "manage:teams" "manage:delete"})
|
||||
|
||||
(def ^:private local-hosts
|
||||
#{"localhost" "127.0.0.1" "[::1]" "::1"})
|
||||
|
||||
(defn- local-url?
|
||||
"True when the url host is a loopback name, ignoring case and a
|
||||
trailing dot."
|
||||
[url]
|
||||
(contains? local-hosts (some-> url u/uri :host str/lower (str/rtrim "."))))
|
||||
|
||||
(defn global-requested?
|
||||
"True when the plugin asks for global scope or a management permission."
|
||||
[{:keys [scope permissions]}]
|
||||
(or (= "global" scope)
|
||||
(boolean (some global-permissions permissions))))
|
||||
|
||||
(defn global-allowed?
|
||||
"True when the plugin may run with global scope and use the management
|
||||
API. The feature is still in progress, so only the bundled MCP plugin
|
||||
and plugins served from localhost get it."
|
||||
[{:keys [plugin-id host]}]
|
||||
(or (= plugin-id mcp-plugin-id)
|
||||
(local-url? host)))
|
||||
|
||||
;; Promise that resolves when plugins runtime is initialized.
|
||||
;; Lives here to avoid circular dependency: workspace.mcp -> app.plugins -> app.plugins.api -> workspace
|
||||
(defonce ^:private runtime-ready-promise (p/deferred))
|
||||
@ -121,9 +149,18 @@
|
||||
:code code
|
||||
:icon icon
|
||||
:permissions (into #{} (map str) permissions)})]
|
||||
(if (sm/validate ctp/schema:registry-entry manifest)
|
||||
manifest
|
||||
(.error js/console (clj->js (sm/explain ctp/schema:registry-entry manifest))))))
|
||||
(cond
|
||||
(not (sm/validate ctp/schema:registry-entry manifest))
|
||||
(.error js/console (clj->js (sm/explain ctp/schema:registry-entry manifest)))
|
||||
|
||||
(and (global-requested? manifest)
|
||||
(not (global-allowed? manifest)))
|
||||
(throw (ex-info "Global plugins can only be loaded from localhost"
|
||||
{:type ::global-not-allowed
|
||||
:host origin}))
|
||||
|
||||
:else
|
||||
manifest)))
|
||||
|
||||
(defn subscribe-registry!
|
||||
"Subscribes f, called with no arguments on every registry change.
|
||||
@ -273,13 +310,16 @@
|
||||
(.error js/console "Failed to remove plugin:" err))))))))
|
||||
|
||||
(defn has-permission?
|
||||
"True when the plugin was granted `permission`."
|
||||
"True when the plugin was granted `permission`. The permissions in
|
||||
`global-permissions` also need the plugin to pass `global-allowed?`."
|
||||
[plugin-id permission]
|
||||
(cond
|
||||
(= plugin-id "00000000-0000-0000-0000-000000000000") true
|
||||
(= plugin-id mcp-plugin-id) (contains? mcp-permissions permission)
|
||||
:else (let [{:keys [permissions]} (dm/get-in @registry [:data plugin-id])]
|
||||
(contains? permissions permission))))
|
||||
:else (let [{:keys [permissions] :as plugin} (dm/get-in @registry [:data plugin-id])]
|
||||
(and (contains? permissions permission)
|
||||
(or (not (contains? global-permissions permission))
|
||||
(global-allowed? plugin))))))
|
||||
|
||||
(defn- read-only-reason
|
||||
"Why plugin writes must fail in `state`, or nil when they can go ahead.
|
||||
|
||||
@ -1535,8 +1535,8 @@
|
||||
(let [objects (u/locate-objects file-id page-id)
|
||||
shape (u/locate-shape file-id page-id id)]
|
||||
(when (ctn/in-any-component? objects shape)
|
||||
(let [[root component] (u/locate-component objects shape)]
|
||||
(shape-proxy plugin-id (:component-file root) (:main-instance-page component) (:id root))))))
|
||||
(let [root (ctn/get-instance-root objects shape)]
|
||||
(shape-proxy plugin-id file-id page-id (:id root))))))
|
||||
|
||||
:componentHead
|
||||
(fn []
|
||||
@ -1551,8 +1551,10 @@
|
||||
(let [objects (u/locate-objects file-id page-id)
|
||||
shape (u/locate-shape file-id page-id id)]
|
||||
(when (ctn/in-any-component? objects shape)
|
||||
(when-let [[head component] (u/locate-head-component objects shape)]
|
||||
(lib-component-proxy plugin-id (:component-file head) (:id component))))))
|
||||
(let [[head component] (u/locate-head-component objects shape)]
|
||||
;; The component is nil when its library is not available
|
||||
(when (some? component)
|
||||
(lib-component-proxy plugin-id (:component-file head) (:id component)))))))
|
||||
|
||||
:detach
|
||||
(fn []
|
||||
|
||||
@ -10,31 +10,35 @@
|
||||
[app.main.store :as st]
|
||||
[app.plugins.register :as r]))
|
||||
|
||||
(defn- get-origin
|
||||
[plugin-id]
|
||||
(if (= plugin-id r/mcp-plugin-id) "mcp" "plugin"))
|
||||
|
||||
(defn- get-context
|
||||
"Returns the audit context of an action run by a plugin: the file it ran
|
||||
in (none on the dashboard) and, for plugins other than the MCP, the
|
||||
plugin name and URL. Events drop the nil values."
|
||||
[state plugin-id]
|
||||
(let [file-id (:current-file-id state)]
|
||||
(if (= plugin-id r/mcp-plugin-id)
|
||||
{:file-id file-id}
|
||||
(let [plugin-data (r/get-plugin-data state plugin-id)]
|
||||
{:file-id file-id
|
||||
:plugin-name (:name plugin-data)
|
||||
:plugin-url (:url plugin-data)}))))
|
||||
|
||||
;; Formats an event from the plugin system
|
||||
(defn event
|
||||
[plugin-id name & {:as props}]
|
||||
(if (= plugin-id r/mcp-plugin-id)
|
||||
(-> props
|
||||
(assoc ::ev/name name)
|
||||
(assoc ::ev/origin "mcp")
|
||||
(ev/event))
|
||||
|
||||
(let [plugin-data (r/get-plugin-data @st/state plugin-id)]
|
||||
(-> props
|
||||
(assoc ::ev/name name)
|
||||
(assoc ::ev/origin "plugin")
|
||||
(assoc ::ev/context
|
||||
{:plugin-name (:name plugin-data)
|
||||
:plugin-url (:url plugin-data)})
|
||||
(ev/event)))))
|
||||
(-> props
|
||||
(assoc ::ev/name name)
|
||||
(assoc ::ev/origin (get-origin plugin-id))
|
||||
(assoc ::ev/context (get-context @st/state plugin-id))
|
||||
(ev/event)))
|
||||
|
||||
(defn add-event
|
||||
[event plugin-id]
|
||||
(let [plugin-data (r/get-plugin-data @st/state plugin-id)]
|
||||
(with-meta
|
||||
event
|
||||
(if (= plugin-id r/mcp-plugin-id)
|
||||
{::ev/origin "mcp"}
|
||||
{::ev/origin "plugin"
|
||||
::ev/context {:plugin-name (:name plugin-data)
|
||||
:plugin-url (:url plugin-data)}}))))
|
||||
(with-meta
|
||||
event
|
||||
{::ev/origin (get-origin plugin-id)
|
||||
::ev/context (get-context @st/state plugin-id)}))
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user