diff --git a/.agents/skills/update-changelog/SKILL.md b/.agents/skills/update-changelog/SKILL.md index 9d0276ef53..b6803bce46 100644 --- a/.agents/skills/update-changelog/SKILL.md +++ b/.agents/skills/update-changelog/SKILL.md @@ -5,16 +5,18 @@ description: Update the project CHANGES.md with issues from a given GitHub miles # Skill: update-changelog -Update `CHANGES.md` with entries for all issues and PRs in a given GitHub -milestone. Each entry references the user-facing issue (not the PR) as the -primary link, with the fix PR inline on the same line. +Entry point for all changelog work. All rules — sections, exclusions, entry +format, the `:boom:` and `:rocket:` label contracts, pre-flight, and the +anomaly report — live in `mem:workflow/updating-changelog` (file: +`.serena/memories/workflow/updating-changelog.md`). This skill routes to that +memory; it does not repeat it. ## When to Use - Before a new release, to populate the changelog with all fixed issues - When new issues are added to an existing milestone and the changelog needs to be refreshed -- To ensure every entry follows the correct format for the changelog +- To audit or repair an existing version section ## Prerequisites @@ -23,289 +25,28 @@ primary link, with the fix PR inline on the same line. - `scripts/gh.py` for GitHub queries, `scripts/changelog.py` for changelog checks (run `python3 scripts/changelog.py --help` for usage) -## Workflow +## What to Read -### 1. Determine the target version +1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth, + layout, entry rules, the `## Breaking change note` comment contract, + pre-flight checks, anomaly types, and pitfalls. +2. `mem:scripts/changelog` — `boom`, `check-merged`, `cross-ref`, `report`. +3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`. -The version is typically a semver string like `2.15.3`. Confirm with the user -if not specified. - -### 2. Fetch all issues in the milestone +## Commands ```bash -# All closed issues (default) -python3 scripts/gh.py issues "2.16.0" - -# Include open issues too -python3 scripts/gh.py issues "2.16.0" --state all - -# Exclude entries that should not go in the changelog -python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" +python3 scripts/gh.py issues "" --exclude "release blocker,no changelog" +python3 scripts/changelog.py check-merged +python3 scripts/changelog.py boom "" # print the :boom: section +python3 scripts/changelog.py boom "" --write # replace it in CHANGES.md +python3 scripts/changelog.py cross-ref "" +python3 scripts/changelog.py report "" ``` -**Exclusion rules (issue-level):** -- `no changelog` label — chore/refactor work, no entry needed -- `release blocker` label — blocked issues not yet ready for changelog -- `Task` issue type — internal chores, not user-facing; excluded by `gh.py` - (use `--include-tasks` to override) -- **Rejected project status** — issues with "Rejected" status on the "Main" - project board are excluded by `gh.py` (use `--include-rejected` to override). - This status is independent of the GitHub issue `state`. +## Hard Rules -**Exclusion rules (PR-level):** PRs with these labels stay out regardless of -their linked issue's labels: `release blocker`, `no issue required`. - -Each entry carries `number`, `title`, `state`, `issue_type`, `labels`, -`closing_prs`, and `project_status`. - -### 3. Identify missing entries (optional) - -```bash -python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --compare CHANGES.md -``` - -Returns only milestone issues not yet referenced in the changelog. Note: it -compares **issues** only. For unreferenced merged **PRs**, use the -cross-reference in step 8. - -### 4. Fetch additional PR details when needed - -```bash -# One or more PR numbers (also: --file prs.txt, or --stdin) -python3 scripts/gh.py prs 9179 9204 9311 - -# All merged PRs in a milestone (default); --state all/open/closed for others -python3 scripts/gh.py prs --milestone "2.16.0" -``` - -Returns `number`, `title`, `body`, `state`, `merged_at`, `author`, `labels`, -and `closing_issues`. Milestone mode uses paginated GraphQL (100 per page). - -### 5. Categorize entries — strictly by issue type, never by labels or emoji - -Use the **Issue Type** field (`issue_type` in the `gh.py` output). No separate -query is needed. - -> **⚠️ CRITICAL: Never use labels or title emoji prefixes for categorization.** -> Labels like `bug`/`enhancement` and prefixes like `:bug:`/`:sparkles:` are -> often wrong or missing. `issue_type` is the single source of truth. - -| `issue_type` value | Changelog section | -|--------------------|-------------------| -| `Bug` | `### :bug: Bugs fixed` | -| `Feature` or `Enhancement` | `### :sparkles: New features & Enhancements` | -| `Task` | **Exclude** — internal chores, not user-facing | -| `null` (not set) | Fallback to labels: `bug` label → bugs, otherwise enhancements | - -**Breaking changes override everything:** an issue with the `breaking change` -label goes under `### :boom: Breaking changes & Deprecations`, no matter its -issue type. This and `release highlight` below are the only label-based -rules — explicit exceptions to the "never by labels" principle above. The -`:boom:` subsection goes first in the version, before `:rocket:` (matching -the existing precedent). - -**Highlights come from the `release highlight` label:** an issue with that -label also gets an entry under `### :rocket: Epics and highlights`, on top of -its regular entry (usually `:sparkles:`). In the section being refreshed, -`:rocket:` lists exactly the labelled issues: never pick highlights yourself -and never keep an unlabelled entry there. Product decides the highlights by -labelling issues on GitHub, so a regeneration cannot undo that choice. Older -version sections are left as they are. - -**Community attribution:** if the issue or its fix PR has the -`community contribution` label, add `(by @)` on the entry -line, **before** the issue/PR references. Use the **PR author** (the `author` -field from step 4), not the issue author: - -```markdown -- Fix description of the bug (by @username) [#](...) (PR: [#](...)) -``` - -**Only closed issues are included**, even if open ones are tracked in the -milestone. - -**Pairing rules:** - -| Pattern | Changelog format | -|---------|-----------------| -| Closed issue + one or more fix PRs | Primary link = issue, PRs inline comma-separated | -| PR with no linked issue | Link the issue if a matching closed one exists in the milestone; otherwise skip (the issue is the changelog unit) | -| Closed issue with no fix PR in milestone | Link the issue directly, no PR reference | - -> **False-positive associations:** a PR may wrongly claim to close an issue -> from another context (ancient PR, cross-project reference). If titles are -> clearly unrelated or the PR predates the issue by years, treat it as a data -> glitch and skip it. - -### 5a. Verify PR merge status before writing - -A closed issue may list closing PRs that were **closed without merging** -(e.g. a superseded community PR). Only **merged** PRs go in the changelog: - -```bash -python3 scripts/changelog.py check-merged -# also accepts: --file prs.txt, or numbers via --stdin -``` - -If a closing PR is closed-unmerged, find the merged PR that superseded it -(other PRs in the issue's closing list, similar titles, or pointers in the -closed PR's timeline) and reference that one instead. - -### 5b. Security advisory (GHSA) entries - -Advisories fixed in a release go in the changelog even though they are -**neither milestone issues nor PRs**. The GHSA ID and description come from -the user or the release notes — never from the milestone fetch. - -```markdown -- Fix (https://github.com/penpot/penpot/security/advisories/GHSA-XXXX-XXXX-XXXX) -``` - -Rules: place under `### :bug: Bugs fixed` with **no issue or PR link**; do -**not** fetch or verify the URL (it may be draft/unpublished and 404); write -the description in imperative mood from the advisory title. These entries are -invisible to the automation — add them by hand, and in step 9 apply only the -backport/duplicate check to them. - -### 6. Read the current CHANGES.md and run pre-flight checks - -Newest version goes at the top, right after the `# CHANGELOG` header: - -```markdown -## - -### :boom: Breaking changes & Deprecations - -- [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) - -### :bug: Bugs fixed - -- Fix description of the bug [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) -- Fix another bug (by @contributor) [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) - -### :sparkles: New features & Enhancements - -- Add new feature description [#](https://github.com/penpot/penpot/issues/) (PR: [#](https://github.com/penpot/penpot/pull/)) -``` - -Format details: entries start with `- ` plus a short imperative description; -PR refs stay inline (`(PR: [#]())`, comma-separated for several); -`(by @)` goes before the issue link; only include non-empty -sections; blank line between a section's last entry and the next title; never -duplicate an entry from an earlier version section (the earlier version wins -for backports). - -**Pre-flight checks — fix violations directly in `CHANGES.md` before writing -the new section.** Reconcile every existing entry (any version section) and -every milestone candidate against the current milestone state (re-fetch, do -not trust cached data): - -1. **Duplicate across versions** → remove from the current section. -2. **Stale milestone assignment** (issue moved out of this milestone) → - remove the entry (or drop it if the section does not exist yet). -3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → - remove the entry. -4. **Issue no longer closed/deleted/Rejected** → remove the entry. -5. **Unmerged or moved PR reference** → fix the reference or remove the - entry (a PR merged in a *different* milestone is a step-9 anomaly, do not - silently remove it). -6. **Issue type changed to `Task`** → remove the entry. -7. **Breaking change misplaced** (issue has the `breaking change` label but - sits in another section) → move the entry to `:boom:`. -8. **Highlight out of sync** (`:rocket:` entry without the `release - highlight` label, or a labelled issue missing from `:rocket:`) → remove - the `:rocket:` entry or add it (step 7b). The regular entry stays. -9. **Missing valid issues** (closed, non-excluded, unreferenced anywhere) → - add them to the current section per step 5. - -### 7. Build the description text - -Derive it from the **issue title**, not the PR title. Strip leading emoji -prefixes (`:bug:`, `:sparkles:`, `:tada:`) and describe the user-facing -behavior: - -| Issue title | Changelog description | -|-------------|----------------------| -| `Plugin API token methods fail with schema validation error on PRO` | `Fix Plugin API token methods failing with schema validation error on PRO` | -| `Comment content is not sanitized before rendering, enabling stored XSS` | `Sanitize comment content on rendering` | -| `Custom uploaded font family names are not sanitized` | `Sanitize font family names on custom uploaded fonts` | - -Insert the new version section right after the `# CHANGELOG` header with the -`edit` tool and enough context for a unique match. - -### 7b. Populate `:rocket: Epics and highlights` from the label - -List the milestone issues that pass step 2 exclusions and carry the -`release highlight` label: - -```bash -python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog" \ - | python3 -c "import sys,json; [print(i['number'], i['title']) for i in json.load(sys.stdin) if 'release highlight' in i['labels']]" -``` - -Create the subsection if missing (place it before `### :sparkles:`) with one -entry per labelled issue, same text and references as its regular entry. -Every `:rocket:` entry MUST carry issue AND PR references. If no issue is -labelled, leave `:rocket:` out and tell the user: the highlights are -product's call, not the agent's. To change them, add or remove the label on -GitHub and re-run; never edit `:rocket:` by hand. - -### 8. Cross-reference milestone PRs against the changelog - -```bash -python3 scripts/changelog.py cross-ref "" [--changes CHANGES.md] -``` - -Lists merged milestone PRs missing from the changelog section (decide per PR: -add it or confirm its exclusion labels) and warns about CLOSED (unmerged) PRs -in the milestone. GHSA entries never appear here — that absence is expected. - -### 9. Generate the anomaly report - -```bash -python3 scripts/changelog.py report "" [--changes CHANGES.md --output CHANGES-ISSUES.md] -``` - -Overwrites `CHANGES-ISSUES.md` with the current state. Every number renders as -a full `[#N](https://github.com/penpot/penpot/issues/N)` or -`[#N](https://github.com/penpot/penpot/pull/N)` link. - -**An anomaly is a milestone mismatch or a `:boom:` mislabel** (the changelog -pairing is misleading and a human must judge intent): - -1. Issue in this milestone, referenced PR in another milestone (or none). -2. PR in this milestone, closed issue in another milestone — except an issue - with *no* milestone, which belongs to another (probably private) project - and is neither anomaly nor changelog candidate. -3. `:boom:` entry whose issue lacks the `breaking change` label. These stay - listed in the report **and** in the changelog: either label the issue or - move the entry to its regular section. -4. `:rocket:` out of sync with the `release highlight` label. Step 6 - pre-flight should have fixed it: if it shows up, re-run the workflow. - -**Highlight gaps are warnings, not anomalies:** missing `:rocket:` on a -released X.Y.0 (patches never carry highlights); `:rocket:` entry without -issue AND PR references. Gaps never count toward the anomaly total. - -**Anything else is a rule violation**, not a report item: fix it in step 6 -pre-flight. If one shows up in the report, re-run the workflow. - -## Key Principles - -- **Issue = changelog unit**, PR = implementation detail inline. -- **Latest version first**, below the `# CHANGELOG` header. -- **Issue Type decides the section — exclusively**, except `breaking change` - label → `:boom:` first, and `release highlight` label → also `:rocket:`. -- **User-facing descriptions** in imperative mood. -- **Community attribution** uses the PR author, placed before the issue link. -- **Only closed issues**; **Rejected** project status excludes. -- **`no changelog` and `Task`** stay out. -- **Multiple fix PRs** go comma-separated inline. -- **Duplicates**: the earlier version section wins. -- **Taiga references**: resolve to the GitHub issue via description text or - PRs mentioning the Taiga URL, then link issue + PR. -- **GHSA entries** live under `:bug:` with only the advisory URL (see 5b). -- **Re-fetch before editing**; prefer `scripts/gh.py` over raw `gh api`. -- **Verify PR merge status** (`check-merged`); PR-level exclusions apply. -- **Cross-reference PRs, not just issues** (`cross-ref`); watch for - false-positive PR-to-issue links. +- `CHANGES.md` is edited only on explicit request or during the release process. +- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline. +- `:boom:` and `:rocket:` are label-generated: never hand-edit them; edit the label or the `## Breaking change note` comment and re-run the command. +- Verify PR merge status before writing; only merged PRs are referenced. diff --git a/.serena/memories/backend/subtleties.md b/.serena/memories/backend/subtleties.md index 1b4fa4ddec..2282c6d792 100644 --- a/.serena/memories/backend/subtleties.md +++ b/.serena/memories/backend/subtleties.md @@ -85,8 +85,8 @@ - A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data. - Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision. - File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only written by the `legacy-db` backend, plus the rows written before the 2025-07 file storage refactor. -- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. +- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. `0157-migrate-legacy-file-data` moved the labelled leftovers to `file_data` and deleted the unlabelled payload rows, so from 2.19 on the label is the only marker there is. - The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again. - Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter. -- A deletion date is assigned once and never moved: `restore-files` (`app.rpc.commands.files`) unmarks only durable history, the `:file` method of `app.tasks.delete-object` dates only rows that have no date yet, and `objects-gc` collects only rows that carry one. Unmarking a revision leaves it invisible to `objects-gc` for ever, because nothing sets that date again. +- A deletion date is assigned once and never moved: `restore-files` (`app.rpc.commands.files`) unmarks only durable history, which is `label IS NOT NULL` (the unlabelled rows that still carry a legacy payload are removed by `0157-migrate-legacy-file-data`, so restore leaves them dated), the `:file` method of `app.tasks.delete-object` dates only rows that have no date yet, and `objects-gc` collects only rows that carry one. Unmarking a revision leaves it invisible to `objects-gc` for ever, because nothing sets that date again. - Binfile v1/v3 import and the v3 export `get-file` step run inside one long transaction. They raise `idle_in_transaction_session_timeout` to 30 min (`app.binfile.common/transaction-idle-timeout-ms`) instead of disabling it, so a connection left idle while the ZIP is decoded or the file data is realized is bounded; `statement_timeout` stays at 300 s. diff --git a/.serena/memories/critical-info.md b/.serena/memories/critical-info.md index 867eecdbd9..43f678a643 100644 --- a/.serena/memories/critical-info.md +++ b/.serena/memories/critical-info.md @@ -15,11 +15,12 @@ You are working on the GitHub project `penpot/penpot`, a monorepo. - Before `git commit` → `mem:workflow/creating-commits` (subject/body format, 80-char body wrapping enforced by `scripts/check-commit`, `AI-assisted-by: model-name` trailer) - Before `gh issue create` → `mem:workflow/creating-issues` (title derivation, body template, labels, Issue Type) - Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, "Note:" line) +- Before updating `CHANGES.md` → `mem:workflow/updating-changelog` (sources of truth, entry rules, the `:boom:`/`:rocket:` label contracts, pre-flight checks, anomaly report) - Before a repo-wide pnpm version update → `mem:workflow/updating-pnpm` (workspace layout, `scripts/sync-pnpm-version` flow, the stamp-missing-field and ignored-builds gotchas, verification steps) - Before a Node version update → `mem:workflow/updating-node` (the six pin sites in `.nvmrc`, the devenv and image Dockerfiles and the plugins-deploy workflows, the DHI tag gotcha, what to verify) - Before bumping a non-JS dependency pinned in `docker/images/` (JDK, ImageMagick) → `mem:workflow/docker-images` - **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked. -- **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via the `update-changelog` skill flow or on explicit user request. +- **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or on explicit user request. - You have access to the GitHub CLI `gh` or corresponding MCP tools. - Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool. - Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`). @@ -59,7 +60,7 @@ The memory is structured in a way that you can get the critical information abou - `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`. - `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`. - `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`. -- `scripts/changelog.py` — Changelog checks for the `update-changelog` skill: `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. +- `scripts/changelog.py` — Changelog checks for `mem:workflow/updating-changelog`: `boom` (generate the `:boom:` section from `breaking change` issues), `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`. - `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`. - `scripts/clean-node-modules` — Remove stale `node_modules` from all pnpm workspaces (root, modules, member packages). Keeps the shared pnpm store at `/.pnpm-store` unless `--store`; ignores `external/` and `.opencode/`. Usage and reinstall steps: `mem:workflow/updating-pnpm`. - `scripts/setup-opencode` — Install the opencode CLI in-container as a global pnpm package in `$PNPM_HOME`. Not part of the devenv image; see `mem:devenv/core`. diff --git a/.serena/memories/scripts/changelog.md b/.serena/memories/scripts/changelog.md index 6dc0dd5c94..e04036ea9a 100644 --- a/.serena/memories/scripts/changelog.md +++ b/.serena/memories/scripts/changelog.md @@ -1,12 +1,13 @@ # Changelog helper -`scripts/changelog.py` holds the checks used by the `update-changelog` skill (extracted from the skill file so the skill stays workflow-only). Calls `scripts/gh.py` via subprocess; never call the GitHub API directly from this script. +`scripts/changelog.py` holds the checks used by the changelog flow. The rules and the step-by-step live in `mem:workflow/updating-changelog`; this memory documents the subcommands. Calls `scripts/gh.py` via subprocess; never call the GitHub API directly from this script. ## Subcommands - `check-merged ` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries. +- `boom [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### ` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`. - `cross-ref [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings; GHSA entries never appear here by design. -- `report [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report (milestone mismatches type A/B plus `:rocket:` gaps C/D); rule violations never belong in the report, they are fixed in CHANGES.md first. +- `report [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place. ## Conventions @@ -15,4 +16,5 @@ - Version-section lookup accepts an optional `(suffix)` after the version header (e.g. `(Unreleased)`); a missing section is an error for `cross-ref` and an empty section for `report`. - PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms. - Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes. -- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry); C/D are `:rocket:` gaps (warnings, never counted). +- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry), G `breaking change` issue missing from `:boom:` (counted, same contract as F); C/D are `:rocket:` gaps (warnings, never counted). +- `issue_excluded` (closed, non-Task, non-Rejected, no `release blocker`/`no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift. diff --git a/.serena/memories/scripts/gh.md b/.serena/memories/scripts/gh.md index 9c53875672..93fd4e37b8 100644 --- a/.serena/memories/scripts/gh.md +++ b/.serena/memories/scripts/gh.md @@ -11,6 +11,7 @@ repository via GraphQL and REST APIs through the authenticated `gh` CLI. - Comparing milestone issues against CHANGES.md to find missing entries. - Explicitly linking a GitHub issue to a pull request. - Listing or inspecting GitHub Security Advisories (GHSA). +- Fetching issue comments (the `## Breaking change note` markdown for the changelog). ## Prerequisites @@ -82,6 +83,16 @@ Fetch issue details by number in batches of 50 (same input styles as `prs`: numb **Output**: JSON array to stdout; progress to stderr. +### `comments` + +Fetch every comment of one or more issues via REST (paginated, 100 per page), oldest first. Same input styles as `issue`: numbers, `--file`, `--stdin`. + +```bash +python3 scripts/gh.py comments 11235 11236 +``` + +**Output fields per comment**: `issue`, `id`, `author`, `created_at`, `body`. JSON array to stdout; progress to stderr. `changelog.py boom` reads the `## Breaking change note` comment from this output. + ### `link-issue` Explicitly assign a GitHub issue to a pull request: diff --git a/.serena/memories/workflow/creating-issues.md b/.serena/memories/workflow/creating-issues.md index dbe55c9241..537e3e73ea 100644 --- a/.serena/memories/workflow/creating-issues.md +++ b/.serena/memories/workflow/creating-issues.md @@ -35,11 +35,35 @@ Command what should be built. Format: `[Imperative verb] [what] in/on [where]`. | Field | Rule | |-------|------| -| **Labels** | `community contribution` (PRs from non-core) · skip workflow labels (`backport candidate`, `team-qa`) · do **not** add `bug` or `enhancement` labels (use Issue Type instead) | +| **Labels** | `community contribution` (PRs from non-core) · `breaking change` when the change breaks compatibility (see below) · skip workflow labels (`backport candidate`, `team-qa`) · do **not** add `bug` or `enhancement` labels (use Issue Type instead) | | **Milestone** | Use the current or next planned milestone. Fetch available milestones: `gh api repos/penpot/penpot/milestones --jq '.[].title'`. If unsure, omit. | | **Project** | Always `Main` (project number 8). Use `--project "Main"` flag. | | **Issue Type** | See Issue Type section below. Cannot be set via `gh issue create` — use GraphQL after creation. | +## Breaking changes + +When the issue introduces a breaking change or a deprecation, it is the unit +`mem:workflow/updating-changelog` turns into a block under +`### :boom: Breaking changes & Deprecations`. The section is generated from +the issue (`scripts/changelog.py boom`), so the issue must carry both: + +1. The `breaking change` label. +2. A `## Breaking change note` **comment** (not a body section). Everything + after that marker line is copied to the changelog: paragraphs, bullet + lists, code blocks and headings are kept (headings are demoted to level 5 + in `CHANGES.md`). Write it for self-hosters: what changes, what they must + do, and what breaks if they do nothing. Keep it short: explain the + implications and link the runbook or docs for the detail, never paste the + runbook itself. Links must point to a long-lived branch (`staging` or + `main`), never to a feature branch. + +Post the note as a new comment. Do not add a breaking-change section to the +issue body and do not hand-write the changelog block. When the issue comes +from a PR, reuse the text of the PR's breaking-change section. + +The full changelog rules (sections, exclusions, `:rocket:`, pre-flight and +anomaly report) live in `mem:workflow/updating-changelog`. + ## Issue Body Template Write the body to a temp file to avoid shell quoting issues: @@ -237,6 +261,9 @@ Identify: description). Omit internal details. Use the templates above. - **Issue Type:** use the mapping table above (also handles `:bug:` / `:sparkles:` / `:tada:` title prefixes). +- **Breaking changes:** when the PR body has a breaking-change section, post + its user-facing text as a `## Breaking change note` comment on the issue and + add the `breaking change` label (see **Breaking changes** above). ### Create the issue @@ -303,6 +330,8 @@ rm -f /tmp/issue-body.md /tmp/pr-body.md label or the author is not a core team member, add the label to the issue. - **Don't put implementation details in the issue body** — the issue is for users, QA, and changelog readers. +- **Drop the PR's `no issue required` label** once the issue exists: it only + marks PRs that need no changelog unit, and it is stale from then on. ## Creating Issues from Draft Body diff --git a/.serena/memories/workflow/creating-prs.md b/.serena/memories/workflow/creating-prs.md index 4d9d0fe8d4..4f2d2bcfa7 100644 --- a/.serena/memories/workflow/creating-prs.md +++ b/.serena/memories/workflow/creating-prs.md @@ -66,7 +66,7 @@ The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR. - **Write for humans.** The diff shows what changed. The description explains why. - **Be concise.** Focus on reasoning: What was the problem? Why did it happen? How did you solve it? - **Prefer bullets over paragraphs.** Short bullet items, grouped by area with bold lead-ins where helpful, are far easier to digest than prose; keep any remaining paragraph to a few sentences. -- **No manual line wraps.** Markdown renders adapting to the viewport; hard-wrapped lines degrade rendering. One line per paragraph or bullet, however long. +- **No manual line wraps.** Markdown renders adapting to the viewport; hard-wrapped lines degrade rendering. One line per paragraph or bullet, however long. This applies to PR descriptions and to PR/issue comments and review replies alike. - **Skip the obvious.** Don't explain what `git diff` already shows. ### What NOT to Include diff --git a/.serena/memories/workflow/updating-changelog.md b/.serena/memories/workflow/updating-changelog.md new file mode 100644 index 0000000000..26ac7624af --- /dev/null +++ b/.serena/memories/workflow/updating-changelog.md @@ -0,0 +1,187 @@ +# Updating the changelog + +> Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human release managers who need full context. That is why it uses plain language and examples. It is the canonical home of the changelog flow; the `update-changelog` skill is only an entry point that routes here. + +`CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory. It is never written from scratch by hand; the only manual parts are the security advisory entries and the wording of a description when the title is not enough. + +The commands live in `scripts/gh.py` (GitHub queries) and `scripts/changelog.py` (changelog checks). Their flags and internals are documented in `mem:scripts/gh` and `mem:scripts/changelog`. + +## 1. When to run it + +- Before a release, to fill the section of the version being released. +- When new issues are added to an existing milestone. +- When asked to audit or refresh `CHANGES.md`. + +Editing `CHANGES.md` happens only on explicit request or as part of the release process; see `mem:critical-info`. + +## 2. Sources of truth + +| Question | Source | +|---|---| +| Which issues belong to the version? | GitHub milestone | +| Which section? | Issue Type | +| Is it a breaking change? | `breaking change` label | +| Is it a highlight? | `release highlight` label | + +- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them. +- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry. +- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge. +- **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status. +- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own. +- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly. + +## 3. Layout of `CHANGES.md` + +The newest version goes at the top, right after the `# CHANGELOG` header. Inside a version, the section order is fixed: + +```markdown +# CHANGELOG + +## 2.19.0 (Unreleased) + +### :boom: Breaking changes & Deprecations + +#### + + + +[#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) + +### :rocket: Epics and highlights + +- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237)) + +### :bug: Bugs fixed + +- Fix [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) + +### :sparkles: New features & Enhancements + +- Add [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR)) +``` + +- `:boom:` first, then `:rocket:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions. +- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. +- One blank line between the last entry of a section and the next heading. +- No manual line wrapping: one line per bullet or paragraph, however long. + +## 4. Regular entries (`:bug:`, `:sparkles:`) + +The entry shape is: + +```markdown +- (by @github_username) [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR), [#PR2](https://github.com/penpot/penpot/pull/PR2)) +``` + +- **Description**: derived from the issue title, not the PR title. Strip emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`) and write the user-facing behaviour in imperative mood. Examples: `Plugin API token methods fail with schema validation error` → `Fix Plugin API token methods failing with schema validation error`; `Custom uploaded font family names are not sanitized` → `Sanitize font family names on custom uploaded fonts`. +- **Issue is the primary link**: the issue is the changelog unit, the PR is the implementation. Multiple fix PRs go comma-separated inline. +- **Community attribution**: when the issue or its PR carries `community contribution`, add `(by @username)` before the issue link. Use the PR author, not the issue author. +- **Closed issue with no PR** in the milestone: link the issue only, no PR reference. +- **PR with no linked issue**: link a matching closed issue in the milestone when one exists; otherwise skip it, because the issue is the changelog unit. +- **False-positive PR links**: a PR may claim to close an unrelated issue (ancient PR, cross-project reference). If the titles are clearly unrelated or the PR predates the issue by years, treat it as a data glitch and ignore it. + +## 5. `:boom:`: breaking changes and deprecations + +Each breaking change is a `####` block, not a bullet. The title is the issue title without its emoji prefix; the content comes from a comment on the issue; the references go on their own line at the end. + +The **issue** must carry: + +1. The `breaking change` label. +2. A comment whose first line is `## Breaking change note`. Everything after that marker line is the changelog content: paragraphs, bullet lists, code blocks, tables, headings (they are demoted to `#####` on write), and so on. The marker itself is case-insensitive and accepts `#`, `##` or `###`; use `##`. + +Example comment: + +```markdown +## Breaking change note + +The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed. + +Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)). +``` + +Generated block: + +```markdown +#### Enable the Admin Console by default + +The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed. + +Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)). + +[#12807](https://github.com/penpot/penpot/issues/12807) (PR: [#12475](https://github.com/penpot/penpot/pull/12475)) +``` + +Rules for the note: + +- **Write it for admins**: what changes, what they must do, and what breaks if they do nothing. Do not paste the full runbook; explain the implications and link the runbook or docs. +- **Link a long-lived branch**, `staging` or `main`, never a feature branch (`issue-1234`). The link may 404 until the PR merges; that is expected. +- **Do not add issue or PR links**: the generator appends them. +- **One note per issue**. If several comments carry the marker, the generator uses the last one and warns. +- **No note**: the generator warns on stderr and emits only the heading and references; fix the issue, not `CHANGES.md`. +- **`####`, not `###`**: `###` is the level of the changelog sections. An entry at `###` would render as a sibling section and the scripts that read and rewrite `:boom:` would cut the section there. The note author never writes the `####`: the generator adds it. +- **Only merged closing PRs** are referenced; unmerged or superseded PRs are skipped. +- **Never hand-edit the section**. Generate it with `boom` (section 7). The note lives on GitHub; edit it there. + +## 6. `:rocket:`: epics and highlights + +- Lists exactly the milestone issues with the `release highlight` label, and nothing else. Product decides highlights by labelling issues; a regeneration cannot undo that choice. +- Every entry copies the text and references of the issue's regular entry and MUST carry both an issue link and a PR reference. +- If no issue is labelled, leave the section out; do not pick highlights yourself. +- Never edit `:rocket:` by hand: add or remove the label and regenerate. + +## 7. Flow + +1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious. +2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`. +3. **Fetch PR details when needed**: `python3 scripts/gh.py prs ` (also `--milestone`). Use it for the PR author, labels and merge state. +4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4). +5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged `. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it. +6. **Reconcile the existing text** (section 8). +7. **Generate `:boom:`**: `python3 scripts/changelog.py boom ""` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent. +8. **Generate `:rocket:`** from the `release highlight` label (section 6). +9. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref ""`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. GHSA entries never appear; that absence is expected. +10. **Generate the anomaly report**: `python3 scripts/changelog.py report ""`. It overwrites `CHANGES-ISSUES.md` with the current state (section 9). +11. **Security advisories (GHSA)**: add them by hand under `:bug:` as `- Fix ()`, with no issue or PR link. Do not fetch or verify the URL: it may be draft and return 404. Write the description in imperative mood from the advisory title. These entries are invisible to the automation; apply only the duplicate/backport check to them. + +## 8. Pre-flight: reconcile before writing + +Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` before adding the new section: + +1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports. +2. **Stale milestone assignment** (the issue moved out of this milestone) → remove the entry. +3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry. +4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry. +5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 9); do not remove it silently. +6. **Issue Type changed to `Task`** → remove the entry. +7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry. +8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it. +9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section. + +## 9. Anomaly report + +`report` classifies problems it cannot decide alone. The total counts A, B, E, F and G; C and D are warnings. + +- **A — issue in this milestone, referenced PR in another milestone (or none).** The claim "this issue is fixed by this PR in this milestone" is inconsistent. Fix the milestone on the issue or the PR, or remove the entry. +- **B — PR in this milestone whose closing issue is in another milestone.** Same fix. An issue with no milestone is not an anomaly (another, probably private, project). +- **E — `:boom:` entry whose issue lacks the `breaking change` label.** The entry stays in the changelog: either label the issue or move the entry to its regular section. +- **F — `:rocket:` out of sync with the `release highlight` label.** A rule violation: `:rocket:` must list exactly the labelled issues. Fix the label or re-run the flow. +- **G — `breaking change` issue missing from `:boom:`.** A rule violation: `:boom:` must list exactly the labelled issues. Regenerate the section or relabel. +- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights). +- **D — `:rocket:` entry without issue AND PR references** (gap). + +Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run. + +## 10. Pitfalls + +- **Open issues with provisional entries**: a PR author may add the entry before the merge. `boom` will not generate it until the issue closes; keep the hand-written block identical to what the generator would produce. +- **Labels, not emoji**: title prefixes are cosmetic and often missing; the section comes from Issue Type and the two labels. +- **Taiga references**: old entries link to Taiga. Resolve them to the GitHub issue via the description text or the PRs that mention the Taiga URL, then link issue and PR. +- **Long-lived links only**: runbooks and docs referenced from a note must point to `staging` or `main`, never to a feature branch that will be deleted. +- **Generated sections**: `:boom:` and `:rocket:` are the only label-generated parts. Never patch them by hand; edit the label or the note comment and re-run. + +## 11. Where the code is + +- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories`. Details: `mem:scripts/gh`. +- `scripts/changelog.py`: `check-merged`, `boom`, `cross-ref`, `report`. Details: `mem:scripts/changelog`. +- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`. +- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**. diff --git a/AGENTS.md b/AGENTS.md index c95fb06920..c39c02dcaf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,8 @@ If the user pushes, treat that commit as final from the agent's side. - **Never edit `CHANGES.md` by hand** in commits or PRs. The changelog is generated from GitHub milestones during the release process; update it only - via the `update-changelog` skill flow or on explicit user request. + via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or + on explicit user request. - **Never pipe test output directly to filters** (`| head`, `| tail`, `| grep`, etc.). Always redirect to a file first: `command > /tmp/output.txt 2>&1`, then read/grep the file. This prevents hiding test failures. See `mem:testing` for details. diff --git a/CHANGES.md b/CHANGES.md index 6da1e66c2d..1fd9718201 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,6 +1,22 @@ # CHANGELOG -## 2.19.0 (Unreleased) +## 2.19.0 + +### :boom: Breaking changes & Deprecations + +#### Migrate legacy file payloads to file_data and delete orphan revisions + +File payloads now live in one place: every file state and snapshot is kept in `file_data`, and the orphan `file_change` rows that kept old payloads alive forever are removed. No read path reached those rows and nothing ever collected them, so the upgrade also reclaims the database and TOAST space they held. There is no API or UI change. + +On small and medium instances this happens automatically on boot and there is nothing to do. On large instances (millions of snapshots, hundreds of GB in TOAST) the work does not fit in the startup transaction: run it by hand **before upgrading to 2.19.0** following the [large-instance runbook](https://github.com/penpot/penpot/blob/staging/operations/2.19.0-migrate-legacy-file-data.md). The runbook tells you whether your instance needs it, keeps the instance online while the data moves, and verifies the result before you deploy. + +[#12784](https://github.com/penpot/penpot/issues/12784) (PR: [#12785](https://github.com/penpot/penpot/pull/12785)) + +#### Enable the Admin Console by default + +The Admin Console is now enabled by default and the `enable-admin-console` flag is no longer needed. Self-hosted instances must deploy the `penpot-admin-console` service and set `PENPOT_ADMIN_CONSOLE_URI` in frontend and backend (check the [upgrade guide](https://help.penpot.app/technical-guide/getting-started/docker/#upgrade-to-2.19)). + +[#12807](https://github.com/penpot/penpot/issues/12807) (PR: [#12475](https://github.com/penpot/penpot/pull/12475)) ### :rocket: Epics and highlights @@ -74,6 +90,37 @@ - Fix thumbnails failing on older Chromium browsers and each thumbnail rendering several times [#11878](https://github.com/penpot/penpot/issues/11878) (PR: [#11879](https://github.com/penpot/penpot/pull/11879)) - Fix internal error when opening the libraries modal [#11943](https://github.com/penpot/penpot/issues/11943) (PR: [#11946](https://github.com/penpot/penpot/pull/11946)) - Fix Plugin API `connectLibrary` linking unpublished files as libraries [#12001](https://github.com/penpot/penpot/issues/12001) (PR: [#12002](https://github.com/penpot/penpot/pull/12002)) +- Fix incorrect padding at the top of the tokens and themes section [#9585](https://github.com/penpot/penpot/issues/9585) (PR: [#12422](https://github.com/penpot/penpot/pull/12422)) +- Fix token tree path handling for repeated and nested segments (by @makesomethingshit) [#11588](https://github.com/penpot/penpot/issues/11588) (PR: [#11591](https://github.com/penpot/penpot/pull/11591)) +- Fix comment input duplicating text when confirming a Japanese IME composition with Enter [#11757](https://github.com/penpot/penpot/issues/11757) (PR: [#11768](https://github.com/penpot/penpot/pull/11768)) +- Fix file not saving after deleting a variant [#11758](https://github.com/penpot/penpot/issues/11758) (PR: [#12216](https://github.com/penpot/penpot/pull/12216)) +- Fix component copies not syncing when a shape is moved out of a main component and another change follows at once [#11978](https://github.com/penpot/penpot/issues/11978) (PR: [#11981](https://github.com/penpot/penpot/pull/11981)) +- Fix MCP plugin showing a version mismatch warning on every patch release (by @robertkodra) [#12007](https://github.com/penpot/penpot/issues/12007) (PR: [#12009](https://github.com/penpot/penpot/pull/12009)) +- Fix workspace crash when switching to the path tool while drawing a shape [#12100](https://github.com/penpot/penpot/issues/12100) (PR: [#12101](https://github.com/penpot/penpot/pull/12101)) +- Fix referential integrity error when swapping a component [#12144](https://github.com/penpot/penpot/issues/12144) (PR: [#12333](https://github.com/penpot/penpot/pull/12333)) +- Fix workspace saving text shapes on every page open when a custom font is missing [#12151](https://github.com/penpot/penpot/issues/12151) (PR: [#12164](https://github.com/penpot/penpot/pull/12164)) +- Fix swapped nested components losing their absolute position after a library update [#12176](https://github.com/penpot/penpot/issues/12176) (PR: [#12556](https://github.com/penpot/penpot/pull/12556)) +- Fix theme selector showing the editor message when the tokens source is a library [#12177](https://github.com/penpot/penpot/issues/12177) (PR: [#12203](https://github.com/penpot/penpot/pull/12203)) +- Fix untranslated texts in the This file tab of the libraries modal [#12178](https://github.com/penpot/penpot/issues/12178) (PR: [#12204](https://github.com/penpot/penpot/pull/12204)) +- Fix workspace staying in the saving state and losing unsaved edits when a save request gets no answer [#12222](https://github.com/penpot/penpot/issues/12222) (PR: [#12223](https://github.com/penpot/penpot/pull/12223)) +- Fix scrub cursor showing on numeric inputs with mixed values in the sidebar [#12227](https://github.com/penpot/penpot/issues/12227) (PR: [#12229](https://github.com/penpot/penpot/pull/12229)) +- Fix internal error when copying a swapped nested component [#12289](https://github.com/penpot/penpot/issues/12289) (PR: [#12434](https://github.com/penpot/penpot/pull/12434)) +- Show an error when uploading a custom font fails [#12304](https://github.com/penpot/penpot/issues/12304) (PR: [#12315](https://github.com/penpot/penpot/pull/12315)) +- Fix React warning about the unknown `tooltipClass` prop in layout-item size inputs [#12311](https://github.com/penpot/penpot/issues/12311) (PR: [#12329](https://github.com/penpot/penpot/pull/12329)) +- Fix guides inside a board still being interactive [#12322](https://github.com/penpot/penpot/issues/12322) (PR: [#12460](https://github.com/penpot/penpot/pull/12460)) +- Fix pasted text not being visible in an empty text box until it is resized [#12438](https://github.com/penpot/penpot/issues/12438) (PR: [#12859](https://github.com/penpot/penpot/pull/12859)) +- Fix files failing referential integrity validation staying unsaveable in every session [#12464](https://github.com/penpot/penpot/issues/12464) (PR: [#12524](https://github.com/penpot/penpot/pull/12524), [#12662](https://github.com/penpot/penpot/pull/12662)) +- Fix unicode characters not rendering in text layers with the new render engine [#12469](https://github.com/penpot/penpot/issues/12469) (PR: [#12471](https://github.com/penpot/penpot/pull/12471)) +- Fix text image fills staying blank when media loads after text layout [#12563](https://github.com/penpot/penpot/issues/12563) (PR: [#12564](https://github.com/penpot/penpot/pull/12564)) +- Fix swapped nested icons moving outside the layout after a library update [#12657](https://github.com/penpot/penpot/issues/12657) (PR: [#12716](https://github.com/penpot/penpot/pull/12716)) +- Fix garbled bullets before the element counts in the libraries modal [#12677](https://github.com/penpot/penpot/issues/12677) (PR: [#12678](https://github.com/penpot/penpot/pull/12678)) +- Fix token dropdown of numeric inputs showing the scrub cursor [#12679](https://github.com/penpot/penpot/issues/12679) (PR: [#12680](https://github.com/penpot/penpot/pull/12680)) +- Fix variant property names and values accepting `,` and `=` [#12683](https://github.com/penpot/penpot/issues/12683) (PR: [#12688](https://github.com/penpot/penpot/pull/12688)) +- Fix changing a variant property from another page leaving the main component label out of date [#12692](https://github.com/penpot/penpot/issues/12692) (PR: [#12693](https://github.com/penpot/penpot/pull/12693)) +- Fix `:referential-integrity` errors when moving variant mains in or out of a set [#12696](https://github.com/penpot/penpot/issues/12696) (PR: [#12706](https://github.com/penpot/penpot/pull/12706)) +- Fix `:referential-integrity` errors when creating, combining or renaming variant sets [#12734](https://github.com/penpot/penpot/issues/12734) (PR: [#12779](https://github.com/penpot/penpot/pull/12779)) +- Fix save errors after masking a variant with Ctrl+M [#12748](https://github.com/penpot/penpot/issues/12748) (PR: [#12749](https://github.com/penpot/penpot/pull/12749)) +- Fix icons in component instances not being shown on the canvas [#12752](https://github.com/penpot/penpot/issues/12752) ### :sparkles: New features & Enhancements @@ -89,7 +136,7 @@ - Apply tokens to stroke per side [#9842](https://github.com/penpot/penpot/issues/9842) (PR: [#10913](https://github.com/penpot/penpot/pull/10913)) - Add individual stroke widths per side (top, right, bottom, left) [#9854](https://github.com/penpot/penpot/issues/9854) (PR: [#10648](https://github.com/penpot/penpot/pull/10648)) - Expand the token tree right after importing it (by @AKnassa) [#9819](https://github.com/penpot/penpot/issues/9819) (PR: [#10664](https://github.com/penpot/penpot/pull/10664)) -- Update the selection size badge live during resize, replacing the resize tooltip [#10259](https://github.com/penpot/penpot/issues/10259) +- Update the selection size badge live during resize, replacing the resize tooltip [#10259](https://github.com/penpot/penpot/issues/10259) (PR: [#11884](https://github.com/penpot/penpot/pull/11884)) - Add color variants, rotation correction, viewport flip and hide-when-too-small to the selection size badge [#10264](https://github.com/penpot/penpot/issues/10264) (PR: [#11884](https://github.com/penpot/penpot/pull/11884)) - Support single-file SVG export with the WebGL render [#10546](https://github.com/penpot/penpot/issues/10546) - Persist the hide resolved comments preference and separate the filter options in the comments menu [#10686](https://github.com/penpot/penpot/issues/10686) (PR: [#10694](https://github.com/penpot/penpot/pull/10694)) @@ -113,13 +160,30 @@ - Make "Fix when scrolling" work in the WebGL render [#11684](https://github.com/penpot/penpot/issues/11684) (PR: [#11752](https://github.com/penpot/penpot/pull/11752)) - Complete Catalan translations [#11739](https://github.com/penpot/penpot/issues/11739) (PR: [#11741](https://github.com/penpot/penpot/pull/11741)) - Remove the optional client-provided id from creation RPC endpoints [#11783](https://github.com/penpot/penpot/issues/11783) (PR: [#11784](https://github.com/penpot/penpot/pull/11784)) -- Allow color tokens for the canvas background [#11791](https://github.com/penpot/penpot/issues/11791) (PR: [#11923](https://github.com/penpot/penpot/pull/11923)) +- Allow color tokens for the canvas background [#11791](https://github.com/penpot/penpot/issues/11791) (PR: [#11923](https://github.com/penpot/penpot/pull/11923), [#11950](https://github.com/penpot/penpot/pull/11950)) - Show a "no assets yet" message for empty files in the libraries modal [#11811](https://github.com/penpot/penpot/issues/11811) (PR: [#11818](https://github.com/penpot/penpot/pull/11818)) - Upgrade the MCP SDK to v2 and adopt the new protocol version [#11827](https://github.com/penpot/penpot/issues/11827) (PR: [#11841](https://github.com/penpot/penpot/pull/11841)) - Remove legacy SSE support from the MCP server [#11846](https://github.com/penpot/penpot/issues/11846) (PR: [#11841](https://github.com/penpot/penpot/pull/11841)) - Delete selected path handles with Backspace or Delete [#11858](https://github.com/penpot/penpot/issues/11858) (PR: [#11896](https://github.com/penpot/penpot/pull/11896)) - Use the tooltip component in the token pill info [#11886](https://github.com/penpot/penpot/issues/11886) (PR: [#11900](https://github.com/penpot/penpot/pull/11900)) - Hide the duplicate cursor when alt-dragging a shape that cannot be duplicated (by @0xTHAC0) [#11905](https://github.com/penpot/penpot/issues/11905) (PR: [#11195](https://github.com/penpot/penpot/pull/11195)) +- Document the team auto-add behavior with `disable-email-verification` [#11376](https://github.com/penpot/penpot/issues/11376) (PR: [#11425](https://github.com/penpot/penpot/pull/11425)) +- Make the workspace recover from interrupted saves and show a proper error instead of the internal error page [#11799](https://github.com/penpot/penpot/issues/11799) +- Add component management operations to the Plugin API [#12301](https://github.com/penpot/penpot/issues/12301) (PR: [#12303](https://github.com/penpot/penpot/pull/12303)) +- Record the imported file ids in the import-binfile audit event [#12485](https://github.com/penpot/penpot/issues/12485) (PR: [#12491](https://github.com/penpot/penpot/pull/12491)) + +## 2.18.6 + +### :bug: Bugs fixed + +- Fix upload endpoints storing server files when the path points outside the temp dir [#12771](https://github.com/penpot/penpot/issues/12771) (PR: [#12772](https://github.com/penpot/penpot/pull/12772)) + +## 2.18.5 + +### :bug: Bugs fixed + +- Fix the Admin Console failing on every URL on Docker deployments by reverting the subpath support [#12358](https://github.com/penpot/penpot/pull/12358) + ## 2.18.4 ### :bug: Bugs fixed diff --git a/backend/src/app/binfile/common.clj b/backend/src/app/binfile/common.clj index 1a010c265b..95241f062f 100644 --- a/backend/src/app/binfile/common.clj +++ b/backend/src/app/binfile/common.clj @@ -472,6 +472,18 @@ {:team-id team-id :deleted-at nil})) +(def ^:private sql:get-files-rels + "SELECT flr.*, + fls.synced_at + FROM file_library_rel AS flr + JOIN file AS l + ON flr.library_file_id = l.id + LEFT JOIN file_library_sync AS fls + ON fls.file_id = flr.file_id + AND fls.library_file_id = flr.library_file_id + WHERE flr.file_id = ANY(?) + AND l.deleted_at IS NULL;") + (defn get-files-rels "Given a set of file-id's, return all matching relations with the libraries" [cfg ids] @@ -480,18 +492,8 @@ (assert (every? uuid? ids) "expected a set of uuids") (db/run! cfg (fn [{:keys [::db/conn]}] - (let [ids (db/create-array conn "uuid" ids) - sql (str "SELECT flr.*," - " fls.synced_at" - " FROM file_library_rel AS flr" - " JOIN file AS l" - " ON flr.library_file_id = l.id" - " LEFT JOIN file_library_sync AS fls" - " ON fls.file_id = flr.file_id" - " AND fls.library_file_id = flr.library_file_id" - " WHERE flr.file_id = ANY(?)" - " AND l.deleted_at IS NULL;")] - (db/exec! conn [sql ids]))))) + (let [ids (db/create-array conn "uuid" ids)] + (db/exec! conn [sql:get-files-rels ids]))))) (def ^:private sql:upsert-file-library-sync "INSERT INTO file_library_sync (file_id, library_file_id, synced_at) diff --git a/backend/src/app/binfile/v3.clj b/backend/src/app/binfile/v3.clj index cd942ccb57..15173a885c 100644 --- a/backend/src/app/binfile/v3.clj +++ b/backend/src/app/binfile/v3.clj @@ -989,20 +989,24 @@ (defn- import-file-relations [{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}] (events/tap :progress {:section :relations}) - (doseq [[file-id libr-id] (:relations manifest)] - - (let [file-id (bfc/lookup-index file-id) - libr-id (bfc/lookup-index libr-id)] - - (when (and file-id libr-id) - (l/dbg :hint "create file library link" - :file-id (str file-id) - :lib-id (str libr-id) - ::l/sync? true) - (let [rel-params {:file-id file-id - :library-file-id libr-id}] - (db/insert! conn :file-library-rel rel-params) - (bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))) + (let [file-ids (into #{} (map :id) (:files manifest))] + (doseq [[file-id libr-id] (:relations manifest)] + (if (and (not= file-id libr-id) + (contains? file-ids file-id) + (contains? file-ids libr-id)) + (let [file-id (bfc/lookup-index file-id) + libr-id (bfc/lookup-index libr-id)] + (l/dbg :hint "create file library link" + :file-id (str file-id) + :lib-id (str libr-id) + ::l/sync? true) + (let [rel-params {:file-id file-id + :library-file-id libr-id}] + (db/insert! conn :file-library-rel rel-params) + (bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp)))) + (l/warn :hint "ignoring invalid file library relation" + :file-id (str file-id) + :lib-id (str libr-id)))))) (defn- import-storage-objects [{:keys [::bfc/input ::entries-index ::bfc/timestamp] :as cfg}] diff --git a/backend/src/app/media.clj b/backend/src/app/media.clj index c00ac00b24..bdf6ed1c49 100644 --- a/backend/src/app/media.clj +++ b/backend/src/app/media.clj @@ -59,8 +59,8 @@ :hint "seems like the url points to resource with unknown size")) (-> {:size size :mtype mtype} - (validation/validate-media-type!) - (validation/validate-media-size!))))] + (validation/check-media-type) + (validation/check-media-size))))] (let [{:keys [body] :as response} (try diff --git a/backend/src/app/media/validation.clj b/backend/src/app/media/validation.clj index 5da86fb99f..df7e534f54 100644 --- a/backend/src/app/media/validation.clj +++ b/backend/src/app/media/validation.clj @@ -32,8 +32,26 @@ (def check-input (sm/check-fn schema:input)) -(defn validate-media-type! - ([upload] (validate-media-type! upload cm/image-types)) +(def allowed-upload-prefixes + ;; Upload temp files are created server-side by the multipart parser + ;; (yetti `default-temp-dir`, `/tmp/undertow/`) plus the tmp dir used + ;; by tests and storage tmp files (`/tmp/penpot/`). Anything else + ;; (e.g. `/etc/passwd` smuggled via Transit `~#path`) is rejected. + ["/tmp/undertow/" "/tmp/penpot/"]) + +(defn check-upload-path + [upload] + (let [real (try (str (fs/real (:path upload))) + (catch Exception _ nil))] + (when (or (nil? real) + (not (some #(str/starts-with? real %) allowed-upload-prefixes))) + (ex/raise :type :validation + :code :params-validation + :hint "invalid upload path")) + upload)) + +(defn check-media-type + ([upload] (check-media-type upload cm/image-types)) ([upload allowed] (when-not (contains? allowed (:mtype upload)) (ex/raise :type :validation @@ -42,7 +60,7 @@ upload)) -(defn validate-media-size! +(defn check-media-size [upload] (let [max-size (cf/get :media-max-file-size)] (when (> (:size upload) max-size) @@ -53,10 +71,10 @@ max-size))) upload)) -(defn validate-font-size! +(defn check-font-size "Validates that the font file `upload` does not exceed the configured `:font-max-file-size` limit. Accepts the same map shape as - `validate-media-size!` — requires a `:size` key in bytes." + `check-media-size` — requires a `:size` key in bytes." [upload] (let [max-size (cf/get :font-max-file-size)] (when (> (:size upload) max-size) diff --git a/backend/src/app/migrations.clj b/backend/src/app/migrations.clj index 34751179b7..4daef3a0ee 100644 --- a/backend/src/app/migrations.clj +++ b/backend/src/app/migrations.clj @@ -526,7 +526,13 @@ :fn (mg/resource "app/migrations/sql/0155-add-idx-to-file-change.sql")} {:name "0156-reset-pg-storage-defaults" - :fn (mg/resource "app/migrations/sql/0156-reset-pg-storage-defaults.sql")}]) + :fn (mg/resource "app/migrations/sql/0156-reset-pg-storage-defaults.sql")} + + {:name "0157-migrate-legacy-file-data" + :fn (mg/resource "app/migrations/sql/0157-migrate-legacy-file-data.sql")} + + {:name "0158-del-forbidden-file-library-rels" + :fn (mg/resource "app/migrations/sql/0158-del-forbidden-file-library-rels.sql")}]) (defn apply-migrations! [pool name migrations] diff --git a/backend/src/app/migrations/sql/0157-migrate-legacy-file-data.sql b/backend/src/app/migrations/sql/0157-migrate-legacy-file-data.sql new file mode 100644 index 0000000000..c7cd6d1fe4 --- /dev/null +++ b/backend/src/app/migrations/sql/0157-migrate-legacy-file-data.sql @@ -0,0 +1,55 @@ +-- Migrate legacy payloads to file_data: labelled file_change.data as +-- snapshots, file.data as mains (backend 'db', deleted_at copied so the +-- GC keeps collecting them), then delete orphan file_change rows with +-- data but no label (no read path reaches them). +-- +-- Idempotent: INSERTs ignore conflicts, UPDATEs only null the legacy +-- column where the file_data row exists. Big instances pre-migrate by +-- hand (operations/2.19.0-migrate-legacy-file-data.md) and fake this +-- step instead: +-- +-- INSERT INTO migrations (module, step) +-- VALUES ('main', '0157-migrate-legacy-file-data'); + +INSERT INTO file_data (file_id, id, created_at, modified_at, deleted_at, + type, backend, metadata, data) +SELECT file_id, id, created_at, updated_at, deleted_at, + 'snapshot', 'db', NULL, data + FROM file_change + WHERE data IS NOT NULL + AND label IS NOT NULL +ON CONFLICT (file_id, id) DO NOTHING; + +UPDATE file_change AS fc + SET data = NULL + WHERE fc.data IS NOT NULL + AND fc.label IS NOT NULL + AND EXISTS (SELECT 1 + FROM file_data AS fd + WHERE fd.file_id = fc.file_id + AND fd.id = fc.id + AND fd.type = 'snapshot'); + +INSERT INTO file_data (file_id, id, created_at, modified_at, deleted_at, + type, backend, metadata, data) +SELECT id, id, created_at, modified_at, deleted_at, + 'main', 'db', NULL, data + FROM file + WHERE data IS NOT NULL +ON CONFLICT (file_id, id) DO NOTHING; + +UPDATE file AS f + SET data = NULL + WHERE f.data IS NOT NULL + AND EXISTS (SELECT 1 + FROM file_data AS fd + WHERE fd.file_id = f.id + AND fd.id = f.id + AND fd.type = 'main'); + +-- Orphan revisions: legacy payloads without label. No file_data row +-- corresponds to them (snapshots are told apart by label), so there is +-- nothing to copy; the row itself is garbage. Delete it. +DELETE FROM file_change + WHERE data IS NOT NULL + AND label IS NULL; diff --git a/backend/src/app/migrations/sql/0158-del-forbidden-file-library-rels.sql b/backend/src/app/migrations/sql/0158-del-forbidden-file-library-rels.sql new file mode 100644 index 0000000000..d7970d34a6 --- /dev/null +++ b/backend/src/app/migrations/sql/0158-del-forbidden-file-library-rels.sql @@ -0,0 +1,38 @@ +-- Remove file-library relations that violate the same-team invariant: +-- relations across teams and self-relations. Leftover from before the +-- binfile export, the view-only bundle, link-file-to-library and the +-- manifest writers enforced same-team libraries. Relations with both +-- endpoints in the same team are kept. +-- +-- The sync rows go first: the second statement reads file_library_rel +-- while it still contains the forbidden rows. + +WITH forbidden AS ( + SELECT flr.file_id, flr.library_file_id + FROM file_library_rel flr + JOIN file f ON f.id = flr.file_id + JOIN project fp ON fp.id = f.project_id + JOIN file l ON l.id = flr.library_file_id + JOIN project lp ON lp.id = l.project_id + WHERE fp.team_id <> lp.team_id + OR flr.file_id = flr.library_file_id +) +DELETE FROM file_library_sync fls +USING forbidden +WHERE fls.file_id = forbidden.file_id + AND fls.library_file_id = forbidden.library_file_id; + +WITH forbidden AS ( + SELECT flr.file_id, flr.library_file_id + FROM file_library_rel flr + JOIN file f ON f.id = flr.file_id + JOIN project fp ON fp.id = f.project_id + JOIN file l ON l.id = flr.library_file_id + JOIN project lp ON lp.id = l.project_id + WHERE fp.team_id <> lp.team_id + OR flr.file_id = flr.library_file_id +) +DELETE FROM file_library_rel flr +USING forbidden +WHERE flr.file_id = forbidden.file_id + AND flr.library_file_id = forbidden.library_file_id; diff --git a/backend/src/app/rpc/commands/binfile.clj b/backend/src/app/rpc/commands/binfile.clj index 9a17dc2da3..c0781e74c4 100644 --- a/backend/src/app/rpc/commands/binfile.clj +++ b/backend/src/app/rpc/commands/binfile.clj @@ -210,6 +210,8 @@ (assoc params :file file)) params) + _ (media.v/check-upload-path (:file params)) + version (or version (case (bfc/parse-file-format (-> params :file :path)) :binfile-v1 1 diff --git a/backend/src/app/rpc/commands/files.clj b/backend/src/app/rpc/commands/files.clj index ad5ebc3e30..d07a9c1280 100644 --- a/backend/src/app/rpc/commands/files.clj +++ b/backend/src/app/rpc/commands/files.clj @@ -1347,16 +1347,16 @@ WHERE file_id = ANY(?::uuid[])") (def ^:private sql:restore-file-changes - ;; Only durable history comes back: a snapshot (carries `label`) or a legacy - ;; payload row (carries `data`, the column the new model replaces with - ;; `file_data`). Revisions have neither and are deliberately left alone: they - ;; are born already scheduled for deletion one hour later, and the collector - ;; only looks at rows that carry a date, so unmarking one here would keep it - ;; in the table for ever. The `data` half is a guard for the rows still - ;; pending migration, not a dependency: drop it with the last one. + ;; Only durable history comes back: a snapshot, the rows that carry `label`. + ;; Everything else is deliberately left alone. Revisions are born already + ;; scheduled for deletion one hour later, and the collector only looks at + ;; rows that carry a date, so unmarking one here would keep it in the table + ;; for ever. Rows that still carry a legacy payload without a label are + ;; garbage that no read path reaches; 0157-migrate-legacy-file-data deletes + ;; them, so they must keep their date and go with the collector. "UPDATE file_change SET deleted_at = null WHERE file_id = ANY(?::uuid[]) - AND (label IS NOT NULL OR data IS NOT NULL)") + AND label IS NOT NULL") (def ^:private sql:restore-file-data "UPDATE file_data SET deleted_at = null diff --git a/backend/src/app/rpc/commands/files_thumbnails.clj b/backend/src/app/rpc/commands/files_thumbnails.clj index 964e3be645..f837750b21 100644 --- a/backend/src/app/rpc/commands/files_thumbnails.clj +++ b/backend/src/app/rpc/commands/files_thumbnails.clj @@ -290,8 +290,9 @@ ::rpc/multipart-only-params [:media]} [cfg {:keys [::rpc/profile-id file-id object-id media tag]}] - (media.v/validate-media-type! media) - (media.v/validate-media-size! media) + (media.v/check-media-type media) + (media.v/check-media-size media) + (media.v/check-upload-path media) (db/run! cfg files/check-edition-permissions! profile-id file-id) (when-let [file (files/get-minimal-file cfg file-id {::db/check-deleted false})] @@ -392,8 +393,9 @@ ::rpc/multipart-only-params [:media]} [cfg {:keys [::rpc/profile-id file-id] :as params}] - (media.v/validate-media-type! (:media params)) - (media.v/validate-media-size! (:media params)) + (media.v/check-media-type (:media params)) + (media.v/check-media-size (:media params)) + (media.v/check-upload-path (:media params)) (db/run! cfg files/check-edition-permissions! profile-id file-id) diff --git a/backend/src/app/rpc/commands/fonts.clj b/backend/src/app/rpc/commands/fonts.clj index a35f4e7ce7..46aa009e20 100644 --- a/backend/src/app/rpc/commands/fonts.clj +++ b/backend/src/app/rpc/commands/fonts.clj @@ -123,8 +123,8 @@ (fn [acc mtype session-id] (let [assembled (assemble-chunks cfg profile-id session-id)] (-> {:mtype mtype :size (:size assembled)} - (media.v/validate-media-type! cm/font-types) - (media.v/validate-font-size!)) + (media.v/check-media-type cm/font-types) + (media.v/check-font-size)) (assoc acc mtype (:path assembled)))) {} uploads)] diff --git a/backend/src/app/rpc/commands/media.clj b/backend/src/app/rpc/commands/media.clj index 3ba28923b0..8a876460fb 100644 --- a/backend/src/app/rpc/commands/media.clj +++ b/backend/src/app/rpc/commands/media.clj @@ -65,8 +65,9 @@ [:process-image/global]]} [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id file-id content] :as params}] (files/check-edition-permissions! pool profile-id file-id) - (media.v/validate-media-type! content) - (media.v/validate-media-size! content) + (media.v/check-media-type content) + (media.v/check-media-size content) + (media.v/check-upload-path content) (let [team-id (:team-id (db/exec-one! pool [sql:get-team-id-for-file file-id]))] ;; TEMPORARILY DISABLED (2026-10-19): media-storage-bytes-per-team @@ -376,6 +377,7 @@ ::sm/result schema:upload-chunk-result} [{:keys [::db/pool] :as cfg} {:keys [::rpc/profile-id session-id index content] :as _params}] + (media.v/check-upload-path content) (let [session (db/tx-run! cfg check-upload-chunk-slot session-id profile-id index content)] (l/trc :hint "upload-chunk" :session-id session-id @@ -576,8 +578,8 @@ content (-> content (assoc :filename (str "upload:" name)) (assoc :mtype mtype) - (media.v/validate-media-type!) - (media.v/validate-media-size!)) + (media.v/check-media-type) + (media.v/check-media-size)) mobj (create-file-media-object cfg (assoc params :from-chunks? true :content content))] diff --git a/backend/src/app/rpc/commands/profile.clj b/backend/src/app/rpc/commands/profile.clj index 339509b1ed..d0e85a8624 100644 --- a/backend/src/app/rpc/commands/profile.clj +++ b/backend/src/app/rpc/commands/profile.clj @@ -313,8 +313,9 @@ ::sm/result :nil} [cfg {:keys [::rpc/profile-id file] :as params}] ;; Validate incoming mime type - (media.v/validate-media-type! file #{"image/jpeg" "image/png" "image/webp"}) - (media.v/validate-media-size! file) + (media.v/check-media-type file #{"image/jpeg" "image/png" "image/webp"}) + (media.v/check-media-size file) + (media.v/check-upload-path file) (update-profile-photo cfg (assoc params :profile-id profile-id))) (defn update-profile-photo diff --git a/backend/src/app/rpc/commands/teams.clj b/backend/src/app/rpc/commands/teams.clj index 3b1785830a..255111c6f9 100644 --- a/backend/src/app/rpc/commands/teams.clj +++ b/backend/src/app/rpc/commands/teams.clj @@ -1092,8 +1092,9 @@ [cfg {:keys [::rpc/profile-id file] :as params}] ;; Validate incoming mime type - (media.v/validate-media-type! file #{"image/jpeg" "image/png" "image/webp"}) - (media.v/validate-media-size! file) + (media.v/check-media-type file #{"image/jpeg" "image/png" "image/webp"}) + (media.v/check-media-size file) + (media.v/check-upload-path file) (update-team-photo cfg (assoc params :profile-id profile-id))) (defn update-team-photo diff --git a/backend/src/app/rpc/management/exporter.clj b/backend/src/app/rpc/management/exporter.clj index 332fe23984..007aa39fb9 100644 --- a/backend/src/app/rpc/management/exporter.clj +++ b/backend/src/app/rpc/management/exporter.clj @@ -34,7 +34,8 @@ ::rpc/multipart-only-params [:content] ::sm/result schema:upload-tempfile-result} [cfg {:keys [::rpc/profile-id content]}] - (media.v/validate-media-type! content cm/tempfile-types) + (media.v/check-media-type content cm/tempfile-types) + (media.v/check-upload-path content) (let [storage (sto/resolve cfg) hash (sto/calculate-hash (:path content)) data (-> (sto/content (:path content)) diff --git a/backend/src/app/rpc/management/nitrate.clj b/backend/src/app/rpc/management/nitrate.clj index bb97a66f37..8c5d1bb658 100644 --- a/backend/src/app/rpc/management/nitrate.clj +++ b/backend/src/app/rpc/management/nitrate.clj @@ -139,7 +139,8 @@ ::sm/result schema:upload-organization-logo-result ::nitrate/sso false} [{:keys [::sto/storage]} {:keys [content organization-id previous-id]}] - (media.v/validate-media-type! content cm/image-types) + (media.v/check-media-type content cm/image-types) + (media.v/check-upload-path content) (when previous-id (sto/touch-object! storage previous-id)) (let [hash (sto/calculate-hash (:path content)) diff --git a/backend/src/app/srepl/procs/fdata_storage.clj b/backend/src/app/srepl/procs/fdata_storage.clj deleted file mode 100644 index 5759aa43b9..0000000000 --- a/backend/src/app/srepl/procs/fdata_storage.clj +++ /dev/null @@ -1,141 +0,0 @@ -;; This Source Code Form is subject to the terms of the Mozilla Public -;; License, v. 2.0. If a copy of the MPL was not distributed with this -;; file, You can obtain one at http://mozilla.org/MPL/2.0/. -;; -;; Copyright (c) KALEIDOS SUBSIDIARY SL - -(ns app.srepl.procs.fdata-storage - (:require - [app.common.logging :as l] - [app.db :as db])) - -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; SNAPSHOTS -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; - -(def sql:get-unmigrated-snapshots - "SELECT fc.id, fc.file_id - FROM file_change AS fc - WHERE fc.data IS NOT NULL - AND fc.label IS NOT NULL - ORDER BY fc.id ASC") - -(def sql:get-migrated-snapshots - "SELECT f.id, f.file_id - FROM file_data AS f - WHERE f.data IS NOT NULL - AND f.type = 'snapshot' - AND f.id != f.file_id - ORDER BY f.id ASC") - -(defn migrate-snapshot-to-storage - "Migrate the current existing files to store data in new storage - tables." - {:query sql:get-unmigrated-snapshots} - [{:keys [::db/conn]} {:keys [id file-id]} & {:as options}] - (let [{:keys [id file-id data created-at updated-at]} - (db/get* conn :file-change {:id id :file-id file-id} - ::db/for-update true - ::db/remove-deleted false)] - (when data - (l/inf :hint "migrating snapshot" :file-id (str file-id) :id (str id)) - (db/update! conn :file-change - {:data nil} - {:id id :file-id file-id} - {::db/return-keys false}) - (db/insert! conn :file-data - {:backend "db" - :metadata nil - :type "snapshot" - :data data - :created-at created-at - :modified-at updated-at - :file-id file-id - :id id} - {::db/return-keys false})))) - -(defn rollback-snapshot-from-storage - "Migrate back to the file table storage." - {:query sql:get-unmigrated-snapshots} - [{:keys [::db/conn]} {:keys [id file-id]} & {:as opts}] - (when-let [{:keys [id file-id data]} - (db/get* conn :file-data {:id id :file-id file-id :type "snapshot"} - ::db/for-update true - ::db/remove-deleted false)] - (l/inf :hint "rollback snapshot" :file-id (str file-id) :id (str id)) - (db/update! conn :file-change - {:data data} - {:id id :file-id file-id} - {::db/return-keys false}) - (db/delete! conn :file-data - {:id id :file-id file-id :type "snapshot"} - {::db/return-keys false}))) - -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; FILES -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; - -(def sql:get-unmigrated-files - "SELECT f.id - FROM file AS f - WHERE f.data IS NOT NULL - ORDER BY f.modified_at ASC") - -(def sql:get-migrated-files - "SELECT f.id, f.file_id - FROM file_data AS f - WHERE f.data IS NOT NULL - AND f.id = f.file_id - ORDER BY f.id ASC") - -(defn migrate-file-to-storage - "Migrate the current existing files to store data in new storage - tables." - {:query sql:get-unmigrated-files} - [{:keys [::db/conn] :as cfg} {:keys [id]} & {:as opts}] - (let [{:keys [id data created-at modified-at]} - (db/get* conn :file {:id id} - ::db/for-update true - ::db/remove-deleted false)] - - (when data - (l/inf :hint "migrating file" :file-id (str id)) - - (db/update! conn :file {:data nil} {:id id} ::db/return-keys false) - (db/insert! conn :file-data - {:backend "db" - :metadata nil - :type "main" - :data data - :created-at created-at - :modified-at modified-at - :file-id id - :id id} - {::db/return-keys false})) - - (let [snapshots-sql - (str "WITH snapshots AS (" sql:get-unmigrated-snapshots ") " - "SELECT s.* FROM snapshots AS s WHERE s.file_id = ?")] - (run! (fn [params] - (migrate-snapshot-to-storage cfg params opts)) - (db/plan cfg [snapshots-sql id]))))) - - -(defn rollback-file-from-storage - "Migrate back to the file table storage." - {:query sql:get-migrated-files} - [{:keys [::db/conn] :as cfg} {:keys [id]} & {:as opts}] - (when-let [{:keys [id data]} - (db/get* conn :file-data {:id id :file-id id :type "main"} - ::db/for-update true - ::db/remove-deleted false)] - (l/inf :hint "rollback file" :file-id (str id)) - (db/update! conn :file {:data data} {:id id} ::db/return-keys false) - (db/delete! conn :file-data {:file-id id :id id :type "main"} ::db/return-keys false) - - (let [snapshots-sql - (str "WITH snapshots AS (" sql:get-migrated-snapshots ") " - "SELECT s.* FROM snapshots AS s WHERE s.file_id = ?")] - (run! (fn [params] - (rollback-snapshot-from-storage cfg params opts)) - (db/plan cfg [snapshots-sql id]))))) diff --git a/backend/test/backend_tests/binfile_svg_test.clj b/backend/test/backend_tests/binfile_svg_test.clj index 446abca995..9d967c827d 100644 --- a/backend/test/backend_tests/binfile_svg_test.clj +++ b/backend/test/backend_tests/binfile_svg_test.clj @@ -146,7 +146,7 @@ (defn- write-temp-svg "Write `text` to a tempfile and return `[path size]`." [text] - (let [path (fs/create-tempfile :prefix "penpot-svg-import-test-" :suffix ".svg") + (let [path (fs/create-tempfile :dir "/tmp/penpot" :prefix "penpot-svg-import-test-" :suffix ".svg") data (utf8bytes text)] (spit (str path) text :encoding "UTF-8") [path (alength data)])) diff --git a/backend/test/backend_tests/binfile_test.clj b/backend/test/backend_tests/binfile_test.clj index 082ef66ba2..78124f0ad7 100644 --- a/backend/test/backend_tests/binfile_test.clj +++ b/backend/test/backend_tests/binfile_test.clj @@ -30,6 +30,7 @@ [app.storage.tmp :as tmp] [backend-tests.helpers :as th] [backend-tests.storage-test :as stt] + [clojure.data.json :as json] [clojure.java.io :as jio] [clojure.test :as t] [cuerdas.core :as str] @@ -37,6 +38,9 @@ [datoteka.io :as io]) (:import java.io.ByteArrayInputStream + java.io.ByteArrayOutputStream + java.io.DataInputStream + java.io.DataOutputStream java.io.OutputStreamWriter java.io.Writer java.util.zip.Deflater @@ -216,6 +220,63 @@ :token-radius token-radius :token-opacity token-opacity})) +(defn- export-file + [file & {:keys [include-libraries]}] + (let [output (tmp/tempfile :suffix ".zip")] + (v3/export-files! + (cond-> (-> th/*system* + (assoc ::bfc/ids #{(:id file)})) + include-libraries + (assoc ::bfc/export-type :include-libraries)) + (io/output-stream output)) + output)) + +(defn- import-bundle + [profile path] + (-> th/*system* + (assoc ::bfc/project-id (:default-project-id profile)) + (assoc ::bfc/team-id (:default-team-id profile)) + (assoc ::bfc/profile-id (:id profile)) + (assoc ::bfc/input path) + (v3/import-files!) + (:file-ids))) + +(defn- read-manifest + [path] + (with-open [zip (ZipFile. (.toFile ^java.nio.file.Path path))] + (let [entry (.getEntry zip "manifest.json")] + (with-open [input (.getInputStream zip entry)] + (json/read-str (slurp input :encoding "UTF-8")))))) + +(defn- tamper-manifest-relations + [src dst relations] + (with-open [zin (ZipFile. (.toFile ^java.nio.file.Path src)) + zout (ZipOutputStream. (io/output-stream dst))] + (doseq [entry (enumeration-seq (.entries zin))] + (let [name (.getName ^ZipEntry entry)] + (if (= name "manifest.json") + (let [manifest (-> (slurp (.getInputStream zin entry) :encoding "UTF-8") + (json/read-str) + (assoc "relations" relations))] + (.putNextEntry zout (ZipEntry. name)) + (.write zout (.getBytes (json/write-str manifest) "UTF-8")) + (.closeEntry zout)) + (do + (.putNextEntry zout (ZipEntry. name)) + (with-open [input (.getInputStream zin entry)] + (io/copy input zout)) + (.closeEntry zout))))))) + +(defn- prepare-victim-file + [n] + (let [profile (th/create-profile* n) + team (th/create-team* n {:profile-id (:id profile)}) + project (th/create-project* n {:team-id (:id team) + :profile-id (:id profile)}) + file (th/create-file* n {:profile-id (:id profile) + :project-id (:id project)})] + [profile file])) + (def ^:private svg-raw-page-id (uuid/custom 1 1)) (def ^:private svg-raw-root-id (uuid/custom 3 1)) (def ^:private svg-raw-child-id (uuid/custom 3 2)) @@ -2898,3 +2959,181 @@ (t/is (= (str (first result) "/" thumb-page-id "/" thumb-frame-id "/" thumb-tag) (:object-id thumb))) (t/is (some? (:media-id thumb))))))) + +;; --- Library relations security tests + +(t/deftest import-v3-ignores-relations-outside-package + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + [_ victim] (prepare-victim-file 3) + bundle (export-file file) + evil (tmp/tempfile :suffix ".zip") + _ (tamper-manifest-relations bundle evil + [[(str (:id file)) (str (:id victim))]]) + [file-id :as result] (import-bundle profile evil)] + (t/is (= 1 (count result))) + (t/is (empty? (th/db-query :file-library-rel {:file-id file-id}))) + (t/is (empty? (th/db-query :file-library-sync {:file-id file-id}))))) + +(defn- prepare-other-team + [n] + (let [profile (th/create-profile* n) + team (th/create-team* n {:profile-id (:id profile)}) + project (th/create-project* n {:team-id (:id team) + :profile-id (:id profile)}) + file (th/create-file* n {:profile-id (:id profile) + :project-id (:id project)}) + library (th/create-file* (inc n) {:profile-id (:id profile) + :project-id (:id project)})] + [profile file library])) + +(t/deftest import-v3-ignores-foreign-file-id + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + [_ victim] (prepare-victim-file 3) + bundle (export-file file) + evil (tmp/tempfile :suffix ".zip") + _ (tamper-manifest-relations bundle evil + [[(str (:id victim)) (str (:id file))]]) + result (import-bundle profile evil)] + (t/is (= 1 (count result))) + (t/is (empty? (th/db-query :file-library-rel {:file-id (:id victim)}))) + (t/is (empty? (th/db-query :file-library-sync {:file-id (:id victim)}))))) + +(t/deftest import-v3-ignores-unknown-library + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + bundle (export-file file) + evil (tmp/tempfile :suffix ".zip") + _ (tamper-manifest-relations bundle evil + [[(str (:id file)) (str (uuid/random))]]) + [file-id :as result] (import-bundle profile evil)] + (t/is (= 1 (count result))) + (t/is (empty? (th/db-query :file-library-rel {:file-id file-id}))))) + +(t/deftest import-v3-ignores-self-library-relation + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + bundle (export-file file) + evil (tmp/tempfile :suffix ".zip") + _ (tamper-manifest-relations bundle evil + [[(str (:id file)) (str (:id file))]]) + [file-id :as result] (import-bundle profile evil)] + (t/is (= 1 (count result))) + (t/is (empty? (th/db-query :file-library-rel {:file-id file-id}))))) + +(t/deftest import-v3-keeps-valid-library-relation + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + library (th/create-file* 2 {:profile-id (:id profile) + :project-id (:default-project-id profile)})] + (th/link-file-to-library* {:file-id (:id file) + :library-id (:id library)}) + (let [bundle (export-file file :include-libraries true) + result (import-bundle profile bundle) + new-file (->> result + (map #(bfc/get-file th/*system* %)) + (some #(when (= (:name %) (:name file)) %)))] + (t/is (= 2 (count result))) + (t/is (some? new-file)) + (t/is (= 1 (count (th/db-query :file-library-rel + {:file-id (:id new-file)}))))))) + +(t/deftest export-multi-file-includes-package-relations + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + library (th/create-file* 2 {:profile-id (:id profile) + :project-id (:default-project-id profile)}) + [_ other-file other-library] (prepare-other-team 3) + _ (th/link-file-to-library* {:file-id (:id file) + :library-id (:id library)}) + _ (th/link-file-to-library* {:file-id (:id other-file) + :library-id (:id other-library)}) + _ (th/db-insert! :file-library-rel + {:file-id (:id file) + :library-file-id (:id other-library)}) + output (tmp/tempfile :suffix ".zip")] + (v3/export-files! + (-> th/*system* + (assoc ::bfc/ids #{(:id file) (:id other-file)}) + (assoc ::bfc/export-type :include-libraries)) + (io/output-stream output)) + (let [rels (set (map vec (get (read-manifest output) "relations")))] + (t/is (contains? rels [(str (:id file)) (str (:id library))])) + (t/is (contains? rels [(str (:id other-file)) (str (:id other-library))])) + (t/is (contains? rels [(str (:id file)) (str (:id other-library))]))))) + +;; --- Forbidden library relations cleanup (migration candidate) +;; +;; The two statements below are the exact cleanup proposed for a future +;; migration (see tmp/CHECK-LIBRARY-RELATIONS.sql, section 6). The sync +;; rows must go first: the second statement reads file_library_rel +;; while it still contains the forbidden rows. + +(def ^:private sql:delete-forbidden-library-sync + "WITH forbidden AS ( + SELECT flr.file_id, flr.library_file_id + FROM file_library_rel flr + JOIN file f ON f.id = flr.file_id + JOIN project fp ON fp.id = f.project_id + JOIN file l ON l.id = flr.library_file_id + JOIN project lp ON lp.id = l.project_id + WHERE fp.team_id <> lp.team_id + OR flr.file_id = flr.library_file_id + ) + DELETE FROM file_library_sync fls + USING forbidden + WHERE fls.file_id = forbidden.file_id + AND fls.library_file_id = forbidden.library_file_id") + +(def ^:private sql:delete-forbidden-library-rel + "WITH forbidden AS ( + SELECT flr.file_id, flr.library_file_id + FROM file_library_rel flr + JOIN file f ON f.id = flr.file_id + JOIN project fp ON fp.id = f.project_id + JOIN file l ON l.id = flr.library_file_id + JOIN project lp ON lp.id = l.project_id + WHERE fp.team_id <> lp.team_id + OR flr.file_id = flr.library_file_id + ) + DELETE FROM file_library_rel flr + USING forbidden + WHERE flr.file_id = forbidden.file_id + AND flr.library_file_id = forbidden.library_file_id") + +(t/deftest cleanup-deletes-forbidden-library-relations + (let [profile (th/create-profile* 1) + file (prepare-simple-file profile) + library (th/create-file* 2 {:profile-id (:id profile) + :project-id (:default-project-id profile)}) + [_ victim] (prepare-victim-file 3) + _ (th/link-file-to-library* {:file-id (:id file) + :library-id (:id library)}) + _ (th/db-insert! :file-library-rel + {:file-id (:id file) + :library-file-id (:id victim)}) + _ (th/db-insert! :file-library-sync + {:file-id (:id file) + :library-file-id (:id victim)}) + _ (th/db-insert! :file-library-rel + {:file-id (:id file) + :library-file-id (:id file)}) + _ (th/db-insert! :file-library-sync + {:file-id (:id file) + :library-file-id (:id file)})] + (t/is (= 3 (count (th/db-query :file-library-rel {:file-id (:id file)})))) + (th/db-exec! [sql:delete-forbidden-library-sync]) + (th/db-exec! [sql:delete-forbidden-library-rel]) + (t/testing "same-team relation and its sync row survive" + (t/is (= [{:file-id (:id file) :library-file-id (:id library)}] + (map #(select-keys % [:file-id :library-file-id]) + (th/db-query :file-library-rel {:file-id (:id file)})))) + (t/is (= [{:file-id (:id file) :library-file-id (:id library)}] + (map #(select-keys % [:file-id :library-file-id]) + (th/db-query :file-library-sync {:file-id (:id file)}))))) + (t/testing "cross-team and self relations and their sync rows are gone" + (t/is (empty? (th/db-query :file-library-rel {:library-file-id (:id victim)}))) + (t/is (empty? (th/db-query :file-library-sync {:library-file-id (:id victim)}))) + (t/is (empty? (th/db-query :file-library-rel {:library-file-id (:id file)}))) + (t/is (empty? (th/db-query :file-library-sync {:library-file-id (:id file)})))))) diff --git a/backend/test/backend_tests/file_change_retention_test.clj b/backend/test/backend_tests/file_change_retention_test.clj index 7cede5a164..3929e0b129 100644 --- a/backend/test/backend_tests/file_change_retention_test.clj +++ b/backend/test/backend_tests/file_change_retention_test.clj @@ -118,8 +118,8 @@ (t/testing "the snapshot comes back" (t/is (nil? (:deleted-at (find-change snapshot-id))))) - (t/testing "the legacy payload row comes back" - (t/is (nil? (:deleted-at (find-change legacy-id)))))))) + (t/testing "the legacy payload row keeps its date" + (t/is (= trash-at (:deleted-at (find-change legacy-id)))))))) (t/deftest deleting-a-file-does-not-push-the-revision-schedule-out (binding [ct/*clock* (ct/fixed-clock test-now)] diff --git a/backend/test/backend_tests/helpers.clj b/backend/test/backend_tests/helpers.clj index 58b36244ea..5148916464 100644 --- a/backend/test/backend_tests/helpers.clj +++ b/backend/test/backend_tests/helpers.clj @@ -86,7 +86,8 @@ :enable-quotes :enable-rpc-climit :enable-auto-file-snapshot - :disable-file-validation]) + :disable-file-validation + :disable-admin-console]) (defn init-config ([next] diff --git a/backend/test/backend_tests/media_validation_test.clj b/backend/test/backend_tests/media_validation_test.clj new file mode 100644 index 0000000000..f4431e1b71 --- /dev/null +++ b/backend/test/backend_tests/media_validation_test.clj @@ -0,0 +1,68 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS INC Sucursal en España SL + +(ns backend-tests.media-validation-test + (:require + [app.media.validation :as media.v] + [backend-tests.helpers :as th] + [clojure.test :as t] + [datoteka.fs :as fs])) + +(t/deftest check-media-type + (t/testing "allowed mtype passes through" + (let [upload {:filename "a.png" :size 1 :path (fs/path "/tmp/penpot/a.png") :mtype "image/png"}] + (t/is (= upload (media.v/check-media-type upload))))) + (t/testing "disallowed mtype is rejected" + (let [upload {:filename "a.txt" :size 1 :path (fs/path "/tmp/penpot/a.txt") :mtype "text/plain"} + error (try (media.v/check-media-type upload) nil + (catch clojure.lang.ExceptionInfo e e))] + (t/is (some? error)) + (t/is (th/ex-of-type? error :validation)) + (t/is (th/ex-of-code? error :media-type-not-allowed))))) + +(t/deftest check-media-size + (t/testing "small file passes through" + (let [upload {:filename "a.png" :size 1 :path (fs/path "/tmp/penpot/a.png") :mtype "image/png"}] + (t/is (= upload (media.v/check-media-size upload))))) + (t/testing "oversized file is rejected" + (let [upload {:filename "a.png" :size (* 1024 1024 31) :path (fs/path "/tmp/penpot/a.png") :mtype "image/png"} + error (try (media.v/check-media-size upload) nil + (catch clojure.lang.ExceptionInfo e e))] + (t/is (some? error)) + (t/is (th/ex-of-type? error :restriction)) + (t/is (th/ex-of-code? error :media-max-file-size-reached))))) + +(t/deftest check-font-size + (t/testing "small font passes through" + (let [upload {:filename "a.ttf" :size 1 :path (fs/path "/tmp/penpot/a.ttf") :mtype "font/ttf"}] + (t/is (= upload (media.v/check-font-size upload))))) + (t/testing "oversized font is rejected" + (let [upload {:filename "a.ttf" :size (* 1024 1024 31) :path (fs/path "/tmp/penpot/a.ttf") :mtype "font/ttf"} + error (try (media.v/check-font-size upload) nil + (catch clojure.lang.ExceptionInfo e e))] + (t/is (some? error)) + (t/is (th/ex-of-type? error :restriction)) + (t/is (th/ex-of-code? error :font-max-file-size-reached))))) + +(t/deftest check-upload-path + (t/testing "tempfile under /tmp/penpot passes through" + (let [path (th/tempfile "backend_tests/test_files/sample.jpg") + upload {:filename "sample.jpg" :size 7923 :path path :mtype "image/jpeg"}] + (t/is (= upload (media.v/check-upload-path upload))))) + (t/testing "system file outside temp is rejected" + (let [upload {:filename "x.png" :size 1 :path (fs/path "/etc/passwd") :mtype "image/png"} + error (try (media.v/check-upload-path upload) nil + (catch clojure.lang.ExceptionInfo e e))] + (t/is (some? error)) + (t/is (th/ex-of-type? error :validation)) + (t/is (th/ex-of-code? error :params-validation)))) + (t/testing "missing file is rejected" + (let [upload {:filename "x.png" :size 1 :path (fs/path "/tmp/penpot/no-such-file-12345.png") :mtype "image/png"} + error (try (media.v/check-upload-path upload) nil + (catch clojure.lang.ExceptionInfo e e))] + (t/is (some? error)) + (t/is (th/ex-of-type? error :validation)) + (t/is (th/ex-of-code? error :params-validation))))) diff --git a/backend/test/backend_tests/migrations_legacy_file_data_test.clj b/backend/test/backend_tests/migrations_legacy_file_data_test.clj new file mode 100644 index 0000000000..e2a31ce2bf --- /dev/null +++ b/backend/test/backend_tests/migrations_legacy_file_data_test.clj @@ -0,0 +1,180 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns backend-tests.migrations-legacy-file-data-test + "Covers the 0157 data migration: legacy file/file_change payloads move + to file_data, keeping timestamps and deleted_at, nulling the legacy + column; unlabelled file_change payloads (orphan revisions) are + deleted." + (:require + [app.common.time :as ct] + [app.common.uuid :as uuid] + [app.db :as db] + [app.util.blob :as blob] + [backend-tests.helpers :as th] + [clojure.java.io :as io] + [clojure.test :as t] + [next.jdbc :as jdbc])) + +(t/use-fixtures :once th/state-init) +(t/use-fixtures :each th/database-reset) + +(defn- run-migration + [] + (let [sql (slurp (io/resource "app/migrations/sql/0157-migrate-legacy-file-data.sql"))] + (jdbc/execute! th/*pool* [sql]))) + +(defn- get-file-data + [file-id id] + (th/db-get :file-data {:file-id file-id :id id} {::db/remove-deleted false})) + +(t/deftest migrates-legacy-file-and-snapshot + (let [profile (th/create-profile* 1) + file (th/create-file* 1 {:profile-id (:id profile) + :project-id (:default-project-id profile)}) + fid (:id file) + + ;; Simulate an unmigrated main: move the payload back to + ;; file.data and drop the file_data row. + main-data (:data (get-file-data fid fid)) + _ (t/is (some? main-data) "expected file_data main row from create-file*") + file-row (th/db-get :file {:id fid} {::db/remove-deleted false}) + _ (th/db-delete! :file-data {:file-id fid :id fid}) + _ (th/db-update! :file {:data main-data} {:id fid}) + + ;; Simulate an unmigrated snapshot (labelled) with its own + ;; timestamps and a deletion date. + snap-id (uuid/next) + snap-data (blob/encode {:snapshot 1}) + created-at (ct/in-past "2h") + updated-at (ct/in-past "1h") + deleted-at (ct/in-past "30m") + _ (th/db-insert! :file-change + {:id snap-id + :file-id fid + :revn 1 + :data snap-data + :label "v1" + :created-at created-at + :updated-at updated-at + :deleted-at deleted-at}) + + ;; Orphan revision (data, no label): deleted by the migration. + rev-id (uuid/next) + rev-data (blob/encode {:rev 1}) + _ (th/db-insert! :file-change + {:id rev-id + :file-id fid + :revn 2 + :data rev-data}) + + ;; Live revision (changes, no data, no label): untouched. + live-id (uuid/next) + _ (th/db-insert! :file-change + {:id live-id + :file-id fid + :revn 3 + :changes (blob/encode [{:type :dummy}])})] + + (run-migration) + + (t/testing "main payload moves to file_data as type main" + (let [fd (get-file-data fid fid)] + (t/is (some? fd)) + (t/is (= "main" (:type fd))) + (t/is (= "db" (:backend fd))) + (t/is (nil? (:metadata fd))) + (t/is (= (seq ^bytes main-data) (seq ^bytes (:data fd)))) + (t/is (= (:created-at file-row) (:created-at fd))) + (t/is (= (:modified-at file-row) (:modified-at fd))) + (t/is (= (:deleted-at file-row) (:deleted-at fd))))) + + (t/testing "snapshot payload moves to file_data as type snapshot" + (let [fd (get-file-data fid snap-id)] + (t/is (some? fd)) + (t/is (= "snapshot" (:type fd))) + (t/is (= "db" (:backend fd))) + (t/is (nil? (:metadata fd))) + (t/is (= (seq ^bytes snap-data) (seq ^bytes (:data fd)))) + ;; NOTE: timestamptz stores microsecond precision while the + ;; test instants carry nanoseconds, so compare at millis. + (t/is (= (inst-ms created-at) (inst-ms (:created-at fd)))) + (t/is (= (inst-ms updated-at) (inst-ms (:modified-at fd)))) + (t/is (= (inst-ms deleted-at) (inst-ms (:deleted-at fd)))))) + + (t/testing "legacy columns are nulled" + (t/is (nil? (:data (th/db-get :file {:id fid} {::db/remove-deleted false})))) + (t/is (nil? (:data (th/db-get :file-change {:id snap-id} {::db/remove-deleted false}))))) + + (t/testing "unlabelled payload row is deleted, live revision stays" + (t/is (nil? (th/db-get :file-change {:id rev-id} {::db/remove-deleted false}))) + (let [row (th/db-get :file-change {:id live-id} {::db/remove-deleted false})] + (t/is (some? row)) + (t/is (some? (:changes row))))) + + (t/testing "second run is a no-op" + (run-migration) + (t/is (some? (get-file-data fid fid))) + (t/is (some? (get-file-data fid snap-id))) + (t/is (nil? (:data (th/db-get :file {:id fid} {::db/remove-deleted false})))) + (t/is (nil? (:data (th/db-get :file-change {:id snap-id} {::db/remove-deleted false})))) + (t/is (nil? (th/db-get :file-change {:id rev-id} {::db/remove-deleted false})))))) + +(t/deftest keeps-preexisting-file-data-row + (let [profile (th/create-profile* 1) + file (th/create-file* 1 {:profile-id (:id profile) + :project-id (:default-project-id profile)}) + fid (:id file) + + ;; file_data already holds the current payload; the legacy + ;; column keeps a stale copy that must not overwrite it. + current (:data (get-file-data fid fid)) + stale (blob/encode {:stale true}) + _ (t/is (some? current)) + _ (t/is (not= (seq ^bytes current) (seq ^bytes stale))) + _ (th/db-update! :file {:data stale} {:id fid})] + + (run-migration) + + (t/testing "file_data keeps its data and legacy is nulled" + (let [fd (get-file-data fid fid)] + (t/is (= (seq ^bytes current) (seq ^bytes (:data fd))))) + (t/is (nil? (:data (th/db-get :file {:id fid} {::db/remove-deleted false}))))))) + +(t/deftest keeps-preexisting-snapshot-row + (let [profile (th/create-profile* 1) + file (th/create-file* 1 {:profile-id (:id profile) + :project-id (:default-project-id profile)}) + fid (:id file) + + ;; file_data already holds the snapshot payload; the legacy column + ;; keeps a stale copy that must not overwrite it. + snap-id (uuid/next) + current (blob/encode {:snapshot :current}) + stale (blob/encode {:snapshot :stale}) + _ (t/is (not= (seq ^bytes current) (seq ^bytes stale))) + _ (th/db-insert! :file-data + {:file-id fid + :id snap-id + :type "snapshot" + :backend "db" + :metadata nil + :data current}) + _ (th/db-insert! :file-change + {:id snap-id + :file-id fid + :revn 1 + :label "v1" + :data stale})] + + (run-migration) + + (t/testing "file_data keeps its data and legacy is nulled" + (let [fd (get-file-data fid snap-id)] + (t/is (= (seq ^bytes current) (seq ^bytes (:data fd))))) + (t/is (nil? (:data (th/db-get :file-change {:id snap-id} + {::db/remove-deleted false}))))))) + diff --git a/backend/test/backend_tests/rpc_auth_test.clj b/backend/test/backend_tests/rpc_auth_test.clj index 802c70ac08..dcc8ffdcbf 100644 --- a/backend/test/backend_tests/rpc_auth_test.clj +++ b/backend/test/backend_tests/rpc_auth_test.clj @@ -122,7 +122,7 @@ (.close ^AutoCloseable conn))))) (t/deftest login-with-password-locks-after-max-attempts - (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout])] + (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :disable-admin-console])] (let [profile (th/create-profile* 200 {:is-active true})] (cleanup-redis (:id profile)) ;; 4 failed attempts — not yet locked @@ -141,7 +141,7 @@ (t/is (th/ex-of-code? (:error out) :account-locked)))))) (t/deftest login-with-password-returns-rate-limit-when-locked - (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout])] + (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :disable-admin-console])] (let [profile (th/create-profile* 201 {:is-active true})] (cleanup-redis (:id profile)) ;; Trigger lockout @@ -161,7 +161,7 @@ (t/is (pos? ttl) "ttl should be a positive number")))))) (t/deftest successful-login-clears-failed-attempts - (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout])] + (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :disable-admin-console])] (let [profile (th/create-profile* 202 {:is-active true})] (cleanup-redis (:id profile)) ;; 3 failed attempts @@ -184,7 +184,7 @@ (t/is (th/ex-of-code? (:error out) :wrong-credentials))))))) (t/deftest lockout-does-not-apply-when-flag-disabled - (with-redefs [app.config/flags (flags/parse flags/default [])] + (with-redefs [app.config/flags (flags/parse flags/default [:disable-admin-console])] (let [profile (th/create-profile* 203 {:is-active true})] (cleanup-redis (:id profile)) ;; 10 failed attempts — no lockout without the flag @@ -199,7 +199,7 @@ "Verify that login-with-ldap short-circuits when account is locked, without attempting LDAP authentication. Calls the actual command function with a mock provider to test the full flow." - (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :login-with-ldap]) + (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :login-with-ldap :disable-admin-console]) ldap/authenticate (constantly nil)] (let [profile (th/create-profile* 204 {:is-active true}) cfg (assoc th/*system* ::ldap/provider {})] @@ -217,7 +217,7 @@ (t/deftest recover-profile-clears-lockout "Verify that clearing attempts (e.g. after password reset) unlocks the account." - (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout])] + (with-redefs [app.config/flags (flags/parse flags/default [:enable-account-lockout :disable-admin-console])] (let [profile (th/create-profile* 205 {:is-active true}) cfg th/*system*] (cleanup-redis (:id profile)) diff --git a/backend/test/backend_tests/rpc_file_thumbnails_test.clj b/backend/test/backend_tests/rpc_file_thumbnails_test.clj index 49345f4ba9..357ebfa60c 100644 --- a/backend/test/backend_tests/rpc_file_thumbnails_test.clj +++ b/backend/test/backend_tests/rpc_file_thumbnails_test.clj @@ -915,39 +915,44 @@ (t/is (= 1 (count rows2))) (t/is (nil? (:deleted-at (first rows2))))))) - -(t/deftest create-file-thumbnail-rejects-forged-media - ;; A transit body like {"~:media": {"~:path": ["~#path", "/etc/passwd"]}} - ;; decodes to this exact map. Without a multipart request it must fail at - ;; params validation, before any file is read. NOTE: plain command! on - ;; purpose, a multipart header would make it legitimate. +(t/deftest rejects-forged-path-on-create-file-thumbnail (let [profile (th/create-profile* 1) file (th/create-file* 1 {:profile-id (:id profile) :project-id (:default-project-id profile) :is-shared false}) - out (th/command! {::th/type :create-file-thumbnail - ::rpc/profile-id (:id profile) - :file-id (:id file) - :revn 0 - :media {:filename "evil.jpg" - :size 123 - :path (fs/path "/etc/passwd") - :mtype "image/jpeg"}})] - (t/is (th/ex-of-code? (:error out) :params-validation)))) + data {::th/type :create-file-thumbnail + ::rpc/profile-id (:id profile) + :file-id (:id file) + :revn 1 + :media {:filename "x.png" + :size 1 + :path (fs/path "/etc/passwd") + :mtype "image/png"}} + out (th/command! data) + error (:error out)] + (t/is (nil? (:result out))) + (t/is (th/ex-info? error)) + (t/is (th/ex-of-type? error :validation)) + (t/is (th/ex-of-code? error :params-validation)) + (t/is (= 0 (count (th/db-query :file-thumbnail {:file-id (:id file)})))))) -(t/deftest create-file-object-thumbnail-rejects-forged-media - ;; Same transit-forgery vector through object thumbnails. NOTE: plain - ;; command! on purpose, a multipart header would make it legitimate. +(t/deftest rejects-forged-path-on-create-file-object-thumbnail (let [profile (th/create-profile* 1) file (th/create-file* 1 {:profile-id (:id profile) :project-id (:default-project-id profile) :is-shared false}) - out (th/command! {::th/type :create-file-object-thumbnail - ::rpc/profile-id (:id profile) - :file-id (:id file) - :object-id "obj" - :media {:filename "evil.jpg" - :size 123 - :path (fs/path "/etc/passwd") - :mtype "image/jpeg"}})] - (t/is (th/ex-of-code? (:error out) :params-validation)))) + data {::th/type :create-file-object-thumbnail + ::rpc/profile-id (:id profile) + :file-id (:id file) + :object-id "test-key-forged" + :media {:filename "x.png" + :size 1 + :path (fs/path "/etc/passwd") + :mtype "image/png"}} + out (th/command! data) + error (:error out)] + (t/is (nil? (:result out))) + (t/is (th/ex-info? error)) + (t/is (th/ex-of-type? error :validation)) + (t/is (th/ex-of-code? error :params-validation)) + (t/is (= 0 (count (th/db-query :file-tagged-object-thumbnail {:file-id (:id file)})))))) diff --git a/common/src/app/common/files/migrations.cljc b/common/src/app/common/files/migrations.cljc index de82f16f9e..6a82e463ef 100644 --- a/common/src/app/common/files/migrations.cljc +++ b/common/src/app/common/files/migrations.cljc @@ -2296,6 +2296,45 @@ (update :pages-index d/update-vals update-container) (d/update-when :components d/update-vals update-container)))) +(defmethod migrate-data "0034-fix-path-content-offset" + [data _] + ;; A path stores its geometry twice: the box (:selrect and :points) and the + ;; drawing (:content), and the box is always derived from the drawing. Some + ;; files have component copies whose drawing and box were moved apart, so + ;; the drawing renders away from where the sidebar, the layouts and the + ;; syncs place the shape. We move the content back onto the box. A path whose drawing + ;; has another size than its box is not a displaced drawing, and is left as + ;; it is. + (let [tolerance 0.1] + (letfn [(fix-path [shape] + (when (and (cfh/path-shape? shape) + (some? (:selrect shape)) + (some? (path.segment/content->selrect (:content shape)))) + (let [selrect (:selrect shape) + derived (:selrect (path/update-geometry shape)) + offset (gpt/subtract (grc/rect->center selrect) + (grc/rect->center derived))] + (when (and (mth/close? (:width selrect) (:width derived) tolerance) + (mth/close? (:height selrect) (:height derived) tolerance) + (or (> (abs (:x offset)) tolerance) + (> (abs (:y offset)) tolerance))) + (update shape :content path/move-content offset))))) + + (fix-objects [objects] + (reduce-kv (fn [objects id shape] + (if-let [shape (fix-path shape)] + (assoc objects id shape) + objects)) + objects + objects)) + + (update-container [container] + (d/update-when container :objects fix-objects))] + + (-> data + (update :pages-index d/update-vals update-container) + (d/update-when :components d/update-vals update-container))))) + (def available-migrations (into (d/ordered-set) ["legacy-2" @@ -2386,4 +2425,5 @@ "0030-remove-stroke-per-side-attr" "0031-migrate-stroke-width-token-attr" "0032-fix-children-not-in-parent" - "0033-normalize-main-instance-names"])) + "0033-normalize-main-instance-names" + "0034-fix-path-content-offset"])) diff --git a/common/src/app/common/flags.cljc b/common/src/app/common/flags.cljc index 164d8c78a9..26f1d54390 100644 --- a/common/src/app/common/flags.cljc +++ b/common/src/app/common/flags.cljc @@ -174,7 +174,7 @@ ;; redis for caching data :redis-cache - ;; Activates the admin-console module + ;; enabled by default. Activates the admin-console module :admin-console ;; disabled by default. When enabled, allows the admin-console @@ -235,7 +235,8 @@ :enable-stroke-per-side :enable-token-combobox :enable-custom-shortcuts - :enable-token-lib-sync]) + :enable-token-lib-sync + :enable-admin-console]) (defn parse [& flags] diff --git a/common/src/app/common/logic/libraries.cljc b/common/src/app/common/logic/libraries.cljc index 3a87e923c4..3afa807ead 100644 --- a/common/src/app/common/logic/libraries.cljc +++ b/common/src/app/common/logic/libraries.cljc @@ -3182,7 +3182,8 @@ (cond-> (ctsl/grid-layout? obj) (ctsl/remap-grid-cells ids-map)) - (cond-> (ctk/is-variant-container? parent) + (cond-> (and (ctk/is-variant-container? parent) + (ctk/main-instance? obj)) (assoc :variant-id parent-id)) (cond-> (and (ctk/is-variant? obj) (not (ctk/is-variant-container? parent))) diff --git a/common/src/app/common/logic/shapes.cljc b/common/src/app/common/logic/shapes.cljc index 30d6067889..8cb30231d8 100644 --- a/common/src/app/common/logic/shapes.cljc +++ b/common/src/app/common/logic/shapes.cljc @@ -10,6 +10,7 @@ [app.common.data.macros :as dm] [app.common.files.changes-builder :as pcb] [app.common.files.helpers :as cfh] + [app.common.files.shapes-helpers :as cfsh] [app.common.geom.shapes :as gsh] [app.common.logic.variant-properties :as clvp] [app.common.types.component :as ctk] @@ -450,6 +451,18 @@ [all-parents changes]))) +(defn- find-empty-variant-containers + [objects excluded-parent-id ids parent-ids] + (let [ids (set ids)] + (reduce (fn [to-delete candidate-id] + (let [parent (get objects candidate-id)] + (if (and (ctk/is-variant-container? parent) + (empty? (remove ids (:shapes parent)))) + (conj to-delete (:id parent)) + to-delete))) + #{} + (remove #(= % excluded-parent-id) parent-ids)))) + (defn generate-relocate [changes parent-id to-index ids & {:keys [cell ignore-parents?]}] (let [objects (pcb/get-objects changes) @@ -553,128 +566,146 @@ ;; Parents that are a variant-container that becomes empty - empty-variant-cont (reduce - (fn [to-delete parent-id] - (let [parent (get objects parent-id)] - (if (and (ctk/is-variant-container? parent) - (empty? (remove (set ids) (:shapes parent)))) - (conj to-delete (:id parent)) - to-delete))) - #{} - (remove #(= % parent-id) all-parents))] + empty-variant-cont (find-empty-variant-containers objects parent-id ids all-parents)] - (-> changes - ;; Remove layout-item properties and tokens when moving a shape outside a layout - (cond-> (not (ctl/any-layout? parent)) - (-> (pcb/update-shapes ids ctl/remove-layout-item-data) - (pcb/update-shapes ids cto/unapply-layout-item-tokens))) + (if (and (ctk/is-variant-container? parent) + (not (every? ctk/main-instance? shapes))) + changes + (-> changes + ;; Remove layout-item properties and tokens when moving a shape outside a layout + (cond-> (not (ctl/any-layout? parent)) + (-> (pcb/update-shapes ids ctl/remove-layout-item-data) + (pcb/update-shapes ids cto/unapply-layout-item-tokens))) - ;; Remove the hide in viewer flag - (cond-> (and (not= uuid/zero parent-id) (cfh/frame-shape? parent)) - (pcb/update-shapes ids #(cond-> % (cfh/frame-shape? %) (assoc :hide-in-viewer true)))) + ;; Remove the hide in viewer flag + (cond-> (and (not= uuid/zero parent-id) (cfh/frame-shape? parent)) + (pcb/update-shapes ids #(cond-> % (cfh/frame-shape? %) (assoc :hide-in-viewer true)))) - ;; Remove the swap slots if it is moving to a different component - (pcb/update-shapes - child-heads-ids - (fn [shape] - (cond-> shape - (not= component-main-parent (ctn/find-component-main objects shape false)) - (ctk/remove-swap-slot)))) + ;; Remove the swap slots if it is moving to a different component + (pcb/update-shapes + child-heads-ids + (fn [shape] + (cond-> shape + (not= component-main-parent (ctn/find-component-main objects shape false)) + (ctk/remove-swap-slot)))) - ;; Remove component-root property when moving a shape inside a component - (cond-> (ctn/get-instance-root objects parent) - (pcb/update-shapes children-ids #(dissoc % :component-root))) + ;; Remove component-root property when moving a shape inside a component + (cond-> (ctn/get-instance-root objects parent) + (pcb/update-shapes children-ids #(dissoc % :component-root))) - ;; Add component-root property when moving a component outside a component - (cond-> (not (ctn/get-instance-root objects parent)) - (pcb/update-shapes child-heads-ids #(assoc % :component-root true))) + ;; Add component-root property when moving a component outside a component + (cond-> (not (ctn/get-instance-root objects parent)) + (pcb/update-shapes child-heads-ids #(assoc % :component-root true))) - ;; Remove variant info and rename when moving outside a variant-container - (cond-> (not (ctk/is-variant-container? parent)) - (clvp/generate-make-shapes-no-variant variant-shapes)) + ;; Remove variant info and rename when moving outside a variant-container + (cond-> (not (ctk/is-variant-container? parent)) + (clvp/generate-make-shapes-no-variant variant-shapes)) - ;; Add variant info and rename when moving into a different variant-container - (cond-> (ctk/is-variant-container? parent) - (clvp/generate-make-shapes-variant child-heads parent)) + ;; Add variant info and rename when moving into a different variant-container + (cond-> (ctk/is-variant-container? parent) + (clvp/generate-make-shapes-variant child-heads parent)) - ;; Move the shapes - (pcb/change-parent parent-id - shapes - to-index) + ;; Move the shapes + (pcb/change-parent parent-id + shapes + to-index) - ;; Remove empty groups - (pcb/remove-objects groups-to-delete) + ;; Remove empty groups + (pcb/remove-objects groups-to-delete) - ;; Unmask groups whose mask have moved outside - (pcb/update-shapes groups-to-unmask - (fn [shape] - (assoc shape :masked-group false))) + ;; Unmask groups whose mask have moved outside + (pcb/update-shapes groups-to-unmask + (fn [shape] + (assoc shape :masked-group false))) - ;; Detach shapes moved out of their component - (pcb/update-shapes shapes-to-detach ctk/detach-shape) + ;; Detach shapes moved out of their component + (pcb/update-shapes shapes-to-detach ctk/detach-shape) - ;; Make non root a component moved inside another one - (pcb/update-shapes shapes-to-deroot - (fn [shape] - (assoc shape :component-root nil))) + ;; Make non root a component moved inside another one + (pcb/update-shapes shapes-to-deroot + (fn [shape] + (assoc shape :component-root nil))) - ;; Make root a subcomponent moved outside its parent component - (pcb/update-shapes shapes-to-reroot - (fn [shape] - (assoc shape :component-root true))) + ;; Make root a subcomponent moved outside its parent component + (pcb/update-shapes shapes-to-reroot + (fn [shape] + (assoc shape :component-root true))) - ;; Reset constraints depending on the new parent - (pcb/update-shapes ids - (fn [shape] - (let [frame-id (if (= (:type parent) :frame) - (:id parent) - (:frame-id parent)) - moved-shape (assoc shape - :parent-id parent-id - :frame-id frame-id)] - (assoc shape - :constraints-h (gsh/default-constraints-h moved-shape) - :constraints-v (gsh/default-constraints-v moved-shape)))) - {:ignore-touched true}) + ;; Reset constraints depending on the new parent + (pcb/update-shapes ids + (fn [shape] + (let [frame-id (if (= (:type parent) :frame) + (:id parent) + (:frame-id parent)) + moved-shape (assoc shape + :parent-id parent-id + :frame-id frame-id)] + (assoc shape + :constraints-h (gsh/default-constraints-h moved-shape) + :constraints-v (gsh/default-constraints-v moved-shape)))) + {:ignore-touched true}) - ;; Fix the sizing when moving a shape - (pcb/update-shapes parents - (fn [parent] - (if (ctl/flex-layout? parent) - (cond-> parent - (ctl/change-h-sizing? (:id parent) objects (:shapes parent)) - (assoc :layout-item-h-sizing :fix) + ;; Fix the sizing when moving a shape + (pcb/update-shapes parents + (fn [parent] + (if (ctl/flex-layout? parent) + (cond-> parent + (ctl/change-h-sizing? (:id parent) objects (:shapes parent)) + (assoc :layout-item-h-sizing :fix) - (ctl/change-v-sizing? (:id parent) objects (:shapes parent)) - (assoc :layout-item-v-sizing :fix)) - parent))) + (ctl/change-v-sizing? (:id parent) objects (:shapes parent)) + (assoc :layout-item-v-sizing :fix)) + parent))) - ;; Change the grid cell in a grid layout - (cond-> (ctl/grid-layout? objects parent-id) - (-> (pcb/update-shapes - [parent-id] - (fn [frame objects] - (let [[row column] cell] - (-> frame - ;; Assign the cell when pushing into a specific grid cell - (cond-> (some? cell) - (-> (ctl/free-cell-shapes ids) - (ctl/push-into-cell ids row column) - (ctl/assign-cells objects))) - (ctl/assign-cell-positions objects)))) - {:with-objects? true}) - (pcb/reorder-grid-children [parent-id]))) + ;; Change the grid cell in a grid layout + (cond-> (ctl/grid-layout? objects parent-id) + (-> (pcb/update-shapes + [parent-id] + (fn [frame objects] + (let [[row column] cell] + (-> frame + ;; Assign the cell when pushing into a specific grid cell + (cond-> (some? cell) + (-> (ctl/free-cell-shapes ids) + (ctl/push-into-cell ids row column) + (ctl/assign-cells objects))) + (ctl/assign-cell-positions objects)))) + {:with-objects? true}) + (pcb/reorder-grid-children [parent-id]))) - ;; If parent locked, lock the added shapes - (cond-> (:blocked parent) - (pcb/update-shapes ids #(assoc % :blocked true))) + ;; If parent locked, lock the added shapes + (cond-> (:blocked parent) + (pcb/update-shapes ids #(assoc % :blocked true))) - ;; Resize parent containers that need to - (pcb/resize-parents parents) + ;; Resize parent containers that need to + (pcb/resize-parents parents) - ;; Remove parents when are a variant-container that becomes empty - (cond-> (seq empty-variant-cont) - (#(second (generate-delete-shapes % empty-variant-cont {}))))))) + ;; Remove parents when are a variant-container that becomes empty + (cond-> (seq empty-variant-cont) + (#(second (generate-delete-shapes % empty-variant-cont {})))))))) + +(defn generate-move-shapes-into-frame + "Reparent `shapes` into `frame-id`, stripping variant records when a + variant main leaves its container. The builder must already have + objects and library data." + [changes frame-id shapes objects remove-layout-data?] + (let [parent-id (dm/get-in objects [frame-id :parent-id]) + ids (remove #(= % parent-id) shapes) + to-move (->> ids + (map (d/getf objects)) + (filter some?)) + new-parent (get objects frame-id) + variant-shapes (filter ctk/is-variant? to-move) + all-parents (into #{frame-id} (map #(cfh/get-parent-id objects %)) ids) + empty-variant-cont (find-empty-variant-containers objects frame-id ids all-parents)] + (if (empty? to-move) + changes + (-> changes + (cond-> (not (ctk/is-variant-container? new-parent)) + (clvp/generate-make-shapes-no-variant variant-shapes)) + (cfsh/prepare-move-shapes-into-frame frame-id shapes objects remove-layout-data?) + (cond-> (seq empty-variant-cont) + (#(second (generate-delete-shapes % empty-variant-cont {})))))))) (defn change-show-in-viewer [shape hide?] diff --git a/common/src/app/common/logic/variant_properties.cljc b/common/src/app/common/logic/variant_properties.cljc index 81dc28399d..3e950a81b8 100644 --- a/common/src/app/common/logic/variant_properties.cljc +++ b/common/src/app/common/logic/variant_properties.cljc @@ -61,15 +61,18 @@ [changes component-id pos value] (if-let [value (ctv/prepare-property-value value)] (let [data (pcb/get-library-data changes) + objects (pcb/get-objects changes) component (ctcl/get-component data component-id true) - main-id (:main-instance-id component) - name (-> (:variant-properties component) - (update pos assoc :value value) - ctv/properties-to-name)] - (-> changes - (pcb/update-component component-id #(assoc-in % [:variant-properties pos :value] value) - {:apply-changes-local-library? true}) - (pcb/update-shapes [main-id] #(assoc % :variant-name name)))) + main-id (:main-instance-id component)] + (if (contains? objects main-id) + (let [name (-> (:variant-properties component) + (update pos assoc :value value) + ctv/properties-to-name)] + (-> changes + (pcb/update-component component-id #(assoc-in % [:variant-properties pos :value] value) + {:apply-changes-local-library? true}) + (pcb/update-shapes [main-id] #(assoc % :variant-name name)))) + changes)) changes)) diff --git a/common/src/app/common/logic/variants.cljc b/common/src/app/common/logic/variants.cljc index 79de230b5a..9a78d6cc20 100644 --- a/common/src/app/common/logic/variants.cljc +++ b/common/src/app/common/logic/variants.cljc @@ -9,12 +9,53 @@ [app.common.logic.libraries :as cll] [app.common.logic.shapes :as cls] [app.common.logic.variant-properties :as clvp] + [app.common.path-names :as cpn] [app.common.types.component :as ctk] [app.common.types.container :as ctn] [app.common.types.file :as ctf] [app.common.types.variant :as ctv] [app.common.uuid :as uuid])) +(defn generate-rename-variant + "Rename a variant set in one change vector. + + The builder already has the mains' page objects and library data. + The stored name is `clean-path` of `name`, or \"(empty)\" when that + path is blank. The board and every direct child take that name. + Each component's path and name come from `split-group-name`." + [changes variant-id name] + (let [objects (pcb/get-objects changes) + data (pcb/get-library-data changes) + clean-name (cpn/clean-path name) + stored-name (if (empty? clean-name) "(empty)" clean-name) + shape-ids (cons variant-id (:shapes (get objects variant-id))) + components (cfv/find-variant-components data objects variant-id)] + (reduce (fn [changes component] + (let [[path item-name] (cpn/split-group-name stored-name)] + (pcb/update-component changes + (:id component) + #(assoc % :path path :name item-name)))) + (pcb/update-shapes changes shape-ids #(assoc % :name stored-name)) + components))) + +(defn generate-define-variant-container + "One change vector. `name` is the string stored on the board. + The component path and name are `split-group-name` of that string. + Library data is refreshed before the first property is added." + [changes variant-id main-id component-id name] + (let [[path item-name] (cpn/split-group-name name)] + (-> changes + (pcb/update-shapes [variant-id] + #(assoc % :is-variant-container true :name name)) + (pcb/update-shapes [main-id] + #(assoc % :name name :variant-id variant-id)) + (pcb/update-component component-id + #(assoc % :variant-id variant-id + :path path + :name item-name) + {:apply-changes-local-library? true}) + (clvp/generate-add-new-property variant-id {:fill-values? true})))) + (defn generate-add-new-variant [changes shape variant-id new-component-id new-shape-id prop-num] (let [data (pcb/get-library-data changes) diff --git a/common/src/app/common/types/container.cljc b/common/src/app/common/types/container.cljc index cc7ea9bd07..d790999db0 100644 --- a/common/src/app/common/types/container.cljc +++ b/common/src/app/common/types/container.cljc @@ -426,6 +426,52 @@ true (has-any-copy-parent? objects (:parent-id shape)))))) +(defn get-variant-boundary-id + "Id of the nearest variant main or variant container for this shape, or nil + when the shape is outside any variant set. + + Nested children resolve to their enclosing main (not the container), so two + shapes under different variants are treated as different boundaries even + when they share a set." + [objects shape] + (some (fn [s] + (cond + (ctk/is-variant? s) (:id s) + (ctk/is-variant-container? s) (:id s) + :else nil)) + (cfh/get-parents-with-self objects (:id shape)))) + +(defn crosses-variant-set-boundary? + "True when the selection mixes shapes that belong to different variant + boundaries — including outside vs in-set, two different variant mains + (even in the same set), or two different variant containers. + + Structure commands (group, mask, board-from-selection, layout) must not + reparent across that boundary." + [objects shapes] + (let [ids (into #{} (map #(get-variant-boundary-id objects %)) shapes)] + (> (count ids) 1))) + +(defn ids-for-structure-op + "Ids allowed for Group, Mask, and Create board from selection. + + Drops copy-children and variant mains, refuses a cross-boundary or + container selection, then removes descendants whose ancestor is + still selected. Mains are dropped before loop-cleaning so a + selected main does not swallow nested children. Layout does not + use this helper." + [objects ids] + (let [shapes (->> ids + (remove #(has-any-copy-parent? objects (get objects %))) + (keep (d/getf objects)))] + (if (or (crosses-variant-set-boundary? objects shapes) + (some ctk/is-variant-container? shapes)) + [] + (->> shapes + (remove ctk/is-variant?) + (map :id) + (cfh/clean-loops objects))))) + (defn has-any-main? "Check if the shape is a main component or has any children or parent that is a main component." [objects shape] diff --git a/common/test/common_tests/files_migrations_0034_test.cljc b/common/test/common_tests/files_migrations_0034_test.cljc new file mode 100644 index 0000000000..82d335e4ff --- /dev/null +++ b/common/test/common_tests/files_migrations_0034_test.cljc @@ -0,0 +1,116 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns common-tests.files-migrations-0034-test + (:require + [app.common.files.migrations :as cfm] + [app.common.geom.matrix :as gmt] + [app.common.geom.point :as gpt] + [app.common.geom.shapes :as gsh] + [app.common.math :as mth] + [app.common.types.modifiers :as ctm] + [app.common.types.path :as path] + [app.common.types.path.segment :as segment] + [app.common.types.shape :as cts] + [app.common.uuid :as uuid] + [clojure.test :as t])) + +;; 0034-fix-path-content-offset +;; A path whose drawing (:content) was moved apart from its box +;; (:selrect and :points) gets its content moved back onto the box. + +(def ^:private migration "0034-fix-path-content-offset") + +(defn- triangle + [x0 y0] + [{:command :move-to :params {:x x0 :y y0}} + {:command :line-to :params {:x (+ x0 14) :y y0}} + {:command :line-to :params {:x (+ x0 7) :y (+ y0 14)}} + {:command :close-path}]) + +(defn- make-path + [shape-id x0 y0] + (cts/setup-shape {:id shape-id :type :path :content (triangle x0 y0)})) + +(defn- rotate + [shape angle] + (gsh/transform-shape shape (ctm/rotation-modifiers shape (gsh/shape->center shape) angle))) + +(defn- displace-content + [shape delta] + (update shape :content path/move-content delta)) + +(defn- migrate-shape + [shape] + (let [page-id (uuid/next) + data {:pages-index {page-id {:objects {(:id shape) shape}}}} + data' (cfm/migrate-data data migration)] + (get-in data' [:pages-index page-id :objects (:id shape)]))) + +(defn- content-matches-box? + [shape] + (let [derived (path/update-geometry shape)] + (and (every? true? (map gpt/close? (:points shape) (:points derived))) + (every? #(mth/close? (get (:selrect shape) %) (get (:selrect derived) %)) + [:x :y :width :height])))) + +(t/deftest migration-0034-moves-displaced-content-onto-its-box + (let [shape (make-path (uuid/next) 1 1) + broken (displace-content shape (gpt/point 0 -182)) + shape' (migrate-shape broken)] + + (t/is (not (content-matches-box? broken)) "the fixture is broken") + (t/is (content-matches-box? shape') "content back onto the box") + (t/is (= (:selrect broken) (:selrect shape')) "box unchanged") + (t/is (= (:points broken) (:points shape')) "points unchanged") + (t/is (= (segment/content->selrect (:content shape)) + (segment/content->selrect (:content shape'))) + "content where it was before being displaced"))) + +(t/deftest migration-0034-moves-displaced-content-of-a-rotated-path + (let [shape (rotate (make-path (uuid/next) 100 100) 30) + broken (displace-content shape (gpt/point 25 -40)) + shape' (migrate-shape broken)] + + (t/is (not (gmt/close? (gmt/matrix) (:transform shape))) "the fixture is rotated") + (t/is (not (content-matches-box? broken)) "the fixture is broken") + (t/is (content-matches-box? shape') "content back onto the box") + (t/is (= (:selrect broken) (:selrect shape')) "box unchanged") + (t/is (= (:transform broken) (:transform shape')) "rotation unchanged"))) + +(t/deftest migration-0034-keeps-paths-whose-content-matches-the-box + (let [shape (make-path (uuid/next) 10 20) + small (displace-content shape (gpt/point 0.05 0))] + + (t/is (identical? shape (migrate-shape shape)) "consistent path untouched") + (t/is (identical? small (migrate-shape small)) "float noise untouched"))) + +(t/deftest migration-0034-keeps-paths-whose-content-has-another-size + (let [shape (make-path (uuid/next) 0 0) + bigger (assoc shape :content (path/content [{:command :move-to :params {:x 40 :y 0}} + {:command :line-to :params {:x 70 :y 0}} + {:command :line-to :params {:x 55 :y 30}} + {:command :close-path}]))] + + (t/is (identical? bigger (migrate-shape bigger)) "size mismatch untouched"))) + +(t/deftest migration-0034-ignores-shapes-other-than-paths + (let [shape-id (uuid/next) + rect (cts/setup-shape {:id shape-id :type :rect :x 0 :y 0 :width 10 :height 10})] + (t/is (identical? rect (migrate-shape rect))))) + +(t/deftest migration-0034-fixes-paths-inside-components + (let [shape (make-path (uuid/next) 1 1) + broken (displace-content shape (gpt/point 0 109)) + component-id (uuid/next) + data {:pages-index {} + :components {component-id {:id component-id + :deleted true + :objects {(:id broken) broken}}}} + data' (cfm/migrate-data data migration) + shape' (get-in data' [:components component-id :objects (:id broken)])] + + (t/is (content-matches-box? shape') "deleted component content fixed"))) diff --git a/common/test/common_tests/logic/variants_reparent_test.cljc b/common/test/common_tests/logic/variants_reparent_test.cljc new file mode 100644 index 0000000000..5ff04f9153 --- /dev/null +++ b/common/test/common_tests/logic/variants_reparent_test.cljc @@ -0,0 +1,316 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns common-tests.logic.variants-reparent-test + (:require + [app.common.files.changes-builder :as pcb] + [app.common.logic.shapes :as cls] + [app.common.test-helpers.components :as thc] + [app.common.test-helpers.compositions :as tho] + [app.common.test-helpers.files :as thf] + [app.common.test-helpers.ids-map :as thi] + [app.common.test-helpers.shapes :as ths] + [app.common.test-helpers.variants :as thv] + [app.common.types.component :as ctk] + [clojure.test :as t])) + +(t/use-fixtures :each thi/test-fixture) + +(defn- relocate + [file parent-id ids & {:keys [to-index] :or {to-index 0}}] + (let [page (thf/current-page file)] + (-> (pcb/empty-changes nil) + (pcb/with-page-id (:id page)) + (pcb/with-objects (:objects page)) + (pcb/with-library-data (:data file)) + (cls/generate-relocate parent-id to-index (set ids))))) + +(defn- move-into-frame + [file frame-id ids] + (let [page (thf/current-page file) + objects (:objects page)] + (-> (pcb/empty-changes nil (:id page)) + (pcb/with-objects objects) + (pcb/with-library-data (:data file)) + (cls/generate-move-shapes-into-frame frame-id ids objects true)))) + +(defn- property-names + [component] + (mapv :name (:variant-properties component))) + +(defn- strip-component-times + "Undo restamps :modified-at on components when mains are updated. + Compare file data without that clock." + [data] + (update data :components update-vals #(dissoc % :modified-at))) + +(t/deftest test-relocate-plain-shape-onto-container-is-rejected + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-rect :rect1)) + container (ths/get-shape file :v01) + rect (ths/get-shape file :rect1) + children (:shapes container) + changes (relocate file (:id container) [(:id rect)]) + file' (thf/apply-changes file changes) + rect' (ths/get-shape file' :rect1) + container' (ths/get-shape file' :v01)] + + (t/is (empty? (:redo-changes changes))) + (t/is (= (:parent-id rect) (:parent-id rect'))) + (t/is (= (:constraints-h rect) (:constraints-h rect'))) + (t/is (= (:constraints-v rect) (:constraints-v rect'))) + (t/is (= children (:shapes container'))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-component-copy-onto-container-is-rejected + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-simple-component-with-copy :c03 :m03 :r03 :copy03)) + container (ths/get-shape file :v01) + copy (ths/get-shape file :copy03) + comp (thc/get-component file :c03) + children (:shapes container) + changes (relocate file (:id container) [(:id copy)]) + file' (thf/apply-changes file changes) + copy' (ths/get-shape file' :copy03) + comp' (thc/get-component file' :c03) + container' (ths/get-shape file' :v01)] + + (t/is (empty? (:redo-changes changes))) + (t/is (= (:parent-id copy) (:parent-id copy'))) + (t/is (nil? (:variant-id copy'))) + (t/is (= (:variant-id comp) (:variant-id comp'))) + (t/is (= (:variant-properties comp) (:variant-properties comp'))) + (t/is (= children (:shapes container'))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-mixed-selection-onto-container-is-rejected + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-rect :rect1) + (tho/add-simple-component :c03 :m03 :r03)) + container (ths/get-shape file :v01) + rect (ths/get-shape file :rect1) + main (ths/get-shape file :m03) + changes (relocate file (:id container) [(:id rect) (:id main)]) + file' (thf/apply-changes file changes) + rect' (ths/get-shape file' :rect1) + main' (ths/get-shape file' :m03)] + + (t/is (empty? (:redo-changes changes))) + (t/is (= (:parent-id rect) (:parent-id rect'))) + (t/is (= (:parent-id main) (:parent-id main'))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-main-into-container-becomes-variant + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-simple-component :c03 :m03 :r03)) + container (ths/get-shape file :v01) + main (ths/get-shape file :m03) + sibling (thc/get-component file :c01) + changes (relocate file (:id container) [(:id main)]) + file' (thf/apply-changes file changes) + file'' (thf/apply-undo-changes file' changes) + main' (ths/get-shape file' :m03) + comp' (thc/get-component file' :c03) + sibling' (thc/get-component file' :c01) + container' (ths/get-shape file' :v01)] + + (t/is (= (:id container') (:parent-id main'))) + (t/is (= (:id container') (:variant-id main'))) + (t/is (= (:id container') (:variant-id comp'))) + (t/is (some? (:variant-name main'))) + (t/is (= (property-names sibling') (property-names comp'))) + (t/is (= (property-names sibling) (property-names sibling'))) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file'')))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-reorder-mains-does-not-rebuild-properties + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02)) + container (ths/get-shape file :v01) + m01 (ths/get-shape file :m01) + m02 (ths/get-shape file :m02) + c01 (thc/get-component file :c01) + c02 (thc/get-component file :c02) + changes (relocate file (:id container) [(:id m01)] :to-index 0) + file' (thf/apply-changes file changes) + file'' (thf/apply-undo-changes file' changes) + m01' (ths/get-shape file' :m01) + m02' (ths/get-shape file' :m02) + c01' (thc/get-component file' :c01) + c02' (thc/get-component file' :c02)] + + (t/is (= (:variant-name m01) (:variant-name m01'))) + (t/is (= (:variant-name m02) (:variant-name m02'))) + (t/is (= (:variant-properties c01) (:variant-properties c01'))) + (t/is (= (:variant-properties c02) (:variant-properties c02'))) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file'')))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-all-mains-within-container-keeps-container + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02)) + container (ths/get-shape file :v01) + m01 (ths/get-shape file :m01) + m02 (ths/get-shape file :m02) + changes (relocate file (:id container) [(:id m01) (:id m02)]) + file' (thf/apply-changes file changes) + container' (ths/get-shape file' :v01)] + + (t/is (= (:id container) (:id container'))) + (t/is (= (set (:shapes container)) (set (:shapes container')))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-main-out-of-container-clears-variant-data + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-frame :board1)) + board (ths/get-shape file :board1) + m01 (ths/get-shape file :m01) + changes (relocate file (:id board) [(:id m01)]) + file' (thf/apply-changes file changes) + file'' (thf/apply-undo-changes file' changes) + m01' (ths/get-shape file' :m01) + c01' (thc/get-component file' :c01) + container' (ths/get-shape file' :v01)] + + (t/is (= (:id board) (:parent-id m01'))) + (t/is (nil? (:variant-id m01'))) + (t/is (nil? (:variant-name m01'))) + (t/is (nil? (:variant-id c01'))) + (t/is (nil? (:variant-properties c01'))) + (t/is (some? container')) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file'')))) + (thf/validate-file! file'))) + +(t/deftest test-relocate-last-mains-out-deletes-empty-container + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-frame :board1)) + board (ths/get-shape file :board1) + m01 (ths/get-shape file :m01) + m02 (ths/get-shape file :m02) + changes (relocate file (:id board) [(:id m01) (:id m02)]) + file' (thf/apply-changes file changes) + file'' (thf/apply-undo-changes file' changes) + container' (ths/get-shape file' :v01) + m01' (ths/get-shape file' :m01) + m02' (ths/get-shape file' :m02) + c01' (thc/get-component file' :c01) + c02' (thc/get-component file' :c02)] + + (t/is (nil? container')) + (t/is (some? m01')) + (t/is (some? m02')) + (t/is (= (:id board) (:parent-id m01'))) + (t/is (= (:id board) (:parent-id m02'))) + (t/is (nil? (:variant-id m01'))) + (t/is (nil? (:variant-id m02'))) + (t/is (nil? (:variant-name m01'))) + (t/is (nil? (:variant-name m02'))) + (t/is (some? c01')) + (t/is (some? c02')) + (t/is (nil? (:variant-id c01'))) + (t/is (nil? (:variant-id c02'))) + (t/is (nil? (:variant-properties c01'))) + (t/is (nil? (:variant-properties c02'))) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file'')))) + (thf/validate-file! file'))) + +(t/deftest test-board-draw-strips-variant-data-when-main-leaves + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-frame :board1)) + board (ths/get-shape file :board1) + m01 (ths/get-shape file :m01) + changes (move-into-frame file (:id board) [(:id m01)]) + file' (thf/apply-changes file changes) + file'' (thf/apply-undo-changes file' changes) + m01' (ths/get-shape file' :m01) + c01' (thc/get-component file' :c01) + container' (ths/get-shape file' :v01)] + + (t/is (= (:id board) (:parent-id m01'))) + (t/is (nil? (:variant-id m01'))) + (t/is (nil? (:variant-name m01'))) + (t/is (nil? (:variant-id c01'))) + (t/is (nil? (:variant-properties c01'))) + (t/is (some? container')) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file'')))) + (thf/validate-file! file'))) + +(t/deftest test-board-draw-last-main-deletes-empty-container + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-frame :board1)) + board (ths/get-shape file :board1) + m01 (ths/get-shape file :m01) + m02 (ths/get-shape file :m02) + file* (thf/apply-changes file (move-into-frame file (:id board) [(:id m01)])) + changes (move-into-frame file* (:id board) [(:id m02)]) + file' (thf/apply-changes file* changes) + container' (ths/get-shape file' :v01) + m01' (ths/get-shape file' :m01) + m02' (ths/get-shape file' :m02) + c01' (thc/get-component file' :c01) + c02' (thc/get-component file' :c02)] + + (t/is (nil? container')) + (t/is (some? m01')) + (t/is (some? m02')) + (t/is (= (:id board) (:parent-id m01'))) + (t/is (= (:id board) (:parent-id m02'))) + (t/is (nil? (:variant-id m01'))) + (t/is (nil? (:variant-id m02'))) + (t/is (nil? (:variant-name m01'))) + (t/is (nil? (:variant-name m02'))) + (t/is (some? c01')) + (t/is (some? c02')) + (t/is (nil? (:variant-id c01'))) + (t/is (nil? (:variant-id c02'))) + (t/is (nil? (:variant-properties c01'))) + (t/is (nil? (:variant-properties c02'))) + (thf/validate-file! file'))) + +(t/deftest test-board-draw-undo-restores-emptied-container + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02) + (tho/add-frame :board1)) + board (ths/get-shape file :board1) + m01 (ths/get-shape file :m01) + m02 (ths/get-shape file :m02) + changes* (move-into-frame file (:id board) [(:id m01)]) + file* (thf/apply-changes file changes*) + changes (move-into-frame file* (:id board) [(:id m02)]) + file' (thf/apply-changes file* changes) + file'' (thf/apply-undo-changes file' changes) + file''' (thf/apply-undo-changes file'' changes*)] + + (t/is (= (strip-component-times (:data file*)) + (strip-component-times (:data file'')))) + (t/is (= (strip-component-times (:data file)) + (strip-component-times (:data file''')))))) + +(t/deftest test-duplicate-variant-main-keeps-variant-id + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02)) + container (ths/get-shape file :v01) + file' (tho/duplicate-shape file :m01) + objects' (:objects (thf/current-page file')) + variant-mains (->> (vals objects') + (filter ctk/main-instance?) + (filter #(= (:id container) (:variant-id %))))] + + (t/is (= 3 (count variant-mains))) + (thf/validate-file! file'))) diff --git a/common/test/common_tests/logic/variants_test.cljc b/common/test/common_tests/logic/variants_test.cljc index d4a4696336..0323df3afb 100644 --- a/common/test/common_tests/logic/variants_test.cljc +++ b/common/test/common_tests/logic/variants_test.cljc @@ -11,11 +11,15 @@ [app.common.logic.libraries :as cll] [app.common.logic.shapes :as cls] [app.common.logic.variant-properties :as clvp] + [app.common.logic.variants :as clv] + [app.common.path-names :as cpn] [app.common.test-helpers.components :as thc] + [app.common.test-helpers.compositions :as tho] [app.common.test-helpers.files :as thf] [app.common.test-helpers.ids-map :as thi] [app.common.test-helpers.shapes :as ths] [app.common.test-helpers.variants :as thv] + [app.common.types.component :as ctk] [app.common.types.variant :as ctv] [clojure.test :as t] [cuerdas.core :as str])) @@ -511,3 +515,174 @@ (thf/validate-file! file') (t/is (some? new-shape')) (t/is (not= (:parent-id new-shape') (:id container))))) + +(defn- two-property-file-with-other-page + [] + (-> (thf/sample-file :file1 :page-label :page1) + (thv/add-variant-two-properties :v01 :c01 :m01 :c02 :m02) + (thf/add-sample-page :page2))) + +(t/deftest test-update-property-value-skips-when-main-not-in-objects + (let [file (two-property-file-with-other-page) + other (thf/current-page file) + comp01 (thc/get-component file :c01) + main (ths/get-shape file :m01 :page-label :page1) + before (:variant-properties comp01) + label (:variant-name main) + + changes (-> (pcb/empty-changes nil) + (pcb/with-page-id (:id other)) + (pcb/with-library-data (:data file)) + (pcb/with-objects (:objects other)) + (clvp/generate-update-property-value (:id comp01) 1 "crítica")) + + file' (thf/apply-changes file changes) + comp01' (thc/get-component file' :c01) + main' (ths/get-shape file' :m01 :page-label :page1)] + + (thf/validate-file! file') + (t/is (empty? (:redo-changes changes))) + (t/is (= before (:variant-properties comp01'))) + (t/is (= label (:variant-name main'))))) + +(t/deftest test-update-property-value-writes-main-page-and-chains + (let [file (two-property-file-with-other-page) + mains (thf/get-page file :page1) + comp01 (thc/get-component file :c01) + + changes (-> (pcb/empty-changes nil) + (pcb/with-page-id (:id mains)) + (pcb/with-library-data (:data file)) + (pcb/with-objects (:objects mains)) + (clvp/generate-update-property-value (:id comp01) 0 "active") + (clvp/generate-update-property-value (:id comp01) 1 "crítica")) + + file' (thf/apply-changes file changes) + comp01' (thc/get-component file' :c01) + main' (ths/get-shape file' :m01 :page-label :page1) + props' (:variant-properties comp01')] + + (thf/validate-file! file') + (t/is (= "active" (-> props' first :value))) + (t/is (= "crítica" (-> props' second :value))) + (t/is (= (ctv/properties-to-name props') (:variant-name main'))) + (t/is (= "active, crítica" (:variant-name main'))))) + +(defn- post-artboard-alpha-file + "A normal frame holding one main named Alpha — before the container flag." + [] + (-> (thf/sample-file :file1) + (tho/add-frame :v01 {:name "Board"}) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:parent-label :v01 :name "Alpha"}))) + +(t/deftest test-define-variant-container-renames-component-in-same-vector + (let [file (post-artboard-alpha-file) + page (thf/current-page file) + variant-id (thi/id :v01) + main-id (thi/id :m01) + component-id (:id (thc/get-component file :c01)) + changes (-> (pcb/empty-changes nil (:id page)) + (pcb/with-objects (:objects page)) + (pcb/with-library-data (:data file)) + (clv/generate-define-variant-container + variant-id main-id component-id "Component")) + file' (thf/apply-changes file changes) + container' (ths/get-shape file' :v01) + main' (ths/get-shape file' :m01) + comp' (thc/get-component file' :c01) + restored (thf/apply-undo-changes file' changes) + container-rest (ths/get-shape restored :v01) + main-rest (ths/get-shape restored :m01) + comp-rest (thc/get-component restored :c01)] + + (t/is (ctk/is-variant-container? container')) + (t/is (= "Component" + (:name container') + (:name main') + (cpn/merge-path-item (:path comp') (:name comp')))) + (t/is (= variant-id (:variant-id main') (:variant-id comp'))) + (t/is (= (ctv/properties-to-name (:variant-properties comp')) + (:variant-name main'))) + (t/is (not (ctk/is-variant-container? container-rest))) + (t/is (= "Alpha" (:name main-rest))) + (t/is (= "Alpha" (cpn/merge-path-item (:path comp-rest) (:name comp-rest)))) + (t/is (nil? (:variant-id main-rest))) + (t/is (nil? (:variant-id comp-rest))))) + +(defn- rename-variant-changes + [file name] + (let [page (thf/current-page file)] + (-> (pcb/empty-changes nil) + (pcb/with-page-id (:id page)) + (pcb/with-library-data (:data file)) + (pcb/with-objects (:objects page)) + (clv/generate-rename-variant (thi/id :v01) name)))) + +(defn- redo-ids + [changes change-type] + (into #{} + (comp (filter #(= change-type (:type %))) + (map :id)) + (:redo-changes changes))) + +(t/deftest test-rename-variant-one-vector + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02)) + board (ths/get-shape file :v01) + main1 (ths/get-shape file :m01) + main2 (ths/get-shape file :m02) + comp1 (thc/get-component file :c01) + comp2 (thc/get-component file :c02) + changes (rename-variant-changes file "Group / Button") + file' (thf/apply-changes file changes) + board' (ths/get-shape file' :v01) + main1' (ths/get-shape file' :m01) + main2' (ths/get-shape file' :m02) + comp1' (thc/get-component file' :c01) + comp2' (thc/get-component file' :c02) + restored (thf/apply-undo-changes file' changes)] + + (t/is (= #{(thi/id :v01) (thi/id :m01) (thi/id :m02)} + (redo-ids changes :mod-obj))) + (t/is (= #{(thi/id :c01) (thi/id :c02)} + (redo-ids changes :mod-component))) + (t/is (= "Group / Button" (:name board') (:name main1') (:name main2'))) + (t/is (= "Group" (:path comp1') (:path comp2'))) + (t/is (= "Button" (:name comp1') (:name comp2'))) + (t/is (= "Group / Button" + (cpn/merge-path-item (:path comp1') (:name comp1')) + (cpn/merge-path-item (:path comp2') (:name comp2')))) + (t/is (= (:id board) (:variant-id comp1') (:variant-id comp2'))) + (t/is (= (:variant-name main1) (:variant-name main1'))) + (t/is (= (:variant-name main2) (:variant-name main2'))) + (t/is (= (:variant-properties comp1) (:variant-properties comp1'))) + (t/is (= (:variant-properties comp2) (:variant-properties comp2'))) + (t/is (= (:name board) (:name (ths/get-shape restored :v01)))) + (t/is (= (:name main1) (:name (ths/get-shape restored :m01)))) + (t/is (= (:name main2) (:name (ths/get-shape restored :m02)))) + (t/is (= (:path comp1) (:path (thc/get-component restored :c01)))) + (t/is (= (:name comp1) (:name (thc/get-component restored :c01)))) + (t/is (= (:path comp2) (:path (thc/get-component restored :c02)))) + (t/is (= (:name comp2) (:name (thc/get-component restored :c02)))))) + +(t/deftest test-rename-variant-slash-only-name + (let [file (-> (thf/sample-file :file1) + (thv/add-variant :v01 :c01 :m01 :c02 :m02)) + changes (rename-variant-changes file "///") + file' (thf/apply-changes file changes) + comp1' (thc/get-component file' :c01) + comp2' (thc/get-component file' :c02)] + + (t/is (= "(empty)" + (:name (ths/get-shape file' :v01)) + (:name (ths/get-shape file' :m01)) + (:name (ths/get-shape file' :m02)) + (:name comp1') + (:name comp2'))) + (t/is (= "" (:path comp1') (:path comp2'))) + (t/is (= "(empty)" + (cpn/merge-path-item (:path comp1') (:name comp1')) + (cpn/merge-path-item (:path comp2') (:name comp2')))) + (t/is (some? (:variant-id comp1'))) + (t/is (some? (:variant-id comp2'))))) diff --git a/common/test/common_tests/runner.cljc b/common/test/common_tests/runner.cljc index 884c9a7356..9a774d271d 100644 --- a/common/test/common_tests/runner.cljc +++ b/common/test/common_tests/runner.cljc @@ -22,6 +22,7 @@ [common-tests.files-changes-test] [common-tests.files-migrations-0025-test] [common-tests.files-migrations-0026-test] + [common-tests.files-migrations-0034-test] [common-tests.files-migrations-test] [common-tests.files.comp-processors-test] [common-tests.files.helpers-test] @@ -77,6 +78,7 @@ [common-tests.logic.text-touched-test] [common-tests.logic.token-apply-test] [common-tests.logic.token-test] + [common-tests.logic.variants-reparent-test] [common-tests.logic.variants-switch-test] [common-tests.logic.variants-test] [common-tests.math-test] @@ -129,6 +131,7 @@ 'common-tests.files-changes-test 'common-tests.files-migrations-0025-test 'common-tests.files-migrations-0026-test + 'common-tests.files-migrations-0034-test 'common-tests.files-migrations-test 'common-tests.files.comp-processors-test 'common-tests.files.helpers-test @@ -184,6 +187,7 @@ 'common-tests.logic.text-touched-test 'common-tests.logic.token-apply-test 'common-tests.logic.token-test + 'common-tests.logic.variants-reparent-test 'common-tests.logic.variants-switch-test 'common-tests.logic.variants-test 'common-tests.math-test diff --git a/common/test/common_tests/types/container_test.cljc b/common/test/common_tests/types/container_test.cljc index e00b1fb9c6..9f7a308c8a 100644 --- a/common/test/common_tests/types/container_test.cljc +++ b/common/test/common_tests/types/container_test.cljc @@ -154,3 +154,140 @@ shape-b (make-shape id-b :parent-id id-a) objects (objects-map shape-a shape-b)] (t/is (nil? (ctc/find-component-main objects shape-a false)))))) + +(t/deftest get-variant-boundary-id-recognizes-container-main-and-nested + (let [page-id uuid/zero + container-id (uuid/next) + main-id (uuid/next) + nested-id (uuid/next) + outside-id (uuid/next) + container (make-shape container-id + :parent-id page-id + :type :frame + :is-variant-container true) + main (make-shape main-id + :parent-id container-id + :type :frame + :variant-id container-id) + nested (make-shape nested-id :parent-id main-id) + outside (make-shape outside-id :parent-id page-id) + objects (objects-map container main nested outside)] + (t/is (= main-id (ctc/get-variant-boundary-id objects nested))) + (t/is (= main-id (ctc/get-variant-boundary-id objects main))) + (t/is (= container-id (ctc/get-variant-boundary-id objects container))) + (t/is (nil? (ctc/get-variant-boundary-id objects outside))))) + +(t/deftest crosses-variant-set-boundary? + (let [page-id uuid/zero + container-id (uuid/next) + main-id (uuid/next) + main-2-id (uuid/next) + nested-id (uuid/next) + nested-2-id (uuid/next) + outside-id (uuid/next) + other-set-id (uuid/next) + other-main-id (uuid/next) + other-nest-id (uuid/next) + container (make-shape container-id + :parent-id page-id + :type :frame + :is-variant-container true) + main (make-shape main-id + :parent-id container-id + :type :frame + :variant-id container-id) + main-2 (make-shape main-2-id + :parent-id container-id + :type :frame + :variant-id container-id) + nested (make-shape nested-id :parent-id main-id) + nested-2 (make-shape nested-2-id :parent-id main-2-id) + outside (make-shape outside-id :parent-id page-id) + other-set (make-shape other-set-id + :parent-id page-id + :type :frame + :is-variant-container true) + other-main (make-shape other-main-id + :parent-id other-set-id + :type :frame + :variant-id other-set-id) + other-nested (make-shape other-nest-id :parent-id other-main-id) + objects (objects-map container main main-2 nested nested-2 + outside other-set other-main other-nested)] + (t/testing "nested child + outside crosses" + (t/is (true? (ctc/crosses-variant-set-boundary? + objects [nested outside])))) + (t/testing "main + outside crosses" + (t/is (true? (ctc/crosses-variant-set-boundary? + objects [main outside])))) + (t/testing "container + outside crosses" + (t/is (true? (ctc/crosses-variant-set-boundary? + objects [container outside])))) + (t/testing "nested children of two variants in the same set cross" + (t/is (true? (ctc/crosses-variant-set-boundary? + objects [nested nested-2])))) + (t/testing "nested children from two different sets cross" + (t/is (true? (ctc/crosses-variant-set-boundary? + objects [nested other-nested])))) + (t/testing "two outside shapes do not cross" + (let [outside-2 (make-shape (uuid/next) :parent-id page-id)] + (t/is (false? (ctc/crosses-variant-set-boundary? + (assoc objects (:id outside-2) outside-2) + [outside outside-2]))))) + (t/testing "nested only does not cross" + (t/is (false? (ctc/crosses-variant-set-boundary? objects [nested])))) + (t/testing "main + nested under that main do not cross" + (t/is (false? (ctc/crosses-variant-set-boundary? + objects [main nested])))) + (t/testing "two nested under the same main do not cross" + (let [nested-b (make-shape (uuid/next) :parent-id main-id)] + (t/is (false? (ctc/crosses-variant-set-boundary? + (assoc objects (:id nested-b) nested-b) + [nested nested-b]))))))) + +(t/deftest ids-for-structure-op + (let [page-id uuid/zero + container-id (uuid/next) + main-id (uuid/next) + main-2-id (uuid/next) + nested-id (uuid/next) + nested-2-id (uuid/next) + outside-id (uuid/next) + outside-2-id (uuid/next) + container (make-shape container-id + :parent-id page-id + :type :frame + :is-variant-container true) + main (make-shape main-id + :parent-id container-id + :type :frame + :variant-id container-id) + main-2 (make-shape main-2-id + :parent-id container-id + :type :frame + :variant-id container-id) + nested (make-shape nested-id :parent-id main-id) + nested-2 (make-shape nested-2-id :parent-id main-id) + outside (make-shape outside-id :parent-id page-id) + outside-2 (make-shape outside-2-id :parent-id page-id) + objects (objects-map container main main-2 nested nested-2 + outside outside-2)] + (t/testing "container-only is refused" + (t/is (= [] (vec (ctc/ids-for-structure-op objects [container-id]))))) + (t/testing "nested + outside is refused" + (t/is (= [] (vec (ctc/ids-for-structure-op objects [nested-id outside-id]))))) + (t/testing "main + outside is refused" + (t/is (= [] (vec (ctc/ids-for-structure-op objects [main-id outside-id]))))) + (t/testing "two mains cross and are refused" + (t/is (= [] (vec (ctc/ids-for-structure-op objects [main-id main-2-id]))))) + (t/testing "single main is dropped to empty" + (t/is (= [] (vec (ctc/ids-for-structure-op objects [main-id]))))) + (t/testing "main + nested under that main keeps only the nested" + (t/is (= [nested-id] + (vec (ctc/ids-for-structure-op objects [main-id nested-id]))))) + (t/testing "two nested under one main are both kept" + (t/is (= [nested-id nested-2-id] + (vec (ctc/ids-for-structure-op objects [nested-id nested-2-id]))))) + (t/testing "two ordinary outside shapes are both kept" + (t/is (= [outside-id outside-2-id] + (vec (ctc/ids-for-structure-op objects [outside-id outside-2-id]))))))) diff --git a/docker/images/Dockerfile.backend b/docker/images/Dockerfile.backend index 4618c1ba91..3edb969257 100644 --- a/docker/images/Dockerfile.backend +++ b/docker/images/Dockerfile.backend @@ -1,4 +1,4 @@ -FROM dhi.io/debian-base:trixie-debian13-dev@sha256:cc4e05ff415ee2d83918f2af45c737983209924ef7362c53f00cfcdaa1c94326 AS build +FROM dhi.io/debian-base:trixie-debian13-dev@sha256:af61a3296b0c92f16969f11ce8c2a93722e84c9e5c1bb42ed6afec796be71b04 AS build LABEL maintainer="Penpot " ENV LANG='C.UTF-8' \ @@ -53,7 +53,7 @@ RUN set -eux; \ --output /opt/jre; -FROM dhi.io/debian-base:trixie-debian13-dev@sha256:cc4e05ff415ee2d83918f2af45c737983209924ef7362c53f00cfcdaa1c94326 AS image +FROM dhi.io/debian-base:trixie-debian13-dev@sha256:af61a3296b0c92f16969f11ce8c2a93722e84c9e5c1bb42ed6afec796be71b04 AS image LABEL maintainer="Penpot " ENV LANG='C.UTF-8' \ diff --git a/docker/images/Dockerfile.exporter b/docker/images/Dockerfile.exporter index b427e0bc80..bb6fa216da 100644 --- a/docker/images/Dockerfile.exporter +++ b/docker/images/Dockerfile.exporter @@ -1,4 +1,4 @@ -FROM dhi.io/node:24.22.2-debian13-dev@sha256:53af56f14166cffcf9c4b9ef6f7025d1b0534edddfa1f333c53bd54766f39e5c +FROM dhi.io/node:24.22.2-debian13-dev@sha256:df819cfcb0de8a9e461bb8437d96d6fcfac341e565129e50597338756a88db5b LABEL maintainer="Penpot " ENV LANG=en_US.UTF-8 \ diff --git a/docker/images/Dockerfile.frontend b/docker/images/Dockerfile.frontend index 05939a1982..aba52d5bcc 100644 --- a/docker/images/Dockerfile.frontend +++ b/docker/images/Dockerfile.frontend @@ -1,4 +1,4 @@ -FROM dhi.io/nginx:1.31.8-debian13-dev@sha256:fbff190f9162256975ed225adcc9a294e333897de543fd18be630de1d6c8de17 +FROM dhi.io/nginx:1.31.8-debian13-dev@sha256:f69a0ea1762c51080fe96ce5f12103fb2977231308ae1784acd915bf63a0a1a2 LABEL maintainer="Penpot " ENV DEBIAN_FRONTEND=noninteractive diff --git a/docker/images/Dockerfile.mcp b/docker/images/Dockerfile.mcp index aac0ff2a38..4054a613b0 100644 --- a/docker/images/Dockerfile.mcp +++ b/docker/images/Dockerfile.mcp @@ -1,4 +1,4 @@ -FROM dhi.io/node:24.22.2-debian13-dev@sha256:53af56f14166cffcf9c4b9ef6f7025d1b0534edddfa1f333c53bd54766f39e5c AS build +FROM dhi.io/node:24.22.2-debian13-dev@sha256:df819cfcb0de8a9e461bb8437d96d6fcfac341e565129e50597338756a88db5b AS build LABEL maintainer="Penpot " ENV DEBIAN_FRONTEND=noninteractive @@ -42,7 +42,7 @@ RUN set -eux; \ RUN ./setup -FROM dhi.io/node:24.22.2-debian13@sha256:24fa6258b891ef66e6c3b7ed82daad550b81cdfcd76f6cc64eee4d2478366747 AS image +FROM dhi.io/node:24.22.2-debian13@sha256:ce582f85254ceba6c572a854b8fb4008ced2f5d365c929f4f580360829037184 AS image LABEL maintainer="Penpot " ENV LANG=en_US.UTF-8 \ diff --git a/docker/images/Dockerfile.media-processor b/docker/images/Dockerfile.media-processor index b2e9eb4af2..218fd84ecd 100644 --- a/docker/images/Dockerfile.media-processor +++ b/docker/images/Dockerfile.media-processor @@ -1,5 +1,5 @@ # syntax=docker/dockerfile:1 -FROM dhi.io/node:24.22.2-debian13-dev@sha256:53af56f14166cffcf9c4b9ef6f7025d1b0534edddfa1f333c53bd54766f39e5c +FROM dhi.io/node:24.22.2-debian13-dev@sha256:df819cfcb0de8a9e461bb8437d96d6fcfac341e565129e50597338756a88db5b LABEL maintainer="Penpot " ENV LANG=en_US.UTF-8 \ diff --git a/docker/images/Dockerfile.storybook b/docker/images/Dockerfile.storybook index 4832c60054..bf67fabf88 100644 --- a/docker/images/Dockerfile.storybook +++ b/docker/images/Dockerfile.storybook @@ -1,4 +1,4 @@ -FROM dhi.io/nginx:1.31.8-alpine3.24@sha256:83489ce440fa56cb5149c088dbf75422362e45c532c814100253e5ab71871366 +FROM dhi.io/nginx:1.31.8-alpine3.24@sha256:7a4e9021737f414ad0f851efb939600690d9d092f51712ce6512bfd0a767ceae LABEL maintainer="Penpot " ARG BUNDLE_PATH="./bundle-storybook/" diff --git a/docker/images/docker-compose.yaml b/docker/images/docker-compose.yaml index e667bc3049..932fda6279 100644 --- a/docker/images/docker-compose.yaml +++ b/docker/images/docker-compose.yaml @@ -24,7 +24,7 @@ # WARNING: if you're exposing Penpot to the internet, you should remove the flags # 'disable-secure-session-cookies' and 'disable-email-verification' x-flags: &penpot-flags - PENPOT_FLAGS: disable-email-verification enable-smtp enable-prepl-server disable-secure-session-cookies enable-mcp enable-admin-console + PENPOT_FLAGS: disable-email-verification enable-smtp enable-prepl-server disable-secure-session-cookies enable-mcp x-uri: &penpot-public-uri PENPOT_PUBLIC_URI: http://localhost:9001 diff --git a/docker/images/files/nginx-entrypoint.sh b/docker/images/files/nginx-entrypoint.sh index 3ec97403a8..e76e581f1c 100644 --- a/docker/images/files/nginx-entrypoint.sh +++ b/docker/images/files/nginx-entrypoint.sh @@ -61,7 +61,7 @@ fi envsubst "\$PENPOT_BACKEND_URI,\$PENPOT_EXPORTER_URI,\$PENPOT_HTTP_SERVER_MAX_BODY_SIZE,\$PENPOT_IPV6_LISTEN_DIRECTIVE" \ < /tmp/nginx.conf.template > /etc/nginx/nginx.conf -if [[ $PENPOT_FLAGS == *"enable-admin-console"* ]]; then +if [[ " $PENPOT_FLAGS " != *" disable-admin-console "* ]]; then export PENPOT_ADMIN_CONSOLE_URI=${PENPOT_ADMIN_CONSOLE_URI:-http://penpot-admin-console:3000} envsubst "\$PENPOT_ADMIN_CONSOLE_URI" \ < /tmp/nginx-admin-console-locations.conf.template > /etc/nginx/overrides/server.d/admin-console-locations.conf diff --git a/docs/img/components/components-show-asset.mp4 b/docs/img/components/components-show-asset.mp4 index d1c16a5999..7d6e46057d 100644 Binary files a/docs/img/components/components-show-asset.mp4 and b/docs/img/components/components-show-asset.mp4 differ diff --git a/docs/img/components/components-show-asset.webp b/docs/img/components/components-show-asset.webp index ecc7c34d62..919b76ebd8 100644 Binary files a/docs/img/components/components-show-asset.webp and b/docs/img/components/components-show-asset.webp differ diff --git a/docs/img/components/components-show-main.mp4 b/docs/img/components/components-show-main.mp4 index cba4d46ed4..7150638803 100644 Binary files a/docs/img/components/components-show-main.mp4 and b/docs/img/components/components-show-main.mp4 differ diff --git a/docs/img/components/components-show-main.webp b/docs/img/components/components-show-main.webp index bd36169c9a..ec285b9549 100644 Binary files a/docs/img/components/components-show-main.webp and b/docs/img/components/components-show-main.webp differ diff --git a/docs/img/customfonts-families.png b/docs/img/customfonts-families.png index 9f2bc0a152..b51c8b1b74 100644 Binary files a/docs/img/customfonts-families.png and b/docs/img/customfonts-families.png differ diff --git a/docs/img/customfonts-upload.png b/docs/img/customfonts-upload.png index 301456584c..4780a5d542 100644 Binary files a/docs/img/customfonts-upload.png and b/docs/img/customfonts-upload.png differ diff --git a/docs/img/customfonts-use.gif b/docs/img/customfonts-use.gif index 4416b268f4..0afeff7003 100644 Binary files a/docs/img/customfonts-use.gif and b/docs/img/customfonts-use.gif differ diff --git a/docs/img/customfonts.png b/docs/img/customfonts.png index fb7b868528..188be15c48 100644 Binary files a/docs/img/customfonts.png and b/docs/img/customfonts.png differ diff --git a/docs/img/design-tokens/01-tokens-cover.webp b/docs/img/design-tokens/01-tokens-cover.webp index 9d163897d7..baf8ff08bb 100644 Binary files a/docs/img/design-tokens/01-tokens-cover.webp and b/docs/img/design-tokens/01-tokens-cover.webp differ diff --git a/docs/img/design-tokens/05-tokens-edit.webp b/docs/img/design-tokens/05-tokens-edit.webp index b77b9cf50e..804081c434 100644 Binary files a/docs/img/design-tokens/05-tokens-edit.webp and b/docs/img/design-tokens/05-tokens-edit.webp differ diff --git a/docs/img/design-tokens/06-tokens-radius.webp b/docs/img/design-tokens/06-tokens-radius.webp index a8d479e769..757260148a 100644 Binary files a/docs/img/design-tokens/06-tokens-radius.webp and b/docs/img/design-tokens/06-tokens-radius.webp differ diff --git a/docs/img/design-tokens/08-tokens-color.webp b/docs/img/design-tokens/08-tokens-color.webp index 2556a9a765..3dfbeb0fc9 100644 Binary files a/docs/img/design-tokens/08-tokens-color.webp and b/docs/img/design-tokens/08-tokens-color.webp differ diff --git a/docs/img/design-tokens/09-tokens-dimensions-sizing.webp b/docs/img/design-tokens/09-tokens-dimensions-sizing.webp index 397508cf36..2077b9a769 100644 Binary files a/docs/img/design-tokens/09-tokens-dimensions-sizing.webp and b/docs/img/design-tokens/09-tokens-dimensions-sizing.webp differ diff --git a/docs/img/design-tokens/10-tokens-dimensions-spacing.webp b/docs/img/design-tokens/10-tokens-dimensions-spacing.webp index 38b7b0f279..dd51893e4b 100644 Binary files a/docs/img/design-tokens/10-tokens-dimensions-spacing.webp and b/docs/img/design-tokens/10-tokens-dimensions-spacing.webp differ diff --git a/docs/img/design-tokens/11-tokens-spacing.webp b/docs/img/design-tokens/11-tokens-spacing.webp index 4680497444..9922d5371a 100644 Binary files a/docs/img/design-tokens/11-tokens-spacing.webp and b/docs/img/design-tokens/11-tokens-spacing.webp differ diff --git a/docs/img/export/export-main-menu.webp b/docs/img/export/export-main-menu.webp index 9844752af6..1043a6041d 100644 Binary files a/docs/img/export/export-main-menu.webp and b/docs/img/export/export-main-menu.webp differ diff --git a/docs/img/export/export-pdf.webp b/docs/img/export/export-pdf.webp index d239f8ebb1..82a2268e57 100644 Binary files a/docs/img/export/export-pdf.webp and b/docs/img/export/export-pdf.webp differ diff --git a/docs/img/interface/dashboard-dark.webp b/docs/img/interface/dashboard-dark.webp index c0a3d53cbd..cb5f385861 100644 Binary files a/docs/img/interface/dashboard-dark.webp and b/docs/img/interface/dashboard-dark.webp differ diff --git a/docs/img/interface/dashboard-light.webp b/docs/img/interface/dashboard-light.webp index e04bbbd4b6..5904c1978b 100644 Binary files a/docs/img/interface/dashboard-light.webp and b/docs/img/interface/dashboard-light.webp differ diff --git a/docs/img/interface/workspace-dark.webp b/docs/img/interface/workspace-dark.webp index 5bfdab77ad..1d7de171fc 100644 Binary files a/docs/img/interface/workspace-dark.webp and b/docs/img/interface/workspace-dark.webp differ diff --git a/docs/img/interface/workspace-light.webp b/docs/img/interface/workspace-light.webp index 83606806de..009337d192 100644 Binary files a/docs/img/interface/workspace-light.webp and b/docs/img/interface/workspace-light.webp differ diff --git a/docs/img/interface/youraccount-notifications.webp b/docs/img/interface/youraccount-notifications.webp index 4abb846785..731afc331a 100644 Binary files a/docs/img/interface/youraccount-notifications.webp and b/docs/img/interface/youraccount-notifications.webp differ diff --git a/docs/img/interface/youraccount-password.webp b/docs/img/interface/youraccount-password.webp index 72979791b0..36d2ce783b 100644 Binary files a/docs/img/interface/youraccount-password.webp and b/docs/img/interface/youraccount-password.webp differ diff --git a/docs/img/interface/youraccount-profile.webp b/docs/img/interface/youraccount-profile.webp index 122242b84c..36be1b4d1c 100644 Binary files a/docs/img/interface/youraccount-profile.webp and b/docs/img/interface/youraccount-profile.webp differ diff --git a/docs/img/interface/youraccount-settings.webp b/docs/img/interface/youraccount-settings.webp index 3fb1bbe93b..fa27757504 100644 Binary files a/docs/img/interface/youraccount-settings.webp and b/docs/img/interface/youraccount-settings.webp differ diff --git a/docs/img/layers-select-menu.gif b/docs/img/layers-select-menu.gif index 365c1d9407..2875134e81 100644 Binary files a/docs/img/layers-select-menu.gif and b/docs/img/layers-select-menu.gif differ diff --git a/docs/img/layers/layers-collapse.mp4 b/docs/img/layers/layers-collapse.mp4 index f2bc9ce5ac..e81818c936 100644 Binary files a/docs/img/layers/layers-collapse.mp4 and b/docs/img/layers/layers-collapse.mp4 differ diff --git a/docs/img/layers/layers-collapse.webp b/docs/img/layers/layers-collapse.webp index 77c9ca90cb..87fb540388 100644 Binary files a/docs/img/layers/layers-collapse.webp and b/docs/img/layers/layers-collapse.webp differ diff --git a/docs/img/layers/layers-create.mp4 b/docs/img/layers/layers-create.mp4 index e5b30e933b..911b4e007a 100644 Binary files a/docs/img/layers/layers-create.mp4 and b/docs/img/layers/layers-create.mp4 differ diff --git a/docs/img/layers/layers-create.webp b/docs/img/layers/layers-create.webp index 87ea736358..576e54bec7 100644 Binary files a/docs/img/layers/layers-create.webp and b/docs/img/layers/layers-create.webp differ diff --git a/docs/img/layers/layers-focus.mp4 b/docs/img/layers/layers-focus.mp4 index 1f4db7bb3b..d7efd7aa20 100644 Binary files a/docs/img/layers/layers-focus.mp4 and b/docs/img/layers/layers-focus.mp4 differ diff --git a/docs/img/layers/layers-focus.webp b/docs/img/layers/layers-focus.webp index 084dd35aca..bd11a51754 100644 Binary files a/docs/img/layers/layers-focus.webp and b/docs/img/layers/layers-focus.webp differ diff --git a/docs/img/layers/pages-create.mp4 b/docs/img/layers/pages-create.mp4 index a5d1ffb2b6..0a59c266c8 100644 Binary files a/docs/img/layers/pages-create.mp4 and b/docs/img/layers/pages-create.mp4 differ diff --git a/docs/img/layers/pages-create.webp b/docs/img/layers/pages-create.webp index 1568ca45f9..2b616a527b 100644 Binary files a/docs/img/layers/pages-create.webp and b/docs/img/layers/pages-create.webp differ diff --git a/docs/img/libraries/add-color.mp4 b/docs/img/libraries/add-color.mp4 index f5b0636982..05307681ef 100644 Binary files a/docs/img/libraries/add-color.mp4 and b/docs/img/libraries/add-color.mp4 differ diff --git a/docs/img/libraries/add-color.webp b/docs/img/libraries/add-color.webp index 21f99547e9..c6ddc2daa8 100644 Binary files a/docs/img/libraries/add-color.webp and b/docs/img/libraries/add-color.webp differ diff --git a/docs/img/libraries/add-component.mp4 b/docs/img/libraries/add-component.mp4 index 932f1a0f9f..ff8e7a9553 100644 Binary files a/docs/img/libraries/add-component.mp4 and b/docs/img/libraries/add-component.mp4 differ diff --git a/docs/img/libraries/add-component.webp b/docs/img/libraries/add-component.webp index d36432d6fb..2b8d97db3e 100644 Binary files a/docs/img/libraries/add-component.webp and b/docs/img/libraries/add-component.webp differ diff --git a/docs/img/libraries/asset-options.mp4 b/docs/img/libraries/asset-options.mp4 index 14c92ee7f1..862abb68ab 100644 Binary files a/docs/img/libraries/asset-options.mp4 and b/docs/img/libraries/asset-options.mp4 differ diff --git a/docs/img/libraries/asset-options.webp b/docs/img/libraries/asset-options.webp index e55e1fc55a..8257ba22a5 100644 Binary files a/docs/img/libraries/asset-options.webp and b/docs/img/libraries/asset-options.webp differ diff --git a/docs/img/libraries/asset-use.mp4 b/docs/img/libraries/asset-use.mp4 index b61f4d02f7..7a6a212c67 100644 Binary files a/docs/img/libraries/asset-use.mp4 and b/docs/img/libraries/asset-use.mp4 differ diff --git a/docs/img/libraries/asset-use.webp b/docs/img/libraries/asset-use.webp index 5908ef7367..7867a19c54 100644 Binary files a/docs/img/libraries/asset-use.webp and b/docs/img/libraries/asset-use.webp differ diff --git a/docs/img/libraries/assets-tab.webp b/docs/img/libraries/assets-tab.webp index 5c412d733f..4cb4956984 100644 Binary files a/docs/img/libraries/assets-tab.webp and b/docs/img/libraries/assets-tab.webp differ diff --git a/docs/img/libraries/libraries-launch.mp4 b/docs/img/libraries/libraries-launch.mp4 index 10dbc5b3aa..4977bced2d 100644 Binary files a/docs/img/libraries/libraries-launch.mp4 and b/docs/img/libraries/libraries-launch.mp4 differ diff --git a/docs/img/libraries/libraries-launch.webp b/docs/img/libraries/libraries-launch.webp index 10c15fcdcb..d3ee665636 100644 Binary files a/docs/img/libraries/libraries-launch.webp and b/docs/img/libraries/libraries-launch.webp differ diff --git a/docs/img/libraries/libraries-sidebar.webp b/docs/img/libraries/libraries-sidebar.webp index e4458888e0..5b56efc0e6 100644 Binary files a/docs/img/libraries/libraries-sidebar.webp and b/docs/img/libraries/libraries-sidebar.webp differ diff --git a/docs/img/objects/board-create.webp b/docs/img/objects/board-create.webp index 606c1dcc54..19d94862ac 100644 Binary files a/docs/img/objects/board-create.webp and b/docs/img/objects/board-create.webp differ diff --git a/docs/img/objects/board-creation.mp4 b/docs/img/objects/board-creation.mp4 index 485b7fdb61..e2ee8b0c3f 100644 Binary files a/docs/img/objects/board-creation.mp4 and b/docs/img/objects/board-creation.mp4 differ diff --git a/docs/img/objects/board-tool.webp b/docs/img/objects/board-tool.webp index b86d537318..b4fd227fac 100644 Binary files a/docs/img/objects/board-tool.webp and b/docs/img/objects/board-tool.webp differ diff --git a/docs/img/objects/rectangles-ellipses.mp4 b/docs/img/objects/rectangles-ellipses.mp4 index 3ced3b26ec..19250eac8a 100644 Binary files a/docs/img/objects/rectangles-ellipses.mp4 and b/docs/img/objects/rectangles-ellipses.mp4 differ diff --git a/docs/img/objects/rectangles-ellipses.webp b/docs/img/objects/rectangles-ellipses.webp index 67d2d3496d..72aea55237 100644 Binary files a/docs/img/objects/rectangles-ellipses.webp and b/docs/img/objects/rectangles-ellipses.webp differ diff --git a/docs/img/objects/text-create.mp4 b/docs/img/objects/text-create.mp4 index c798e755c0..d04e1728a2 100644 Binary files a/docs/img/objects/text-create.mp4 and b/docs/img/objects/text-create.mp4 differ diff --git a/docs/img/objects/text-create.webp b/docs/img/objects/text-create.webp index 466f1c507a..b39518d6b2 100644 Binary files a/docs/img/objects/text-create.webp and b/docs/img/objects/text-create.webp differ diff --git a/docs/img/plugins/03_toolbar_plugins.png b/docs/img/plugins/03_toolbar_plugins.png index 3cd5a8d51b..71cf62e971 100644 Binary files a/docs/img/plugins/03_toolbar_plugins.png and b/docs/img/plugins/03_toolbar_plugins.png differ diff --git a/docs/img/plugins/04_menu_plugins.png b/docs/img/plugins/04_menu_plugins.png index d546948d20..880d730e59 100644 Binary files a/docs/img/plugins/04_menu_plugins.png and b/docs/img/plugins/04_menu_plugins.png differ diff --git a/docs/img/plugins/plugins-menu.mp4 b/docs/img/plugins/plugins-menu.mp4 index e0fb360676..52a93af645 100644 Binary files a/docs/img/plugins/plugins-menu.mp4 and b/docs/img/plugins/plugins-menu.mp4 differ diff --git a/docs/img/plugins/plugins-menu.png b/docs/img/plugins/plugins-menu.png index 1230554d3f..472d4fbbe7 100644 Binary files a/docs/img/plugins/plugins-menu.png and b/docs/img/plugins/plugins-menu.png differ diff --git a/docs/img/plugins/plugins-toolbar.mp4 b/docs/img/plugins/plugins-toolbar.mp4 index 0b99b9c081..9412bccaf8 100644 Binary files a/docs/img/plugins/plugins-toolbar.mp4 and b/docs/img/plugins/plugins-toolbar.mp4 differ diff --git a/docs/img/plugins/plugins-toolbar.png b/docs/img/plugins/plugins-toolbar.png index c8212d32df..a230f9c299 100644 Binary files a/docs/img/plugins/plugins-toolbar.png and b/docs/img/plugins/plugins-toolbar.png differ diff --git a/docs/img/styling/color-picker-gradient.webp b/docs/img/styling/color-picker-gradient.webp index eec479e702..50e9af2790 100644 Binary files a/docs/img/styling/color-picker-gradient.webp and b/docs/img/styling/color-picker-gradient.webp differ diff --git a/docs/img/styling/color-picker.webp b/docs/img/styling/color-picker.webp index 6c7e9584e1..a83e98254f 100644 Binary files a/docs/img/styling/color-picker.webp and b/docs/img/styling/color-picker.webp differ diff --git a/docs/img/tech-guide/access-tokens-create-1.webp b/docs/img/tech-guide/access-tokens-create-1.webp index 8aff3a665c..bb6f333c62 100644 Binary files a/docs/img/tech-guide/access-tokens-create-1.webp and b/docs/img/tech-guide/access-tokens-create-1.webp differ diff --git a/docs/img/tech-guide/access-tokens-create-2.webp b/docs/img/tech-guide/access-tokens-create-2.webp index c7176a3d4a..980448fde7 100644 Binary files a/docs/img/tech-guide/access-tokens-create-2.webp and b/docs/img/tech-guide/access-tokens-create-2.webp differ diff --git a/docs/img/tech-guide/access-tokens-create-3.webp b/docs/img/tech-guide/access-tokens-create-3.webp index 2e84e15368..24db9ead20 100644 Binary files a/docs/img/tech-guide/access-tokens-create-3.webp and b/docs/img/tech-guide/access-tokens-create-3.webp differ diff --git a/docs/img/tech-guide/access-tokens.webp b/docs/img/tech-guide/access-tokens.webp index 6a7d616d80..39947de1e7 100644 Binary files a/docs/img/tech-guide/access-tokens.webp and b/docs/img/tech-guide/access-tokens.webp differ diff --git a/docs/img/workspace-basics/guides-snap.mp4 b/docs/img/workspace-basics/guides-snap.mp4 index 447d385aee..3048542b85 100644 Binary files a/docs/img/workspace-basics/guides-snap.mp4 and b/docs/img/workspace-basics/guides-snap.mp4 differ diff --git a/docs/img/workspace-basics/guides-snap.webp b/docs/img/workspace-basics/guides-snap.webp index 821db4edba..13cd1fb908 100644 Binary files a/docs/img/workspace-basics/guides-snap.webp and b/docs/img/workspace-basics/guides-snap.webp differ diff --git a/docs/img/workspace-basics/main-menu.webp b/docs/img/workspace-basics/main-menu.webp index 8f91da0fa9..a308d0067c 100644 Binary files a/docs/img/workspace-basics/main-menu.webp and b/docs/img/workspace-basics/main-menu.webp differ diff --git a/docs/img/workspace-basics/nudge-menu.webp b/docs/img/workspace-basics/nudge-menu.webp index cbae6c7081..2e70a7aaf8 100644 Binary files a/docs/img/workspace-basics/nudge-menu.webp and b/docs/img/workspace-basics/nudge-menu.webp differ diff --git a/docs/img/workspace-basics/ruler-guides.mp4 b/docs/img/workspace-basics/ruler-guides.mp4 index 3e98354266..9eb9362751 100644 Binary files a/docs/img/workspace-basics/ruler-guides.mp4 and b/docs/img/workspace-basics/ruler-guides.mp4 differ diff --git a/docs/img/workspace-basics/ruler-guides.webp b/docs/img/workspace-basics/ruler-guides.webp index 819078c3fc..90e72a0f99 100644 Binary files a/docs/img/workspace-basics/ruler-guides.webp and b/docs/img/workspace-basics/ruler-guides.webp differ diff --git a/docs/img/workspace-basics/viewport.webp b/docs/img/workspace-basics/viewport.webp index 1a89970007..8659546af6 100644 Binary files a/docs/img/workspace-basics/viewport.webp and b/docs/img/workspace-basics/viewport.webp differ diff --git a/docs/img/workspace-basics/workspace-snap.webp b/docs/img/workspace-basics/workspace-snap.webp index 3ea9a225a4..f7b366a9b8 100644 Binary files a/docs/img/workspace-basics/workspace-snap.webp and b/docs/img/workspace-basics/workspace-snap.webp differ diff --git a/docs/img/workspace-basics/workspace-zoom-layer.mp4 b/docs/img/workspace-basics/workspace-zoom-layer.mp4 index c8eee212d9..b9eed42016 100644 Binary files a/docs/img/workspace-basics/workspace-zoom-layer.mp4 and b/docs/img/workspace-basics/workspace-zoom-layer.mp4 differ diff --git a/docs/img/workspace-basics/workspace-zoom-layer.webp b/docs/img/workspace-basics/workspace-zoom-layer.webp index fe5b5114b5..86c59660dd 100644 Binary files a/docs/img/workspace-basics/workspace-zoom-layer.webp and b/docs/img/workspace-basics/workspace-zoom-layer.webp differ diff --git a/docs/technical-guide/getting-started/docker.md b/docs/technical-guide/getting-started/docker.md index 4d192aee11..3d204f6a60 100644 --- a/docs/technical-guide/getting-started/docker.md +++ b/docs/technical-guide/getting-started/docker.md @@ -122,6 +122,13 @@ This will fetch the latest images. When you do docke It is strongly recommended to update the Penpot version in small increments, rather than updating between two distant versions.

+#### Upgrade to 2.19 + +The Admin Console is now enabled by default, so the `enable-admin-console` flag is no longer +needed and can be removed from `PENPOT_FLAGS`. The `penpot-admin-console` service and the +`PENPOT_ADMIN_CONSOLE_URI` variable are now required: if you haven't set them up yet, follow +steps 2–4 of the [2.18 upgrade](#upgrade-to-2.18). + #### Upgrade to 2.18 This version deploys a new service, **`penpot-admin-console`**, in the official `docker-compose.yaml` @@ -132,10 +139,9 @@ example. If you maintain your own compose file, you need to replicate the follow **NOTE:** Enabling the Admin Console is not required in this version, but it will be in a future release. We recommend setting it up now to avoid a more complex upgrade later. -This step (the `enable-admin-console` flag) only applies to this version 2.18. In a future release, the -Admin Console will be enabled by default and this flag will be removed. If you're upgrading directly to a -version where the flag has already been removed, skip step 1 and go straight to steps 2–4, which remain -required. +This step (the `enable-admin-console` flag) only applies to this version 2.18. Since 2.19 the Admin +Console is enabled by default: if you're upgrading directly to 2.19 or later, skip step 1 and go straight +to steps 2–4, which remain required. Wherever you set `PENPOT_FLAGS` (frontend and backend): diff --git a/docs/technical-guide/integration.md b/docs/technical-guide/integration.md index 2d8d1c5a0a..9bd3aad3ba 100644 --- a/docs/technical-guide/integration.md +++ b/docs/technical-guide/integration.md @@ -67,13 +67,13 @@ Personal access tokens function like an alternative to our login/password authen

Important: Treat your access tokens like passwords as they provide access to our account.

### Manage access tokens -In Penpot, access tokens are configured at user account level. To manage your access tokens, go to Your account > Access tokens. +In Penpot, access tokens are configured at user account level. To manage your access tokens, go to Your account > Integrations. You'll find them in the Personal access tokens section. ![Access tokens](/img/tech-guide/access-tokens.webp) ### Generate access tokens -1. Press the "Generate new token" button. +1. Press the "Create new access token" button. ![Creating token](/img/tech-guide/access-tokens-create-1.webp) @@ -83,7 +83,7 @@ In Penpot, access tokens are configured at user account level. To manage your ac ![Token expiration](/img/tech-guide/access-tokens-create-2.webp) -4. Once you're happy with the name and the expiration date, press "Create token". At this step you will be able to copy the token to your clipboard. +4. Once you're happy with the name and the expiration date, press "Create access token". At this step you will be able to copy the token to your clipboard. Keep it somewhere safe: once you close the dialog you will not be able to see it again. ![Token created](/img/tech-guide/access-tokens-create-3.webp) diff --git a/docs/user-guide/account-teams/your-account.njk b/docs/user-guide/account-teams/your-account.njk index 3267ac5886..f738f82361 100644 --- a/docs/user-guide/account-teams/your-account.njk +++ b/docs/user-guide/account-teams/your-account.njk @@ -30,6 +30,19 @@ desc: Learn how to import and export files in Penpot, the free, open-source desi

After several failed login attempts in a short time, your account can be locked for some minutes. Penpot shows how long to wait. Resetting your password unlocks it.

On self-hosted installations, the administrator sets these limits. See Session expiration and Account lockout in the Technical Guide.

+

If you lose access to your account + # +

+

If you can no longer sign in to your Penpot cloud account, for example because your Google account was suspended or you lost access to your email, write to support@penpot.app. Once we confirm that the account is yours, we can give you access to its files from an account you can use.

+

Include in your message:

+
    +
  • The email of the account you can no longer access.
    • +
  • The email you can use now, if you already have another Penpot account.
    • +
  • Anything that helps us confirm the account is yours: names of projects, files or boards, file links, or screenshots of your designs.
    • +
+

To avoid losing work, export your important files from time to time and keep the .penpot files somewhere safe.

+

On self-hosted installations, contact the administrator of your instance.

+

Notifications #

@@ -76,6 +89,6 @@ desc: Learn how to import and export files in Penpot, the free, open-source desi

Access tokens #

-

At the Asset tokens section you can manage your access tokens. Read more about access tokens here.

+

At the Integrations section, under Personal access tokens, you can manage your access tokens. Read more about access tokens here.

diff --git a/docs/user-guide/design-systems/libraries.njk b/docs/user-guide/design-systems/libraries.njk index d4b66e692e..d31debde9c 100644 --- a/docs/user-guide/design-systems/libraries.njk +++ b/docs/user-guide/design-systems/libraries.njk @@ -24,7 +24,7 @@ desc: Use Penpot's libraries for reusable design elements! Learn to create, mana

There are two ways to publish a library:

  • Using the file main menu.
    • -
  • From the libraries panel, that you can launch by clicking on the "Libraries" button that is found at the assets tab.
    • +
  • From the libraries panel, that you can launch from the button at the top of the assets tab. It reads "Add library" when the file has no connected libraries and "Manage library" when it has.
Publish library diff --git a/docs/user-guide/designing/workspace-basics.njk b/docs/user-guide/designing/workspace-basics.njk index bcd23eb546..1f00bb725b 100644 --- a/docs/user-guide/designing/workspace-basics.njk +++ b/docs/user-guide/designing/workspace-basics.njk @@ -25,7 +25,7 @@ desc: Master Penpot's workspace basics! Learn interface navigation, zoom tools,

The menu

-

There's a main menu at the workspace where you will find groups with all the actions that you can do at file level. File, View, Edit, Preferences and Help.

+

There's a main menu at the workspace where you will find groups with all the actions that you can do at file level. File, Edit, View, Preferences, Plugins and Help & info. If the MCP server is enabled, you'll also find an MCP server entry.

Main menu
diff --git a/docs/user-guide/first-steps/the-interface.njk b/docs/user-guide/first-steps/the-interface.njk index c670c0f6c6..81e469c92f 100644 --- a/docs/user-guide/first-steps/the-interface.njk +++ b/docs/user-guide/first-steps/the-interface.njk @@ -131,8 +131,8 @@ desc: Discover Penpot's free user guide! Learn the interface, workspace basics,
  • Search: If you are looking for a specific file just type its title at the search box.
  • Projects: A project allows you to group design files. It works pretty much like a folder in a file system. You can create as many projects as you need. If you are going to work with more people in a project, you should create it inside a team.
  • Drafts: The drafts section is where you can find the design files that are not inside any project.
    • -
  • Shared Libraries: In this section you will find all the design files that have been added as shared libraries. That way you will be able to better control the files that are sharing their assets.
    • -
  • Custom fonts: If you have purchased or own personal fonts that are not included in the catalog provided by Penpot, you can upload them from your computer and use them across the files of a team.
    • +
  • Libraries: In this section you will find all the design files that have been added as shared libraries. That way you will be able to better control the files that are sharing their assets.
    • +
  • Fonts: If you have purchased or own personal fonts that are not included in the catalog provided by Penpot, you can upload them from your computer and use them across the files of a team.
  • Pinned projects: If you want to keep some projects handy (for instance because you’re currently working on them) you can pin them to make them quickly available at the sidebar.
  • User area: This must be you! Access your profile settings, Penpot tutorials, the Penpot Community and more. You can also find here a way to leave us feedback. We’d love to read your thoughts :).
  • Comments notifications: Here you will be able to see if you have unread comments inside the files of the team. There's also a button to mark all notifications as read.
    • diff --git a/frontend/playwright/ui/pages/BasePage.js b/frontend/playwright/ui/pages/BasePage.js index 670e8a2766..f8f246058d 100644 --- a/frontend/playwright/ui/pages/BasePage.js +++ b/frontend/playwright/ui/pages/BasePage.js @@ -101,7 +101,11 @@ export class BasePage { static async mockConfigFlags(page, flags, extras = {}) { const url = "**/js/config.js*"; - const config = { penpotFlags: flags.join(" "), ...extras }; + // The e2e suite mocks the backend and has no admin console behind it. + const config = { + penpotFlags: ["disable-admin-console", ...flags].join(" "), + ...extras, + }; const body = Object.entries(config) .map(([key, value]) => `var ${key} = ${JSON.stringify(value)};`) .join("\n"); diff --git a/frontend/playwright/ui/visual-specs/workspace.spec.js b/frontend/playwright/ui/visual-specs/workspace.spec.js index df766736a0..a134ecf245 100644 --- a/frontend/playwright/ui/visual-specs/workspace.spec.js +++ b/frontend/playwright/ui/visual-specs/workspace.spec.js @@ -164,3 +164,25 @@ test.describe("Palette", () => { ).toBeVisible(); }); }); + +test.describe("Shortcuts panel", () => { + const shot = { maxDiffPixelRatio: 0, threshold: 0 }; + + test("Wraps long key lists between combinations", async ({ page }) => { + const workspace = new WorkspacePage(page); + await workspace.setupEmptyFile(); + await workspace.goToWorkspace(); + + await page.keyboard.press("Shift+?"); + const list = page.locator("[class*='shortcuts-list']"); + await expect(list).toBeVisible(); + await page.getByRole("textbox").first().fill("board"); + + const next = page.getByRole("listitem", { name: "Next board" }); + const prev = page.getByRole("listitem", { name: "Previous board" }); + await prev.scrollIntoViewIfNeeded(); + await page.mouse.move(640, 300); + await expect(next).toHaveScreenshot(shot); + await expect(prev).toHaveScreenshot(shot); + }); +}); diff --git a/frontend/src/app/main/data/workspace/groups.cljs b/frontend/src/app/main/data/workspace/groups.cljs index f60ae89ba8..53175a61f2 100644 --- a/frontend/src/app/main/data/workspace/groups.cljs +++ b/frontend/src/app/main/data/workspace/groups.cljs @@ -30,6 +30,14 @@ reverse (map #(get objects %)))) +(defn- shapes-for-group-or-mask + "Shapes allowed for Group and Mask after the shared variant gates. + + `ids-for-structure-op` drops mains before `clean-loops` so a selected + variant main does not swallow nested children." + [objects ids] + (shapes-for-grouping objects (ctn/ids-for-structure-op objects ids))) + (defn- get-empty-groups-after-group-creation "An auxiliary function that finds and returns a set of ids that corresponds to groups that should be deleted after a group creation. @@ -188,18 +196,11 @@ (ptk/reify ::group-shapes ptk/WatchEvent (watch [it state _] - (let [id (d/nilv id (uuid/next)) - page-id (:current-page-id state) - objects (dsh/lookup-page-objects state page-id) - - shapes - (->> ids - (cfh/clean-loops objects) - (remove #(ctn/has-any-copy-parent? objects (get objects %))) - (shapes-for-grouping objects) - (remove ctk/is-variant?)) - - parents (into #{} (map :parent-id) shapes)] + (let [id (d/nilv id (uuid/next)) + page-id (:current-page-id state) + objects (dsh/lookup-page-objects state page-id) + shapes (shapes-for-group-or-mask objects ids) + parents (into #{} (map :parent-id) shapes)] (when-not (empty? shapes) (let [[group changes] (prepare-create-group (pcb/empty-changes it) id objects page-id shapes "Group" false)] @@ -213,7 +214,9 @@ (ptk/reify ::group-selected ptk/WatchEvent (watch [_ state _] - (let [selected (dsh/lookup-selected state)] + ;; Raw ids: lookup-selected already clean-loops, which would drop + ;; nested children of a selected variant main before we can. + (let [selected (dsh/get-selected-ids state)] (rx/of (group-shapes nil selected :change-selection? true)))))) (defn ungroup-shapes @@ -287,10 +290,9 @@ (watch [it state _] (let [page-id (:current-page-id state) objects (dsh/lookup-page-objects state page-id) - selected (->> (or ids (dsh/lookup-selected state)) - (cfh/clean-loops objects) - (remove #(ctn/has-any-copy-parent? objects (get objects %)))) - shapes (shapes-for-grouping objects selected) + shapes (shapes-for-group-or-mask + objects + (or ids (dsh/get-selected-ids state))) first-shape (first shapes)] (when-not (empty? shapes) (let [;; If the selected shape is a group, we can use it. If not, diff --git a/frontend/src/app/main/data/workspace/libraries.cljs b/frontend/src/app/main/data/workspace/libraries.cljs index bc52e63736..8d2aec1597 100644 --- a/frontend/src/app/main/data/workspace/libraries.cljs +++ b/frontend/src/app/main/data/workspace/libraries.cljs @@ -1057,67 +1057,123 @@ (cfh/get-children-with-self (:objects page) (:id root)) (cfh/get-parents-with-self objects (:parent-id shape)))))) +(def component-swap-layout-ids-type + "Companion DataEvent type emitted by `component-swap` when `:skip-layout?` + is set. Batch callers filter this exact keyword (libraries ns) — never a + bare `::component-swap-layout-ids` in another namespace." + ::component-swap-layout-ids) + +(def ^:private component-swap-batch-done-type + ::component-swap-batch-done) + +(defn coalesce-skipped-layout + "While `swap-source` runs, collect companion layout-id DataEvents from + `stream`, then emit one `:layout/update` with the union. + + `swap-source` must emit WatchEvents already configured with + `:skip-layout? true`. Uses a done marker (not `(rx/take (count …))`) so + no-op swaps cannot hang the fold. Reads the buffer inside `rx/defer` so + the id set is not frozen empty when `watch` returns." + [stream swap-source] + (let [layout-acc (volatile! [])] + (rx/concat + (rx/merge + (->> stream + (rx/take-until (rx/filter (ptk/type? component-swap-batch-done-type) stream)) + (rx/filter (ptk/type? component-swap-layout-ids-type)) + (rx/tap (fn [ev] + (vswap! layout-acc into (:ids (deref ev))))) + (rx/ignore)) + (rx/concat + swap-source + (rx/of (ptk/data-event component-swap-batch-done-type {})))) + (rx/defer + #(rx/of (ptk/data-event :layout/update {:ids @layout-acc})))))) + (defn component-swap - "Swaps a component with another one" - [shape file-id id-new-component keep-touched?] - (dm/assert! (uuid? id-new-component)) - (dm/assert! (uuid? file-id)) - (ptk/reify ::component-swap - ptk/WatchEvent - (watch [it state _] - ;; First delete shapes so we have space in the layout otherwise we can have problems - ;; in the grid creating new rows/columns to make space - (let [libraries (dsh/lookup-libraries state) - page (dsh/lookup-page state) - objects (:objects page) - parent (get objects (:parent-id shape)) + "Swaps a component with another one. - ldata (dsh/lookup-file-data state file-id) - orig-shapes (when keep-touched? (cfh/get-children-with-self objects (:id shape))) + Optional `opts`: + - `:skip-select?` — do not emit `select-shape` after the swap (batch callers + restore selection once). + - `:skip-layout?` — do not emit `:layout/update`; emit a companion + `component-swap-layout-ids-type` DataEvent instead so the batch can + coalesce one layout update." + ([shape file-id id-new-component keep-touched?] + (component-swap shape file-id id-new-component keep-touched? nil)) + ([shape file-id id-new-component keep-touched? + {:keys [skip-select? skip-layout?] :or {skip-select? false skip-layout? false}}] + (dm/assert! (uuid? id-new-component)) + (dm/assert! (uuid? file-id)) + (ptk/reify ::component-swap + ptk/WatchEvent + (watch [it state _] + ;; First delete shapes so we have space in the layout otherwise we can have problems + ;; in the grid creating new rows/columns to make space + (let [libraries (dsh/lookup-libraries state) + page (dsh/lookup-page state) + objects (:objects page)] + ;; Re-read from live state: sibling swaps / layout can move this head. + ;; Gone from objects → no-op (do not swap a closed-over stale shape). + (if-let [shape (get objects (:id shape))] + (let [parent (get objects (:parent-id shape)) - ;; If the target parent is a grid layout we need to pass the target cell - target-cell (when (ctsl/grid-layout? parent) - (ctsl/get-cell-by-shape-id parent (:id shape))) + ldata (dsh/lookup-file-data state file-id) + orig-shapes (when keep-touched? (cfh/get-children-with-self objects (:id shape))) - index (find-shape-index objects (:parent-id shape) (:id shape)) + ;; If the target parent is a grid layout we need to pass the target cell + target-cell (when (ctsl/grid-layout? parent) + (ctsl/get-cell-by-shape-id parent (:id shape))) - ;; Store the properties that need to be maintained when the component is swapped - keep-props-values (select-keys shape ctk/swap-keep-attrs) + index (find-shape-index objects (:parent-id shape) (:id shape)) - undo-id (js/Symbol) - undo-group (uuid/next) + ;; Store the properties that need to be maintained when the component is swapped + keep-props-values (select-keys shape ctk/swap-keep-attrs) - [new-shape all-parents changes] - (-> (pcb/empty-changes it (:id page)) - (pcb/set-undo-group undo-group) - (cll/generate-component-swap objects shape ldata page libraries id-new-component - index target-cell keep-props-values keep-touched?)) + undo-id (js/Symbol) + undo-group (uuid/next) - updated-objects (pcb/get-objects changes) - new-children-ids (cfh/get-children-ids-with-self updated-objects (:id new-shape)) - new-text-ids (->> new-children-ids - (keep (fn [id] - (when-let [child (get updated-objects id)] - (when (and (cfh/text-shape? child) - (not= :fixed (:grow-type child))) - id)))) - (vec)) + [new-shape all-parents changes] + (-> (pcb/empty-changes it (:id page)) + (pcb/set-undo-group undo-group) + (cll/generate-component-swap objects shape ldata page libraries id-new-component + index target-cell keep-props-values keep-touched?)) - [changes parents-of-swapped] - (if keep-touched? - (clv/generate-keep-touched changes new-shape shape orig-shapes page libraries ldata) - [changes []]) - update-layout-ids (concat all-parents parents-of-swapped new-children-ids)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwtp/propagate-workspace-tokens) ;; Make the new instance get the token values from the current file, not from the component's library - (when (and (features/active-feature? state "render-wasm/v1") - (seq new-text-ids)) - (dwwt/resize-wasm-text-all new-text-ids {:skip-component-sync? true})) - (ptk/data-event :layout/update {:ids update-layout-ids :undo-group undo-group}) - (dwu/commit-undo-transaction undo-id) - (dws/select-shape (:id new-shape) false)))))) + updated-objects (pcb/get-objects changes) + new-children-ids (cfh/get-children-ids-with-self updated-objects (:id new-shape)) + new-text-ids (->> new-children-ids + (keep (fn [id] + (when-let [child (get updated-objects id)] + (when (and (cfh/text-shape? child) + (not= :fixed (:grow-type child))) + id)))) + (vec)) + + [changes parents-of-swapped] + (if keep-touched? + (clv/generate-keep-touched changes new-shape shape orig-shapes page libraries ldata) + [changes []]) + update-layout-ids (concat all-parents parents-of-swapped new-children-ids)] + (cond-> (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + ;; Make the new instance get the token values from the current file, + ;; not from the component's library + (dwtp/propagate-workspace-tokens) + (when (and (features/active-feature? state "render-wasm/v1") + (seq new-text-ids)) + (dwwt/resize-wasm-text-all new-text-ids {:skip-component-sync? true}))) + (not skip-layout?) + (rx/concat (rx/of (ptk/data-event :layout/update + {:ids update-layout-ids :undo-group undo-group}))) + skip-layout? + (rx/concat (rx/of (ptk/data-event component-swap-layout-ids-type + {:ids update-layout-ids}))) + true + (rx/concat (rx/of (dwu/commit-undo-transaction undo-id))) + (not skip-select?) + (rx/concat (rx/of (dws/select-shape (:id new-shape) false))))) + (rx/empty))))))) (defn component-multi-swap "Swaps several components with another one" @@ -1131,17 +1187,21 @@ {::ev/name "component-swap"}) ptk/WatchEvent - (watch [_ state _] - (let [undo-id (js/Symbol)] + (watch [_ state stream] + (let [undo-id (js/Symbol) + swap-opts {:skip-select? true :skip-layout? true}] (log/info :msg "COMPONENT-SWAP" :file (debug-pretty-file file-id state) :id-new-component id-new-component :undo-id undo-id) (rx/concat (rx/of (dwu/start-undo-transaction undo-id)) - (rx/map #(component-swap % file-id id-new-component false) (rx/from shapes)) - (rx/of (dwu/commit-undo-transaction undo-id)) - (rx/of (dwsp/open-specialized-panel :component-swap))))))) + (coalesce-skipped-layout + stream + (->> (rx/from shapes) + (rx/map #(component-swap % file-id id-new-component false swap-opts)))) + (rx/of (dwu/commit-undo-transaction undo-id) + (dwsp/open-specialized-panel :component-swap))))))) (def valid-asset-types #{:colors :components :typographies}) diff --git a/frontend/src/app/main/data/workspace/shape_layout.cljs b/frontend/src/app/main/data/workspace/shape_layout.cljs index df614dfe7a..784036857f 100644 --- a/frontend/src/app/main/data/workspace/shape_layout.cljs +++ b/frontend/src/app/main/data/workspace/shape_layout.cljs @@ -17,6 +17,7 @@ [app.common.logic.libraries :as cll] [app.common.types.color :as clr] [app.common.types.component :as ctc] + [app.common.types.container :as ctn] [app.common.types.modifiers :as ctm] [app.common.types.shape.layout :as ctl] [app.common.uuid :as uuid] @@ -216,13 +217,15 @@ has-component? (some true? (map ctc/instance-root? selected-shapes)) is-component? (and single? has-component?) has-variant? (some ctc/is-variant? selected-shapes) + has-container? (some ctc/is-variant-container? selected-shapes) + crosses-set? (ctn/crosses-variant-set-boundary? objects selected-shapes) has-layout? (and single? (ctl/any-layout? (first selected-shapes))) new-shape-id (uuid/next) undo-id (js/Symbol)] - (if has-variant? + (if (or has-variant? has-container? crosses-set?) (rx/empty) (rx/concat (rx/of (dwu/start-undo-transaction undo-id)) @@ -294,12 +297,17 @@ undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (if (and single? is-frame? (not has-layout?)) - (create-layout-from-id (first selected) type :from-frame? true) - (create-layout-from-selection type)) - (dwu/commit-undo-transaction undo-id)))))) + ;; Refuse before undo: container or cross-boundary. Leave single + ;; mains on the from-id branch (selection refuses has-variant?). + (if (or (some ctc/is-variant-container? selected-shapes) + (ctn/crosses-variant-set-boundary? objects selected-shapes)) + (rx/empty) + (rx/of + (dwu/start-undo-transaction undo-id) + (if (and single? is-frame? (not has-layout?)) + (create-layout-from-id (first selected) type :from-frame? true) + (create-layout-from-selection type)) + (dwu/commit-undo-transaction undo-id))))))) (defn toggle-layout [type] diff --git a/frontend/src/app/main/data/workspace/shapes.cljs b/frontend/src/app/main/data/workspace/shapes.cljs index 447a192a7d..491f7a2c2d 100644 --- a/frontend/src/app/main/data/workspace/shapes.cljs +++ b/frontend/src/app/main/data/workspace/shapes.cljs @@ -336,10 +336,11 @@ (remove #(dm/get-in objects [% :blocked])) (cfh/order-by-indexed-shapes objects)) + data (dsh/lookup-file-data state) changes (-> (pcb/empty-changes it page-id) - (pcb/with-objects objects)) - - changes (cfsh/prepare-move-shapes-into-frame changes frame-id shapes objects true)] + (pcb/with-objects objects) + (pcb/with-library-data data) + (cls/generate-move-shapes-into-frame frame-id shapes objects true))] (if (some? changes) (rx/of (dch/commit-changes changes)) @@ -475,16 +476,14 @@ (ptk/reify ::create-artboard-from-selection ptk/WatchEvent (watch [_ state _] - (let [page-id (:current-page-id state) - objects (dsh/lookup-page-objects state page-id) - selected (->> (dsh/lookup-selected state) - (cfh/clean-loops objects) - (remove #(ctn/has-any-copy-parent? objects (get objects %))) - (remove #(->> % - (get objects) - (ctc/is-variant?))))] + (let [page-id (:current-page-id state) + objects (dsh/lookup-page-objects state page-id) + selected (vec (ctn/ids-for-structure-op + objects + (dsh/get-selected-ids state)))] - (rx/of (create-artboard-from-shapes selected id parent-id index name delta))))))) + (when (seq selected) + (rx/of (create-artboard-from-shapes selected id parent-id index name delta)))))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; Shape Flags diff --git a/frontend/src/app/main/data/workspace/variants.cljs b/frontend/src/app/main/data/workspace/variants.cljs index 09f6003a9d..46730ee112 100644 --- a/frontend/src/app/main/data/workspace/variants.cljs +++ b/frontend/src/app/main/data/workspace/variants.cljs @@ -38,6 +38,34 @@ [beicon.v2.core :as rx] [potok.v2.core :as ptk])) +(defn- get-variant-mains-page + "Return the page that holds the variant mains. + + Pass `:component-id` or `:variant-id`. Finds nothing when that id has + no main page in the file." + [data {:keys [component-id variant-id]}] + (let [page-id (cond + (some? component-id) + (:main-instance-page (ctkl/get-component data component-id)) + + (some? variant-id) + (-> (cfv/find-variant-components data variant-id) + first + :main-instance-page)) + objects (when page-id + (dm/get-in data [:pages-index page-id :objects]))] + (when (and page-id objects) + {:page-id page-id + :objects objects}))) + +(defn- with-main-page + "Run `f` with the page id and objects that hold the component/variant mains. + Returns an empty stream when the file cannot resolve them." + [data opts f] + (if-let [{:keys [page-id objects]} (get-variant-mains-page data opts)] + (f page-id objects) + (rx/empty))) + (defn update-properties-names-and-values "Compares the previous properties with the updated ones and executes the correspondent action for each one depending on if it needs to be removed, updated or added" @@ -49,53 +77,51 @@ ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:component-id component-id} + (fn [page-id objects] + (let [updated-properties (ctv/update-number-in-repeated-prop-names updated-properties) - updated-properties (ctv/update-number-in-repeated-prop-names updated-properties) + properties-to-remove (ctv/find-properties-to-remove previous-properties updated-properties) + properties-to-add (ctv/find-properties-to-add previous-properties updated-properties) + properties-to-update (ctv/find-properties-to-update previous-properties updated-properties) - properties-to-remove (ctv/find-properties-to-remove previous-properties updated-properties) - properties-to-add (ctv/find-properties-to-add previous-properties updated-properties) - properties-to-update (ctv/find-properties-to-update previous-properties updated-properties) + changes (-> (pcb/empty-changes it page-id) + (pcb/with-objects objects) + (pcb/with-library-data data)) - changes (-> (pcb/empty-changes it page-id) - (pcb/with-objects objects) - (pcb/with-library-data data)) + changes (reduce + (fn [changes {:keys [name]}] + (-> changes + (clvp/generate-update-property-value component-id (ctv/find-index-for-property-name previous-properties name) ""))) + changes + properties-to-remove) - changes (reduce - (fn [changes {:keys [name]}] - (-> changes - (clvp/generate-update-property-value component-id (ctv/find-index-for-property-name previous-properties name) ""))) - changes - properties-to-remove) + changes (reduce + (fn [changes {:keys [name value]}] + (-> changes + (clvp/generate-update-property-value component-id (ctv/find-index-for-property-name previous-properties name) value))) + changes + properties-to-update) - changes (reduce - (fn [changes {:keys [name value]}] - (-> changes - (clvp/generate-update-property-value component-id (ctv/find-index-for-property-name previous-properties name) value))) - changes - properties-to-update) + changes (reduce + (fn [changes [idx {:keys [name value]}]] + (-> changes + (clvp/generate-add-new-property variant-id {:property-name name}) + (clvp/generate-update-property-value component-id (+ idx (count previous-properties)) value))) + changes + (map-indexed vector properties-to-add)) - changes (reduce - (fn [changes [idx {:keys [name value]}]] - (-> changes - (clvp/generate-add-new-property variant-id {:property-name name}) - (clvp/generate-update-property-value component-id (+ idx (count previous-properties)) value))) - changes - (map-indexed vector properties-to-add)) + undo-id (js/Symbol)] - undo-id (js/Symbol)] - - (rx/of - (when (or (seq properties-to-remove) (seq properties-to-update)) - (ev/event {::ev/name "variant-edit-property-value" ::ev/origin "workspace:rename-in-layers"})) - (when (seq properties-to-add) - (ev/event {::ev/name "variant-add-property" ::ev/origin "workspace:rename-in-layers"})) - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) + (rx/of + (when (or (seq properties-to-remove) (seq properties-to-update)) + (ev/event {::ev/name "variant-edit-property-value" ::ev/origin "workspace:rename-in-layers"})) + (when (seq properties-to-add) + (ev/event {::ev/name "variant-add-property" ::ev/origin "workspace:rename-in-layers"})) + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) (defn update-property-name "Update the variant property name on the position pos @@ -105,44 +131,39 @@ ptk/UpdateEvent (update [_ state] (let [file-id (:current-file-id state) - data (dsh/lookup-file-data state) - objects (dsh/lookup-page-objects state) - - related-components (cfv/find-variant-components data objects variant-id)] - - (reduce - (fn [s related-component] - (update-in s - [:files file-id :data :components (:id related-component) :variant-properties] - (fn [props] (mapv #(with-meta % nil) props)))) - state - related-components))) + data (dsh/lookup-file-data state)] + (if-let [{:keys [objects]} (get-variant-mains-page data {:variant-id variant-id})] + (let [related-components (cfv/find-variant-components data objects variant-id)] + (reduce + (fn [s related-component] + (update-in s + [:files file-id :data :components (:id related-component) :variant-properties] + (fn [props] (mapv #(with-meta % nil) props)))) + state + related-components)) + state))) ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - - related-components (cfv/find-variant-components data objects variant-id) - - props (-> related-components last :variant-properties) - valid-pos? (> (count props) pos) - prop-name (when valid-pos? (-> props (nth pos) :name)) - - changes (when valid-pos? - (-> (pcb/empty-changes it page-id) - (pcb/with-objects objects) - (pcb/with-library-data data) - (clvp/generate-update-property-name variant-id pos new-name))) - undo-id (js/Symbol)] - (when (and valid-pos? (not= prop-name new-name)) - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id) - (ev/event {::ev/name "variant-edit-property-name" ::ev/origin trigger}))))))) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [related-components (cfv/find-variant-components data objects variant-id) + props (-> related-components last :variant-properties) + valid-pos? (> (count props) pos) + prop-name (when valid-pos? (-> props (nth pos) :name)) + changes (when valid-pos? + (-> (pcb/empty-changes it page-id) + (pcb/with-objects objects) + (pcb/with-library-data data) + (clvp/generate-update-property-name variant-id pos new-name))) + undo-id (js/Symbol)] + (when (and valid-pos? (not= prop-name new-name)) + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id) + (ev/event {::ev/name "variant-edit-property-name" ::ev/origin trigger})))))))))) (defn update-property-value "Updates the variant property value on the position pos in a component" @@ -150,20 +171,18 @@ (ptk/reify ::update-property-value ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects) - (clvp/generate-update-property-value component-id pos value)) - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:component-id component-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clvp/generate-update-property-value component-id pos value)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) (defn update-error "Sets or unsets an error for a component" @@ -173,20 +192,18 @@ (ptk/reify ::update-error ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects) - (clvp/generate-set-variant-error component-id value)) - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id))))))) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:component-id component-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clvp/generate-set-variant-error component-id value)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id)))))))))) (defn remove-property "Remove the variant property on the position pos @@ -195,21 +212,18 @@ (ptk/reify ::remove-property ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects) - (clvp/generate-remove-property variant-id pos)) - - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clvp/generate-remove-property variant-id pos)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) (defn remove-empty-properties "Remove every empty property for all components when their respective values are empty @@ -218,45 +232,39 @@ (ptk/reify ::remove-empty-properties ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [variant-components (cfv/find-variant-components data objects variant-id) + properties-empty-pos (->> variant-components + (mapcat :variant-properties) + (group-by :name) + (map-indexed + (fn [i [_ v]] + [i (->> v + (map :value) + (remove empty?) + empty?)])) + (reverse)) + changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects)) + changes (reduce + (fn [changes [pos property-empty?]] + (if property-empty? + (-> changes + (clvp/generate-remove-property variant-id pos)) + changes)) + changes + properties-empty-pos) + undo-id (js/Symbol)] - variant-components (cfv/find-variant-components data objects variant-id) - - properties-empty-pos (->> variant-components - (mapcat :variant-properties) - (group-by :name) - (map-indexed - (fn [i [_ v]] - [i (->> v - (map :value) - (remove empty?) - empty?)])) - (reverse)) - - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects)) - - changes (reduce - (fn [changes [pos property-empty?]] - (if property-empty? - (-> changes - (clvp/generate-remove-property variant-id pos)) - changes)) - changes - properties-empty-pos) - - undo-id (js/Symbol)] - - (when (seq (:redo-changes changes)) - (rx/of - (ev/event {::ev/name "variant-remove-property" ::ev/origin "workspace:rename-in-layers"}) - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id))))))) + (when (seq (:redo-changes changes)) + (rx/of + (ev/event {::ev/name "variant-remove-property" ::ev/origin "workspace:rename-in-layers"}) + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id)))))))))) (defn add-new-property "Add a new variant property to all the components with this variant-id" @@ -264,21 +272,18 @@ (ptk/reify ::add-new-property ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects) - (clvp/generate-add-new-property variant-id options)) - - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clvp/generate-add-new-property variant-id options)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) (defn reorder-variant-poperties "Reorder properties by moving a property from some position to some space between positions" @@ -286,45 +291,45 @@ (ptk/reify ::reorder-variant-properties ptk/WatchEvent (watch [it state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clvp/generate-reorder-variant-poperties variant-id from-pos to-space-between-pos)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) - changes (-> (pcb/empty-changes it page-id) - (pcb/with-library-data data) - (pcb/with-objects objects) - (clvp/generate-reorder-variant-poperties variant-id from-pos to-space-between-pos)) - - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) - -(defn- set-variant-id - "Sets the variant-id on a component" - [component-id variant-id] - (ptk/reify ::set-variant-id +(defn- define-variant-container + "Read page objects after the artboard exists, then commit the defining + variant fields in one change." + [variant-id main-id component-id name] + (ptk/reify ::define-variant-container ptk/WatchEvent (watch [it state _] (let [page-id (:current-page-id state) data (dsh/lookup-file-data state) + objects (dsh/lookup-page-objects state page-id) changes (-> (pcb/empty-changes it page-id) + (pcb/with-objects objects) (pcb/with-library-data data) - (pcb/update-component component-id #(assoc % :variant-id variant-id))) - undo-id (js/Symbol)] - (rx/of - (dwu/start-undo-transaction undo-id) - (dch/commit-changes changes) - (dwu/commit-undo-transaction undo-id)))))) + (clv/generate-define-variant-container + variant-id main-id component-id name))] + (rx/of (dch/commit-changes changes)))))) (defn- focus-property [variant-id] (ptk/reify ::focus-property ptk/EffectEvent (effect [_ _ _] - (dom/focus! (dom/get-element (str "variant-prop-" variant-id "-0")))))) + ;; Unit tests run without a DOM; skip rather than throw into the store. + (when (exists? js/document) + (some-> (dom/get-element (str "variant-prop-" variant-id "-0")) + (dom/focus!)))))) (defn- resposition-and-resize-variant "Resize the variant container, and move the shape (that is a variant) to the right" @@ -395,6 +400,92 @@ (->> (rx/of (focus-property (:id variant-container))) (rx/delay 250)))))))))) +(defn- transform-in-variant-stream + "Event stream that turns `main-instance-id` into a variant set. + The defining fields share one commit (`define-variant-container`). + Combine concatenates this stream before relocating the other mains. + `page-id` is the page that holds the main; nil means the open page." + [state main-instance-id variant-id delta prefix add-wrapper? duplicate? flex? page-id] + (let [variant-id (or variant-id (uuid/next)) + variant-vec [variant-id] + file-id (:current-file-id state) + page-id (or page-id (:current-page-id state)) + objects (dsh/lookup-page-objects state file-id page-id) + main (get objects main-instance-id) + parent (get objects (:parent-id main)) + component-id (:component-id main) + name (if add-wrapper? + (str "Component/" (:name main)) + (:name main)) + ;; If there is a prefix, set is as first item of path + cpath (-> name + cpn/split-path + (cond-> + (seq prefix) + (->> (drop (count prefix)) + (cons (cpn/join-path prefix)) + vec))) + + name (first cpath) + num-props (max 1 (dec (count cpath))) + style-props (cond-> {:r1 20 + :r2 20 + :r3 20 + :r4 20 + :layout-item-absolute true} + flex? + (into {:layout-item-h-sizing :auto + :layout-item-v-sizing :auto + :layout-padding {:p1 30 :p2 30 :p3 30 :p4 30} + :layout-gap {:row-gap 0 :column-gap 20}})) + + stroke-props {:stroke-alignment :inner + :stroke-style :solid + :stroke-color "#bb97d8" ;; todo use color var? + :stroke-opacity 1 + :stroke-width 2} + + ;; Move the position of the variant container so the main shape doesn't + ;; change its position + delta (or delta + (if (ctsl/any-layout? parent) + (gpt/point 0 0) + (gpt/point -30 -30))) + undo-id (js/Symbol)] + + (rx/concat + (rx/of + (dwu/start-undo-transaction undo-id) + + ;; Create the frame while it is still a normal board + (dwsh/create-artboard-from-shapes [main-instance-id] variant-id nil nil nil delta flex?) + (cl/remove-all-fills variant-vec {:color clr/black :opacity 1}) + (when flex? (dwsl/create-layout-from-id variant-id :flex)) + (dwsh/update-shapes variant-vec #(merge % style-props)) + (cl/add-stroke variant-vec stroke-props)) + + (rx/of (define-variant-container variant-id main-instance-id component-id name)) + + ;; First property is in the defining commit + (when (> num-props 1) + (rx/from + (repeatedly (dec num-props) + #(add-new-property variant-id {:fill-values? true})))) + + ;; When the component has path, set the path items as properties values + (when (> (count cpath) 1) + (rx/from + (map + #(update-property-value component-id % (nth cpath (inc %))) + (range num-props)))) + + (rx/of + (when duplicate? (add-new-variant main-instance-id)) + (dwsh/update-shapes variant-vec #(dissoc % :layout-item-absolute)) + (dwu/commit-undo-transaction undo-id) + (when flex? + (ptk/data-event :layout/update {:ids [variant-id]})))))) + (defn transform-in-variant "Given the id of a main shape of a component, creates a variant structure for that component" @@ -404,98 +495,7 @@ (ptk/reify ::transform-in-variant ptk/WatchEvent (watch [_ state _] - (let [variant-id (or variant-id (uuid/next)) - variant-vec [variant-id] - file-id (:current-file-id state) - page-id (:current-page-id state) - objects (dsh/lookup-page-objects state file-id page-id) - main (get objects main-instance-id) - parent (get objects (:parent-id main)) - component-id (:component-id main) - name (if add-wrapper? - (str "Component/" (:name main)) - (:name main)) - ;; If there is a prefix, set is as first item of path - cpath (-> name - cpn/split-path - (cond-> - (seq prefix) - (->> (drop (count prefix)) - (cons (cpn/join-path prefix)) - vec))) - - name (first cpath) - num-props (max 1 (dec (count cpath))) - base-props {:is-variant-container true - :name name - :r1 20 - :r2 20 - :r3 20 - :r4 20 - :layout-item-absolute true} - flex-props {:layout-item-h-sizing :auto - :layout-item-v-sizing :auto - :layout-padding {:p1 30 :p2 30 :p3 30 :p4 30} - :layout-gap {:row-gap 0 :column-gap 20}} - cont-props (if flex? - (into base-props flex-props) - base-props) - main-props {:name name - :variant-id variant-id} - - stroke-props {:stroke-alignment :inner - :stroke-style :solid - :stroke-color "#bb97d8" ;; todo use color var? - :stroke-opacity 1 - :stroke-width 2} - - ;; Move the position of the variant container so the main shape doesn't - ;; change its position - delta (or delta - (if (ctsl/any-layout? parent) - (gpt/point 0 0) - (gpt/point -30 -30))) - undo-id (js/Symbol)] - - - ;;TODO Refactor all called methods in order to be able to - ;;generate changes instead of call the events - - - (rx/concat - (rx/of - (dwu/start-undo-transaction undo-id) - - (when (not= name (:name main)) - (dwl/rename-component component-id name)) - - ;; Create variant container - (dwsh/create-artboard-from-shapes [main-instance-id] variant-id nil nil nil delta flex?) - (cl/remove-all-fills variant-vec {:color clr/black :opacity 1}) - (when flex? (dwsl/create-layout-from-id variant-id :flex)) - (dwsh/update-shapes variant-vec #(merge % cont-props)) - (dwsh/update-shapes [main-instance-id] #(merge % main-props)) - (cl/add-stroke variant-vec stroke-props) - (set-variant-id component-id variant-id)) - - ;; Add the necessary number of new properties, with default values - (rx/from - (repeatedly num-props - #(add-new-property variant-id {:fill-values? true}))) - - ;; When the component has path, set the path items as properties values - (when (> (count cpath) 1) - (rx/from - (map - #(update-property-value component-id % (nth cpath (inc %))) - (range num-props)))) - - (rx/of - (when duplicate? (add-new-variant main-instance-id)) - (dwsh/update-shapes variant-vec #(dissoc % :layout-item-absolute)) - (dwu/commit-undo-transaction undo-id) - (when flex? - (ptk/data-event :layout/update {:ids [variant-id]}))))))))) + (transform-in-variant-stream state main-instance-id variant-id delta prefix add-wrapper? duplicate? flex? nil))))) (defn add-component-or-variant "Manage the shared shortcut, and do the pertinent action" @@ -565,22 +565,19 @@ (ptk/reify ::rename-variant ptk/WatchEvent - (watch [_ state _] - (let [page-id (:current-page-id state) - data (dsh/lookup-file-data state) - objects (-> (dsh/get-page data page-id) - (get :objects)) - variant-components (cfv/find-variant-components data objects variant-id) - clean-name (cpn/clean-path name) - undo-id (js/Symbol)] - - (rx/concat - (rx/of (dwu/start-undo-transaction undo-id) - (dwsh/update-shapes [variant-id] #(assoc % :name clean-name))) - (rx/from (map - #(dwl/rename-component-and-main-instance (:id %) clean-name) - variant-components)) - (rx/of (dwu/commit-undo-transaction undo-id))))))) + (watch [it state _] + (let [data (dsh/lookup-file-data state)] + (with-main-page data {:variant-id variant-id} + (fn [page-id objects] + (let [changes (-> (pcb/empty-changes it page-id) + (pcb/with-library-data data) + (pcb/with-objects objects) + (clv/generate-rename-variant variant-id name)) + undo-id (js/Symbol)] + (rx/of + (dwu/start-undo-transaction undo-id) + (dch/commit-changes changes) + (dwu/commit-undo-transaction undo-id))))))))) (defn rename-comp-or-variant-and-main "If the component is in a variant, rename the variant. @@ -696,9 +693,11 @@ (rx/of (dcm/go-to-workspace :page-id page-id)) (rx/empty)) - (rx/of (dwu/start-undo-transaction undo-id) - (transform-in-variant (first ids) variant-id delta prefix add-wrapper? false false) - (dwsh/relocate-shapes (into #{} (rest ids)) variant-id 0) + (rx/of (dwu/start-undo-transaction undo-id)) + + (transform-in-variant-stream state (first ids) variant-id delta prefix add-wrapper? false false current-page) + + (rx/of (dwsh/relocate-shapes (into #{} (rest ids)) variant-id 0) ;; relocate-shapes inserts the shapes in layer-tree ;; order, so restore the intended order afterwards (order-variant-children variant-id ids) @@ -781,60 +780,72 @@ (rx/of (combine-as-variants selected options)))))) (defn- variant-switch - "Switch the shape (that must be a variant copy head) for the closest one with the property value passed as parameter" - [shape {:keys [pos val] :as params}] + "Switch the shape (that must be a variant copy head) for the closest one with the property value passed as parameter. + + `params` may include `:swap-opts` forwarded to `component-swap` (e.g. + `{:skip-select? true :skip-layout? true}` for a multi batch)." + [shape {:keys [pos val swap-opts] :as params}] (ptk/reify ::variant-switch ptk/WatchEvent (watch [it state _] - (let [libraries (dsh/lookup-libraries state) - component-id (:component-id shape) - component (ctf/get-component libraries (:component-file shape) component-id :include-deleted? false)] - ;; If the value is already val, do nothing - (when (not= val (dm/get-in component [:variant-properties pos :value])) - (let [current-page-objects (dsh/lookup-page-objects state) - variant-id (:variant-id component) - component-file-data (dm/get-in libraries [(:component-file shape) :data]) - component-page-objects (-> (dsh/get-page component-file-data (:main-instance-page component)) - (get :objects)) - variant-comps (cfv/find-variant-components component-file-data component-page-objects variant-id) - target-props (-> (:variant-properties component) - (update pos assoc :value val)) - valid-comps (->> variant-comps - (remove #(= (:id %) component-id)) - (filter #(= (dm/get-in % [:variant-properties pos :value]) val)) - (reverse)) - nearest-comp (apply min-key #(ctv/distance target-props (:variant-properties %)) valid-comps) - shape-parents (cfh/get-parents-with-self current-page-objects (:parent-id shape)) - nearest-comp-children (cfh/get-children-with-self component-page-objects (:main-instance-id nearest-comp)) - comps-nesting-loop? (seq? (cfh/components-nesting-loop? nearest-comp-children shape-parents)) + (let [objects (dsh/lookup-page-objects state)] + ;; Sibling swaps / layout can move this head before our turn. + ;; Gone from objects → no-op (do not switch a closed-over stale shape). + (if-let [shape (get objects (:id shape))] + (let [libraries (dsh/lookup-libraries state) + component-id (:component-id shape) + component (ctf/get-component libraries (:component-file shape) component-id :include-deleted? false)] + ;; If the value is already val, do nothing + (if (not= val (dm/get-in component [:variant-properties pos :value])) + (let [variant-id (:variant-id component) + component-file-data (dm/get-in libraries [(:component-file shape) :data]) + component-page-objects (-> (dsh/get-page component-file-data (:main-instance-page component)) + (get :objects)) + variant-comps (cfv/find-variant-components component-file-data component-page-objects variant-id) + target-props (-> (:variant-properties component) + (update pos assoc :value val)) + valid-comps (->> variant-comps + (remove #(= (:id %) component-id)) + (filter #(= (dm/get-in % [:variant-properties pos :value]) val)) + (reverse)) + nearest-comp (apply min-key #(ctv/distance target-props (:variant-properties %)) valid-comps) + shape-parents (cfh/get-parents-with-self objects (:parent-id shape)) + nearest-comp-children (cfh/get-children-with-self component-page-objects (:main-instance-id nearest-comp)) + comps-nesting-loop? (seq? (cfh/components-nesting-loop? nearest-comp-children shape-parents)) - {:keys [on-error] - :or {on-error rx/throw}} (meta params)] + {:keys [on-error] + :or {on-error rx/throw}} (meta params)] - ;; If there is no nearest-comp, do nothing - (when nearest-comp - (if comps-nesting-loop? - (do - (on-error) - (rx/empty)) - (rx/of - (dwl/component-swap shape (:component-file shape) (:id nearest-comp) true) - (ev/event (-> {::ev/name "variant-switch" ::ev/origin "workspace:design-tab"} - (merge (meta it))))))))))))) + ;; If there is no nearest-comp, do nothing + (if nearest-comp + (if comps-nesting-loop? + (do + (on-error) + (rx/empty)) + (rx/of + (dwl/component-swap shape (:component-file shape) (:id nearest-comp) true swap-opts) + (ev/event (-> {::ev/name "variant-switch" ::ev/origin "workspace:design-tab"} + (merge (meta it)))))) + (rx/empty))) + (rx/empty))) + (rx/empty)))))) (defn variants-switch "Switch each shape (that must be a variant copy head) for the closest one with the property value passed as parameter" [{:keys [shapes] :as params}] (ptk/reify ::variants-switch ptk/WatchEvent - (watch [it _ _] - (let [ids (into (d/ordered-set) d/xf:map-id shapes) - undo-id (js/Symbol)] + (watch [it _ stream] + (let [ids (into (d/ordered-set) d/xf:map-id shapes) + undo-id (js/Symbol) + swap-opts {:skip-select? true :skip-layout? true}] (rx/concat (rx/of (dwu/start-undo-transaction undo-id)) - (->> (rx/from shapes) - (rx/map (fn [data] - (-> (variant-switch data params) - (with-meta (meta it)))))) + (dwl/coalesce-skipped-layout + stream + (->> (rx/from shapes) + (rx/map (fn [shape] + (-> (variant-switch shape (assoc params :swap-opts swap-opts)) + (with-meta (meta it))))))) (rx/of (dwu/commit-undo-transaction undo-id) (dws/select-shapes ids))))))) diff --git a/frontend/src/app/main/ui/shortcuts.cljs b/frontend/src/app/main/ui/shortcuts.cljs index 4166314a37..64df9a10bf 100644 --- a/frontend/src/app/main/ui/shortcuts.cljs +++ b/frontend/src/app/main/ui/shortcuts.cljs @@ -539,39 +539,31 @@ (mf/defc shortcuts-keys* [{:keys [content command is-customized has-conflict? light-shortcut]}] - (let [managed-list (if (coll? content) - content - (conj () content)) - chars-list (map ds/split-sc managed-list) - last-element (last chars-list) - short-char-list (if (= 1 (count chars-list)) - chars-list - (drop-last chars-list)) - penultimate (last short-char-list)] + (let [managed-list (if (coll? content) + content + (conj () content)) + chars-list (map ds/split-sc managed-list) + last-index (dec (count chars-list)) + key-class (cond + has-conflict? (stl/css :conflict-key) + is-customized (stl/css :customized-key) + light-shortcut (stl/css :light-key))] + ;; Each combination is a group that never breaks; the list wraps + ;; between groups. The separator stays inside a group so no line + ;; starts with a comma. [:span {:class (stl/css :keys)} - (for [chars short-char-list] - [:* {:key (str/join chars)} + (for [[index chars] (d/enumerate chars-list)] + [:span {:key (dm/str index "-" (str/join chars)) + :class (stl/css :key-group)} + (when (and (pos? index) (= index last-index)) + [:span {:class (stl/css :space)} (tr "shortcuts.or")]) (for [char chars] [:> converted-chars* {:key (dm/str char "-" (name command)) :char char :command command - :class (cond - has-conflict? (stl/css :conflict-key) - is-customized (stl/css :customized-key) - light-shortcut (stl/css :light-key))}]) - - (when (not= chars penultimate) [:span {:class (stl/css :space)} ","])]) - (when (not= last-element penultimate) - [:* - [:span {:class (stl/css :space)} (tr "shortcuts.or")] - (for [char last-element] - [:> converted-chars* {:key (dm/str char "-" (name command)) - :char char - :command command - :class (cond - has-conflict? (stl/css :conflict-key) - is-customized (stl/css :customized-key) - light-shortcut (stl/css :light-key))}])])])) + :class key-class}]) + (when (< index (dec last-index)) + [:span {:class (stl/css :space)} ","])])])) (mf/defc shortcut-row-editable* [{:keys [elements custom-shortcuts command-translate conflicts section-key]}] @@ -863,7 +855,7 @@ :data-conflict (str has-conflict?) :aria-label command-translate :key command-translate} - [:span + [:span {:class (stl/css :command-label)} [:span {:class (stl/css-case :command-name true :not-customizable-label (not customizable?)) :id (dm/str command-translate "-label")} diff --git a/frontend/src/app/main/ui/shortcuts.scss b/frontend/src/app/main/ui/shortcuts.scss index 11eef8a482..dea29e9aba 100644 --- a/frontend/src/app/main/ui/shortcuts.scss +++ b/frontend/src/app/main/ui/shortcuts.scss @@ -4,12 +4,10 @@ // // Copyright (c) KALEIDOS SUBSIDIARY SL -@use "refactor/common-refactor.scss" as deprecated; @use "ds/_borders.scss" as *; @use "ds/_sizes.scss" as *; @use "ds/_utils.scss" as *; -@use "ds/mixins.scss" as *; -@use "ds/typography.scss" as t; +@use "ds/typography.scss" as *; .section { margin: 0; @@ -21,7 +19,7 @@ .section-title, .subsection-title { - @include t.use-typography("headline-small"); + @include use-typography("headline-small"); display: flex; align-items: center; @@ -60,6 +58,7 @@ display: flex; align-items: center; justify-content: space-between; + column-gap: var(--sp-s); inline-size: 100%; min-block-size: $sz-32; padding: px2rem(6); @@ -69,7 +68,7 @@ } .command-name { - @include t.use-typography("body-small"); + @include use-typography("body-small"); margin-inline-start: var(--sp-xxs); color: var(--color-foreground-primary); @@ -78,6 +77,16 @@ text-align: start; } +.command-label { + flex: 0 1 auto; +} + +// The keys take the space the label leaves and wrap between groups. +.shortcuts-name .shortcut-actions { + flex: 1 1 0; + justify-content: flex-end; +} + .not-customizable-label { padding-inline-start: px2rem(6); } @@ -163,13 +172,20 @@ .keys { display: flex; - justify-content: center; + flex-wrap: wrap; + justify-content: flex-end; + align-items: center; + gap: var(--sp-xxs); +} + +.key-group { + display: flex; align-items: center; gap: var(--sp-xxs); } .key { - @include t.use-typography("body-small"); + @include use-typography("body-small"); display: flex; justify-content: center; @@ -203,7 +219,7 @@ } .shortcut-placeholder { - @include t.use-typography("body-small"); + @include use-typography("body-small"); color: var(--color-foreground-secondary); } diff --git a/frontend/src/app/main/ui/workspace/context_menu.cljs b/frontend/src/app/main/ui/workspace/context_menu.cljs index 7cb59b75e5..6a11dfd988 100644 --- a/frontend/src/app/main/ui/workspace/context_menu.cljs +++ b/frontend/src/app/main/ui/workspace/context_menu.cljs @@ -398,6 +398,8 @@ objects (deref refs/workspace-page-objects) any-in-copy? (some #(ctn/has-any-copy-parent? objects %) shapes) any-is-variant? (some ctk/is-variant? shapes) + any-is-container? (some ctk/is-variant-container? shapes) + crosses-set? (ctn/crosses-variant-set-boundary? objects shapes) ;; components can't be ungrouped has-frame? (->> shapes (d/seek #(and (cfh/frame-shape? %) (not (ctk/instance-head? %)) (not (ctk/is-variant-container? %))))) @@ -416,7 +418,7 @@ #(st/emit! (dwsh/create-artboard-from-selection))] [:* - (when (not (or any-in-copy? any-is-variant?)) + (when (not (or any-in-copy? any-is-variant? any-is-container? crosses-set?)) [:* (when (or has-bool? has-group? has-mask? has-frame?) [:> menu-entry* {:title (tr "workspace.shape.menu.ungroup") @@ -621,6 +623,10 @@ (and single? (every? ctl/grid-layout? shapes)) any-is-variant? (some ctk/is-variant? shapes) + ;; shape-context-menu* already omits this menu for container-only; + ;; gate here too so a future caller cannot show Add flex/grid. + any-is-container? (some ctk/is-variant-container? shapes) + crosses-set? (ctn/crosses-variant-set-boundary? objects shapes) on-add-layout (mf/use-fn @@ -653,7 +659,9 @@ :shortcut-key :toggle-layout-grid :on-click on-remove-layout}])] - (when (or single? (not any-is-variant?)) + (when (and (not crosses-set?) + (not any-is-container?) + (or single? (not any-is-variant?))) [:div [:> menu-separator* {}] [:> menu-entry* {:title (tr "workspace.shape.menu.add-flex") diff --git a/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs b/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs index 770a1ebec7..f7e55cd8be 100644 --- a/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs +++ b/frontend/src/app/main/ui/workspace/shapes/text/editor.cljs @@ -256,7 +256,10 @@ :on-blur on-blur :on-focus on-focus :handle-return handle-return - :strip-pasted-styles true + ;; Do not set strip-pasted-styles. When true, draft-js never calls + ;; handlePastedText, so caret/block styles are not applied on paste. + ;; handle-pasted-text always returns "handled", so pasted HTML/styles + ;; are still stripped. :handle-pasted-text handle-pasted-text :handle-drop handle-drop :custom-style-fn (partial styles-fn shape) diff --git a/frontend/src/app/plugins/flex.cljs b/frontend/src/app/plugins/flex.cljs index 57bf9f1953..0b5f9243b2 100644 --- a/frontend/src/app/plugins/flex.cljs +++ b/frontend/src/app/plugins/flex.cljs @@ -358,6 +358,9 @@ (not (r/check-permission plugin-id "content:write")) (u/not-valid plugin-id :appendChild "Plugin doesn't have 'content:write' permission") + (u/non-main-into-variant-container? shape child) + (u/not-valid plugin-id :appendChild "Only main instances can be added to a variant container") + :else (st/emit! (dwsh/relocate-shapes #{child-id} id index) diff --git a/frontend/src/app/plugins/grid.cljs b/frontend/src/app/plugins/grid.cljs index f15980bf2a..8860aa5d58 100644 --- a/frontend/src/app/plugins/grid.cljs +++ b/frontend/src/app/plugins/grid.cljs @@ -574,6 +574,9 @@ (u/changes-component-copy-structure? objects shape child-shape) (u/not-valid plugin-id :appendChild "Cannot change the structure of a component copy") + (u/non-main-into-variant-container? shape child) + (u/not-valid plugin-id :appendChild "Only main instances can be added to a variant container") + :else (st/emit! (dwt/move-shapes-to-frame #{child-id} id nil [row column]) (ptk/data-event :layout/update {:ids [id]}))))))) diff --git a/frontend/src/app/plugins/shape.cljs b/frontend/src/app/plugins/shape.cljs index aea764dc28..408cf84b25 100644 --- a/frontend/src/app/plugins/shape.cljs +++ b/frontend/src/app/plugins/shape.cljs @@ -1263,6 +1263,9 @@ (u/changes-component-copy-structure? objects shape child-shape) (u/not-valid plugin-id :appendChild "Cannot change the structure of a component copy") + (u/non-main-into-variant-container? shape child) + (u/not-valid plugin-id :appendChild "Only main instances can be added to a variant container") + :else (st/emit! (dwsh/relocate-shapes #{child-id} id index) @@ -1302,6 +1305,9 @@ (u/changes-component-copy-structure? objects shape child-shape) (u/not-valid plugin-id :insertChild "Cannot change the structure of a component copy") + (u/non-main-into-variant-container? shape child) + (u/not-valid plugin-id :insertChild "Only main instances can be added to a variant container") + :else (st/emit! (dwsh/relocate-shapes #{child-id} id index) diff --git a/frontend/src/app/plugins/utils.cljs b/frontend/src/app/plugins/utils.cljs index 22f57d45fd..581a41afe2 100644 --- a/frontend/src/app/plugins/utils.cljs +++ b/frontend/src/app/plugins/utils.cljs @@ -604,3 +604,10 @@ [p] (when-let [shape (proxy->shape p)] (ctk/main-instance? shape))) + +(defn non-main-into-variant-container? + "True when `parent` is a variant container and `child` (a shape proxy) + is not a main instance." + [parent child] + (and (ctk/is-variant-container? parent) + (not (is-main-component-proxy? child)))) diff --git a/frontend/test/frontend_tests/data/workspace_variant_rename_test.cljs b/frontend/test/frontend_tests/data/workspace_variant_rename_test.cljs index 5e177b1b21..038d0ba112 100644 --- a/frontend/test/frontend_tests/data/workspace_variant_rename_test.cljs +++ b/frontend/test/frontend_tests/data/workspace_variant_rename_test.cljs @@ -6,6 +6,7 @@ (ns frontend-tests.data.workspace-variant-rename-test (:require + [app.common.path-names :as cpn] [app.common.test-helpers.components :as cthc] [app.common.test-helpers.files :as cthf] [app.common.test-helpers.ids-map :as cthi] @@ -13,9 +14,13 @@ [app.common.test-helpers.variants :as thv] [app.common.types.file :as ctf] [app.common.types.pages-list :as ctpl] + [app.main.data.changes :as dch] [app.main.data.workspace :as dw] + [app.main.data.workspace.variants :as dwv] + [beicon.v2.core :as rx] [cljs.test :as t :include-macros true] - [frontend-tests.helpers.state :as ths])) + [frontend-tests.helpers.state :as ths] + [potok.v2.core :as ptk])) (t/use-fixtures :each {:before cthi/reset-idmap!}) @@ -28,6 +33,15 @@ :variant2-params {:name "Button"}) (cths/update-shape :variant :name "Button"))) +(defn- variant-file-on-other-page + [] + (-> (cthf/sample-file :file1 :page-label :page1) + (thv/add-variant :variant :comp1 :root1 :comp2 :root2 + :variant1-params {:name "Button"} + :variant2-params {:name "Button"}) + (cths/update-shape :variant :name "Button") + (cthf/add-sample-page :page2))) + (defn- without-variant-name [file label] (let [page-id (cthf/current-page-id file) @@ -47,16 +61,22 @@ "Run workspace events, then validate the file before assertions. Pass `:validate? false` only when the fixture is intentionally incomplete - (e.g. a variant main without `:variant-name`)." - [file events done assert-fn & {:keys [validate?] :or {validate? true}}] - (ths/run-store - (ths/setup-store file) - done - events - (fn [state] - (when validate? - (cthf/validate-file! (ths/get-file-from-state state))) - (assert-fn state)))) + (e.g. a variant main without `:variant-name`). Pass `:commits` as an atom + to collect each `dch/commit?` event from the store input." + [file events done assert-fn & {:keys [validate? commits] :or {validate? true}}] + (let [store (ths/setup-store file)] + (when commits + (->> (ptk/input-stream store) + (rx/filter dch/commit?) + (rx/subs! #(swap! commits conj %)))) + (ths/run-store + store + done + events + (fn [state] + (when validate? + (cthf/validate-file! (ths/get-file-from-state state))) + (assert-fn state))))) (t/deftest end-rename-shape-does-not-apply-variant-formula (t/async @@ -230,3 +250,61 @@ (t/is (= "Renamed" (:name (cthc/get-component file :comp)))) (t/is (= "Box" (:name (cths/get-shape file :rect)))) (t/is (nil? (:shape-for-rename (:workspace-local state)))))))))) + +(t/deftest end-rename-shape-renames-variant-container-in-one-commit + (t/async + done + (let [commits (atom [])] + (run (variant-file) + [(dw/end-rename-shape (cthi/id :variant) "Renamed set")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + comp1 (cthc/get-component file :comp1) + comp2 (cthc/get-component file :comp2)] + (t/is (= 1 (count @commits))) + (cthf/validate-file! file) + (t/is (= "Renamed set" (:name (cths/get-shape file :variant)))) + (t/is (= "Renamed set" (:name (cths/get-shape file :root1)))) + (t/is (= "Renamed set" (:name (cths/get-shape file :root2)))) + (t/is (= "Renamed set" + (cpn/merge-path-item (:path comp1) (:name comp1)) + (cpn/merge-path-item (:path comp2) (:name comp2)))))) + :commits commits)))) + +(t/deftest rename-variant-from-another-page-keeps-the-open-page + (t/async + done + (let [file (variant-file-on-other-page) + open-page-id (cthf/current-page-id file)] + (run file + [(dwv/rename-variant (cthi/id :variant) "Renamed")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + comp1 (cthc/get-component file :comp1) + comp2 (cthc/get-component file :comp2)] + (t/is (= open-page-id (:current-page-id state))) + (t/is (= "Renamed" (:name (cths/get-shape file :variant :page-label :page1)))) + (t/is (= "Renamed" (:name (cths/get-shape file :root1 :page-label :page1)))) + (t/is (= "Renamed" (:name (cths/get-shape file :root2 :page-label :page1)))) + (t/is (= "Renamed" + (cpn/merge-path-item (:path comp1) (:name comp1)) + (cpn/merge-path-item (:path comp2) (:name comp2)))))))))) + +(t/deftest rename-variant-stores-empty-for-a-slash-only-name + (t/async + done + (run (variant-file) + [(dwv/rename-variant (cthi/id :variant) "///")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + comp1 (cthc/get-component file :comp1) + comp2 (cthc/get-component file :comp2)] + (t/is (= "(empty)" (:name (cths/get-shape file :variant)))) + (t/is (= "(empty)" (:name (cths/get-shape file :root1)))) + (t/is (= "(empty)" (:name (cths/get-shape file :root2)))) + (t/is (= "" (:path comp1) (:path comp2))) + (t/is (= "(empty)" (:name comp1) (:name comp2))) + (t/is (= (cthi/id :variant) (:variant-id comp1) (:variant-id comp2)))))))) diff --git a/frontend/test/frontend_tests/data/workspace_variants_test.cljs b/frontend/test/frontend_tests/data/workspace_variants_test.cljs new file mode 100644 index 0000000000..20c2fa7acf --- /dev/null +++ b/frontend/test/frontend_tests/data/workspace_variants_test.cljs @@ -0,0 +1,558 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns frontend-tests.data.workspace-variants-test + (:require + [app.common.files.changes-builder :as pcb] + [app.common.files.variant :as cfv] + [app.common.logic.variants :as clv] + [app.common.path-names :as cpn] + [app.common.test-helpers.components :as cthc] + [app.common.test-helpers.compositions :as tho] + [app.common.test-helpers.files :as cthf] + [app.common.test-helpers.ids-map :as cthi] + [app.common.test-helpers.shapes :as cths] + [app.common.test-helpers.variants :as thv] + [app.common.types.component :as ctk] + [app.common.types.variant :as ctv] + [app.common.uuid :as uuid] + [app.main.data.workspace :as dw] + [app.main.data.workspace.variants :as dwv] + [beicon.v2.core :as rx] + [cljs.test :as t :include-macros true] + [frontend-tests.helpers.state :as ths] + [potok.v2.core :as ptk])) + +(t/use-fixtures :each + {:before cthi/reset-idmap!}) + +(defn- two-property-file-on-other-page + [] + (-> (cthf/sample-file :file1 :page-label :page1) + (thv/add-variant-two-properties :v01 :c01 :m01 :c02 :m02) + (cthf/add-sample-page :page2))) + +(defn- run + [file events done assert-fn] + (ths/run-store + (ths/setup-store file) + done + events + (fn [state] + (cthf/validate-file! (ths/get-file-from-state state)) + (assert-fn state)))) + +(defn- variant-name + [file label] + (:variant-name (cths/get-shape file label :page-label :page1))) + +(t/deftest update-property-value-from-another-page + (t/async + done + (let [file (two-property-file-on-other-page) + component-id (:id (cthc/get-component file :c01)) + other-id (cthi/id :page2)] + (run file + [(dwv/update-property-value component-id 0 "active") + (dwv/update-property-value component-id 1 "crítica")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + props (:variant-properties (cthc/get-component file :c01))] + (t/is (= other-id (:current-page-id state))) + (t/is (= "crítica" (-> props second :value))) + (t/is (= "active, crítica" (variant-name file :m01))) + (t/is (= (ctv/properties-to-name props) (variant-name file :m01))))))))) + +(t/deftest update-property-value-on-main-page + (t/async + done + (let [file (-> (two-property-file-on-other-page) + (cthf/switch-to-page :page1)) + component-id (:id (cthc/get-component file :c01))] + (run file + [(dwv/update-property-value component-id 1 "crítica")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + props (:variant-properties (cthc/get-component file :c01))] + (t/is (= (cthi/id :page1) (:current-page-id state))) + (t/is (= "crítica" (-> props second :value))) + (t/is (= (ctv/properties-to-name props) (variant-name file :m01))))))))) + +(t/deftest add-remove-reorder-from-another-page + (t/async + done + (let [file (two-property-file-on-other-page) + variant-id (cthi/id :v01)] + (run file + [(dwv/add-new-property variant-id {:fill-values? true}) + (dwv/reorder-variant-poperties variant-id 0 3) + (dwv/remove-property variant-id 0)] + done + (fn [state] + (let [file (ths/get-file-from-state state) + comp1 (cthc/get-component file :c01) + comp2 (cthc/get-component file :c02) + names1 (mapv :name (:variant-properties comp1)) + names2 (mapv :name (:variant-properties comp2))] + (t/is (= names1 names2)) + (t/is (= (ctv/properties-to-name (:variant-properties comp1)) + (variant-name file :m01))) + (t/is (= (ctv/properties-to-name (:variant-properties comp2)) + (variant-name file :m02))))))))) + +(t/deftest formula-rename-from-another-page + (t/async + done + (let [file (two-property-file-on-other-page) + comp (cthc/get-component file :c01) + previous (:variant-properties comp) + updated [{:name "Property 1" :value "active"} + {:name "Property 2" :value "crítica"}]] + (run file + [(dwv/update-properties-names-and-values (:id comp) (:variant-id comp) previous updated) + (dwv/remove-empty-properties (:variant-id comp))] + done + (fn [state] + (let [file (ths/get-file-from-state state) + props (:variant-properties (cthc/get-component file :c01))] + (t/is (= "active" (-> props first :value))) + (t/is (= "crítica" (-> props second :value))) + (t/is (= (ctv/properties-to-name props) (variant-name file :m01))))))))) + +(t/deftest update-property-name-from-another-page + (t/async + done + (let [file (two-property-file-on-other-page) + variant-id (cthi/id :v01)] + (run file + [(dwv/update-property-name variant-id 0 "Status" {:trigger "test"})] + done + (fn [state] + (let [file (ths/get-file-from-state state) + names (mapv :name (:variant-properties (cthc/get-component file :c01)))] + (t/is (= names (mapv :name (:variant-properties (cthc/get-component file :c02))))) + (t/is (= "Status" (first names))))))))) + +(t/deftest rename-variant-from-another-page + (t/async + done + (let [file (two-property-file-on-other-page) + variant-id (cthi/id :v01) + other-id (cthi/id :page2) + clean-name (cpn/clean-path "Renamed set")] + (run file + [(dwv/rename-variant variant-id "Renamed set")] + done + (fn [state] + (let [file (ths/get-file-from-state state) + container (cths/get-shape file :v01 :page-label :page1) + main1 (cths/get-shape file :m01 :page-label :page1) + main2 (cths/get-shape file :m02 :page-label :page1) + comp1 (cthc/get-component file :c01) + comp2 (cthc/get-component file :c02)] + (t/is (= other-id (:current-page-id state))) + (t/is (= clean-name (:name container))) + (t/is (= clean-name (:name main1))) + (t/is (= clean-name (:name main2))) + (t/is (= clean-name (cpn/merge-path-item (:path comp1) (:name comp1)))) + (t/is (= clean-name (cpn/merge-path-item (:path comp2) (:name comp2)))))))))) + +;; --------------------------------------------------------------------------- +;; Create-set / combine: every commit must be a valid file +;; --------------------------------------------------------------------------- + +(def ^:private apply-local? + (ptk/type? :app.main.data.changes/apply-changes-localy)) + +(defn- simple-main-file + [] + (-> (cthf/sample-file :file1) + (tho/add-simple-component :c01 :m01 :r01))) + +(defn- two-mains-file + [] + (-> (cthf/sample-file :file1) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:name "Alpha" :x 0 :y 0 :width 100 :height 80}) + (tho/add-simple-component :c02 :m02 :r02 + :root-params {:name "Beta" :x 150 :y 0 :width 100 :height 80}))) + +(defn- two-mains-shared-prefix-file + [] + (-> (cthf/sample-file :file1) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:name "UI / Alpha" :x 0 :y 0 :width 100 :height 80}) + (tho/add-simple-component :c02 :m02 :r02 + :root-params {:name "UI / Beta" :x 150 :y 0 :width 100 :height 80}))) + +(defn- path-named-main-file + [] + (-> (cthf/sample-file :file1) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:name "Group / Button"}))) + +(defn- two-mains-on-page1-with-page2 + "Two mains on :page1; the open page is :page2 (Assets combine while elsewhere)." + [] + (-> (cthf/sample-file :file1 :page-label :page1) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:name "Alpha" :x 0 :y 0 :width 100 :height 80}) + (tho/add-simple-component :c02 :m02 :r02 + :root-params {:name "Beta" :x 150 :y 0 :width 100 :height 80}) + (cthf/add-sample-page :page2) + (cthf/switch-to-page :page2))) + +(defn- post-artboard-file + "A normal frame holding one main — the state after create-artboard-from-shapes + and before the container flag." + [] + (-> (cthf/sample-file :file1) + (tho/add-frame :v01 {:name "Set"}) + (tho/add-simple-component :c01 :m01 :r01 + :root-params {:parent-label :v01 :name "Set"}))) + +(defn- component-full-name + [component] + (cpn/merge-path-item (:path component) (:name component))) + +(defn- page-objects + [file] + (:objects (cthf/current-page file))) + +(defn- page-objects-on + [file page-label] + (:objects (cthf/get-page file page-label))) + +(defn- commit-error + [file] + (try + (cthf/validate-file! file) + nil + (catch :default cause + (or (ex-message cause) (str cause))))) + +(defn- watch-applied-commits + "Record the file after every local apply. Keep the subscription for the + whole run-store; assertions run in the completed callback." + [store snapshots] + (->> (ptk/input-stream store) + (rx/filter apply-local?) + (rx/subs! (fn [_] + (let [file (ths/get-file-from-state @store)] + (swap! snapshots conj {:file file + :error (commit-error file)})))))) + +(defn- after-commits-idle + "Stop once applied commits have been idle. Combine schedules an async + undo commit after the main stream." + [stream] + (->> stream + (rx/filter apply-local?) + (rx/debounce 100))) + +(defn- assert-valid-snapshots + [snapshots] + (doseq [{:keys [error]} snapshots] + (t/is (nil? error) error))) + +(defn- first-flagged + [snapshots container-id] + (first (filter #(ctk/is-variant-container? + (get (page-objects (:file %)) container-id)) + snapshots))) + +(t/deftest define-variant-container-commit-validates + (let [file (post-artboard-file) + page (cthf/current-page file) + variant-id (cthi/id :v01) + main-id (cthi/id :m01) + component (cthc/get-component file :c01) + component-id (:id component) + name (:name (cths/get-shape file :v01)) + changes (-> (pcb/empty-changes nil (:id page)) + (pcb/with-objects (:objects page)) + (pcb/with-library-data (:data file)) + (clv/generate-define-variant-container + variant-id main-id component-id name)) + file' (cthf/apply-changes file changes) + container' (cths/get-shape file' :v01) + main' (cths/get-shape file' :m01) + comp' (cthc/get-component file' :c01)] + + (t/is (ctk/is-variant-container? container')) + (t/is (= [main-id] (:shapes container'))) + (t/is (= variant-id (:variant-id main'))) + (t/is (= variant-id (:variant-id comp'))) + (t/is (some? (:variant-id comp'))) + (t/is (= (ctv/properties-to-name (:variant-properties comp')) + (:variant-name main'))))) + +(t/deftest transform-in-variant-validates-after-every-commit + (t/async + done + (let [file (simple-main-file) + main-id (cthi/id :m01) + variant-id (uuid/next) + snapshots (atom []) + store (ths/setup-store file)] + (watch-applied-commits store snapshots) + (ths/run-store + store + done + [(dwv/transform-in-variant main-id variant-id nil [] false true true)] + (fn [state] + (let [snaps @snapshots + flagged (first-flagged snaps variant-id) + file' (ths/get-file-from-state state) + objects' (page-objects file') + container' (get objects' variant-id) + main' (get objects' main-id) + comp' (cthc/get-component file' :c01) + before-flag (take-while + #(not (ctk/is-variant-container? + (get (page-objects (:file %)) variant-id))) + snaps)] + (assert-valid-snapshots snaps) + (t/is (some? flagged) "a commit must set :is-variant-container") + (doseq [snap before-flag] + (t/is (not (ctk/is-variant-container? + (get (page-objects (:file snap)) variant-id))))) + (when flagged + (let [ffile (:file flagged) + fobjs (page-objects ffile) + fcont (get fobjs variant-id) + fmain (get fobjs main-id) + fcomp (cthc/get-component ffile :c01)] + (t/is (= [main-id] (:shapes fcont))) + (t/is (ctk/main-instance? fmain)) + (t/is (= variant-id (:variant-id fmain))) + (t/is (= variant-id (:variant-id fcomp))) + (t/is (= (ctv/properties-to-name (:variant-properties fcomp)) + (:variant-name fmain))))) + (t/is (ctk/is-variant-container? container')) + (t/is (= variant-id (:variant-id main'))) + (t/is (= variant-id (:variant-id comp'))))) + after-commits-idle)))) + +(t/deftest transform-in-variant-path-name-validates-after-every-commit + (t/async + done + (let [file (path-named-main-file) + main-id (cthi/id :m01) + variant-id (uuid/next) + snapshots (atom []) + store (ths/setup-store file)] + (watch-applied-commits store snapshots) + (ths/run-store + store + done + [(dwv/transform-in-variant main-id variant-id nil [] false true true)] + (fn [state] + (let [snaps @snapshots + flagged (first-flagged snaps variant-id) + file' (ths/get-file-from-state state) + objects' (page-objects file') + container' (get objects' variant-id) + main' (get objects' main-id) + comp' (cthc/get-component file' :c01) + before-flag (take-while + #(not (ctk/is-variant-container? + (get (page-objects (:file %)) variant-id))) + snaps)] + (assert-valid-snapshots snaps) + (t/is (some? flagged) "a commit must set :is-variant-container") + (doseq [snap before-flag] + (let [ffile (:file snap) + fmain (get (page-objects ffile) main-id) + fcomp (cthc/get-component ffile :c01)] + (t/is (= "Group / Button" (:name fmain))) + (t/is (= "Group / Button" (component-full-name fcomp))))) + (when flagged + (let [ffile (:file flagged) + fobjs (page-objects ffile) + fcont (get fobjs variant-id) + fmain (get fobjs main-id) + fcomp (cthc/get-component ffile :c01)] + (t/is (= "Group" (:name fcont) (:name fmain) (component-full-name fcomp))) + (t/is (= variant-id (:variant-id fmain) (:variant-id fcomp))) + (t/is (= (ctv/properties-to-name (:variant-properties fcomp)) + (:variant-name fmain))))) + (t/is (ctk/is-variant-container? container')) + (t/is (= "Group" (:name container') (:name main') (component-full-name comp'))) + (t/is (= variant-id (:variant-id main') (:variant-id comp'))))) + after-commits-idle)))) + +(t/deftest add-new-variant-on-existing-set-still-validates + (t/async + done + (let [file (-> (two-property-file-on-other-page) + (cthf/switch-to-page :page1)) + main-id (cthi/id :m01) + c01-before (cthc/get-component file :c01) + c02-before (cthc/get-component file :c02) + last-c01 (-> c01-before :variant-properties last) + last-c02 (-> c02-before :variant-properties last)] + (run file + [(dwv/add-new-variant main-id)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + container' (cths/get-shape file' :v01 :page-label :page1) + c01' (cthc/get-component file' :c01) + c02' (cthc/get-component file' :c02) + new-id (first (:shapes container')) + new-shape (cths/get-shape-by-id file' new-id :page-label :page1) + new-comp (cthc/get-component-by-id file' (:component-id new-shape))] + (t/is (= 3 (count (:shapes container')))) + (t/is (= last-c01 (-> c01' :variant-properties last))) + (t/is (= last-c02 (-> c02' :variant-properties last))) + (t/is (not= last-c01 (-> new-comp :variant-properties last))) + (t/is (= (ctv/properties-to-name (:variant-properties new-comp)) + (:variant-name new-shape))))))))) + +(t/deftest combine-as-variants-validates-after-every-commit + (t/async + done + (let [file (two-mains-file) + id1 (cthi/id :m01) + id2 (cthi/id :m02) + variant-id (uuid/next) + snapshots (atom []) + store (ths/setup-store file)] + (watch-applied-commits store snapshots) + (ths/run-store + store + done + [(dwv/combine-as-variants [id1 id2] {:variant-id variant-id :trigger "test"})] + (fn [state] + (let [snaps @snapshots + file' (ths/get-file-from-state state) + objects (page-objects file') + container (get objects variant-id) + m01' (get objects id1) + m02' (get objects id2) + c01' (cthc/get-component file' :c01) + c02' (cthc/get-component file' :c02) + names1 (mapv :name (:variant-properties c01')) + names2 (mapv :name (:variant-properties c02')) + ordered (mapv :main-instance-id + (cfv/find-variant-components (:data file') objects variant-id)) + before-flag (take-while + #(not (ctk/is-variant-container? + (get (page-objects (:file %)) variant-id))) + snaps) + flagged (first-flagged snaps variant-id)] + (assert-valid-snapshots snaps) + (doseq [snap before-flag] + (let [ffile (:file snap) + fmain (get (page-objects ffile) id1) + fcomp (cthc/get-component ffile :c01)] + (t/is (= "Alpha" (:name fmain))) + (t/is (= "Alpha" (component-full-name fcomp))))) + (when flagged + (let [ffile (:file flagged) + fobjs (page-objects ffile) + fcont (get fobjs variant-id) + fmain (get fobjs id1) + fcomp (cthc/get-component ffile :c01)] + (t/is (= "Component" (:name fcont) (:name fmain) (component-full-name fcomp))))) + (doseq [{:keys [file]} snaps] + (let [objs (page-objects file) + cont (get objs variant-id) + kids (set (:shapes cont))] + (when (contains? kids id2) + (t/is (ctk/is-variant-container? cont) + "other mains must not enter the frame before it is a variant container")))) + (t/is (ctk/is-variant-container? container)) + (t/is (= names1 names2)) + (t/is (= [id1 id2] ordered)) + (t/is (= variant-id (:variant-id m01'))) + (t/is (= variant-id (:variant-id m02'))) + (t/is (= variant-id (:parent-id m01'))) + (t/is (= variant-id (:parent-id m02'))))) + after-commits-idle)))) + +(t/deftest combine-as-variants-shared-prefix-validates-after-every-commit + (t/async + done + (let [file (two-mains-shared-prefix-file) + id1 (cthi/id :m01) + id2 (cthi/id :m02) + variant-id (uuid/next) + snapshots (atom []) + store (ths/setup-store file)] + (watch-applied-commits store snapshots) + (ths/run-store + store + done + [(dwv/combine-as-variants [id1 id2] {:variant-id variant-id :trigger "test"})] + (fn [state] + (let [snaps @snapshots + file' (ths/get-file-from-state state) + objects (page-objects file') + container (get objects variant-id) + m01' (get objects id1) + m02' (get objects id2) + c01' (cthc/get-component file' :c01) + c02' (cthc/get-component file' :c02) + flagged (first-flagged snaps variant-id)] + (assert-valid-snapshots snaps) + (doseq [{:keys [file]} snaps] + (let [objs (page-objects file) + cont (get objs variant-id) + kids (set (:shapes cont))] + (when (contains? kids id2) + (t/is (ctk/is-variant-container? cont) + "other mains must not enter the frame before it is a variant container")))) + (when flagged + (let [ffile (:file flagged) + fobjs (page-objects ffile) + fcont (get fobjs variant-id) + fmain (get fobjs id1) + fcomp (cthc/get-component ffile :c01)] + (t/is (= "UI" (:name fcont) (:name fmain) (component-full-name fcomp))))) + (t/is (ctk/is-variant-container? container)) + (t/is (= "UI" (:name container) (:name m01') (:name m02') + (component-full-name c01') (component-full-name c02'))) + (t/is (= variant-id (:variant-id m01') (:variant-id m02') + (:variant-id c01') (:variant-id c02'))) + (t/is (= variant-id (:parent-id m01') (:parent-id m02'))))) + after-commits-idle)))) + +(t/deftest combine-as-variants-from-another-page-validates-after-every-commit + (t/async + done + (let [file (two-mains-on-page1-with-page2) + id1 (cthi/id :m01) + id2 (cthi/id :m02) + page1 (cthi/id :page1) + variant-id (uuid/next) + snapshots (atom []) + store (ths/setup-store file)] + (watch-applied-commits store snapshots) + (ths/run-store + store + done + [(dwv/combine-as-variants [id1 id2] {:page-id page1 :variant-id variant-id :trigger "test"}) + (dw/initialize-page (:id file) page1)] + (fn [state] + (let [snaps @snapshots + file' (ths/get-file-from-state state) + objects (page-objects-on file' :page1) + container (get objects variant-id) + m01' (get objects id1) + m02' (get objects id2) + c01' (cthc/get-component file' :c01) + c02' (cthc/get-component file' :c02)] + (assert-valid-snapshots snaps) + (t/is (ctk/is-variant-container? container)) + (t/is (= variant-id (:variant-id m01'))) + (t/is (= variant-id (:variant-id m02'))) + (t/is (= variant-id (:variant-id c01'))) + (t/is (= variant-id (:variant-id c02'))))) + after-commits-idle)))) diff --git a/frontend/test/frontend_tests/logic/groups_test.cljs b/frontend/test/frontend_tests/logic/groups_test.cljs index 600e076096..33590b3bc0 100644 --- a/frontend/test/frontend_tests/logic/groups_test.cljs +++ b/frontend/test/frontend_tests/logic/groups_test.cljs @@ -6,19 +6,90 @@ (ns frontend-tests.logic.groups-test (:require [app.common.data :as d] - [app.common.test-helpers.compositions :as ctho] [app.common.test-helpers.files :as cthf] + [app.common.test-helpers.ids-map :as cthi] [app.common.test-helpers.shapes :as cths] - [app.common.uuid :as uuid] - [app.main.data.workspace :as dw] + [app.common.test-helpers.variants :as thv] + [app.common.types.component :as ctk] + [app.common.types.container :as ctn] + [app.common.types.shape.layout :as ctl] + [app.main.data.changes :as dch] [app.main.data.workspace.groups :as dwgr] [app.main.data.workspace.selection :as dws] + [app.main.data.workspace.shape-layout :as dwsl] + [app.main.data.workspace.shapes :as dwsh] + [beicon.v2.core :as rx] [cljs.test :as t :include-macros true] [frontend-tests.helpers.pages :as thp] - [frontend-tests.helpers.state :as ths])) + [frontend-tests.helpers.state :as ths] + [potok.v2.core :as ptk])) (t/use-fixtures :each - {:before thp/reset-idmap!}) + {:before (fn [] + (thp/reset-idmap!) + (cthi/reset-idmap!))}) + +(defn- variant-file + [] + (-> (cthf/sample-file :file1) + (thv/add-variant :variant :comp1 :root1 :comp2 :root2 + :variant1-params {:name "Button"} + :variant2-params {:name "Button"}) + (cths/update-shape :variant :name "Button"))) + +(defn- variant-file-with-rect + [] + (cths/add-sample-shape (variant-file) :rect)) + +(defn- variant-file-with-nested-and-outside + [] + (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1) + (cths/add-sample-shape :outside))) + +(defn- variant-file-with-two-nested + [] + (-> (variant-file) + (cths/add-sample-shape :nested-a :parent-label :root1) + (cths/add-sample-shape :nested-b :parent-label :root1))) + +(defn- variant-file-with-nested-in-each-main + [] + (-> (variant-file) + (cths/add-sample-shape :nested-a :parent-label :root1) + (cths/add-sample-shape :nested-b :parent-label :root2))) + +(defn- masked-groups + [file] + (->> (cthf/current-page file) + :objects + vals + (filter :masked-group))) + +(defn- groups + [file] + (->> (cthf/current-page file) + :objects + vals + (filter #(= :group (:type %))))) + +(defn- run + "Run workspace events, then validate the file before assertions. + + Pass `:commits` as an atom to collect each `dch/commit?` event." + [file events done assert-fn & {:keys [commits]}] + (let [store (ths/setup-store file)] + (when commits + (->> (ptk/input-stream store) + (rx/filter dch/commit?) + (rx/subs! #(swap! commits conj %)))) + (ths/run-store + store + done + events + (fn [state] + (cthf/validate-file! (ths/get-file-from-state state)) + (assert-fn state))))) (t/deftest test-create-group @@ -49,3 +120,584 @@ ;; ==== Check ;; Group has been created and is selected (t/is (= (get-in new-state [:workspace-local :selected]) #{group-id})))))))) + +(t/deftest mask-group-of-main-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-rect) + rect (cths/get-shape file :rect) + main (cths/get-shape file :root1) + commits (atom [])] + (run file + [(dwgr/mask-group [(cthi/id :root1) (cthi/id :rect)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (masked-groups file'))) + (t/is (= (:parent-id main) (:parent-id (cths/get-shape file' :root1)))) + (t/is (= (:parent-id rect) (:parent-id (cths/get-shape file' :rect)))))) + :commits commits)))) + +(t/deftest mask-group-of-nested-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-and-outside) + nested (cths/get-shape file :nested) + outside (cths/get-shape file :outside) + commits (atom [])] + (t/is (ctn/crosses-variant-set-boundary? + (:objects (cthf/current-page file)) + [nested outside])) + (run file + [(dwgr/mask-group [(cthi/id :nested) (cthi/id :outside)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (masked-groups file'))) + (t/is (= (:parent-id nested) + (:parent-id (cths/get-shape file' :nested)))) + (t/is (= (:parent-id outside) + (:parent-id (cths/get-shape file' :outside)))))) + :commits commits)))) + +(t/deftest mask-group-of-only-variant-mains-does-not-commit + (t/async + done + (let [file (variant-file) + commits (atom [])] + (run file + [(dwgr/mask-group [(cthi/id :root1) (cthi/id :root2)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + board (cths/get-shape file' :variant)] + (t/is (= 0 (count @commits))) + (t/is (empty? (masked-groups file'))) + (t/is (= (:id board) (:parent-id (cths/get-shape file' :root1)))) + (t/is (= (:id board) (:parent-id (cths/get-shape file' :root2)))))) + :commits commits)))) + +(t/deftest mask-group-of-ordinary-shapes-creates-a-mask + (t/async + done + (let [file (-> (cthf/sample-file :file1) + (cths/add-sample-shape :a) + (cths/add-sample-shape :b))] + (run file + [(dwgr/mask-group [(cthi/id :a) (cthi/id :b)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + group (first (masked-groups file'))] + (t/is (= 1 (count (masked-groups file')))) + (t/is (:masked-group group)) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :a)))) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :b)))))))))) + +(t/deftest mask-group-marks-a-single-plain-group + (t/async + done + (let [file (-> (cthf/sample-file :file1) + (cths/add-sample-shape :g :type :group) + (cths/add-sample-shape :child :parent-label :g)) + group-id (cthi/id :g)] + (run file + [(dwgr/mask-group [group-id])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + group' (cths/get-shape file' :g)] + (t/is (:masked-group group')) + (t/is (= [group-id] (map :id (groups file')))) + (t/is (= group-id (:parent-id (cths/get-shape file' :child)))))))))) + +(t/deftest group-shapes-of-main-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-rect) + rect (cths/get-shape file :rect) + main (cths/get-shape file :root1) + commits (atom [])] + (run file + [(dwgr/group-shapes nil [(cthi/id :root1) (cthi/id :rect)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (groups file'))) + (t/is (= (:parent-id main) (:parent-id (cths/get-shape file' :root1)))) + (t/is (= (:parent-id rect) (:parent-id (cths/get-shape file' :rect)))))) + :commits commits)))) + +(t/deftest group-shapes-of-nested-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-and-outside) + nested (cths/get-shape file :nested) + outside (cths/get-shape file :outside) + commits (atom [])] + (run file + [(dwgr/group-shapes nil [(cthi/id :nested) (cthi/id :outside)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (groups file'))) + (t/is (= (:parent-id nested) + (:parent-id (cths/get-shape file' :nested)))) + (t/is (= (:parent-id outside) + (:parent-id (cths/get-shape file' :outside)))))) + :commits commits)))) + +(t/deftest group-shapes-of-two-nested-children-creates-a-group + (t/async + done + (let [file (variant-file-with-two-nested) + main (cths/get-shape file :root1)] + (run file + [(dwgr/group-shapes nil [(cthi/id :nested-a) (cthi/id :nested-b)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + group (first (groups file'))] + (t/is (some? group)) + (t/is (= (:id main) (:parent-id group))) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :nested-a)))) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :nested-b)))))))))) + +(t/deftest group-shapes-of-main-and-nested-keeps-main-groups-nested + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + main (cths/get-shape file :root1) + nested (cths/get-shape file :nested) + commits (atom [])] + (run file + [(dwgr/group-shapes nil [(cthi/id :root1) (cthi/id :nested)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1) + nested' (cths/get-shape file' :nested) + group (first (groups file'))] + (t/is (pos? (count @commits))) + (t/is (some? group)) + (t/is (= (:parent-id main) (:parent-id main'))) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id main') (:parent-id group))) + (t/is (= (:id group) (:parent-id nested'))) + (t/is (= (:parent-id nested) (:id main))))) + :commits commits)))) + +(t/deftest mask-group-of-main-and-nested-keeps-main-masks-nested + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + main (cths/get-shape file :root1) + nested (cths/get-shape file :nested) + commits (atom [])] + (run file + [(dwgr/mask-group [(cthi/id :root1) (cthi/id :nested)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1) + nested' (cths/get-shape file' :nested) + group (first (masked-groups file'))] + (t/is (pos? (count @commits))) + (t/is (some? group)) + (t/is (:masked-group group)) + (t/is (= (:parent-id main) (:parent-id main'))) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id main') (:parent-id group))) + (t/is (= (:id group) (:parent-id nested'))) + (t/is (= (:parent-id nested) (:id main))))) + :commits commits)))) + +(t/deftest group-shapes-of-nested-in-different-variants-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-in-each-main) + nested-a (cths/get-shape file :nested-a) + nested-b (cths/get-shape file :nested-b) + commits (atom [])] + (t/is (ctn/crosses-variant-set-boundary? + (:objects (cthf/current-page file)) + [nested-a nested-b])) + (run file + [(dwgr/group-shapes nil [(cthi/id :nested-a) (cthi/id :nested-b)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (groups file'))) + (t/is (= (:parent-id nested-a) + (:parent-id (cths/get-shape file' :nested-a)))) + (t/is (= (:parent-id nested-b) + (:parent-id (cths/get-shape file' :nested-b)))))) + :commits commits)))) + +(t/deftest mask-group-of-nested-in-different-variants-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-in-each-main) + nested-a (cths/get-shape file :nested-a) + nested-b (cths/get-shape file :nested-b) + commits (atom [])] + (run file + [(dwgr/mask-group [(cthi/id :nested-a) (cthi/id :nested-b)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (masked-groups file'))) + (t/is (= (:parent-id nested-a) + (:parent-id (cths/get-shape file' :nested-a)))) + (t/is (= (:parent-id nested-b) + (:parent-id (cths/get-shape file' :nested-b)))))) + :commits commits)))) + +(t/deftest create-artboard-of-nested-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-and-outside) + nested (cths/get-shape file :nested) + outside (cths/get-shape file :outside) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :nested) (cthi/id :outside))) + (dwsh/create-artboard-from-selection)] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (= (:parent-id nested) + (:parent-id (cths/get-shape file' :nested)))) + (t/is (= (:parent-id outside) + (:parent-id (cths/get-shape file' :outside)))))) + :commits commits)))) + +(t/deftest create-artboard-of-main-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-rect) + rect (cths/get-shape file :rect) + main (cths/get-shape file :root1) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :root1) (cthi/id :rect))) + (dwsh/create-artboard-from-selection)] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (= (:parent-id main) (:parent-id (cths/get-shape file' :root1)))) + (t/is (= (:parent-id rect) (:parent-id (cths/get-shape file' :rect)))))) + :commits commits)))) + +(t/deftest create-artboard-of-two-nested-children-commits + (t/async + done + (let [file (variant-file-with-two-nested) + main (cths/get-shape file :root1) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :nested-a) (cthi/id :nested-b))) + (dwsh/create-artboard-from-selection)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + nested-a' (cths/get-shape file' :nested-a) + nested-b' (cths/get-shape file' :nested-b) + board (get (:objects (cthf/current-page file')) + (:parent-id nested-a'))] + (t/is (pos? (count @commits))) + (t/is (= :frame (:type board))) + (t/is (not (ctk/is-variant-container? board))) + (t/is (= (:id main) (:parent-id board))) + (t/is (= (:id board) (:parent-id nested-a'))) + (t/is (= (:id board) (:parent-id nested-b'))))) + :commits commits)))) + +(t/deftest create-artboard-of-main-and-nested-keeps-main-boards-nested + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + main (cths/get-shape file :root1) + nested (cths/get-shape file :nested) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :root1) (cthi/id :nested))) + (dwsh/create-artboard-from-selection)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1) + nested' (cths/get-shape file' :nested) + board (get (:objects (cthf/current-page file')) + (:parent-id nested'))] + (t/is (pos? (count @commits))) + (t/is (= :frame (:type board))) + (t/is (not (ctk/is-variant-container? board))) + (t/is (= (:parent-id main) (:parent-id main'))) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id main') (:parent-id board))) + (t/is (= (:id board) (:parent-id nested'))) + (t/is (= (:parent-id nested) (:id main))))) + :commits commits)))) + +(t/deftest group-selected-of-main-and-nested-keeps-main-groups-nested + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + main (cths/get-shape file :root1) + nested (cths/get-shape file :nested) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :root1) (cthi/id :nested))) + (dwgr/group-selected)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1) + nested' (cths/get-shape file' :nested) + group (first (groups file'))] + (t/is (pos? (count @commits))) + (t/is (some? group)) + (t/is (= (:parent-id main) (:parent-id main'))) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id main') (:parent-id group))) + (t/is (= (:id group) (:parent-id nested'))) + (t/is (= (:parent-id nested) (:id main))))) + :commits commits)))) + +(t/deftest mask-group-from-selection-of-main-and-nested-keeps-main-masks-nested + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + main (cths/get-shape file :root1) + nested (cths/get-shape file :nested) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :root1) (cthi/id :nested))) + (dwgr/mask-group)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1) + nested' (cths/get-shape file' :nested) + group (first (masked-groups file'))] + (t/is (pos? (count @commits))) + (t/is (some? group)) + (t/is (:masked-group group)) + (t/is (= (:parent-id main) (:parent-id main'))) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id main') (:parent-id group))) + (t/is (= (:id group) (:parent-id nested'))) + (t/is (= (:parent-id nested) (:id main))))) + :commits commits)))) + +(t/deftest create-layout-of-nested-and-outside-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-and-outside) + nested (cths/get-shape file :nested) + outside (cths/get-shape file :outside) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :nested) (cthi/id :outside))) + (dwsl/create-layout-from-selection :flex)] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (= (:parent-id nested) + (:parent-id (cths/get-shape file' :nested)))) + (t/is (= (:parent-id outside) + (:parent-id (cths/get-shape file' :outside)))))) + :commits commits)))) + +(t/deftest create-layout-of-nested-in-different-variants-does-not-commit + (t/async + done + (let [file (variant-file-with-nested-in-each-main) + nested-a (cths/get-shape file :nested-a) + nested-b (cths/get-shape file :nested-b) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :nested-a) (cthi/id :nested-b))) + (dwsl/create-layout :flex)] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (= (:parent-id nested-a) + (:parent-id (cths/get-shape file' :nested-a)))) + (t/is (= (:parent-id nested-b) + (:parent-id (cths/get-shape file' :nested-b)))))) + :commits commits)))) + +(t/deftest group-shapes-of-variant-container-only-does-not-commit + (t/async + done + (let [file (variant-file) + board (cths/get-shape file :variant) + commits (atom [])] + (run file + [(dwgr/group-shapes nil [(cthi/id :variant)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (groups file'))) + (t/is (= (:parent-id board) + (:parent-id (cths/get-shape file' :variant)))) + (t/is (ctk/is-variant-container? + (cths/get-shape file' :variant))))) + :commits commits)))) + +(t/deftest mask-group-of-variant-container-only-does-not-commit + (t/async + done + (let [file (variant-file) + board (cths/get-shape file :variant) + commits (atom [])] + (run file + [(dwgr/mask-group [(cthi/id :variant)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (empty? (masked-groups file'))) + (t/is (= (:parent-id board) + (:parent-id (cths/get-shape file' :variant)))) + (t/is (ctk/is-variant-container? + (cths/get-shape file' :variant))))) + :commits commits)))) + +(t/deftest create-artboard-of-variant-container-only-does-not-commit + (t/async + done + (let [file (variant-file) + board (cths/get-shape file :variant) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :variant))) + (dwsh/create-artboard-from-selection)] + done + (fn [state] + (let [file' (ths/get-file-from-state state)] + (t/is (= 0 (count @commits))) + (t/is (= (:parent-id board) + (:parent-id (cths/get-shape file' :variant)))) + (t/is (ctk/is-variant-container? + (cths/get-shape file' :variant))))) + :commits commits)))) + +(t/deftest mask-group-of-two-nested-children-creates-a-mask + (t/async + done + (let [file (variant-file-with-two-nested) + main (cths/get-shape file :root1)] + (run file + [(dwgr/mask-group [(cthi/id :nested-a) (cthi/id :nested-b)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + group (first (masked-groups file'))] + (t/is (= 1 (count (masked-groups file')))) + (t/is (:masked-group group)) + (t/is (= (:id main) (:parent-id group))) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :nested-a)))) + (t/is (= (:id group) (:parent-id (cths/get-shape file' :nested-b)))))))))) + +(t/deftest create-layout-of-variant-container-only-does-not-commit + (t/async + done + (let [file (variant-file) + board (cths/get-shape file :variant) + commits (atom [])] + (t/is (not (ctl/any-layout? board))) + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :variant))) + ;; Pin the create-layout entry (from-id hole), not only + ;; create-layout-from-selection which already no-ops. + (dwsl/create-layout :flex)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + board' (cths/get-shape file' :variant)] + (t/is (= 0 (count @commits))) + (t/is (not (ctl/any-layout? board'))) + (t/is (ctk/is-variant-container? board')) + (t/is (= (:parent-id board) (:parent-id board'))))) + :commits commits)))) + +(t/deftest create-layout-of-variant-main-commits + (t/async + done + (let [file (variant-file) + main (cths/get-shape file :root1) + board (cths/get-shape file :variant) + commits (atom [])] + (t/is (not (ctl/any-layout? main))) + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :root1))) + ;; Locks create-layout → create-layout-from-id for a single main. + (dwsl/create-layout :flex)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + main' (cths/get-shape file' :root1)] + (t/is (pos? (count @commits))) + (t/is (ctl/flex-layout? main')) + (t/is (ctk/is-variant? main')) + (t/is (= (:variant-id main) (:variant-id main'))) + (t/is (= (:id board) (:parent-id main'))))) + :commits commits)))) + +(t/deftest create-layout-of-one-nested-child-commits + (t/async + done + (let [file (-> (variant-file) + (cths/add-sample-shape :nested :parent-label :root1)) + nested (cths/get-shape file :nested) + main (cths/get-shape file :root1) + commits (atom [])] + (run file + [(dws/select-shapes (d/ordered-set (cthi/id :nested))) + ;; Same Shift+A / menu entry as the container refuse and main allow. + (dwsl/create-layout :flex)] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + nested' (cths/get-shape file' :nested) + ;; Layout wraps the child in a new board under the main. + parent' (get (:objects (cthf/current-page file')) + (:parent-id nested'))] + (t/is (pos? (count @commits))) + (t/is (not= (:parent-id nested) (:parent-id nested'))) + (t/is (= (:id main) (:parent-id parent'))) + (t/is (ctl/any-layout? parent')))) + :commits commits)))) + +(t/deftest ungroup-shapes-leaves-a-variant-container + (t/async + done + (let [file (variant-file)] + (run file + [(dwgr/ungroup-shapes [(cthi/id :variant)])] + done + (fn [state] + (let [file' (ths/get-file-from-state state) + board (cths/get-shape file' :variant)] + (t/is (ctk/is-variant-container? board)) + (t/is (= (:id board) (:parent-id (cths/get-shape file' :root1)))) + (t/is (= (:id board) (:parent-id (cths/get-shape file' :root2)))))))))) diff --git a/frontend/test/frontend_tests/logic/multi_variant_switch_test.cljs b/frontend/test/frontend_tests/logic/multi_variant_switch_test.cljs new file mode 100644 index 0000000000..3bcd56d3a5 --- /dev/null +++ b/frontend/test/frontend_tests/logic/multi_variant_switch_test.cljs @@ -0,0 +1,254 @@ +;; This Source Code Form is subject to the terms of the Mozilla Public +;; License, v. 2.0. If a copy of the MPL was not distributed with this +;; file, You can obtain one at http://mozilla.org/MPL/2.0/. +;; +;; Copyright (c) KALEIDOS SUBSIDIARY SL + +(ns frontend-tests.logic.multi-variant-switch-test + "Regression tests for #12829: multi-select variants-switch / + component-multi-swap must not emit one select-shape per copy, and must + coalesce skipped per-swap layout into one :layout/update." + (:require + [app.common.data :as d] + [app.common.test-helpers.components :as cthc] + [app.common.test-helpers.compositions :as ctho] + [app.common.test-helpers.files :as cthf] + [app.common.test-helpers.ids-map :as cthi] + [app.common.test-helpers.shapes :as cths] + [app.common.test-helpers.variants :as cthv] + [app.common.uuid :as uuid] + [app.main.data.workspace.libraries :as dwl] + [app.main.data.workspace.selection :as-alias dws] + [app.main.data.workspace.variants :as dwv] + [beicon.v2.core :as rx] + [cljs.test :as t :include-macros true] + [frontend-tests.helpers.pages :as thp] + [frontend-tests.helpers.state :as ths] + [potok.v2.core :as ptk])) + +(t/use-fixtures :each + {:before thp/reset-idmap!}) + +(def ^:private fill-value1 "#2152e5") +(def ^:private fill-value2 "#e52152") + +(defn- count-events + "Atom counting how many events of `type` land on the store input-stream + (nested WatchEvents only appear there — not on a bare ptk/watch of + variants-switch)." + [store type] + (let [counter (atom 0)] + (->> (ptk/input-stream store) + (rx/filter (ptk/type? type)) + (rx/tap (fn [_] (swap! counter inc))) + (rx/subs! (fn [_] nil))) + counter)) + +(defn- collect-layout-updates + "Atom accumulating deref'd `:layout/update` payloads from the store + input-stream (same subscribe-before-emit pattern as `count-events`)." + [store] + (let [acc (atom [])] + (->> (ptk/input-stream store) + (rx/filter (ptk/type? :layout/update)) + (rx/tap #(swap! acc conj (deref %))) + (rx/subs! (fn [_] nil))) + acc)) + +(defn- assert-coalesced-layout + [layout-updates flex-board-id] + (t/is (= 1 (count @layout-updates)) + (str "expected one coalesced :layout/update; got " + (count @layout-updates))) + (let [ids (set (:ids (first @layout-updates)))] + (t/is (seq ids) + "coalesced :layout/update must carry non-empty :ids") + (t/is (contains? ids flex-board-id) + (str "coalesced :ids must include flex board; got " ids)))) + +(defn- fill-color + [shape] + (-> shape :fills first :fill-color)) + +(defn- setup-flex-variant-file + "Two variant members with distinct fills and sizes." + [] + (-> (cthf/sample-file :file1) + (cthv/add-variant :v01 :c01 :m01 :c02 :m02 + :variant1-params {:fills (cths/sample-fills-color + :fill-color fill-value1) + :width 40 + :height 40} + :variant2-params {:fills (cths/sample-fills-color + :fill-color fill-value2) + :width 50 + :height 50}) + (ctho/add-frame :flex-board + :name "Toggle Group" + :layout :flex + :layout-flex-dir :row + :width 400 + :height 80))) + +(defn- setup-flex-multi-copies + "Three Value1 copies as children of a flex board." + [] + (-> (setup-flex-variant-file) + (cthc/instantiate-component :c01 :copy1 :parent-label :flex-board) + (cthc/instantiate-component :c01 :copy2 :parent-label :flex-board) + (cthc/instantiate-component :c01 :copy3 :parent-label :flex-board))) + +(defn- setup-flex-mixed-copies + "Two Value1 copies and one already on Value2 (no-op on switch to Value2)." + [] + (-> (setup-flex-variant-file) + (cthc/instantiate-component :c01 :copy1 :parent-label :flex-board) + (cthc/instantiate-component :c01 :copy2 :parent-label :flex-board) + (cthc/instantiate-component :c02 :copy3 :parent-label :flex-board))) + +(t/deftest multi-variants-switch-emits-no-intermediate-select-shape + (t/async + done + (let [file (setup-flex-multi-copies) + store (ths/setup-store file) + copy1 (cths/get-shape file :copy1) + copy2 (cths/get-shape file :copy2) + copy3 (cths/get-shape file :copy3) + flex-board-id (:id (cths/get-shape file :flex-board)) + target-comp-id (cthi/id :c02) + copy-ids (d/ordered-set (:id copy1) (:id copy2) (:id copy3)) + select-shape-n (count-events store ::dws/select-shape) + layout-updates (collect-layout-updates store) + events + [(dwv/variants-switch {:shapes [copy1 copy2 copy3] + :pos 0 + :val "Value2"})]] + + (ths/run-store + store done events + (fn [new-state] + (let [file' (ths/get-file-from-state new-state) + copy1' (cths/get-shape file' :copy1) + copy2' (cths/get-shape file' :copy2) + copy3' (cths/get-shape file' :copy3) + selected (get-in new-state [:workspace-local :selected])] + + (t/is (zero? @select-shape-n) + (str "multi variants-switch must not emit intermediate " + "select-shape; got " @select-shape-n)) + (assert-coalesced-layout layout-updates flex-board-id) + + (t/is (= target-comp-id (:component-id copy1'))) + (t/is (= target-comp-id (:component-id copy2'))) + (t/is (= target-comp-id (:component-id copy3'))) + (t/is (= fill-value2 (fill-color copy1'))) + (t/is (= fill-value2 (fill-color copy2'))) + (t/is (= fill-value2 (fill-color copy3'))) + (t/is (= (set copy-ids) selected)))))))) + +;; Done-marker collection must not hang when some selected copies already +;; have the target value (no companion from those no-ops). +(t/deftest multi-variants-switch-coalesces-layout-when-some-noop + (t/async + done + (let [file (setup-flex-mixed-copies) + store (ths/setup-store file) + copy1 (cths/get-shape file :copy1) + copy2 (cths/get-shape file :copy2) + copy3 (cths/get-shape file :copy3) + flex-board-id (:id (cths/get-shape file :flex-board)) + target-comp-id (cthi/id :c02) + copy-ids (d/ordered-set (:id copy1) (:id copy2) (:id copy3)) + select-shape-n (count-events store ::dws/select-shape) + layout-updates (collect-layout-updates store) + events + [(dwv/variants-switch {:shapes [copy1 copy2 copy3] + :pos 0 + :val "Value2"})]] + + (ths/run-store + store done events + (fn [new-state] + (let [file' (ths/get-file-from-state new-state) + copy1' (cths/get-shape file' :copy1) + copy2' (cths/get-shape file' :copy2) + copy3' (cths/get-shape file' :copy3) + selected (get-in new-state [:workspace-local :selected])] + + (t/is (zero? @select-shape-n)) + (assert-coalesced-layout layout-updates flex-board-id) + + (t/is (= target-comp-id (:component-id copy1'))) + (t/is (= target-comp-id (:component-id copy2'))) + (t/is (= target-comp-id (:component-id copy3'))) + (t/is (= fill-value2 (fill-color copy1'))) + (t/is (= fill-value2 (fill-color copy2'))) + (t/is (= fill-value2 (fill-color copy3'))) + (t/is (= (set copy-ids) selected)))))))) + +(t/deftest component-multi-swap-skips-select-and-coalesces-layout + (t/async + done + (let [file (setup-flex-multi-copies) + store (ths/setup-store file) + copy1 (cths/get-shape file :copy1) + copy2 (cths/get-shape file :copy2) + copy3 (cths/get-shape file :copy3) + flex-board-id (:id (cths/get-shape file :flex-board)) + target-comp-id (cthi/id :c02) + select-shape-n (count-events store ::dws/select-shape) + layout-updates (collect-layout-updates store) + events + [(dwl/component-multi-swap [copy1 copy2 copy3] + (:id file) + target-comp-id)]] + + (ths/run-store + store done events + (fn [new-state] + (let [file' (ths/get-file-from-state new-state) + copy1' (cths/get-shape file' :copy1) + copy2' (cths/get-shape file' :copy2) + copy3' (cths/get-shape file' :copy3)] + + (t/is (zero? @select-shape-n) + (str "component-multi-swap must not emit intermediate " + "select-shape; got " @select-shape-n)) + (assert-coalesced-layout layout-updates flex-board-id) + + (t/is (= target-comp-id (:component-id copy1'))) + (t/is (= target-comp-id (:component-id copy2'))) + (t/is (= target-comp-id (:component-id copy3'))) + (t/is (= fill-value2 (fill-color copy1'))) + (t/is (= fill-value2 (fill-color copy2'))) + (t/is (= fill-value2 (fill-color copy3'))))))))) + +(t/deftest component-swap-noop-when-shape-missing + (t/async + done + (let [file (setup-flex-multi-copies) + store (ths/setup-store file) + copy1 (cths/get-shape file :copy1) + ghost (assoc copy1 :id (uuid/next)) + before (ths/get-file-from-state @store)] + (ths/run-store + store done + [(dwl/component-swap ghost (:id file) (cthi/id :c02) true)] + (fn [new-state] + (t/is (= before (ths/get-file-from-state new-state)) + "component-swap must no-op when shape id is absent from objects")))))) + +(t/deftest variant-switch-noop-when-shape-missing + (t/async + done + (let [file (setup-flex-multi-copies) + store (ths/setup-store file) + copy1 (cths/get-shape file :copy1) + ghost (assoc copy1 :id (uuid/next)) + before (ths/get-file-from-state @store)] + (ths/run-store + store done + [(#'dwv/variant-switch ghost {:pos 0 :val "Value2"})] + (fn [new-state] + (t/is (= before (ths/get-file-from-state new-state)) + "variant-switch must no-op when shape id is absent from objects")))))) diff --git a/frontend/test/frontend_tests/plugins/shape_bugfixes_test.cljs b/frontend/test/frontend_tests/plugins/shape_bugfixes_test.cljs index 7dca8b7af8..3d05d8abc1 100644 --- a/frontend/test/frontend_tests/plugins/shape_bugfixes_test.cljs +++ b/frontend/test/frontend_tests/plugins/shape_bugfixes_test.cljs @@ -18,6 +18,8 @@ [app.main.data.workspace.variants :as dwv] [app.main.store :as st] [app.plugins.api :as api] + [app.plugins.flex :as flex] + [app.plugins.grid :as grid] [app.plugins.public-utils :as public-utils] [app.plugins.register :as r] [app.plugins.shape :as shape] @@ -410,3 +412,125 @@ (t/is (= 1 (count @errors))) (t/is (= [plugin-id :combineAsVariants "Plugin doesn't have 'content:write' permission"] (first @errors))))))) + +;; --------------------------------------------------------------------------- +;; Variant container append / insert +;; --------------------------------------------------------------------------- + +(def ^:private variant-append-msg + "Only main instances can be added to a variant container") + +(defn- variant-append-objects + [child-attrs] + (let [container-id (uuid/next) + child-id (uuid/next)] + {:container-id container-id + :child-id child-id + :objects {container-id {:id container-id + :type :frame + :is-variant-container true + :shapes []} + child-id (merge {:id child-id + :type :rect + :parent-id uuid/zero} + child-attrs)}})) + +(defn- with-append-stubs + [objects f] + (let [errors (atom []) + emitted (atom [])] + (with-redefs [u/locate-shape (fn [_file _page id] (get objects id)) + ;; Multi-arity: CLJS callers hit IFn$_invoke$arity$2; `constantly` + ;; only installs arity-variadic and breaks those calls. + u/locate-objects (fn + ([] objects) + ([_file _page] objects)) + u/page-active? (constantly true) + u/natural-child-ordering? (constantly false) + r/check-permission (constantly true) + u/not-valid (mock/stub (fn [pid prop msg] + (swap! errors conj [pid prop msg]))) + st/emit! (fn [& args] + (swap! emitted conj args) + nil)] + (f errors emitted)))) + +(t/deftest append-child-rejects-plain-shape-into-variant-container + (let [{:keys [container-id child-id objects]} (variant-append-objects {})] + (with-append-stubs + objects + (fn [errors emitted] + (let [container (shape/shape-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.appendChild container child) + (t/is (= 1 (count @errors))) + (t/is (= variant-append-msg (last (first @errors)))) + (t/is (empty? @emitted))))))) + +(t/deftest append-child-rejects-copy-into-variant-container + (let [{:keys [container-id child-id objects]} + (variant-append-objects {:type :frame + :component-id (uuid/next) + :component-root true})] + (with-append-stubs + objects + (fn [errors emitted] + (let [container (shape/shape-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.appendChild container child) + (t/is (= 1 (count @errors))) + (t/is (= variant-append-msg (last (first @errors)))) + (t/is (empty? @emitted))))))) + +(t/deftest append-child-allows-main-into-variant-container + (let [{:keys [container-id child-id objects]} + (variant-append-objects {:type :frame + :main-instance true + :component-id (uuid/next) + :component-root true})] + (with-append-stubs + objects + (fn [errors emitted] + (let [container (shape/shape-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.appendChild container child) + (t/is (empty? @errors)) + (t/is (pos? (count @emitted)))))))) + +(t/deftest insert-child-rejects-plain-shape-into-variant-container + (let [{:keys [container-id child-id objects]} (variant-append-objects {})] + (with-append-stubs + objects + (fn [errors emitted] + (let [container (shape/shape-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.insertChild container 0 child) + (t/is (= 1 (count @errors))) + (t/is (= [plugin-id :insertChild variant-append-msg] (first @errors))) + (t/is (empty? @emitted))))))) + +(t/deftest flex-append-child-rejects-plain-shape-into-variant-container + (let [{:keys [container-id child-id objects]} (variant-append-objects {})] + (with-append-stubs + objects + (fn [errors emitted] + (with-redefs [flex/shape-proxy? (constantly true)] + (let [layout (flex/flex-layout-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.appendChild layout child) + (t/is (= 1 (count @errors))) + (t/is (= variant-append-msg (last (first @errors)))) + (t/is (empty? @emitted)))))))) + +(t/deftest grid-append-child-rejects-plain-shape-into-variant-container + (let [{:keys [container-id child-id objects]} (variant-append-objects {})] + (with-append-stubs + objects + (fn [errors emitted] + (with-redefs [grid/shape-proxy? (constantly true)] + (let [layout (grid/grid-layout-proxy plugin-id (uuid/next) (uuid/next) container-id) + child (shape/shape-proxy plugin-id (uuid/next) (uuid/next) child-id)] + (.appendChild layout child 0 0) + (t/is (= 1 (count @errors))) + (t/is (= variant-append-msg (last (first @errors)))) + (t/is (empty? @emitted)))))))) diff --git a/frontend/test/frontend_tests/runner.cljs b/frontend/test/frontend_tests/runner.cljs index c96f1e4983..df1a89ef60 100644 --- a/frontend/test/frontend_tests/runner.cljs +++ b/frontend/test/frontend_tests/runner.cljs @@ -38,6 +38,7 @@ [frontend-tests.data.workspace-texts-test] [frontend-tests.data.workspace-thumbnails-test] [frontend-tests.data.workspace-variant-rename-test] + [frontend-tests.data.workspace-variants-test] [frontend-tests.data.workspace-versions-test] [frontend-tests.debug-logging-test] [frontend-tests.errors-governor-test] @@ -51,6 +52,7 @@ [frontend-tests.logic.copying-and-duplicating-test] [frontend-tests.logic.frame-guides-test] [frontend-tests.logic.groups-test] + [frontend-tests.logic.multi-variant-switch-test] [frontend-tests.logic.nudge-selected-shapes-test] [frontend-tests.logic.pasting-in-containers-test] [frontend-tests.logic.path-actions-test] @@ -189,6 +191,7 @@ 'frontend-tests.data.workspace-texts-test 'frontend-tests.data.workspace-thumbnails-test 'frontend-tests.data.workspace-variant-rename-test + 'frontend-tests.data.workspace-variants-test 'frontend-tests.data.workspace-versions-test 'frontend-tests.debug-logging-test 'frontend-tests.errors-governor-test @@ -202,6 +205,7 @@ 'frontend-tests.logic.copying-and-duplicating-test 'frontend-tests.logic.frame-guides-test 'frontend-tests.logic.groups-test + 'frontend-tests.logic.multi-variant-switch-test 'frontend-tests.logic.nudge-selected-shapes-test 'frontend-tests.logic.path-actions-test 'frontend-tests.logic.path-clipboard-test diff --git a/library/CHANGES.md b/library/CHANGES.md index b1f7acdc70..61112d6081 100644 --- a/library/CHANGES.md +++ b/library/CHANGES.md @@ -1,5 +1,10 @@ # CHANGELOG +## 1.3.0 + +- Update deps and build it with penpot 2.19.0 + + ## 1.2.0 - Update deps and build it with penpot 2.18.3 diff --git a/library/package.json b/library/package.json index 6b1911e2bd..0ddb8634a7 100644 --- a/library/package.json +++ b/library/package.json @@ -1,6 +1,6 @@ { "name": "@penpot/library", - "version": "1.2.0", + "version": "1.3.0", "license": "MPL-2.0", "author": "KALEIDOS SUBSIDIARY SL", "packageManager": "pnpm@12.16.1+sha512.091d1dee7927575661b25dab3c2ae31cafc1c54d6b71ed6a6600cf0c1cb10434a447eae14015f1b23e2e6bedeed79df94a932e2164d2832280dce3fb32f656d1", diff --git a/mcp/README.md b/mcp/README.md index bc13252c16..32530bc869 100644 --- a/mcp/README.md +++ b/mcp/README.md @@ -309,9 +309,10 @@ you may set the following environment variables to configure the two servers * Auto-formatting: Use `pnpm run fmt` * Generating API type data: See [types-generator/README.md](types-generator/README.md) * Versioning: Use `bash scripts/set-version` to set the version for the MCP package (in `package.json`). - - Ensure that at least the major, minor and patch components of the version are always up-to-date. - - The MCP plugin assumes that a mismatch between the MCP version and the Penpot version (as returned by the API) - indicates incompatibility, resulting in the display of a warning message in the plugin UI. + - Ensure that at least the major and minor components of the version are always up-to-date. + - The MCP plugin assumes that a mismatch between the major.minor components of the MCP version and the Penpot + version (as returned by the API) indicates incompatibility, resulting in the display of a warning message in + the plugin UI. Patch releases are treated as compatible. * Packaging and publishing: 1. Ensure release version is set correctly in package.json (call `bash scripts/set-version` to update it automatically) 2. Create npm package: `bash scripts/pack` (creates `penpot-mcp-.tgz` for publishing) diff --git a/mcp/package.json b/mcp/package.json index 97012b38b0..d1d1fcc72a 100644 --- a/mcp/package.json +++ b/mcp/package.json @@ -1,6 +1,6 @@ { "name": "@penpot/mcp", - "version": "2.17.0", + "version": "2.19.0", "description": "MCP server for Penpot integration", "license": "MPL-2.0", "bin": { diff --git a/mcp/packages/plugin/src/VersionCompatibility.test.ts b/mcp/packages/plugin/src/VersionCompatibility.test.ts new file mode 100644 index 0000000000..2e40bb7cf0 --- /dev/null +++ b/mcp/packages/plugin/src/VersionCompatibility.test.ts @@ -0,0 +1,28 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { extractMinorVersion, isCompatibleVersion } from "./VersionCompatibility.ts"; + +test("extracts major.minor from release and pre-release versions", () => { + assert.equal(extractMinorVersion("2.18.1"), "2.18"); + assert.equal(extractMinorVersion("2.17.0"), "2.17"); + assert.equal(extractMinorVersion("2.15.0-rc.1.116"), "2.15"); + assert.equal(extractMinorVersion("2.18.1-dev.140"), "2.18"); + assert.equal(extractMinorVersion("2.18"), "2.18"); +}); + +test("returns non-matching versions unchanged", () => { + assert.equal(extractMinorVersion("<2.15"), "<2.15"); + assert.equal(extractMinorVersion("develop"), "develop"); +}); + +test("treats patch releases of the same minor version as compatible", () => { + assert.equal(isCompatibleVersion(extractMinorVersion("2.17.2"), extractMinorVersion("2.17.0")), true); +}); + +test("reports a different minor version as incompatible", () => { + assert.equal(isCompatibleVersion(extractMinorVersion("2.18.1"), extractMinorVersion("2.17.0")), false); +}); + +test("treats local development builds of Penpot as compatible", () => { + assert.equal(isCompatibleVersion(extractMinorVersion("0.0.0"), extractMinorVersion("2.17.0")), true); +}); diff --git a/mcp/packages/plugin/src/VersionCompatibility.ts b/mcp/packages/plugin/src/VersionCompatibility.ts new file mode 100644 index 0000000000..f529491ea3 --- /dev/null +++ b/mcp/packages/plugin/src/VersionCompatibility.ts @@ -0,0 +1,27 @@ +/** + * Extracts the major.minor prefix from a version string. + * + * Patch releases of Penpot do not change the plugin API the MCP server is + * written against, and the MCP package version is only set per minor + * release, so the patch component is not part of the compatibility check. + * + * @param version - a version string starting with major.minor (e.g. "2.18.1") + * @returns the major.minor prefix, or the original string if it does not match + */ +export function extractMinorVersion(version: string): string { + const match = version.match(/^(\d+\.\d+)(?:\.|$)/); + return match ? match[1] : version; +} + +/** + * Returns whether an MCP build is intended for the running Penpot version. + * + * Local development builds of Penpot report "0.0.0" and are always treated + * as compatible. + * + * @param penpotVersion - the running Penpot version, as major.minor prefix + * @param mcpVersion - the version the MCP build targets, as major.minor prefix + */ +export function isCompatibleVersion(penpotVersion: string, mcpVersion: string): boolean { + return penpotVersion === "0.0" || penpotVersion === mcpVersion; +} diff --git a/mcp/packages/plugin/src/plugin.ts b/mcp/packages/plugin/src/plugin.ts index 502116707f..60f09ccc03 100644 --- a/mcp/packages/plugin/src/plugin.ts +++ b/mcp/packages/plugin/src/plugin.ts @@ -1,6 +1,7 @@ import { ExecuteCodeTaskHandler } from "./task-handlers/ExecuteCodeTaskHandler"; import { Task, TaskHandler } from "./TaskHandler"; import { formatTaskError } from "./ErrorUtils"; +import { extractMinorVersion, isCompatibleVersion } from "./VersionCompatibility"; /** * indicates whether the plugin is running in an environment with the Penpot-integrated remote MCP server @@ -8,17 +9,6 @@ import { formatTaskError } from "./ErrorUtils"; */ const isIntegratedRemoteMcp = !!mcp; -/** - * Extracts the major.minor.patch prefix from a version string. - * - * @param version - a version string starting with major.minor.patch - * @returns the major.minor.patch prefix, or the original string if it does not match - */ -function extractVersionPrefix(version: string): string { - const match = version.match(/^(\d+\.\d+\.\d+)/); - return match ? match[1] : version; -} - mcp?.setMcpStatus("connecting"); /** @@ -42,11 +32,10 @@ penpot.ui.onMessage + +The result has to be **exactly** the end state of the migration, because on a +big instance the migration is marked as applied and never runs. This procedure +runs the same statements by hand, in small batched transactions. + +Use this only if the automatic migration would take too long on the instance +(millions of snapshots, hundreds of GB in TOAST). Small and medium instances +run the migration automatically on boot and need nothing from here. + +Run it before upgrading to 2.19.0. + +## What changes + +- Every `file_change` row with `data IS NOT NULL AND label IS NOT NULL` + (a snapshot) gets a `file_data` row with `type='snapshot'`, + `backend='db'`, `metadata=NULL`, `created_at` from `created_at`, + `modified_at` from `updated_at`, and `deleted_at` copied as is. Then the + legacy `file_change.data` is set to `NULL`. +- Every `file` row with `data IS NOT NULL` gets a `file_data` row with + `type='main'` (and `file_id = id`), the same `backend`/`metadata` mapping + and the timestamps copied one to one. Then the legacy `file.data` is set + to `NULL`. +- Every `file_change` row with `data IS NOT NULL AND label IS NULL` (an + orphan revision) is deleted outright. No read path reaches these rows: + snapshots resolve through `label IS NOT NULL`, lagged changes read the + `changes` column (not `data`), and nothing references `file_change` rows + from other tables. Rows carrying `deleted_at` would be collected by + `objects-gc` anyway; undated ones would sit in the table and its TOAST + for ever. + +`deleted_at` is copied on purpose: the write path always writes it to +`file_data`, the GC only collects `file_data` rows carrying it, and reads +never filter on `file_data.deleted_at`. Without the copy, migrated rows of +deleted files and snapshots would be orphaned: nobody reads them and nobody +collects them. + +## Why this mechanism + +A single-statement copy inside the transaction of the startup does not fit a +big instance, so the copy runs here in small batched transactions: + +- One statement is one transaction. Never wrap several batches in one + transaction. Run a single stream of batches at a time. +- `FOR UPDATE SKIP LOCKED`: the batches never block the application and are + never blocked by it. A row skipped by a lock stays in the index and the + next execution picks it up. +- Partial temp indexes hold only the pending rows, so every execution finds + its next batch without rescanning what is done, and migrated rows + (`data = NULL`) and deleted orphans drop out of the index on their own. + No cursor is needed. Do not `VACUUM` mid-run to clean them: it would walk + the whole TOAST. Drop them at the end. +- No `ON CONFLICT DO NOTHING` here on purpose: a duplicate key must abort + the batch (atomically: it migrates nothing) so the conflict is + investigated instead of silently skipped. The automatic migration is + lenient so a small instance never fails to boot; here strictness is what + keeps the data auditable. + +--- + +## Phase 0 — Before anything + +### 0.1 Mark the migration as applied + +Do this first: it registers the automatic migration as done, so a backend +restart mid-procedure never runs it on boot (it would copy the whole legacy +payload inside the startup transaction). + +```sql +INSERT INTO migrations (module, step) +VALUES ('main', '0157-migrate-legacy-file-data'); +``` + +The module is `main` and the step has no `.sql` (see +`backend/src/app/util/migrations.clj`). + +### 0.2 Counters + +Repeat the counters any time to follow the progress. Each batch loop ends +when it returns 0; the check in phase 3 confirms nothing is pending. + +```sql +-- Remaining legacy rows. +SELECT (SELECT count(*) FROM file_change + WHERE data IS NOT NULL AND label IS NOT NULL) AS snapshots_remaining, + (SELECT count(*) FROM file_change + WHERE data IS NOT NULL AND label IS NULL) AS orphans_remaining, + (SELECT count(*) FROM file + WHERE data IS NOT NULL) AS files_remaining; +``` + +Optional, reference only: how much is already migrated. Skip it unless +curious: it is a full count over the partitioned table and can take a +while on a big instance. For progress, the remaining counters above and +the per-batch counts are enough. + +```sql +SELECT (SELECT count(*) FROM file_data + WHERE type = 'snapshot' AND backend = 'db') AS snapshots_in_file_data, + (SELECT count(*) FROM file_data + WHERE type = 'main' AND backend = 'db') AS mains_in_file_data; +``` + +## Phase 1 — Temp indexes + +Required by the batch loops, created before them. `CONCURRENTLY` so writes +are not blocked; each statement is its own transaction. + +```sql +CREATE INDEX CONCURRENTLY tmp_file_change_mig ON file_change (id) + WHERE data IS NOT NULL AND label IS NOT NULL; +CREATE INDEX CONCURRENTLY tmp_file_change_orphan_mig ON file_change (id) + WHERE data IS NOT NULL AND label IS NULL; +CREATE INDEX CONCURRENTLY tmp_file_mig ON file (id) + WHERE data IS NOT NULL; +``` + +A failed `CONCURRENTLY` build stays `INVALID` and the planner ignores it +silently, so verify all three indexes exist and are valid: + +```sql +SELECT indexrelid::regclass AS index, indisvalid AS valid + FROM pg_index + WHERE indexrelid::regclass::text LIKE 'tmp_%mig'; +``` + +## Phase 2 — Batch loops + +Repeat each statement until it returns 0. Lower `LIMIT` if a batch hits +`statement_timeout` or memory pressure (`SET statement_timeout = '15min'` +per batch is the usual guard). + +### 2.1 Delete orphans + +Orphans first (or interleaved with the other two; the row sets are +disjoint, and deleting TOAST-heavy rows early frees space for the rest): + +```sql +WITH doomed AS ( + SELECT id + FROM file_change + WHERE data IS NOT NULL + AND label IS NULL + ORDER BY id + LIMIT 10000 + FOR UPDATE SKIP LOCKED +), +del AS ( + DELETE FROM file_change fc + USING doomed + WHERE fc.id = doomed.id + RETURNING fc.id +) +SELECT count(*) AS deleted + FROM del; +``` + +### 2.2 Migrate snapshots + +Repeat until it returns 0 (`~120 MB` per transaction at `LIMIT 10000`): + +```sql +WITH batch AS ( + SELECT id, file_id, data, created_at, updated_at, deleted_at + FROM file_change + WHERE data IS NOT NULL + AND label IS NOT NULL + ORDER BY id + LIMIT 10000 + FOR UPDATE SKIP LOCKED +), +-- NO ON CONFLICT on purpose: duplicates must abort (see above). +-- upd reads from ins (RETURNING), not from batch. +ins AS ( + INSERT INTO file_data (file_id, id, created_at, modified_at, deleted_at, + type, backend, metadata, data) + SELECT file_id, id, created_at, updated_at, deleted_at, + 'snapshot', 'db', NULL, data + FROM batch + RETURNING file_id, id +), +-- Only the origin of what was really inserted is nulled. On conflict +-- the whole transaction aborts, so ins == batch whenever this runs. +upd AS ( + UPDATE file_change fc + SET data = NULL + FROM ins + WHERE ins.file_id = fc.file_id + AND ins.id = fc.id + RETURNING fc.id +) +SELECT count(*) AS migrated + FROM upd; +``` + +### 2.3 Migrate files + +Repeat until it returns 0 (tune `LIMIT` to the average `file.data` size; +start at 5000): + +```sql +WITH batch AS ( + SELECT id, data, created_at, modified_at, deleted_at + FROM file + WHERE data IS NOT NULL + ORDER BY id + LIMIT 5000 + FOR UPDATE SKIP LOCKED +), +ins AS ( + INSERT INTO file_data (file_id, id, created_at, modified_at, deleted_at, + type, backend, metadata, data) + SELECT id, id, created_at, modified_at, deleted_at, + 'main', 'db', NULL, data + FROM batch + RETURNING file_id, id +), +upd AS ( + UPDATE file f + SET data = NULL + FROM ins + WHERE ins.file_id = f.id + AND ins.id = f.id + RETURNING f.id +) +SELECT count(*) AS migrated + FROM upd; +``` + +### 2.4 Duplicate keys + +If a batch aborts on duplicate key, stop and do not blind-retry it: it +means `file_data` already holds that key. Locate the overlap: + +```sql +-- Snapshots with legacy data AND a file_data key already present. +SELECT c.file_id, c.id, c.created_by, c.deleted_at, + pg_size_pretty(length(c.data)) AS legacy_size, + pg_size_pretty(length(fd.data)) AS file_data_size, + fd.modified_at AS file_data_modified_at + FROM file_change c + JOIN file_data fd ON fd.file_id = c.file_id + AND fd.id = c.id + AND fd.type = 'snapshot' + WHERE c.data IS NOT NULL + ORDER BY c.id + LIMIT 20; + +-- Files with legacy data AND a file_data key already present. +SELECT f.id, + pg_size_pretty(length(f.data)) AS legacy_size, + pg_size_pretty(length(fd.data)) AS file_data_size, + fd.modified_at AS file_data_modified_at + FROM file f + JOIN file_data fd ON fd.file_id = f.id + AND fd.id = f.id + AND fd.type = 'main' + WHERE f.data IS NOT NULL + ORDER BY f.id + LIMIT 20; +``` + +Resolve manually, one row at a time, only when `file_data` holds equal or +newer data (compare `modified_at` and sizes above): the legacy copy is +redundant, null the origin, leave `file_data` untouched. + +```sql +-- UPDATE file_change SET data = NULL WHERE file_id = '...' AND id = '...'; +-- UPDATE file SET data = NULL WHERE id = '...'; +``` + +## Phase 3 — Verification and cleanup + +Each loop above already ended returning 0; confirm nothing is pending (if +anything shows up, repeat that loop): + +```sql +-- Remaining legacy rows: all three must read 0. +SELECT (SELECT count(*) FROM file_change + WHERE data IS NOT NULL AND label IS NOT NULL) AS snapshots_remaining, + (SELECT count(*) FROM file_change + WHERE data IS NOT NULL AND label IS NULL) AS orphans_remaining, + (SELECT count(*) FROM file + WHERE data IS NOT NULL) AS files_remaining; +``` + +```sql +-- Spot check: migrated rows exist in file_data with backend 'db'. +SELECT fd.file_id, fd.id, fd.type, fd.backend, + pg_size_pretty(length(fd.data)) AS size_in_file_data + FROM file_data fd + ORDER BY fd.file_id, fd.id + LIMIT 5; +``` + +Drop the temp indexes first, so the vacuum below skips them (each batch +left its dead entries behind), then reclaim the dead tuples and refresh +planner stats. Plain `VACUUM` does not block writes; `VERBOSE` prints +progress, which matters on big tables: + +```sql +DROP INDEX CONCURRENTLY tmp_file_change_mig; +DROP INDEX CONCURRENTLY tmp_file_change_orphan_mig; +DROP INDEX CONCURRENTLY tmp_file_mig; +VACUUM (ANALYZE ON, VERBOSE ON) file_change; +VACUUM (ANALYZE ON, VERBOSE ON) file; +-- file_data is partitioned into 16 tables: the statement walks all of +-- them, so it can take a while. Without it the planner stats of the fresh +-- inserts stay stale until autovacuum catches up. +VACUUM (ANALYZE ON, VERBOSE ON) file_data; +``` + +## Phase 4 — Upgrade + +The migration was marked as applied in phase 0, so the new version skips +it on boot. Deploy normally. diff --git a/plugins/apps/composable-test-suite/ci/run-ci.ts b/plugins/apps/composable-test-suite/ci/run-ci.ts index d9403c6793..69186e9436 100644 --- a/plugins/apps/composable-test-suite/ci/run-ci.ts +++ b/plugins/apps/composable-test-suite/ci/run-ci.ts @@ -146,7 +146,7 @@ async function setupMockedRoutes(page: Page): Promise { route.fulfill({ status: 200, contentType: "application/javascript", - body: 'var penpotFlags = "";\n', + body: 'var penpotFlags = "disable-admin-console";\n', }) ); diff --git a/plugins/apps/plugin-api-test-suite/ci/run-ci.ts b/plugins/apps/plugin-api-test-suite/ci/run-ci.ts index 559e6f0aea..8b97cf088a 100644 --- a/plugins/apps/plugin-api-test-suite/ci/run-ci.ts +++ b/plugins/apps/plugin-api-test-suite/ci/run-ci.ts @@ -242,7 +242,7 @@ async function setupMockedRoutes(page: Page): Promise { route.fulfill({ status: 200, contentType: 'application/javascript', - body: 'var penpotFlags = "";\n', + body: 'var penpotFlags = "disable-admin-console";\n', }), ); diff --git a/scripts/changelog.py b/scripts/changelog.py index fbba459fbd..e964bacc18 100644 --- a/scripts/changelog.py +++ b/scripts/changelog.py @@ -1,18 +1,21 @@ #!/usr/bin/env python3 """ -changelog.py — Helper commands for the ``update-changelog`` skill. +changelog.py — Helper commands for the changelog flow. Uses ``scripts/gh.py`` (via subprocess) and the local ``CHANGES.md`` so the -skill file itself stays free of inline programs. +flow memory (`mem:workflow/updating-changelog`) stays free of inline programs. Subcommands: check-merged Verify that every given PR is merged (warns otherwise) + boom Generate the :boom: section from `breaking change` issues cross-ref Compare merged milestone PRs against the changelog section report Generate the anomaly report (CHANGES-ISSUES.md) Usage: python3 scripts/changelog.py check-merged 9179 9204 9311 cat prs.txt | python3 scripts/changelog.py check-merged --stdin + python3 scripts/changelog.py boom "2.19.0" + python3 scripts/changelog.py boom "2.19.0" --write python3 scripts/changelog.py cross-ref "2.16.0" python3 scripts/changelog.py report "2.16.0" python3 scripts/changelog.py report "2.16.0" --changes CHANGES.md --output CHANGES-ISSUES.md @@ -34,6 +37,21 @@ from pathlib import Path REPO = "penpot/penpot" GH_PY = Path(__file__).resolve().parent / "gh.py" +BOOM_HEADING = "### :boom: Breaking changes & Deprecations" +BREAKING_LABEL = "breaking change" +EXCLUDED_LABELS = {"release blocker", "no changelog"} + +# Marker line of the issue comment that carries the changelog text of a +# breaking change. Everything after the marker line is the entry content. +BREAKING_NOTE_RE = re.compile(r"(?im)^#{1,3}[ \t]+Breaking change note[ \t]*$") + +# Level used for the headings found inside a breaking change note. Entries +# themselves are level-4 headings, so note headings must live below them. +NOTE_HEADING = "#####" + +# Emoji prefixes used at the start of issue titles (`:bug:`, `:sparkles:`). +TITLE_PREFIX_RE = re.compile(r"^(?::[A-Za-z0-9_+-]+:\s*)+") + # ───────────────────────────────────────────── # Shared helpers @@ -125,6 +143,84 @@ def collect_changelog_issues(section: str) -> set[int]: return numbers +def issue_excluded(issue: dict | None) -> bool: + """Return True when an issue must stay out of the changelog. + + Shared by the ``report`` and ``boom`` subcommands so the exclusions + cannot drift apart. + """ + if not issue: + return True + if issue.get("state") != "CLOSED": + return True + if issue.get("issue_type") in {"Task"}: + return True + if issue.get("project_status") in {"Rejected"}: + return True + if EXCLUDED_LABELS & set(issue.get("labels", [])): + return True + return False + + +def extract_breaking_note(comment_body: str | None) -> str | None: + """Return the content that follows a ``## Breaking change note`` marker. + + The whole comment is used (paragraphs, lists, code blocks), not just its + first paragraph, so a breaking change can be as detailed as needed. + """ + if not comment_body: + return None + match = BREAKING_NOTE_RE.search(comment_body) + if not match: + return None + content = demote_headings(comment_body[match.end():].strip("\n")) + return content.strip() or None + + +def demote_headings(markdown: str) -> str: + """Push note headings below the entry heading, leaving code fences alone.""" + lines: list[str] = [] + in_fence = False + for line in markdown.splitlines(): + if line.lstrip().startswith(("```", "~~~")): + in_fence = not in_fence + lines.append(line) + continue + if not in_fence: + match = re.match(r"^#{1,4}([ \t]+.*)$", line) + if match: + line = NOTE_HEADING + match.group(1) + lines.append(line) + return "\n".join(lines) + + +def strip_title_prefix(title: str) -> str: + """Remove leading emoji shortcodes from a title.""" + return TITLE_PREFIX_RE.sub("", title).strip() + + +def closing_pr_numbers(issue: dict) -> list[int]: + """Return the sorted closing PR numbers of an issue.""" + return sorted(set(issue.get("closing_prs", []))) + + +def render_boom_entry( + issue: dict, + note: str | None, + pr_numbers: list[int], + author: str | None = None, +) -> str: + """Render one ``:boom:`` entry: title, note content and references.""" + title = strip_title_prefix(issue["title"]) + attribution = f"(by @{author}) " if author else "" + refs = attribution + issue_link(issue["number"]) + if pr_numbers: + refs += " (PR: " + ", ".join(pr_link(n) for n in pr_numbers) + ")" + if note: + return f"#### {title}\n\n{note}\n\n{refs}" + return f"#### {title}\n\n{refs}" + + # ───────────────────────────────────────────── # Subcommand: check-merged # ───────────────────────────────────────────── @@ -155,6 +251,153 @@ def cmd_check_merged(args: argparse.Namespace) -> None: print(f"OK: all {len(prs)} PRs are merged", file=sys.stderr) +# ───────────────────────────────────────────── +# Subcommand: boom +# ───────────────────────────────────────────── + + +def version_section_bounds(content: str, milestone: str) -> tuple[int, int] | None: + """Return the ``[start, end)`` offsets of a ``## `` section.""" + match = re.search( + rf"(?m)^## {re.escape(milestone)}(?:[ \t]*\([^)]*\))?[ \t]*$", content + ) + if not match: + return None + start = match.start() + next_heading = re.search(r"(?m)^## ", content[match.end():]) + end = match.end() + next_heading.start() if next_heading else len(content) + return start, end + + +def boom_body(entries: list[str]) -> str: + """Return the body text that follows the ``:boom:`` heading line.""" + if entries: + return "\n\n" + "\n\n".join(entries) + "\n\n" + return "\n\n" + + +def replace_boom_section(content: str, milestone: str, entries: list[str]) -> str: + """Return ``content`` with its ``:boom:`` subsection replaced. + + The subsection is inserted right after the version header when missing, + which keeps ``:boom:`` as the first section. + """ + bounds = version_section_bounds(content, milestone) + if bounds is None: + raise ValueError(f'no "## {milestone}" section found') + start, end = bounds + section = content[start:end] + heading = re.search(rf"(?m)^{re.escape(BOOM_HEADING)}[ \t]*$", section) + if heading: + body_start = start + heading.end() + next_heading = re.search(r"(?m)^### ", content[body_start:end]) + body_end = body_start + next_heading.start() if next_heading else end + return content[:body_start] + boom_body(entries) + content[body_end:] + header = re.match(r"(?m)^## [^\n]*\n", section) + if header is None: + raise ValueError(f'version "## {milestone}" header has no trailing newline') + insert_at = start + header.end() + block = "\n" + BOOM_HEADING + boom_body(entries)[:-1] + return content[:insert_at] + block + content[insert_at:] + + +def fetch_prs_by_issue(issues: list[dict]) -> dict[int, dict]: + """Fetch every closing PR of ``issues``, keyed by PR number.""" + numbers = sorted({n for issue in issues for n in issue.get("closing_prs", [])}) + if not numbers: + return {} + return {pr["number"]: pr for pr in fetch_prs_by_numbers(numbers)} + + +def fetch_breaking_notes(issues: list[dict]) -> dict[int, str]: + """Return the last ``## Breaking change note`` comment of each issue.""" + if not issues: + return {} + comments = json.loads( + run_gh_py("comments", *[str(issue["number"]) for issue in issues]) + ) + notes: dict[int, str] = {} + for comment in comments: + note = extract_breaking_note(comment.get("body")) + if note is None: + continue + number = comment["issue"] + if number in notes: + print( + f'WARNING: issue #{number} has several "## Breaking change note" ' + "comments; using the last one", + file=sys.stderr, + ) + notes[number] = note + return notes + + +def cmd_boom(args: argparse.Namespace) -> None: + """Generate the ``:boom:`` section from ``breaking change`` issues.""" + all_issues = json.loads(run_gh_py("issues", args.milestone, "--state", "all")) + labelled = sorted( + ( + issue + for issue in all_issues + if BREAKING_LABEL in issue.get("labels", []) + and not issue_excluded(issue) + ), + key=lambda issue: issue["number"], + ) + + notes: dict[int, str] = {} + prs: dict[int, dict] = {} + if labelled: + notes = fetch_breaking_notes(labelled) + prs = fetch_prs_by_issue(labelled) + + entries: list[str] = [] + for issue in labelled: + note = notes.get(issue["number"]) + if not note: + print( + f'WARNING: issue #{issue["number"]} has no "## Breaking change note" ' + "comment; the entry will have no description", + file=sys.stderr, + ) + merged = [ + prs[num] + for num in closing_pr_numbers(issue) + if num in prs and prs[num].get("state") == "MERGED" + ] + author = None + if "community contribution" in issue.get("labels", []): + authors = [pr["author"] for pr in merged if pr.get("author")] + author = authors[0] if authors else None + entries.append( + render_boom_entry(issue, note, [pr["number"] for pr in merged], author) + ) + + block = BOOM_HEADING + boom_body(entries) + print(block.rstrip("\n")) + + if not labelled: + print( + f"No closed issues labelled `{BREAKING_LABEL}` in {args.milestone}", + file=sys.stderr, + ) + + if args.write: + with open(args.changes) as f: + content = f.read() + try: + updated = replace_boom_section(content, args.milestone, entries) + except ValueError as err: + print(f"ERROR: {err}", file=sys.stderr) + sys.exit(1) + if updated == content: + print(f"{args.changes}: :boom: section already up to date", file=sys.stderr) + else: + with open(args.changes, "w") as f: + f.write(updated) + print(f"{args.changes}: :boom: section updated", file=sys.stderr) + + # ───────────────────────────────────────────── # Subcommand: cross-ref # ───────────────────────────────────────────── @@ -298,21 +541,6 @@ def cmd_report(args: argparse.Namespace) -> None: """Return the milestone title for an issue, or None if unassigned.""" return issue_milestone_cache.get(issue_num) - excluded_labels = {"release blocker", "no changelog"} - - def issue_excluded(issue: dict | None) -> bool: - if not issue: - return True - if issue.get("state") != "CLOSED": - return True - if issue.get("issue_type") in {"Task"}: - return True - if issue.get("project_status") in {"Rejected"}: - return True - if excluded_labels & set(issue.get("labels", [])): - return True - return False - # Type A: issue in milestone, referenced PR in different milestone or none anomalies_a = [] for issue_num in sorted(changelog_issues): @@ -361,13 +589,14 @@ def cmd_report(args: argparse.Namespace) -> None: # These stay in the changelog (they are preserved, not removed) — the # human decides whether to label the issue or move the entry elsewhere. anomalies_e = [] - boom_body = extract_subsection(section, r"### :boom:") - if boom_body: - for issue_num in sorted(collect_changelog_issues(boom_body)): + boom_section_body = extract_subsection(section, r"### :boom:") + boom_issues = collect_changelog_issues(boom_section_body) + if boom_section_body: + for issue_num in sorted(boom_issues): labels = get_issue_labels(issue_num) if labels is None: continue - if "breaking change" not in labels: + if BREAKING_LABEL not in labels: issue = issue_by_num.get(issue_num) or external_issues.get(issue_num) or {} anomalies_e.append( { @@ -406,6 +635,20 @@ def cmd_report(args: argparse.Namespace) -> None: } ) + # Type G: `breaking change` issue missing from :boom:. Same contract as + # type F for highlights: the label is the only source of the section. + anomalies_g = [] + for issue_num, issue in sorted(issue_by_num.items()): + if issue_excluded(issue) or issue_num in boom_issues: + continue + if BREAKING_LABEL in issue.get("labels", []): + anomalies_g.append( + { + "issue": issue_num, + "issue_title": issue.get("title", ""), + } + ) + # Type C: released X.Y.0 sections without a :rocket: subsection. # Patches (X.Y.Z with Z != 0) never carry :rocket: — only minors/majors. anomalies_c = [] @@ -459,13 +702,14 @@ def cmd_report(args: argparse.Namespace) -> None: ) f.write("---\n\n") - n_a, n_b, n_c, n_d, n_e, n_f = ( + n_a, n_b, n_c, n_d, n_e, n_f, n_g = ( len(anomalies_a), len(anomalies_b), len(anomalies_c), len(anomalies_d), len(anomalies_e), len(anomalies_f), + len(anomalies_g), ) f.write("## Summary\n\n") @@ -476,14 +720,15 @@ def cmd_report(args: argparse.Namespace) -> None: f"- **PR in {milestone}, closing issue in a different milestone:** {n_b}\n" ) f.write(f"- **:boom: entry without breaking change label:** {n_e}\n") + f.write(f"- **:boom: out of sync with breaking change label:** {n_g}\n") f.write(f"- **:rocket: out of sync with release highlight label:** {n_f}\n") - f.write(f"- **Total anomalies:** {n_a + n_b + n_e + n_f}\n") + f.write(f"- **Total anomalies:** {n_a + n_b + n_e + n_f + n_g}\n") f.write( f"- **Released X.Y.0 version missing :rocket: section (gap):** {n_c}\n" ) f.write(f"- **:rocket: entry without issue AND PR references (gap):** {n_d}\n\n") - if n_a or n_b or n_e or n_f: + if n_a or n_b or n_e or n_f or n_g: f.write("## Anomalies\n\n") f.write( "Types A and B are milestone mismatches between an issue in the changelog " @@ -494,8 +739,9 @@ def cmd_report(args: argparse.Namespace) -> None: "removing the misleading entry from the changelog.\n\n" "Type E entries stay in the changelog: either label the issue " "as `breaking change` or move the entry out of `:boom:`.\n\n" - "Type F is a rule violation: `:rocket:` must list exactly the " - "issues labelled `release highlight`. Fix the label on GitHub " + "Types F and G are rule violations: `:rocket:` must list exactly " + "the issues labelled `release highlight`, and `:boom:` exactly " + "the issues labelled `breaking change`. Fix the label on GitHub " "or re-run the workflow.\n\n" ) @@ -556,6 +802,18 @@ def cmd_report(args: argparse.Namespace) -> None: f"- ⚠️ {issue_link_title(e['issue'], e['issue_title'][:80])}: {e['reason']}\n" ) f.write("\n") + + if n_g: + f.write("\n### :boom: out of sync with breaking change label\n\n") + f.write( + "These milestone issues carry the `breaking change` label but " + "have no entry under `### :boom: Breaking changes & Deprecations`. " + "Regenerate the section (`changelog.py boom --write`) or add " + "the entry.\n\n" + ) + for e in anomalies_g: + f.write(f"- ⚠️ {issue_link_title(e['issue'], e['issue_title'][:80])}\n") + f.write("\n") else: f.write( "✅ No anomalies found. All (issue, PR) pairs in the changelog have aligned milestone assignments.\n\n" @@ -590,7 +848,7 @@ def cmd_report(args: argparse.Namespace) -> None: for d in anomalies_d: f.write(f"- **{d['version']}**: `{d['line']}`\n") f.write("\n") - elif not (n_a or n_b or n_e or n_f): + elif not (n_a or n_b or n_e or n_f or n_g): f.write( "✅ No highlight gaps found. All released X.Y.0 versions have properly referenced :rocket: entries.\n\n" ) @@ -618,7 +876,7 @@ def cmd_report(args: argparse.Namespace) -> None: def main() -> None: parser = argparse.ArgumentParser( - description="Helper commands for the update-changelog skill" + description="Helper commands for the changelog flow" ) sub = parser.add_subparsers(dest="command", required=True, title="subcommands") @@ -633,6 +891,16 @@ def main() -> None: help="Read PR numbers from stdin (one per line)") p_check.set_defaults(func=cmd_check_merged) + p_boom = sub.add_parser( + "boom", help="Generate the :boom: section from `breaking change` issues" + ) + p_boom.add_argument("milestone", help="Milestone title (e.g. '2.16.0')") + p_boom.add_argument("--changes", default="CHANGES.md", + help="Path to the changelog file (default: CHANGES.md)") + p_boom.add_argument("--write", action="store_true", + help="Replace the :boom: section in CHANGES.md in place") + p_boom.set_defaults(func=cmd_boom) + p_cross = sub.add_parser( "cross-ref", help="Compare merged milestone PRs against the changelog section", diff --git a/scripts/gh.py b/scripts/gh.py index c430fbe843..2ec3eb8dc3 100755 --- a/scripts/gh.py +++ b/scripts/gh.py @@ -6,6 +6,8 @@ Uses GitHub GraphQL and REST APIs via the authenticated ``gh`` CLI. Subcommands: issues List issues in a milestone (or unassigned with milestone=none) + issue Fetch issue details by number (batched) + comments Fetch every comment of one or more issues prs Fetch details for one or more PRs (by number or milestone) advisories List or inspect GitHub security advisories link-issue Explicitly link a GitHub issue to a pull request @@ -26,6 +28,7 @@ Usage: python3 scripts/gh.py prs --milestone "2.16.0" (default: state=merged) python3 scripts/gh.py prs --milestone "2.16.0" --state all python3 scripts/gh.py issue 11235 11236 + python3 scripts/gh.py comments 11235 11236 python3 scripts/gh.py advisories (list all advisories) python3 scripts/gh.py advisories --severity critical (filter by severity) python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7 (single advisory detail) @@ -888,6 +891,51 @@ def cmd_issue(args: argparse.Namespace) -> None: print(json.dumps(all_results, indent=2)) +# ───────────────────────────────────────────── +# Subcommand: comments +# ───────────────────────────────────────────── + +COMMENTS_PER_PAGE = 100 + + +def fetch_issue_comments(issue_number: int) -> list[dict]: + """Fetch every comment of an issue via REST, oldest first.""" + comments: list[dict] = [] + page = 1 + while True: + batch = run_gh_rest( + f"repos/{REPO}/issues/{issue_number}/comments" + f"?per_page={COMMENTS_PER_PAGE}&page={page}" + ) + for comment in batch: + comments.append({ + "issue": issue_number, + "id": comment["id"], + "author": (comment.get("user") or {}).get("login"), + "created_at": comment.get("created_at"), + "body": comment.get("body"), + }) + if len(batch) < COMMENTS_PER_PAGE: + return comments + page += 1 + + +def cmd_comments(args: argparse.Namespace) -> None: + """Handle the ``comments`` subcommand (REST, paginated per issue).""" + issue_numbers = read_numbers_from_args(args) + if not issue_numbers: + print("ERROR: no issue numbers provided (pass numbers, --file, or --stdin)", + file=sys.stderr) + sys.exit(1) + + all_comments: list[dict] = [] + for num in issue_numbers: + print(f"Fetching comments for issue #{num}...", file=sys.stderr) + all_comments.extend(fetch_issue_comments(num)) + + print(json.dumps(all_comments, indent=2)) + + # ───────────────────────────────────────────── # Subcommand: advisories # ───────────────────────────────────────────── @@ -1080,6 +1128,24 @@ def main() -> None: ) p_issue.set_defaults(func=cmd_issue) + # --- comments --- + p_comments = sub.add_parser( + "comments", help="Fetch every comment of one or more issues" + ) + p_comments.add_argument( + "numbers", type=int, nargs="*", + help="Issue numbers to fetch (space-separated)" + ) + p_comments.add_argument( + "--file", type=str, + help="File with one issue number per line" + ) + p_comments.add_argument( + "--stdin", action="store_true", + help="Read issue numbers from stdin (one per line)" + ) + p_comments.set_defaults(func=cmd_comments) + # --- advisories --- p_adv = sub.add_parser("advisories", help="List or inspect GitHub security advisories") p_adv.add_argument( diff --git a/scripts/test_changelog.py b/scripts/test_changelog.py new file mode 100644 index 0000000000..5d6a19946e --- /dev/null +++ b/scripts/test_changelog.py @@ -0,0 +1,383 @@ +#!/usr/bin/env python3 +"""Tests for scripts/changelog.py. + +Run with: + + python3 scripts/test_changelog.py +""" + +import argparse +import contextlib +import importlib.machinery +import importlib.util +import io +import json +import pathlib +import sys +import tempfile +import unittest +from unittest.mock import patch + + +# Loading scripts/changelog.py should not emit scripts/__pycache__/. +sys.dont_write_bytecode = True + +SCRIPT_PATH = pathlib.Path(__file__).resolve().parent / "changelog.py" + + +def load_changelog(): + """Load scripts/changelog.py as a module without running its CLI.""" + loader = importlib.machinery.SourceFileLoader("changelog_helper", str(SCRIPT_PATH)) + spec = importlib.util.spec_from_loader("changelog_helper", loader) + module = importlib.util.module_from_spec(spec) + loader.exec_module(module) + return module + + +ch = load_changelog() + +CHANGES = """# CHANGELOG + +## 2.19.0 (Unreleased) + +### :boom: Breaking changes & Deprecations + +- Old entry. + +### :rocket: Epics and highlights + +- Add highlights [#1](https://github.com/penpot/penpot/issues/1) +""" + + +class ExtractBreakingNoteTests(unittest.TestCase): + def test_uses_the_whole_comment_after_the_marker(self): + body = ( + "## Breaking change note\n\n" + "First paragraph.\n\n" + "- bullet one\n" + "- bullet two\n\n" + "```sql\nSELECT 1;\n```\n" + ) + self.assertEqual( + ch.extract_breaking_note(body), + "First paragraph.\n\n- bullet one\n- bullet two\n\n" + "```sql\nSELECT 1;\n```", + ) + + def test_accepts_any_heading_level_for_the_marker(self): + self.assertEqual( + ch.extract_breaking_note("# breaking change note\n\nText."), "Text." + ) + self.assertEqual( + ch.extract_breaking_note("### Breaking Change Note\n\nText."), "Text." + ) + + def test_returns_none_without_marker(self): + self.assertIsNone(ch.extract_breaking_note("### Description\n\ntext\n")) + self.assertIsNone(ch.extract_breaking_note("## Breaking change notes\n\ntext")) + self.assertIsNone(ch.extract_breaking_note(None)) + self.assertIsNone(ch.extract_breaking_note("")) + + def test_returns_none_for_a_marker_without_content(self): + self.assertIsNone(ch.extract_breaking_note("## Breaking change note\n")) + + +class DemoteHeadingsTests(unittest.TestCase): + def test_demotes_headings_below_the_entry_level(self): + markdown = "# One\n## Two\n### Three\n#### Four\n##### Five\n" + self.assertEqual( + ch.demote_headings(markdown), + "##### One\n##### Two\n##### Three\n##### Four\n##### Five", + ) + + def test_keeps_headings_inside_code_fences(self): + markdown = ( + "```bash\n# not a heading\n```\n" + "## Real heading\n" + "~~~\n### not a heading\n~~~\n" + ) + self.assertEqual( + ch.demote_headings(markdown), + "```bash\n# not a heading\n```\n" + "##### Real heading\n" + "~~~\n### not a heading\n~~~", + ) + + +class StripTitlePrefixTests(unittest.TestCase): + def test_strips_shortcodes(self): + self.assertEqual(ch.strip_title_prefix(":boom: Enable it"), "Enable it") + self.assertEqual(ch.strip_title_prefix(":bug: :tada: Fix it"), "Fix it") + self.assertEqual(ch.strip_title_prefix("Plain title"), "Plain title") + + +class IssueExcludedTests(unittest.TestCase): + def issue(self, **overrides): + base = { + "state": "CLOSED", + "issue_type": "Enhancement", + "project_status": "In progress", + "labels": [], + } + base.update(overrides) + return base + + def test_keeps_a_valid_closed_issue(self): + self.assertFalse(ch.issue_excluded(self.issue())) + + def test_excludes_open_task_rejected_and_labelled(self): + self.assertTrue(ch.issue_excluded(None)) + self.assertTrue(ch.issue_excluded(self.issue(state="OPEN"))) + self.assertTrue(ch.issue_excluded(self.issue(issue_type="Task"))) + self.assertTrue(ch.issue_excluded(self.issue(project_status="Rejected"))) + self.assertTrue(ch.issue_excluded(self.issue(labels=["release blocker"]))) + self.assertTrue(ch.issue_excluded(self.issue(labels=["no changelog"]))) + + +class RenderBoomEntryTests(unittest.TestCase): + def test_entry_is_a_heading_plus_note_plus_references(self): + issue = {"number": 7, "title": ":boom: Drop the flag", "labels": []} + self.assertEqual( + ch.render_boom_entry(issue, "One.\n\nTwo.", [8, 9]), + "#### Drop the flag\n\nOne.\n\nTwo.\n\n" + "[#7](https://github.com/penpot/penpot/issues/7)" + " (PR: [#8](https://github.com/penpot/penpot/pull/8)," + " [#9](https://github.com/penpot/penpot/pull/9))", + ) + + def test_entry_without_a_note_keeps_the_heading(self): + issue = {"number": 7, "title": "Drop the flag", "labels": []} + self.assertEqual( + ch.render_boom_entry(issue, None, []), + "#### Drop the flag\n\n[#7](https://github.com/penpot/penpot/issues/7)", + ) + + def test_community_attribution_goes_before_the_issue_link(self): + issue = {"number": 7, "title": "Drop the flag", "labels": []} + entry = ch.render_boom_entry(issue, "Text.", [8], "someone") + self.assertEqual( + entry, + "#### Drop the flag\n\nText.\n\n(by @someone)" + " [#7](https://github.com/penpot/penpot/issues/7)" + " (PR: [#8](https://github.com/penpot/penpot/pull/8))", + ) + + +class ReplaceBoomSectionTests(unittest.TestCase): + def test_replaces_the_existing_subsection(self): + entry = "#### New entry\n\nSome text.\n\n[#9](https://example.test/9)" + updated = ch.replace_boom_section(CHANGES, "2.19.0", [entry]) + self.assertIn( + "### :boom: Breaking changes & Deprecations\n\n" + f"{entry}\n\n### :rocket:", + updated, + ) + self.assertNotIn("Old entry.", updated) + + def test_replacement_is_idempotent(self): + entry = "#### New entry\n\nSome text.\n\n[#9](https://example.test/9)" + once = ch.replace_boom_section(CHANGES, "2.19.0", [entry]) + self.assertEqual(once, ch.replace_boom_section(once, "2.19.0", [entry])) + + def test_empty_section_keeps_the_heading(self): + updated = ch.replace_boom_section(CHANGES, "2.19.0", []) + self.assertIn( + "### :boom: Breaking changes & Deprecations\n\n### :rocket:", updated + ) + + def test_inserts_the_subsection_when_missing(self): + without = CHANGES.replace( + "### :boom: Breaking changes & Deprecations\n\n- Old entry.\n\n", "" + ) + updated = ch.replace_boom_section(without, "2.19.0", ["#### New entry"]) + self.assertIn( + "## 2.19.0 (Unreleased)\n\n" + "### :boom: Breaking changes & Deprecations\n\n#### New entry\n\n" + "### :rocket:", + updated, + ) + + def test_unknown_version_raises(self): + with self.assertRaises(ValueError): + ch.replace_boom_section(CHANGES, "9.9.9", ["#### Entry"]) + + +ISSUES = [ + { + "number": 21, + "title": ":boom: New default", + "state": "CLOSED", + "issue_type": "Enhancement", + "labels": ["breaking change"], + "closing_prs": [31], + "project_status": "Done", + }, + { + "number": 22, + "title": "No note", + "state": "CLOSED", + "issue_type": "Enhancement", + "labels": ["breaking change"], + "closing_prs": [], + "project_status": "Done", + }, + { + "number": 23, + "title": "Still open", + "state": "OPEN", + "issue_type": "Enhancement", + "labels": ["breaking change"], + "closing_prs": [], + "project_status": "In progress", + }, + { + "number": 24, + "title": "Not breaking", + "state": "CLOSED", + "issue_type": "Bug", + "labels": [], + "closing_prs": [], + "project_status": "Done", + }, +] + +COMMENTS = [ + {"issue": 21, "body": "## Breaking change note\n\nDeploy the new service.\n"}, + {"issue": 22, "body": "## Description\n\nNo marker here.\n"}, + {"issue": 21, "body": "## Breaking change note\n\nDeploy the new service.\n"}, +] + +PRS = [{"number": 31, "author": "someone", "labels": [], "state": "MERGED"}] + + +class CmdBoomTests(unittest.TestCase): + def setUp(self): + self.calls = [] + self.issues = ISSUES + self.comments = COMMENTS + self.prs = PRS + + def fake_run_gh_py(*args, input_text=None): + self.calls.append(args) + if args[0] == "issues": + return json.dumps(self.issues) + if args[0] == "comments": + return json.dumps(self.comments) + if args[0] == "prs": + return json.dumps(self.prs) + raise AssertionError(f"unexpected gh.py call: {args}") + + patcher = patch.object(ch, "run_gh_py", side_effect=fake_run_gh_py) + patcher.start() + self.addCleanup(patcher.stop) + + def run_boom(self, **kwargs): + args = argparse.Namespace( + milestone="2.19.0", + changes=CHANGES if "changes" not in kwargs else kwargs["changes"], + write=kwargs.get("write", False), + ) + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + ch.cmd_boom(args) + return out.getvalue(), err.getvalue() + + def test_renders_only_closed_breaking_issues(self): + out, err = self.run_boom() + expected = ( + "### :boom: Breaking changes & Deprecations\n\n" + "#### New default\n\n" + "Deploy the new service.\n\n" + "[#21](https://github.com/penpot/penpot/issues/21)" + " (PR: [#31](https://github.com/penpot/penpot/pull/31))\n\n" + "#### No note\n\n" + "[#22](https://github.com/penpot/penpot/issues/22)\n" + ) + self.assertEqual(out, expected) + self.assertIn("issue #22 has no", err) + self.assertIn("using the last one", err) + + def test_unmerged_closing_pr_is_not_referenced(self): + self.prs = [{"number": 31, "author": "someone", "state": "CLOSED"}] + out, _ = self.run_boom() + self.assertNotIn("pull/31", out) + self.assertIn("[#21](https://github.com/penpot/penpot/issues/21)", out) + + def test_no_labelled_issue_clears_the_section_on_write(self): + self.issues = [ISSUES[3]] + with tempfile.TemporaryDirectory() as tmp: + path = pathlib.Path(tmp) / "CHANGES.md" + path.write_text(CHANGES) + out, err = self.run_boom(changes=str(path), write=True) + updated = path.read_text() + self.assertEqual(out, "### :boom: Breaking changes & Deprecations\n") + self.assertNotIn("Old entry.", updated) + self.assertIn(":boom: section updated", err) + + def test_writes_the_section_in_place(self): + with tempfile.TemporaryDirectory() as tmp: + path = pathlib.Path(tmp) / "CHANGES.md" + path.write_text(CHANGES) + out, err = self.run_boom(changes=str(path), write=True) + updated = path.read_text() + self.assertIn("#### New default", updated) + self.assertNotIn("Old entry.", updated) + self.assertIn(":boom: section updated", err) + self.assertIn("Deploy the new service.", out) + + +class ReportTests(unittest.TestCase): + """Focused check of the `:boom:` sync anomaly (type G) in the report.""" + + def test_reports_breaking_issue_missing_from_boom(self): + issues = [ + { + "number": 1, + "title": "Unlabelled entry", + "state": "CLOSED", + "issue_type": "Enhancement", + "labels": [], + "closing_prs": [], + "project_status": "Done", + }, + { + "number": 2, + "title": "Labelled, not listed", + "state": "CLOSED", + "issue_type": "Enhancement", + "labels": ["breaking change"], + "closing_prs": [], + "project_status": "Done", + }, + ] + changes = ( + "# CHANGELOG\n\n## 2.19.0 (Unreleased)\n\n" + "### :boom: Breaking changes & Deprecations\n\n" + "#### Something changed\n\n" + "[#1](https://github.com/penpot/penpot/issues/1)\n" + ) + with tempfile.TemporaryDirectory() as tmp: + changes_path = pathlib.Path(tmp) / "CHANGES.md" + output_path = pathlib.Path(tmp) / "ISSUES.md" + changes_path.write_text(changes) + args = argparse.Namespace( + milestone="2.19.0", + changes=str(changes_path), + output=str(output_path), + ) + with patch.object(ch, "run_gh_py", return_value=json.dumps(issues)), \ + patch.object(ch, "fetch_milestone_prs", return_value=[]), \ + contextlib.redirect_stdout(io.StringIO()), \ + contextlib.redirect_stderr(io.StringIO()): + ch.cmd_report(args) + report = output_path.read_text() + + self.assertIn("**:boom: out of sync with breaking change label:** 1", report) + self.assertIn("### :boom: out of sync with breaking change label", report) + self.assertIn("issues/2", report) + self.assertIn("**:boom: entry without breaking change label:** 1", report) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/scripts/test_gh.py b/scripts/test_gh.py index 5fb6840a22..3255e8b495 100644 --- a/scripts/test_gh.py +++ b/scripts/test_gh.py @@ -110,5 +110,51 @@ class LinkIssueTests(unittest.TestCase): self.assertIn("link missing", stderr.getvalue()) +class CommentsTests(unittest.TestCase): + def test_paginates_until_a_short_page(self): + page_one = [ + {"id": i, "user": {"login": "u"}, "created_at": "t", "body": f"b{i}"} + for i in range(100) + ] + page_two = [{"id": 100, "user": None, "created_at": "t", "body": "last"}] + calls = [] + + def fake_rest(path): + calls.append(path) + return page_one if len(calls) == 1 else page_two + + with patch.object(gh, "run_gh_rest", side_effect=fake_rest): + comments = gh.fetch_issue_comments(7) + + self.assertEqual(len(comments), 101) + self.assertEqual( + comments[0], + {"issue": 7, "id": 0, "author": "u", "created_at": "t", "body": "b0"}, + ) + self.assertIsNone(comments[-1]["author"]) + self.assertIn("page=1", calls[0]) + self.assertIn("page=2", calls[1]) + + @patch.object(gh, "fetch_issue_comments") + def test_cmd_comments_outputs_every_issue(self, fetch_comments): + fetch_comments.side_effect = lambda num: [ + {"issue": num, "id": num, "author": "u", "created_at": "t", "body": "x"} + ] + args = types.SimpleNamespace(numbers=[1, 2], file=None, stdin=False) + stdout = io.StringIO() + + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(io.StringIO()): + gh.cmd_comments(args) + + self.assertEqual([c["issue"] for c in json.loads(stdout.getvalue())], [1, 2]) + + def test_cmd_comments_requires_numbers(self): + args = types.SimpleNamespace(numbers=[], file=None, stdin=False) + with contextlib.redirect_stderr(io.StringIO()): + with self.assertRaises(SystemExit) as error: + gh.cmd_comments(args) + self.assertEqual(error.exception.code, 1) + + if __name__ == "__main__": unittest.main(verbosity=2)