Compare commits

..

No commits in common. "develop" and "2.20.0-RC1" have entirely different histories.

752 changed files with 10205 additions and 63492 deletions

View File

@ -337,32 +337,28 @@ small. To delegate without switching models, delegate to `general`.
report with your intent: "investigate this and find the possible cause",
"investigate and tell me where this points", or "does this still apply?".
Explore until you and the agent roughly agree on the problem.
2. **Write the plan.** Invoke `@make-a-plan`. Skills use the current agent;
select Build mode first when the skill needs to write files. Use Plan mode
only when you want a hard guarantee that the agent modifies no file under
any circumstance. If you
2. **Write the plan.** Run `/make-a-plan` — it executes in Build mode.
If you need to step in and answer something yourself, press TAB to
leave Build mode. Use Plan mode only when you want a hard guarantee
that the agent modifies no file under any circumstance. If you
explored with a weaker model but want a stronger one to write the
plan, switch models first or delegate:
`@make-a-plan delegate to @engineer-glm`.
`/make-a-plan delegate to @engineer-glm`.
3. **Iterate on the plan.** The plan is saved to `.agents/plans/`, so you
never depend on LLM memory: read the file directly, or invoke `@review-plan`
for a second opinion (delegation works here too). It applies clear plan
improvements and discusses consequential decisions one by one before
applying them. Complex plans deserve a review; simple ones can skip it.
4. **Execute.** Invoke `@implement-plan`. It first prints the full picture —
never depend on LLM memory: read the file directly, or run `/review-plan`
for a second opinion (delegation works here too). Complex plans deserve a
review; simple ones can skip it.
4. **Execute.** Run `/implement-plan`. It first prints the full picture —
whether it will create an issue and a branch, the execution style, and a
task checklist — and waits for your go-ahead. Say "step by step" to stop
after each task (one commit per task) so you can verify as it goes;
the default runs all tasks with one final commit. For a plan based on a
code review, it then asks whether to have the original reviewer verify the
corrections. If that session is unavailable, it tells you and uses a new
reviewer with the review and implementation context.
5. **Land the work.** When it finishes, either push yourself and invoke
`@create-pr`, or loop `@review-code` → `@make-a-plan` →
`@implement-plan` until the findings are addressed, then push and invoke
`@create-pr`. Nothing pushes for you — you always push from your shell.
the default runs all tasks with one final commit.
5. **Land the work.** When it finishes, either push yourself and run
`/create-pr`, or loop `/review-code` → `/make-a-plan` →
`/implement-plan` until the findings are addressed, then push and
`/create-pr`. Nothing pushes for you — you always push from your shell.
> Note: `@implement-plan` checks the current branch. On a base branch
> Note: `/implement-plan` checks the current branch. On a base branch
> (`main`, `develop`, `staging`) it creates a GitHub issue and a branch
> `issue-NNNN`; on an existing feature branch it continues there and
> creates nothing. The pre-run summary tells you which applies. Read the
@ -371,14 +367,14 @@ small. To delegate without switching models, delegate to `general`.
### Big feature with multiple plans
When the work is too large for a single plan, tell `@make-a-plan`
When the work is too large for a single plan, tell `/make-a-plan`
up front: produce a high-level roadmap where each task will get its
own execution plan, and the roadmap doubles as the progress tracker.
From there the flow mirrors the issue flow above, one level down:
take each roadmap task in turn, write its own plan (`@make-a-plan`,
take each roadmap task in turn, write its own plan (`/make-a-plan`,
delegating when it helps), review it when the task is complex
(`@review-plan`), implement it (`@implement-plan`), and mark progress
(`/review-plan`), implement it (`/implement-plan`), and mark progress
on the roadmap as you land each piece.
### Plan file naming (base + derivatives)
@ -396,7 +392,7 @@ keeps everything adjacent in `ls`:
`2026-09-20-upload-pipeline-roadmap--task-01-chunk-upload.md`
- Chained: `...--task-02-gc--review-01.md`
While a plan is still unimplemented, `@make-a-plan` revises it in
While a plan is still unimplemented, `/make-a-plan` revises it in
place. Once implemented and reviewed, it writes a new followup file.
Full rules live in the `planner` skill.
@ -405,10 +401,11 @@ Full rules live in the `planner` skill.
Every plan carries `Status: draft | reviewed | done` plus an
append-only `Review Log` (UTC ISO 8601, one line per entry):
- `draft` — fresh from `@make-a-plan`.
- `reviewed` — review feedback incorporated. `@review-plan` applies clear
improvements and agreed decisions, then updates the status and log.
- `done` — `@implement-plan` closes it on completion, logging the
- `draft` — fresh from `/make-a-plan`.
- `reviewed` — review feedback incorporated. `/review-plan` stays
read-only; only your explicit "apply" makes `/make-a-plan` apply
the changes, flip the status, and log one line.
- `done` — `/implement-plan` closes it on completion, logging the
issue URL when one exists (never commit hashes).
A plan approved with no changes goes `draft` → `done` directly.
@ -508,6 +505,7 @@ JSON, REPL access, and so on.
| [`ripgrep`](skills/ripgrep/SKILL.md) | Fast content search with regular expressions. |
| [`jq-json-processor`](skills/jq-json-processor/SKILL.md) | Slice, filter, and reshape JSON output. |
| [`nrepl-eval`](skills/nrepl-eval/SKILL.md) | Run Clojure or ClojureScript code in the live REPL sessions (backend and frontend). |
| [`taiga`](skills/taiga/SKILL.md) | Look up Penpot issues, user stories, and tasks in Taiga. |
| [`testing`](skills/testing/SKILL.md) | The repo's testing rules and TDD workflow, loaded before writing tests. |
| [`local-ci`](skills/local-ci/SKILL.md) | Run CI-style lint, test, and format checks for the modules you touched with `scripts/ci`, and read the logs when they fail. |
| [`security-and-hardening`](skills/security-and-hardening/SKILL.md) | Security checks for code that handles user input, auth, or external services. |
@ -518,11 +516,11 @@ JSON, REPL access, and so on.
### A typical round
1. `@make-a-plan` — you get a plan and a saved file in `.agents/plans/`.
2. `@review-plan` — a second opinion; approve or request changes.
3. `@implement-plan` — the code gets written and committed. Starting from a base branch, it also opens the GitHub issue and the `issue-NNNN` branch; the plans that follow continue on that same branch.
4. `@review-code` — a reviewer checks the commit.
5. `@create-pr` — the branch goes up as a pull request.
1. `/make-a-plan` — you get a plan and a saved file in `.agents/plans/`.
2. `/review-plan` — a second opinion; approve or request changes.
3. `/implement-plan` — the code gets written and committed. Starting from a base branch, it also opens the GitHub issue and the `issue-NNNN` branch; the plans that follow continue on that same branch.
4. `/review-code` — a reviewer checks the commit.
5. `/create-pr` — the branch goes up as a pull request.
Every step also works on its own, and you can always say what you want
in plain words — the agents pick the right skill from what you say.

View File

@ -1,95 +1,246 @@
---
name: code-review-criteria
description: Code review criteria for correctness, clarity, architecture, security and performance. Loaded by the review-code subagent.
description: Code review criteria — the five review axes, core principles, severity format, and verdict for reviewing code changes. Loaded by the reviewer subagent of the review-code flow. Not a user-facing flow — to review code, use the review-code flow.
---
# Code Review Criteria
# Code Review Criteria and Quality
Assess whether the change improves the codebase and is safe to merge. Review
the diff in context, its intent and relevant tests. Follow project conventions
and module memories. Do not block on personal preferences or invent problems.
Apply each axis in proportion to the change and its risks.
## Overview
Follow `review-code` for delegation and its read-only contract. Never run tests,
builds, linters or other state-changing checks during a review or follow-up.
Inspect prior test results or CI evidence when available. Identify their source
and scope; if evidence is missing, stale or inaccessible, say what is
unverified. Never claim a suite passed without evidence.
Multi-dimensional code review with quality gates. Every change gets reviewed before merge — no exceptions. Review covers five axes: correctness, readability, architecture, security, and performance.
## Review axes
**The approval standard:** Approve a change when it definitely improves overall code health, even if it isn't perfect. Perfect code doesn't exist — the goal is continuous improvement. Don't block a change because it isn't exactly how you would have written it. If it improves the codebase and follows the project's conventions, approve it.
### Correctness and tests
## When to Use
Check requirements, relevant boundaries, error paths, concurrency and state.
Use the module's contracts to decide where validation belongs; do not demand
redundant guards inside functions with established preconditions. Check that
new functionality has tests covering its behavior and important failure
cases, using the appropriate layer and project testing guidance. Missing
required tests are findings; documentation-only changes need not add tests.
Do not infer that a test failed before a fix without evidence.
- The reviewer subagent of the `review-code` flow loads this skill to perform
the review of a code change.
- To review code, always go through the `review-code` flow — never load this
skill directly for that. This is the criteria reference, not the flow.
### Readability and simplicity
## Core Principles
Check names, control flow and maintainability against existing conventions.
Flag duplication when it creates a concrete risk of drift or maintenance
cost, not merely because code appears more than once. In tests, prefer
self-contained, readable cases even when they repeat setup. Suggest the
simplest remedy that preserves behavior.
These principles underpin every axis. When in doubt, default to them.
### Architecture and compatibility
- **DRY (Don't Repeat Yourself):** Every piece of knowledge has one authoritative representation. If the same logic appears in two places, extract it into a shared helper, model, or type. Reviewers: flag duplicated logic as a required change — it's not "just similar," it's drift that will diverge.
- **KISS (Keep It Simple, Stupid):** The simplest solution that works is the best solution. Complexity must earn its place. Reviewers: if you need more than one sentence to explain what a piece of code does, it's too complex — push for simplification before merge.
- **YAGNI (You Aren't Gonna Need It):** Don't add abstractions, hooks, or generalizations for hypothetical future use cases. Generalize on the third occurrence, not the first. Reviewers: delete speculative generality.
- **Don't invent problems:** Do not manufacture issues to produce more feedback. Every finding must be a real risk, a real readability barrier, or a real architectural concern — not a hypothetical or a stylistic preference disguised as a problem.
Check module boundaries, existing abstractions, contracts and unnecessary
complexity. Review surrounding code when needed to understand the change.
Do not label a compatibility shim dead code without checking its purpose.
For file formats or data model changes, consult the relevant compatibility,
migration and feature memories; for Clojure, logging and translations, read
the corresponding guidance when relevant.
## The Five-Axis Review
### Security
Every review evaluates code across these dimensions.
Evaluate relevant input boundaries, authorization, secrets, data handling and
external integrations. Consult project security guidance for the affected
technology; do not assume a generic framework or tool applies to this module.
### 1. Correctness
### Performance
Does the code do what it claims to do?
Assess relevant hot paths, resource bounds, data access and scaling. Explain
the workload or conditions in which a problem matters; do not flag speculative
optimizations without a concrete risk.
- Does it match the spec or task requirements?
- Are edge cases handled (null, empty, boundary values)?
- Are error paths handled (not just the happy path)?
- Does it pass all tests? Are the tests actually testing the right things?
- Are there off-by-one errors, race conditions, or state inconsistencies?
## Scope and dependencies
### 2. Readability & Simplicity
Review relevant generated files or lockfile changes when they affect the
contract, dependencies or security; do not skip a lockfile-only dependency
change. Check dependency changes against available changelogs, project tools
and relevant lockfile diffs. Treat size as a signal to assess reviewability,
not an automatic finding. Identify unrelated work in the target diff when it
materially impedes review; do not review unrelated working-tree changes.
Can another engineer (or agent) understand this code without the author explaining it?
## Findings and verdict
- Are names descriptive and consistent with project conventions? (No `temp`, `data`, `result` without context)
- Is the control flow straightforward (avoid nested ternaries, deep callbacks)?
- Are there any "clever" tricks that should be simplified?
- **KISS check:** Is this the simplest approach that solves the problem? A 20-line straightforward function beats a 5-line clever one that requires a comment to explain.
- Could this be done in fewer lines? (1000 lines where 100 suffice is a failure)
- Are abstractions earning their complexity? (Don't generalize until the third use case)
- Is a new conditional bolted onto an unrelated flow? Push the logic into its own helper, state, or policy.
- Do repeated conditionals on the same shape appear? They signal a missing model or dispatcher.
- Are there dead code artifacts: no-op variables, backwards-compat shims, or `// removed` comments?
Assign each finding a stable ID (`F1`, `F2`, …), independent of severity.
Never renumber it within the review or follow-up; severity may change as
evidence changes. New findings receive unused IDs.
### 3. Architecture
For each finding state severity, file and location, concrete circumstances,
impact, remedy and the tests needed to verify it; explain when no new test is
needed. If a consequential decision requires user input, give the real options,
consequences and recommendation within that finding, without pausing to ask.
Does the change fit the system's design?
- **Critical:** security incident, data loss or comparably severe breakage;
blocks merge.
- **High:** a significant defect or missing required test; must be addressed.
- **Medium:** a substantive issue worth fixing but not a merge blocker.
- **Low / Suggestion:** an optional improvement with a concrete benefit.
- Does it follow existing patterns or introduce a new one? If new, is it justified?
- Does it maintain clean module boundaries?
- **DRY check:** Is there existing code that does the same thing? Reuse the canonical helper instead of writing a near-duplicate. If two branches do nearly the same thing, collapse them.
- Are dependencies flowing in the right direction (no circular dependencies)?
- Is the abstraction level appropriate (not over-engineered, not too coupled)?
- Does this refactor reduce complexity or just relocate it? Count the concepts a reader must hold. Prefer the restructuring that makes whole branches disappear over one that re-centralizes the same logic. Prefer deleting an abstraction to polishing it.
- Is feature-specific logic leaking into a shared or general-purpose module?
- Are type boundaries explicit? Question gratuitous `any`/`unknown`/optional/casts and silent fallbacks.
- **Structural remedies:** When you flag a problem, propose the move — not just the problem. Replace conditionals with dispatchers, collapse duplicate branches, separate orchestration from business logic, extract helpers, split large files. Prefer the remedy that removes moving pieces over one that spreads the same complexity around.
Base severity on impact and project requirements. Missing tests required for
new functionality must be addressed; do not invent a test requirement for
changes where none applies.
### 4. Security
Return a short summary, findings ordered by impact and a verdict with the
relevant IDs: **Approve**, **Approve with minor changes**, or **Request
changes**. State verification evidence and limitations. Do not duplicate
remedies or tests in separate sections. If no findings exist, say so. In
follow-up, retain original IDs and
report resolved, pending and not-addressed findings as `review-code` requires.
For detailed security guidance, see `security-and-hardening`.
- Is user input validated and sanitized?
- Are secrets kept out of code, logs, and version control?
- Is authentication/authorization checked where needed?
- Are SQL queries parameterized (no string concatenation)?
- Are outputs encoded to prevent XSS?
- Are dependencies from trusted sources with no known vulnerabilities?
- Is data from external sources (APIs, logs, user content, config files) treated as untrusted?
### 5. Performance
- Any N+1 query patterns?
- Any unbounded loops or unconstrained data fetching?
- Any synchronous operations that should be async?
- Any unnecessary re-renders in UI components?
- Any missing pagination on list endpoints?
- Any large objects created in hot paths?
## Review Process
1. **Understand the intent** — What is this change trying to accomplish? What spec or task does it implement?
2. **Review tests first** — Tests reveal intent and coverage. Do they test behavior, not implementation details? Are edge cases covered?
3. **Review the implementation** — Walk through each file with the five axes in mind.
4. **Categorize findings** — Label every comment with its severity:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| **Critical:** | Blocks merge | Security vulnerability, data loss, broken functionality |
| **High:** | Required change | Must address before merge |
| **Medium:** | Should fix | Strongly recommended, not a blocker |
| **Low:** | Minor, optional | Author may ignore — formatting, style preferences |
| **Suggestion:** | Worth considering | Not required, but improves the code |
**Finding IDs.** Assign every finding a stable identifier by severity class: `R1`, `R2`, … for required findings (Critical, High), `M1`, `M2`, … for minor findings (Medium, Low), and `S1`, `S2`, … for suggestions. Number within each class in review order. Use the ID everywhere the finding is mentioned — in section headers, in the verdict, in follow-up discussion. Never renumber within a review. Example: `**R2** — `app/validate.cljs:42` — duplicate branch logic…`.
For each finding, describe the circumstances under which it could fail: specific inputs, load conditions, timing, or user actions that trigger the problem. "This crashes when input is null" is actionable; "this might crash" is not.
Lead with what matters: correctness and security first, then structural issues, then everything else. A few high-conviction comments beat a long list.
5. **Verify the verification** — What tests were run? Did the build pass? Was the change tested manually? Screenshots for UI changes?
## Review Output
Structure every review using this format:
### Summary
Briefly explain what the code does and give an overall assessment.
### Critical and High-Priority Issues
List problems that could cause security incidents, data loss, crashes, incorrect behavior, or major performance degradation. Each finding gets its unique ID (`R1`, `R2`, …). For each: state the severity, identify the file/function/code section, explain why it's a problem, describe failure circumstances, and provide a concrete improvement with corrected code when useful.
### Other Findings
List medium- and low-priority issues, including maintainability and design concerns. Use `M1`, `M2`, … for Medium/Low and `S1`, `S2`, … for Suggestions.
### Suggested Refactoring
Provide focused code changes or revised snippets. Preserve existing behavior unless a behavior change is explicitly justified.
### Testing Recommendations
Identify missing tests and describe specific test cases, including edge cases and failure scenarios.
### Positive Observations
Mention implementation choices that are clear, safe, efficient, or well designed. This is not fluff — it reinforces good patterns and tells the author what to keep doing.
### Final Verdict
Choose one:
- **Approve** — Ready to merge
- **Approve with minor changes** — Good to merge after addressing low/medium issues
- **Request changes** — Critical or high issues must be resolved before merge
List the finding IDs the verdict depends on (e.g. "Request changes: R1, R2").
## Change Sizing
Small, focused changes are easier to review, faster to merge, and safer to deploy.
```
~100 lines changed → Good. Reviewable in one sitting.
~300 lines changed → Acceptable if it's a single logical change.
~1000 lines changed → Too large. Split it.
```
**Watch file size, not just diff size.** Around 1000 *total* lines in a single file is a common inspection signal. When a change materially grows an already-large file, decompose first.
**Splitting strategies:**
| Strategy | How | When |
|----------|-----|------|
| **Stack** | Submit a small change, start the next one based on it | Sequential dependencies |
| **By file group** | Separate changes for groups needing different reviewers | Cross-cutting concerns |
| **Horizontal** | Create shared code/stubs first, then consumers | Layered architecture |
| **Vertical** | Break into smaller full-stack slices of the feature | Feature work |
**Separate refactoring from feature work.** A change that refactors and adds new behavior is two changes — submit them separately.
## Change Descriptions
- **First line:** Short, imperative, standalone. "Delete the FizzBuzz RPC" not "Deleting the FizzBuzz RPC."
- **Body:** What is changing and why. Include context and reasoning not visible in the code itself.
- **Anti-patterns:** "Fix bug," "Fix build," "Add patch," "Phase 1."
## Dependencies
Before adding any dependency:
1. Does the existing stack solve this? (Often it does.)
2. How large is the dependency? (Check bundle impact.)
3. Is it actively maintained? (Check last commit, open issues.)
4. Does it have known vulnerabilities? (`npm audit`)
5. What's the license? (Must be compatible with the project.)
**Rule:** Prefer standard library and existing utilities over new dependencies. Every dependency is a liability.
**Upgrading dependencies:**
- Read the changelog, not just the version number. Semver is a promise the maintainer may not have kept.
- One dependency per change. When a bulk bump breaks the build, you've lost which package did it.
- Let the tests decide — a green suite before *and* after, not just "it installed."
- Review the lockfile diff, not just `package.json`. Commit it and never hand-edit it.
For supply-chain risk triage, follow the `security-and-hardening` skill.
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "It works, that's good enough" | Working code that's unreadable, insecure, or architecturally wrong creates debt that compounds. |
| "I wrote it, so I know it's correct" | Authors are blind to their own assumptions. Every change benefits from another set of eyes. |
| "We'll clean it up later" | Later never comes. The review is the quality gate — use it. |
| "AI-generated code is probably fine" | AI code needs more scrutiny, not less. It's confident and plausible, even when wrong. |
| "The tests pass, so it's good" | Tests are necessary but not sufficient. They don't catch architecture, security, or readability problems. |
| "The refactor makes it cleaner" | Relocating complexity isn't reducing it. If the reader still holds the same number of concepts, the structure didn't improve. |
| "It's only a small addition to this file" | Small diffs still push files past healthy size and bolt branches onto unrelated flows. |
| "It's just a version bump" | A bump is a behavior change you didn't write. Read the changelog. |
| "I'll upgrade everything in one PR" | A bulk bump hides which package broke the build. One per change. |
| "It's duplicated but it's only two places" | Two becomes three becomes five. Extract now, before the copies diverge. |
| "The abstraction is future-proof" | YAGNI. Delete speculative generality — generalize on the third occurrence, not the first. |
| "It's clever but efficient" | Cleverness is a readability tax. If it needs a comment to understand, simplify it. |
## Red Flags
- PRs merged without any review
- Review that only checks if tests pass (ignoring other axes)
- "LGTM" without evidence of actual review
- Security-sensitive changes without security-focused review
- Large PRs that are "too big to review properly" (split them)
- No regression tests with bug fix PRs
- Accepting "I'll fix it later" — it never happens
- A refactor that moves code around without reducing the number of concepts a reader must hold
- New conditionals scattered into unrelated code paths (a missing abstraction)
- A bespoke helper that duplicates an existing canonical one
- A bulk "bump dependencies" PR with no changelog review
## Verification
Before emitting the verdict, verify the change as it stands. This is the reviewer's own due diligence — it covers the state of the code at review time, not the later resolution of findings (fixing findings is the author's job; confirming them is a new review):
- [ ] Tests pass — run them yourself, don't trust the claim
- [ ] Build succeeds
- [ ] The verification story is documented (what changed, how it was verified)
- [ ] Dependency upgrades reviewed against changelog, isolated per package, verified by green suite
## See Also
- For detailed security review guidance, see `security-and-hardening`

View File

@ -7,7 +7,7 @@ slash: true
# Skill: create-commit
Produce a git commit that follows Penpot's commit message conventions. This
skill handles staging review and safety checks — it does not
skill owns the commit format, staging review, and safety checks — it does not
implement features or push.
## When to Use
@ -23,9 +23,15 @@ subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
## Iron Rules (non-negotiable)
Follow the message rules and exceptions in `mem:workflow/creating-commits`.
**Run `./scripts/check-commit` and require exit code 0.** A non-zero exit is a
hard blocker: fix the message and re-check it before reporting success.
1. **Wrap every body line at 80 characters or fewer.** Count characters, do
not eyeball. Exceptions: `Signed-off-by:` / `AI-assisted-by:` trailers and
lines carrying a URL. This is the rule agents skip most often.
2. **Subject ≤80 chars**, imperative, capitalized, no trailing period.
`Merge`/`Revert`/`Reapply` commits skip the length check.
3. **Blank line between subject and body.**
4. **Run `./scripts/check-commit` and require exit code 0.** It mechanically
checks rules 1–3. A non-zero exit is a hard blocker: fix the message and
re-commit. Never report the commit as done with a failing checker.
## Workflow
@ -57,8 +63,6 @@ hard blocker: fix the message and re-check it before reporting success.
- Do not pass `--author`. Author identity comes from the local git config.
- Do not amend a commit you did not create in this session, unless explicitly asked.
- Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
- Add `Co-authored-by:` only for a human co-author. AI providers, agents and
bots are rejected by `./scripts/check-commit`; for AI participation use the
`AI-assisted-by:` trailer.
- Never add a `Co-authored-by:` trailer. It is forbidden and fails `./scripts/check-commit`.
- Do not add untracked files that were not created in this session.
- Do not skip the `scripts/check-commit` verification step.
- Do not skip the `scripts/check-commit` verification step (Iron Rule 4).

View File

@ -18,7 +18,7 @@ push — the user pushes from their own shell.
- The user asks to open or create a NEW PR for the current task branch, in
any phrasing ("open a PR", "create the pull request", "put this up for
review") — or invokes `@create-pr`. → **Open mode**.
review") — or runs `/create-pr`. → **Open mode**.
- The user asks to fix or update an EXISTING PR's title or description to
match conventions. → **Update mode**.
@ -59,7 +59,7 @@ Run all checks before reporting, then report every failure together:
5. **Branch not pushed.** The remote-tracking ref `origin/<branch>` is
missing, or `git rev-parse origin/<branch>` differs from HEAD — the
branch was never pushed, or has commits the remote does not have. Never
push yourself; ask the user to push and to invoke `@create-pr` again
push yourself; ask the user to push and to run `/create-pr` again
afterwards, then stop.
### 3. Already-open PR

View File

@ -26,17 +26,12 @@ in the local working tree.
- The user asks to implement or execute a plan, in any phrasing:
"implement the plan", "execute it", "go build it" — or runs
`@implement-plan`.
`/implement-plan`.
- A ready, reviewed plan is in the session context or a plan file path
was given (typically after `@make-a-plan` or `@review-plan`).
was given (typically after `/make-a-plan` or `/review-plan`).
Do not use it to produce plans — that is the `make-a-plan` flow.
For a plan derived from a code review, retain its review target, reviewer
session ID, original findings and decisions. Record the starting revision and
the exact commits or diff produced by this implementation, plus test evidence,
in session context for follow-up. Do not put commit hashes in the plan file.
## 1. Read the plan first
Identify the plan to execute — from the file path the user gave, the
@ -116,20 +111,19 @@ Implement every task in one go. Work methodically, keeping changes
focused on what the issue requires. Respect the plan's proposed
parallelization when it applies.
When the implementation is complete, load the **`create-commit`** skill and
follow its workflow to commit
When the implementation is complete, close the plan file first: flip its
`Status` to `done` and append one UTC ISO 8601 line to its `Review Log`
with the issue URL when one exists (standalone mode, e.g.
`https://github.com/penpot/penpot/issues/NNNN`); in continue/direct
mode with no issue, just `done` with no invented identifier. Never
record commit hashes. Leave the closed plan file local and uncommitted.
Then load the **`create-commit`** skill and follow its workflow to commit
only the implementation, tests, memory, and documentation changes. Before
committing, verify that the staged file list does not contain the plan
file. Provide a brief summary of what was implemented and why, the issue
reference (`issue-NNNN`) when there is one, and the model name you are
running as so the `AI-assisted-by` trailer is set correctly.
Only after the commit exists and passes its checks, close the plan: flip
`Status` to `done` and append one UTC ISO 8601 line to `Review Log` with the
issue URL when one exists; otherwise record `done` without inventing an
identifier. Never record commit hashes. Leave the plan local and uncommitted.
If the commit or its checks fail, do not close the plan.
### Step-by-step mode (on request)
When the user asks for it — "step by step", "task by task", "one commit
@ -152,23 +146,12 @@ stop and ask the user before splitting it.
## When you are done
After the entire plan is implemented and verified, including the final task in
step-by-step mode, if it addresses code review findings, ask whether the user
wants the original reviewer to verify the corrections. Use the `question` tool
if available; otherwise ask in chat. Wait for the answer. Do not start the
follow-up without consent, and do not ask after each intermediate task.
If the user accepts, follow `@review-code`'s follow-up procedure with the
review context, decisions, exact correction range or diff and test evidence.
If the original reviewer cannot be resumed, say so and use a new reviewer with
the same context. If the user declines, stop without starting a review. The
follow-up is read-only; further fixes require the user's direction.
End by suggesting the next steps (suggestions, not a required pipeline — any
instruction from me overrides them):
- `@review-code` — for a new independent review of the changes.
- `@create-pr` — when the task is done and the branch is ready to merge.
- `/review-code` — to review the changes just committed; it routes to
`/make-a-plan` by itself if the findings need one.
- `/create-pr` — when the task is done and the branch is ready to merge.
## User input, overrides and additional context

View File

@ -86,7 +86,7 @@ log path.
./scripts/ci --lint frontend
# Changed common/ — validate its consumers too
./scripts/ci common frontend backend exporter
./scripts/ci frontend backend exporter
# Preview what would run, without running it
./scripts/ci --dry-run --all

View File

@ -16,18 +16,19 @@ stop — this skill needs the build agent to save the plan.
- The user asks to plan, design, or break down a task, in any phrasing:
"make a plan", "how would we build X", "design an approach for Y" —
or invokes `@make-a-plan`.
or runs `/make-a-plan`.
- The user asks to rework or extend an existing plan. While the plan is
still unimplemented (pre-`implement-plan` iteration, e.g. after
`@review-plan` findings or user feedback), revise the saved plan file
`/review-plan` findings or user feedback), revise the saved plan file
in place. Once the plan has been implemented and reviewed (post
`@review-code` findings on committed work), write a new derived plan
`/review-code` findings on committed work), write a new derived plan
file instead — never rewrite the executed plan. The `planner` skill
defines the derived naming (`--review-NN`, `--task-NN`).
- The user says to apply additional review feedback to a still-unimplemented
plan: apply the agreed changes, flip `Status` to `reviewed`, and append one
UTC ISO 8601 line to `Review Log`. `review-plan` can also update the plan
and its metadata as part of its own review flow.
- The user says to apply `review-plan` findings to a still-unimplemented
plan: apply the agreed changes, flip `Status` to `reviewed`, and
append one UTC ISO 8601 line to `Review Log` describing what changed.
This apply step is the only pre-implementation write to those fields —
`review-plan` itself stays read-only and never writes.
Do not use it to execute a plan — that is the `implement-plan` flow.
@ -38,11 +39,6 @@ Do not use it to execute a plan — that is the `implement-plan` flow.
example, `delegated` in the user context), delegate to the `general` subagent
instead — the delegate must also follow the `planner` skill and receive all
the relevant session context (a review, user feedback, and so on).
When the plan addresses a code review, preserve the review target, reviewer
session ID if available, original findings and IDs, and decisions including
deferred or rejected findings and their reasons in the local plan. Map
accepted findings to tasks. If context is unavailable, mark it unknown; do
not invent it. Pass this context to a delegated planner too.
2. Before asking me to decide anything, explain the plan and every open question in
plain language. Assume I know only the high-level project goal, not the codebase,
architecture, implementation terms, or the problem this task solves.
@ -105,8 +101,8 @@ constraints and risks, what is deliberately out of scope, and the path where the
is saved. Never assume that a short list of task names is enough context. End
the final response by suggesting the next steps, in this order:
1. `@review-plan` — to get a second opinion on the plan before executing it.
2. `@implement-plan` — to execute the plan from the current session context.
1. `/review-plan` — to get a second opinion on the plan before executing it.
2. `/implement-plan` — to execute the plan from the current session context.
These are suggestions, not a required pipeline — any instruction from me
overrides them (for example, asking you to implement the plan directly).

View File

@ -1,105 +1,315 @@
---
name: plan-review-criteria
description: Criteria for reviewing implementation plans, including evidence, severity and verdicts. Loaded by the review-plan subagent.
description: Plan review criteria — the six review axes, severity rubric, approval standard, and output format for reviewing implementation plans. Loaded by the reviewer subagent of the review-plan flow. Not a user-facing flow — to review a plan, use the review-plan flow.
---
# Plan Review Criteria
Evaluate whether the plan can deliver its stated goal safely and whether a
skilled implementer can execute and verify it without guessing. Perfect plans
do not exist: do not block on personal preferences or missing sections that
add no useful information. Apply each criterion in proportion to the plan's
scope and risks; explain when a criterion is not applicable.
## Overview
Follow `review-plan` for delegation, permitted edits, decisions and handover.
Do not run tests, builds or other state-changing checks during the review.
Multi-dimensional plan review with quality gates. Every plan gets reviewed before implementation starts — no exceptions. Review covers six axes: completeness, task quality, architecture & sequencing, risk coverage, actionability, and proposed code quality.
## Evidence and scope
**The approval standard:** Approve a plan when it is specific enough that a skilled implementer could execute it without guessing, the task ordering is sound, and risks are acknowledged. Perfect plans don't exist — the goal is confidence that implementation won't derail. Don't block a plan because it isn't exactly how you would have structured it. If it's executable and well-organized, approve it.
- Read the plan, relevant code, project guidance and module memories. Check
that cited paths, symbols and proposed commands exist and are appropriate;
distinguish verified facts from assumptions and unknowns. Do not invent
findings or claim checks were run when they were not.
- Check that tasks serve the stated goal and respect existing requirements and
decisions. Identify scope creep, contradictions and dependencies on external
work. A consequential change in scope, behavior or tradeoffs needs the user's
decision.
- Evaluate the proposed approach even without code snippets: does it solve the
problem, fit existing conventions and avoid unnecessary complexity? If the
plan includes code, also check its types, signatures and examples.
## When to Use
## Six review axes
- The reviewer subagent of the `review-plan` flow loads this skill to perform
the review of a plan.
- To review a plan, always go through the `review-plan` flow — never load this
skill directly for that. This is the criteria reference, not the flow.
**Do NOT use for:** Single-file changes with obvious scope, or when the task is trivial enough to just do.
## The Six-Axis Review
Every plan gets evaluated across these dimensions:
### 1. Completeness
Are the problem, intended outcome, affected areas and relevant constraints
clear? Does the plan explain important decisions and include an appropriate
testing strategy? Identify missing information that would change the outcome
or force an implementer to guess. Require risk, rollback, migration or
parallelization details only when relevant.
Does the plan cover everything needed to implement successfully?
### 2. Task quality
- Is the **context** clear? (What problem, why now, what's the goal?)
- Are **affected modules** identified with paths?
- Are **architecture decisions** documented with rationale?
- Is there a **testing strategy**?
- Are **verification commands** explicit (not "run the tests")?
- Are **open questions** listed (not buried in someone's head)?
- Is there a **parallelization** assessment for multi-task plans?
Are tasks scoped, ordered and specific enough to execute? Check likely files,
dependencies, rationale, acceptance criteria and verification where relevant.
Acceptance criteria should describe observable behavior, including important
failure cases. Split work when it is too large to understand or verify safely,
not merely because it crosses an arbitrary task count.
**Missing any of these is a gap, not a nit.**
### 3. Architecture and sequencing
### 2. Task Quality
Does the approach fit the existing architecture and reuse appropriate
facilities? Do dependencies precede their consumers, and can the work be
verified as it progresses? Prefer independently testable slices where useful;
use checkpoints where risk or coordination warrants them, not at a fixed
interval. Flag unrelated work or abstractions that add complexity without
helping the goal.
Are the tasks well-defined and independently executable?
### 4. Risk coverage
- Does every task have **acceptance criteria**? (Testable, not vague)
- Does every task have **verification steps**?
- Are tasks **sized appropriately**? (XS–M is ideal, L is acceptable, XL must be split)
- Are **dependencies** between tasks explicitly stated?
- Are **files likely touched** listed?
- Is each task a **single, self-contained change**? (Not "implement the whole feature")
- Could a skilled implementer pick up any task and execute it without asking clarifying questions?
Consider relevant edge cases, security, performance, compatibility, data
integrity, migrations and external dependencies. Where failure has meaningful
consequences, check detection, mitigation and rollback or recovery. Do not
invent risks or demand a rollback section for changes that do not need one.
### 3. Architecture & Sequencing
### 5. Actionability and verification
Is the plan structured so implementation flows correctly?
Can the implementer identify the affected code, follow project conventions and
run the stated project-specific checks? Are commands and prerequisites correct?
Do tests and other checks demonstrate the intended behavior and important
failure cases? Surface unknowns honestly rather than presenting an unverified
command or assumption as fact.
- Does implementation order follow the **dependency graph** (foundations first)?
- Are tasks **vertically sliced** (feature paths) rather than horizontally layered?
- Does each task leave the system in a **working state**?
- Are there **checkpoints** between major phases?
- Are **high-risk tasks early** (fail fast)?
- Is the total plan a reasonable number of tasks? (More than ~15 tasks suggests the scope should be split into multiple plans)
### 6. Proposed solution quality
### 4. Risk Coverage
Assess correctness, clarity, architecture, security and performance of the
proposed solution, even when the plan contains no code. For any code sketches,
also check boundary cases, names, types and contracts. Preserve intended
behavior unless a change is explicitly agreed; prefer the simplest solution
that meets the requirements.
Are the hard parts acknowledged and mitigated?
## Findings and verdict
- Are **edge cases** identified?
- Are **breaking changes** or **migration concerns** noted?
- Are **security implications** considered?
- Are **performance implications** considered?
- Are **external dependencies** or integration risks flagged?
- Is there a plan for **rollback** if something goes wrong?
- Are **data integrity** risks addressed (what happens if a migration fails mid-way)?
Give each reported item a stable ID (`P1`, `P2`, …); retain it throughout the
review and any decision walkthrough. For each item state the evidence, impact,
concrete remedy and verification needed. For a consequential user decision,
include real options, consequences and a recommendation.
### 5. Actionability
- **Blocking:** implementation cannot proceed safely or an essential user
decision remains open. Explain what blocks it.
- **Improvement:** a clear correction that preserves the agreed goal, scope
and behavior. Apply it under `review-plan`'s permitted-edit rules.
- **Optional:** a suggestion that does not prevent implementation. Do not
silently expand scope or override the user's decisions.
Can an implementer actually execute this?
Report applied improvements and any outstanding items with their IDs. Give a
provisional verdict before any walkthrough and a final verdict after applying
the decisions:
- Are **file paths** specific (not "update the relevant files")?
- Are **function/method names** mentioned where applicable?
- Are **verification commands** copy-pasteable (not "run the linter")?
- Are **test commands** project-specific (not generic)?
- Is the **code shape** described where the implementation isn't obvious?
- Are **conventions** referenced (naming, patterns, existing utilities to reuse)?
- Does the plan reference **existing code** the implementer should read first?
- **Ready:** no blocking issue or essential decision remains.
- **Awaiting decisions:** user decisions are needed before the plan is ready.
- **Blocked:** required work or evidence is missing and cannot be resolved by
applying clear improvements or obtaining the current decisions alone.
### 6. Proposed Code Quality *(when the plan includes implementation details)*
Do not approve an unresolved blocker. A plan may be ready with optional
suggestions; identify them without treating them as required.
If the plan proposes code shapes, function signatures, data structures, or API designs, evaluate those proposals against `code-review-criteria`:
- **Correctness:** Do the proposed types/signatures handle edge cases (null, empty, boundaries)?
- **Readability:** Are proposed names descriptive and consistent with project conventions?
- **Architecture:** Do proposed abstractions follow existing patterns? Are they justified (not over-engineered)?
- **Security:** Do proposed APIs validate input at boundaries? Any injection/XSS vectors in the design?
- **Performance:** Do proposed data structures avoid N+1 patterns? Any unbounded operations in the design?
**When to apply:** Only when the plan includes specific code snippets, type definitions, API contracts, or function signatures. Plans that only describe "what" without showing "how" skip this axis.
## Structural Remedies
When you flag a structural problem in a plan, propose the fix — not just the problem:
- **A task is too large (XL):** Split it into vertical slices. Each slice should be independently testable.
- **Missing acceptance criteria:** Draft 2–3 specific, testable conditions for the task.
- **Wrong sequencing:** Identify the dependency and propose the correct order.
- **No checkpoints:** Suggest where checkpoints should go (typically after every 2–3 tasks).
- **Vague verification:** Replace "run tests" with the actual project command.
- **Horizontal slicing:** Restructure into vertical feature paths.
- **Missing risk section:** Draft the risks you can identify from the plan content.
Prefer the remedy that makes the plan immediately actionable over one that just flags the gap.
## Plan Sizing
Plans should be scoped to a single deliverable:
```
1–5 tasks → Good. A focused feature or bug fix.
6–10 tasks → Acceptable for a moderate feature.
11–15 tasks → Large. Consider splitting into phases.
15+ tasks → Too large. Split into multiple plans.
```
**What counts as "one plan":** A self-contained set of changes that delivers a single coherent capability. If you can describe the goal in one sentence, it's one plan.
## Categorize Findings
Label every comment with its severity so the author knows what's required vs optional:
| Prefix | Meaning | Author Action |
|--------|---------|---------------|
| *(no prefix)* | Required change | Must address before implementation starts |
| **Critical:** | Blocks implementation | Missing security consideration, data integrity risk, fundamentally wrong approach |
| **Nit:** | Minor, optional | Author may ignore — wording, formatting |
| **Optional:** / **Consider:** | Suggestion | Worth considering but not required |
| **FYI** | Informational only | No action needed — context for future reference |
**Lead with what matters.** Order findings by leverage: missing risks and wrong sequencing first, then task quality gaps, then completeness, then nits. If you have one critical sequencing problem and ten nits, the sequencing problem *is* the review.
## Review Process
### Step 1: Understand the Goal
Before evaluating structure, understand intent:
```
- What is this plan trying to accomplish?
- What problem does it solve?
- What does "done" look like?
```
### Step 2: Check Completeness First
Scan for missing sections before diving into content:
```
- Context present?
- Affected modules listed?
- Architecture decisions documented?
- Risks acknowledged?
- Testing strategy defined?
- Verification commands explicit?
```
### Step 3: Review Task Quality
Walk through each task:
```
For each task:
1. Can I tell exactly what to build?
2. Are acceptance criteria specific and testable?
3. Is the size reasonable (not XL)?
4. Are dependencies clear?
5. Would I know which files to touch?
```
### Step 4: Validate Sequencing
Check the dependency graph:
```
- Are foundations built first?
- Does each task leave the system working?
- Are checkpoints placed correctly?
- Are high-risk items early?
- Is it vertically sliced?
```
### Step 5: Assess Actionability
Put yourself in the implementer's shoes:
```
- Could I pick up task 1 and start coding without asking any questions?
- Are the verification commands copy-pasteable?
- Are file paths and function names specific?
- Is existing code referenced where I'd need to read it?
```
### Step 6: Verify the Verification Story
Check that the plan can actually confirm it worked:
```
- What tests should pass after implementation?
- What build/compile commands are relevant?
- What manual checks are needed?
- How do we know the feature works end-to-end?
```
### Step 7: Evaluate Proposed Code Quality *(if applicable)*
If the plan includes code snippets, types, or API designs:
```
- Load code-review-criteria skill for criteria
- Check proposed signatures for edge cases
- Verify naming follows project conventions
- Confirm abstractions follow existing patterns
- Scan for security vectors in proposed APIs
- Check for performance issues in proposed data structures
```
## Review Checklist
```markdown
## Review: [Plan title]
### Completeness
- [ ] Context explains the problem and goal
- [ ] Affected modules are listed with paths
- [ ] Architecture decisions have rationale
- [ ] Testing strategy is defined
- [ ] Verification commands are explicit and project-specific
- [ ] Open questions are listed
### Task Quality
- [ ] Every task has acceptance criteria
- [ ] Every task has verification steps
- [ ] Tasks are sized XS–M (L acceptable, XL must be split)
- [ ] Task dependencies are stated
- [ ] Files likely touched are listed
### Architecture & Sequencing
- [ ] Order follows dependency graph (foundations first)
- [ ] Vertically sliced (not horizontal layers)
- [ ] Each task leaves system working
- [ ] Checkpoints exist between phases
- [ ] High-risk tasks are early
### Risk Coverage
- [ ] Edge cases identified
- [ ] Breaking changes / migrations noted
- [ ] Security implications considered
- [ ] Performance implications considered
- [ ] Rollback strategy exists (if applicable)
### Actionability
- [ ] File paths are specific
- [ ] Verification commands are copy-pasteable
- [ ] Existing code to read is referenced
- [ ] Conventions and patterns are noted
### Proposed Code Quality *(if plan includes implementation details)*
- [ ] Proposed types/signatures handle edge cases
- [ ] Proposed names follow project conventions
- [ ] Proposed abstractions follow existing patterns
- [ ] No security vectors in proposed APIs
- [ ] No performance issues in proposed structures
### Verdict
- [ ] **Approve** — Ready to implement
- [ ] **Request changes** — Gaps must be addressed
```
## Common Rationalizations
| Rationalization | Reality |
|---|---|
| "I'll figure out the details during implementation" | That's how you discover blocking dependencies mid-task. Surface them now. |
| "The tasks are obvious, no need for criteria" | Write them anyway. Explicit criteria surface hidden assumptions. |
| "It's just a small feature, it doesn't need a plan" | Small features have edge cases too. 3 tasks with criteria takes 5 minutes. |
| "The plan is good enough" | "Good enough" without acceptance criteria means the implementer defines "done" — and they might define it differently. |
| "I'll add verification steps later" | Later never comes. The plan is the contract — define verification now. |
| "Risks are minimal" | Every change has risks. If you can't name them, you haven't thought about them. |
| "The file paths are obvious" | They're obvious to the author. The implementer might not know the codebase. |
| "The code in the plan is fine, it'll get reviewed later" | Plan-level code review catches design problems before implementation — fixing them after coding is more expensive. |
## Red Flags
- No acceptance criteria on any task
- Tasks that say "implement the feature" without specifics
- No verification steps anywhere in the plan
- All tasks are XL-sized
- No checkpoints between phases
- Dependency order isn't considered (e.g., API handler before domain model)
- No testing strategy
- Verification commands are generic ("run tests") instead of project-specific
- Plan has 20+ tasks (scope too large for one plan)
- No risk section on a plan with migrations, breaking changes, or security implications
- Horizontal slicing (all domain, then all services, then all API)
- File paths are vague ("update the relevant files")
- Missing open questions section despite stated unknowns
- Proposed code ignores project conventions or existing patterns
- Proposed types use gratuitous `any`/`unknown`/optional without justification
- Proposed APIs don't validate input at boundaries
## See Also
- For producing plans, use the `planner` skill
- For reviewing implemented code, use `code-review-criteria` — also the criteria source for axis 6
- For security-specific concerns, see `security-and-hardening`
- For testing strategy guidance, see `testing`

View File

@ -42,10 +42,9 @@ Skipping this step is the #1 cause of incorrect or incomplete plans.
only file you may write is the plan itself, and only when the command or
user explicitly instructs you to save it.
- You do **not** run builds, tests, linters, or any commands that modify state.
- You do **not** create commits or run commands that modify version control.
Read-only inspection such as `git status`, `git log`, `git show` and
`git diff` is allowed.
- Shell commands must be read-only.
- You do **not** create git commits or interact with version control.
- You do **not** execute shell commands beyond read-only searches (`rg`, `ls`,
`find`, `cat`, `bat`).
- Your output is a structured plan or analysis, ready for handoff to an
engineer agent or developer.
@ -193,8 +192,8 @@ Source: review-code over `<commit>` (branch `<branch>`) | task `NN` of roadmap `
Created: YYYY-MM-DD
```
End the response by suggesting the next steps: `@review-plan` to get a second
opinion on the plan and `@implement-plan` to execute it.
End the response by suggesting the next steps: `/review-plan` to get a second
opinion on the plan and `/implement-plan` to execute it.
### Plan Structure
@ -224,16 +223,16 @@ Review Log:
log. `reviewed` never means "a review was emitted" — it means
"review feedback was incorporated". A plan approved with no changes
goes `draft` → `done` without passing through `reviewed`.
- Before implementation, `review-plan` applies clear review feedback and
agreed decisions, flips to `reviewed` and appends one UTC ISO 8601 entry
when the plan changed. `make-a-plan` can also do this for additional
feedback when the user asks.
- Only a user-triggered apply step writes them before implementation:
when the user says to apply `review-plan` findings, `make-a-plan`
applies the changes, flips to `reviewed`, and appends one UTC
ISO 8601 line describing what changed.
- `implement-plan` flips to `done` on completion and appends one line
with the issue URL when one exists (standalone mode); otherwise
just `done`. Never record commit hashes — they rot on amend/rebase
and git already links the commit.
- The log is append-only: never rewrite or delete lines.
- `review-code` never writes these fields.
- `review-plan` and `review-code` never write these fields.
Omit empty sections only when they do not apply. Every implementation task
still requires acceptance criteria, verification, dependencies, likely files,
@ -262,5 +261,5 @@ Before delivering the plan, confirm:
- [ ] No task is XL or larger — break it down instead
- [ ] Checkpoints exist after every 2-3 tasks
- [ ] The response states the plan's path (saved or suggested) and suggests
`@review-plan` and `@implement-plan`
`/review-plan` and `/implement-plan`
- [ ] The plan is ready for human review

View File

@ -12,7 +12,7 @@ Act as a senior software engineer and perform a thorough code review.
- The user asks to review code, in any phrasing: "review this diff",
"review the PR", "check my changes", "code review" — or runs
`@review-code`.
`/review-code`.
- A commit, branch, PR, or diff is ready and the user wants it assessed
before merge.
@ -22,61 +22,17 @@ Act as a senior software engineer and perform a thorough code review.
diff, a commit range, a branch, a PR (number or URL), or specific files. If
the target is ambiguous, ask before reviewing.
2. Delegate the review to the `general` subagent (via the task tool), unless the
user specifies another agent. The prompt carries the **`code-review-criteria`**
skill name, all user context, and these requirements:
- Load `code-review-criteria` and follow its process and output format.
- Read `AGENTS.md` (if present) and the module memories it points to for the
code being reviewed.
- Return the COMPLETE review, verbatim, exactly as the skill instructs it to
be produced. Never summarize it.
- Read-only: modify no file, create no commit, and run no tests, builds,
linters or other state-changing checks. Use prior results or CI evidence;
state explicitly when evidence is unavailable.
- Do not invent problems: every finding must be real and actionable.
- Missing tests are an issue, not a suggestion — a severity-tagged finding.
- For each finding, give a concrete remedy and the tests needed to verify it.
If a user decision is needed, include the real options and a recommendation
in that finding. Do not pause to ask during the review.
- Prioritize by impact: one structural issue outweighs ten nits, and "this
could be better" is not a finding — say why and how.
- Inspect generated files and lockfiles when relevant to the change, its
dependencies or security; exclude unrelated working-tree modifications.
user specifies another agent. Include in the prompt the
**`code-review-criteria`** skill name and all user context.
3. When the subagent returns, output the review to the user verbatim. Do not
summarize it and do not act on its findings. Retain the subagent session ID,
review target and complete findings in the session context for follow-up;
do not invent an ID if the tool does not provide one.
summarize it and do not act on its findings.
4. Right after the review, suggest how to proceed based on the findings. These
are suggestions — the user decides:
- **Approve (no required changes):** say so — there is nothing to address.
- **Minor findings:** suggest direct fixes after the user ends the review if
appropriate; no plan is required.
- **Substantive findings:** suggest `@make-a-plan` to make a plan to address
- **Minor findings (nits):** applying them directly as-is is fine once the
review is done — no plan needed.
- **Substantive findings:** suggest `/make-a-plan` to make a plan to address
them.
- If the user wants to discuss findings one by one, use `@issue-walkthrough`
on request; do not start it automatically. The walkthrough returns its
decisions to `@make-a-plan`, not to automatic code editing.
### Follow-up after implementation
When the user agrees to verify corrections, resume the original reviewer using
its session ID if the tool supports it. Give it the original review target and
findings, the decisions and plan, the exact correction commit range or diff,
and the test evidence. Require it to inspect the actual corrections and
evidence, not rely on its earlier context alone. If the original session is
unavailable or cannot be resumed, tell the user and delegate to a new reviewer
with the same context; never claim it is the original reviewer. If the review,
decisions or correction range cannot be established, ask for the missing
information rather than inventing it.
The reviewer follows `code-review-criteria` and remains read-only. It reports
each original finding with its unchanged ID as **resolved**, **pending**, or
**not addressed** (including the agreed reason); report any new problems with
new, non-conflicting IDs. For pending and new problems, give evidence, a remedy,
tests and any user decision as in the original review. Summarize separately
whether all agreed corrections are resolved and whether any findings remain,
including deferred findings. Do not equate deferred with resolved. Present the
result without implementing fixes; the user chooses whether to iterate, request
an independent review, or stop.
### Hard rule — read-only while reviewing
@ -84,26 +40,50 @@ This flow is read-only **for the duration of the review**: from the moment it
starts until the user considers the review finished (including any feedback,
questions, or clarifications about it). During that period, never fix,
implement, edit files or create commits — not even "obvious" fixes derived from
the findings. Never run tests, builds, linters or state-changing checks during
the review; inspect existing evidence instead. Once the user explicitly states
the review is done (or moves on to a different task), this rule no longer
applies and you act as a normal build agent again.
the findings. Once the user explicitly states the review is done (or moves on to
a different task), this rule no longer applies and you act as a normal build
agent again.
## Instructions for the subagent
1. Load the **`code-review-criteria`** skill and follow its process and output
format.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and
reading all related testing documentation from memories before reviewing.
3. Return in your final message the COMPLETE review, verbatim, exactly as the
skill instructs it to be produced. Do not summarize it — include the full
structured review.
### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing
never writes.
3. Be specific and constructive. "This could be better" is not helpful — explain
why and how.
4. Prioritize by impact. One structural issue outweighs ten nits.
5. Missing tests are an issue, not a suggestion. Report as a severity-tagged
finding — never as a recommendation.
6. Skip generated files, lockfile-only changes, and unrelated modifications
unless they introduce security risks.
## Posting as a GitHub comment
Post only when the user asks for it explicitly; otherwise keep the review in the
session. PR comments go in English, with this minimal structure:
Only post the review as a comment when the user asks for it explicitly. Keep
it in the session otherwise.
When posting: PR comments go in English, with this minimal structure:
1. **Intro line** — one sentence: what was found and the overall tone. No
greetings.
2. **`## Main issues`** — one subsection per required finding: what breaks, the
failure circumstances, and the minimal fix (code snippet when useful). A
missing test belongs here, not under minor suggestions.
3. **`## Minor suggestions`** — one bullet per minor finding: file/function,
problem, concrete remedy.
2. **`## Main issues`** — one subsection per required finding: what breaks,
the failure circumstances, and the minimal fix (code snippet when useful).
A missing test belongs here, not under minor suggestions.
3. **`## Minor suggestions`** — one bullet per minor finding or suggestion:
file/function, problem, concrete remedy.
4. **Closing** — optional and short. Cut it when it adds nothing.
Reference every finding by its ID (`F1`, `F2`, … from
Reference every finding by its ID (`R1`, `M1`, `S1`, … from
`code-review-criteria`) so follow-up replies can point to findings
unambiguously.

View File

@ -1,60 +1,46 @@
---
name: review-plan
description: Review an implementation plan with a subagent, apply clear improvements and resolve consequential decisions with the user.
description: Plan review flow — evaluate an implementation plan before it is executed, delegating the review to a subagent that follows the plan-review-criteria skill. Use it when the user asks to review a plan, in any phrasing.
slash: true
---
# Review Plan
Review an unimplemented plan with a subagent and improve the plan without
implementing it. Use this flow when the user asks for a plan review or invokes
`@review-plan`.
Act as a senior software engineer and perform a thorough review of an implementation plan.
## Flow
## When to use
1. Identify and read the complete plan and its file. If the target or path is
ambiguous, ask. If no saved file exists, ask for a path or have the user save
it first. Do not edit a completed plan: propose a derived plan instead.
2. Delegate to `general`, unless the user specifies a different subagent or
model. Supply the plan path, relevant session context and these instructions:
- Load `plan-review-criteria`, read `AGENTS.md` and the relevant module and
testing memories, and review the plan against the criteria.
- Apply clear, evidence-based improvements directly to that plan file only.
Preserve the goal, scope, agreed decisions and intended behavior. Examples:
clarify tasks, add missing tests, correct dependencies and specify actual
verification commands. Do not guess commands or invent findings.
- A change that affects the goal, user-visible behavior, scope, tradeoffs or
an existing decision needs the user's decision. Leave it unresolved; report
concrete options, consequences and a recommendation. Do not ask questions
from the subagent.
- Return a concise summary of applied changes, remaining decisions with
stable IDs, evidence and recommendations, and an honest provisional
verdict. Retain the subagent session ID for follow-up if available.
- Do not change source files, other plans or documents, run state-changing
checks, commit, or implement the plan. If the plan is not writable, report
this and return the proposed edits instead.
3. Show the summary. If decisions remain, start `@issue-walkthrough` immediately
in the parent session with those questions and their IDs. Follow its one
question per turn and read-only rules. Do not ask questions that the code or
existing decisions settle. Wait for every answer.
4. Resume the same subagent with the decision table and plan path. It applies
the decisions to the plan, rechecks consistency and reports changes and any
newly discovered consequential decisions. Repeat the walkthrough for new
decisions, then apply those answers. If the original session cannot be
resumed, tell the user and apply the changes in the parent or delegate a new
subagent with the full context. If the reviewer cannot write, apply its
proposed edits in the parent only if the plan file is writable; otherwise
explain the blocker and stop.
5. Once all decisions are resolved and the plan is coherent, update its
metadata: when review feedback changed the plan, set `Status: reviewed` and
append one UTC ISO 8601 `Review Log` entry for this review, summarizing the
changes. Preserve prior log entries. If no feedback changed the plan, leave
status and log unchanged. Never record commit hashes. If decisions or
required changes remain unresolved, do not claim the plan is ready or mark
it reviewed.
6. Show the final summary, plan path and verdict. Suggest `@implement-plan` only
when the plan is ready. Never execute the plan automatically.
- The user asks to review a plan, in any phrasing: "review this plan", "does this plan look right?", "second opinion on the plan" — or runs `/review-plan`.
- A plan was just produced (typically by `/make-a-plan`) and the user wants it evaluated before executing it.
The only permitted write in this flow is the specified local plan file,
including its status and log. Do not stage or commit it. If a user instruction
overrides automatic editing, respect it and report findings without editing.
## Instructions
1. **Determine the plan under review** from the session context (for example, a plan just produced by `/make-a-plan`) or from a plan file path given by the user (typically under `.agents/plans/`). If a file path is given, read the file first so the complete plan is in context.
2. Delegate the review to the `general` subagent (via the task tool), unless the user specifies another agent. Include in the prompt the **`plan-review-criteria`** skill name and all user context.
3. When the subagent returns, output the review to the user verbatim. Do not summarize it and do not act on its findings.
4. Right after the review, suggest the next step based on the verdict. These are suggestions — the user decides, and any instruction overrides them:
- **Approve** → suggest `/implement-plan` to execute it.
- **Request changes** → suggest `/make-a-plan` to make a plan to address the findings.
### Hard rule — read-only while reviewing
This flow is read-only **for the duration of the review**: from the moment it starts until the user considers the review finished (including any feedback, questions, or clarifications about it). During that period, never fix, implement, edit files or create commits — not even "obvious" fixes derived from the findings. Once the user explicitly states the review is done (or moves on to a different task), this rule no longer applies and you act as a normal build agent again.
## Instructions for the subagent
1. Load the **`plan-review-criteria`** skill and follow its process and output format.
2. Read `AGENTS.md` (if present) and follow its instructions for finding and reading all related documentation and testing memories before reviewing.
3. Cada hallazgo debe tener un identificador único y estable, con el formato `F1`, `F2`, etc. El identificador debe aparecer en el título del hallazgo y no debe reutilizarse dentro de la misma revisión. Esto permite que el usuario pueda responder sobre un hallazgo concreto.
4. Return in your final message the COMPLETE review, verbatim, exactly as the skill instructs it to be produced. Do not summarize it — include the full structured review.
### Strong rules for the subagent
1. Do not invent problems. Every finding must be real and actionable.
2. Read-only: do not modify any file and do not create a commit — reviewing never writes.
3. Be specific and constructive. "This could be better" is not helpful — explain why and how.
4. Prioritize by impact. One structural issue outweighs ten nits.
5. Judge the plan as the implementer would: every task executable without guessing, ordering follows the dependency graph, risks named.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -6,8 +6,9 @@ slash: true
# Skill: rewrite-commit-message
Rewrite HEAD's title and description from its own diff. This also handles a
squash message with several subjects, bodies and repeated trailers.
HEAD's message is a concatenation of older messages (typical after a
squash): several subjects, bodies and repeated trailers stacked in a
single message. Replace it with one proper message and amend HEAD.
## When to Use
@ -16,11 +17,9 @@ squash message with several subjects, bodies and repeated trailers.
## Steps
1. Check `git diff --cached` first. If any changes are staged, stop and ask
the user to handle them; an amend must never include those changes. Read
the full current message (`git log -1 --format=%B HEAD`) and HEAD's own
diff (`git show --format= --stat HEAD` and the patch). Respect the project
rule on amending pushed commits.
1. Read the full current message (`git log -1 --format=%B HEAD`) and
the branch diff (`git diff develop..HEAD --stat`, plus the code
when the summary needs grounding).
2. Collect every DISTINCT `AI-assisted-by:` value in the current
message. All of them survive into the new message, one line each:
attributions are never dropped in a rewrite, even when most of
@ -28,7 +27,7 @@ squash message with several subjects, bodies and repeated trailers.
3. Draft one message per `mem:workflow/creating-commits` (read it
end-to-end first): single subject (imperative, capitalized, no
trailing period, ≤80 chars), blank line, body wrapped at ≤80
chars covering HEAD's diff (not the old per-commit slices),
chars covering the whole diff (not the old per-commit slices),
then the preserved trailers.
4. Write it to a file (a heredoc keeps real newlines; inline `-m`
with `\n` does not) and `git commit --amend -F <file>`.

View File

@ -67,7 +67,7 @@ rg "user_\d+"
rg -F "function()"
# Multiline search
rg -U '(?s)start.*end'
rg -U "start.*end"
# Only show matches, not lines
rg -o "https?://[^\s]+"

View File

@ -1,6 +1,6 @@
---
name: ste
description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "@ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it.
description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "/ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it.
---
# ASD-STE100 Simplified Technical English
@ -11,7 +11,7 @@ Compliance note (for you, not for output): the official specification and its di
## When to use
Only when the user explicitly invokes it: they type `@ste`, or say "use
Only when the user explicitly invokes it: they type `/ste`, or say "use
the ste skill" / "apply ASD-STE100". Requests like "simplify this",
"make it clearer", or "shorter sentences" do NOT invoke it — respond
normally unless it is named.

View File

@ -0,0 +1,116 @@
---
name: taiga
description: Fetch information from Taiga public API for the Penpot project (id 345963) — issues, user stories, and tasks, without authentication.
metadata: {"clawdbot":{"requires":{"bins":["python3"]}}}
---
# Taiga API Skill
Fetch information from Taiga public API for the **Penpot** project
(project id: `345963`, slug: `penpot`).
**No authentication required** — only public project data is accessed.
## When to use
- The user asks about Penpot issues, user stories, or tasks tracked in
Taiga — fetch them via the public API (project id 345963), no
authentication needed.
## Prerequisites
- `python3` — the `scripts/taiga.py` CLI script is self-contained (stdlib only)
## Quick Start
The easiest way is to use the bundled Python script:
```bash
# Pass a Taiga URL directly
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# Or use "<type> <ref>" syntax
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Add --json for raw output
python3 scripts/taiga.py --json issue 13714
# See full usage
python3 scripts/taiga.py --help
```
## URL Pattern Reference
Taiga web URLs follow these patterns:
| Type | Web URL Pattern |
|------|----------------|
| Issue | `https://tree.taiga.io/project/penpot/issue/<REF>` |
| User Story | `https://tree.taiga.io/project/penpot/us/<REF>` |
| Task | `https://tree.taiga.io/project/penpot/task/<REF>` |
To extract the **type** and **ref** from a URL:
- `issue/13714` → type=`issue`, ref=`13714`
- `us/14128` → type=`us`, ref=`14128`
- `task/13648` → type=`task`, ref=`13648`
## Python Script Reference
The `scripts/taiga.py` script wraps the Taiga API into a single convenient CLI
with sensible defaults.
### Usage
```
python3 scripts/taiga.py <taiga-url>
python3 scripts/taiga.py <type> <ref>
python3 scripts/taiga.py [--json] <taiga-url>
python3 scripts/taiga.py [--json] <type> <ref>
```
### Examples
```bash
# By URL (recommended — no need to think about type/ref)
python3 scripts/taiga.py https://tree.taiga.io/project/penpot/issue/13714
# By type and ref
python3 scripts/taiga.py us 14128
python3 scripts/taiga.py task 13648
# Raw JSON output
python3 scripts/taiga.py --json issue 13714
```
### Output
The script prints a clean, structured summary:
```text
User Story #11964 — 🔴 [DESIGN TOKENS] Typography Composite Input
================================
Status: Defining
Milestone: design-systems-sprint-26
Points: 3 role(s)
Assignee: Natacha Menjibar
Author: Natacha Menjibar
Created: 2025-09-01
Tags: iop-design-tokens
URL: https://tree.taiga.io/project/penpot/us/11964
================================
<full description text, unmodified>
```
The fields section includes type-specific information:
- **Issues:** Status, Type ID, Severity ID, Priority ID
- **User Stories:** Status, Milestone, Points
- **Tasks:** Status, Milestone, Parent US
## Reference
- API docs: https://docs.taiga.io/api.html
- Taiga instance: https://tree.taiga.io
- API base: https://api.taiga.io/api/v1
- Penpot project id: `345963`
- Penpot project slug: `penpot`

View File

@ -6,7 +6,7 @@ description: Update the project CHANGES.md with issues from a given GitHub miles
# Skill: update-changelog
Entry point for all changelog work. All rules — sections, exclusions, entry
format, the `:boom:`, `:rocket:` and `:lock:` contracts, pre-flight, and the
format, the `:boom:` and `:rocket:` label contracts, pre-flight, and the
anomaly report — live in `mem:workflow/updating-changelog` (file:
`.serena/memories/workflow/updating-changelog.md`). This skill routes to that
memory; it does not repeat it.
@ -28,25 +28,18 @@ memory; it does not repeat it.
## What to Read
1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth,
layout, entry rules, the `## Breaking change note` comment contract, the
advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
`dev-only-bugs`, `report`.
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
layout, entry rules, the `## Breaking change note` comment contract,
pre-flight checks, anomaly types, and pitfalls.
2. `mem:scripts/changelog` — `boom`, `check-merged`, `cross-ref`, `report`.
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`.
## Commands
```bash
python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"
python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog" # closed non-bug residue to review by hand
python3 scripts/gh.py advisories --patched-version "<VERSION>"
python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"
python3 scripts/changelog.py check-merged <PR numbers...>
python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py security "<VERSION>" # print the :lock: section
python3 scripts/changelog.py security "<VERSION>" --write # replace it in CHANGES.md
python3 scripts/changelog.py cross-ref "<VERSION>"
python3 scripts/changelog.py report "<VERSION>"
```
@ -55,7 +48,5 @@ python3 scripts/changelog.py report "<VERSION>"
- `CHANGES.md` is edited only on explicit request or during the release process.
- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
- `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
- `:boom:` and `:rocket:` are label-generated: never hand-edit them; edit the label or the `## Breaking change note` comment and re-run the command.
- Verify PR merge status before writing; only merged PRs are referenced.
- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.

View File

@ -9,8 +9,7 @@
app.common.data.macros/get-in clojure.core/get-in
app.common.data.macros/with-open clojure.core/with-open
app.common.data.macros/select-keys clojure.core/select-keys
app.common.logging/with-context clojure.core/do
benches.render-wasm.errors/attempt clojure.core/do}
app.common.logging/with-context clojure.core/do}
:hooks
{:analyze-call
@ -25,11 +24,7 @@
shadow.lazy/loadable hooks.export/rumext-lazycomponent
app.util.i18n/tr hooks.i18n/tr-dynamic
app.common.i18n/tr hooks.i18n/tr-dynamic
benches.render-wasm.declarations/defscene hooks.benches/defscene
benches.render-wasm.declarations/defcase hooks.benches/defcase
benches.render-wasm.runtime.commands/defcommand hooks.benches/defcommand
benches.render-wasm.runtime.protocol/resolve-or-reject hooks.benches/resolve-or-reject
benches.render-wasm.runtime.protocol/resolve-or-reject-with hooks.benches/resolve-or-reject-with}}
}}
:output
{:exclude-files
@ -41,7 +36,7 @@
"^(?:backend|frontend|exporter|common)/deps.edn"
"^(?:backend|frontend|exporter|common)/scripts/"
"^(?:backend|frontend|exporter|common)/dev/"
"^(?:backend|frontend|exporter|common)/test/(?!performance_tests/)"]
"^(?:backend|frontend|exporter|common)/test/"]
:linter-name true}
@ -106,4 +101,6 @@
:unused-binding
{:exclude-destructured-as true
:exclude-destructured-keys-in-fn-args false}}}
:exclude-destructured-keys-in-fn-args false
}
}}

View File

@ -1,77 +0,0 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.benches
(:require [clj-kondo.hooks-api :as api]))
(defn defscene
"Analyzes scene options and its compiled builder body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcase
"Analyzes case options and its compiled operation body."
[{:keys [node]}]
(let [[_ _ opts argv & body] (:children node)]
(if (and (seq body) argv (api/vector-node? argv))
{:node (api/list-node [(api/token-node (quote do)) opts
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
{:node node})))
(defn defcommand
"Analyzes command options and both generated functions."
[{:keys [node]}]
(let [[_ command & forms] (:children node)
[doc forms] (if (string? (api/sexpr (first forms)))
[(first forms) (next forms)]
[nil forms])
[options forms] (if (api/map-node? (first forms))
[(first forms) (next forms)]
[nil forms])
[argv & body] forms
[conditions body] (if (and (api/map-node? (first body)) (next body))
[(first body) (next body)]
[nil body])
handlers (:async (some-> options api/sexpr))
core (api/list-node
(into [(api/token-node 'fn)
(api/vector-node (mapv api/token-node handlers))]
body))
starred (api/token-node (symbol (str (api/sexpr command) "*")))
measured (get (some-> options api/sexpr) :measured-variant true)
definition (api/list-node
(into [(api/token-node 'defn) command]
(concat (when doc [doc]) [argv]
(when conditions [conditions]) [core])))]
{:node (api/list-node
(into [(api/token-node 'do)]
(concat (when options [(api/coerce (dissoc (api/sexpr options) :async))])
[definition]
(when measured
[(api/list-node [(api/token-node 'def)
starred command])]))))}))
(defn resolve-or-reject
"Analyzes the body of a Promise-returning form."
[{:keys [node]}]
(let [[_ & body] (:children node)]
{:node (api/list-node (into [(api/token-node (quote do))] body))}))
(defn resolve-or-reject-with
"Analyzes the cause handler and body. Binds `cause` in the handler only."
[{:keys [node]}]
(let [[_ cause on-error & body] (:children node)]
{:node (api/list-node
(into [(api/token-node (quote do))
(api/list-node
[(api/token-node (quote let))
(api/vector-node [cause (api/token-node nil)])
on-error])]
body))}))

View File

@ -1,9 +1,3 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns hooks.export
(:require [clj-kondo.hooks-api :as api]))

View File

@ -10,7 +10,5 @@
:extra-indents {rumext.v2/fnc [[:inner 0]]
cljs.test/async [[:inner 0]]
promesa.exec/thread [[:inner 0]]
specify! [[:inner 0] [:inner 1]]
benches.render-wasm.runtime.commands/defcommand [[:inner 0]]
benches.render-wasm.runtime.protocol/resolve-or-reject-with [[:inner 0]]}
specify! [[:inner 0] [:inner 1]]}
}

View File

@ -1,7 +1,7 @@
description: An unexpected problem or behavior
labels: ["needs triage"]
name: Bug
description: Create a report to help us improve
name: Bug report
type: Bug
labels: ["needs triage"]
body:
- type: markdown
@ -15,7 +15,7 @@ body:
- type: textarea
attributes:
label: Steps To Reproduce
description: Steps to reproduce the behavior
description: Steps to reproduce the behavior.
placeholder: |
Steps to reproduce the behavior:
1. Go to '...'
@ -26,22 +26,22 @@ body:
- type: textarea
id: expected
attributes:
description: A clear and concise description of what you expected to happen.
label: Expected behavior
description: A clear and concise description of what you expected to happen
validations:
required: true
- type: textarea
id: actual
attributes:
description: A clear and concise description of what happens instead; what the bug is.
label: Actual behavior
description: A clear and concise description of what happens instead; what the bug is
validations:
required: true
- type: textarea
id: screenshots
attributes:
description: If applicable, add screenshots to help explain your problem.
label: Screenshots or video
description: If applicable, add screenshots to help explain your problem
- type: textarea
id: desktop
attributes:
@ -87,4 +87,4 @@ body:
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the bug here
description: Any other context about the problem.

View File

@ -1 +1 @@
blank_issues_enabled: false
blank_issues_enabled: true

View File

@ -1,38 +0,0 @@
description: Improvement to existing behavior
labels: ["needs triage"]
name: "Enhancement"
type: Enhancement
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: problem
attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
validations:
required: true
- type: textarea
id: solution
attributes:
label: Describe the solution you'd like
description: A clear and concise description of what you want to happen
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the enhancement here

View File

@ -1,26 +0,0 @@
description: An epic is a large user story that cannot be delivered as defined within a single iteration
labels: ["needs triage"]
name: "EPIC"
type: EPIC
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: description
attributes:
label: Epic description
description: Tell us which epic feature do you want to discuss
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the feature request here

View File

@ -1,7 +1,7 @@
description: Suggest an idea for this project
description: Suggest an idea for this project.
labels: ["needs triage"]
name: "Feature request"
type: Feature
type: Enhancement
body:
- type: markdown
@ -15,24 +15,24 @@ body:
- type: textarea
id: problem
attributes:
label: Is your feature request related to a problem? Please describe
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
label: Is your feature request related to a problem? Please describe.
validations:
required: true
- type: textarea
id: solution
attributes:
label: Describe the solution you'd like
description: A clear and concise description of what you want to happen
description: A clear and concise description of what you want to happen.
label: Describe the solution you'd like.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered
label: Describe alternatives you've considered.
description: A clear and concise description of any alternative solutions or features you've considered.
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the feature request here
description: Add any other context or screenshots about the feature request here.

View File

@ -1,26 +0,0 @@
description: A specific piece of work
labels: []
name: "Task"
type: Task
body:
- type: markdown
attributes:
value: |
## Before you start
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
---
- type: textarea
id: task
attributes:
description: A clear and concise description of the task
label: Please describe
validations:
required: true
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Add any other context or screenshots about the task here

View File

@ -5,7 +5,7 @@ name: Auto Label and Add to Project
# code, it only reads ids from the event payload.
on:
issues:
types: [opened, closed]
types: [opened]
pull_request_target:
types: [opened]
@ -43,9 +43,6 @@ jobs:
// === 1. CONFIGURATION ===
const PROJECT_NUMBER = 8; // <--- Replace with your project board number
const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization
const STATUS_FIELD_ID = 'PVTSSF_lADOAcyBPM4BV2pqzhRPY48';
const STATUS_BACKLOG = 'f75ad846';
const STATUS_TECH_REVIEW = 'b0f45db9';
const OWNER = context.repo.owner;
const REPO = context.repo.repo;
@ -53,100 +50,6 @@ jobs:
const isPR = !!context.payload.pull_request;
const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id;
// A completed issue is already closed by GitHub. Route its Main
// project status according to whether QA needs to verify it.
if (context.eventName === 'issues' && context.payload.action === 'closed') {
const issue = context.payload.issue;
if (issue.state_reason !== 'completed') {
console.log(`Issue #${issue.number} was closed as ${issue.state_reason}; leaving project status unchanged.`);
return;
}
const targetStatus = issue.labels.some(({ name }) => name === 'team-qa')
? 'Ready for QA'
: 'Product Review';
const projectQuery = `
query($owner: String!, $number: Int!) {
organization(login: $owner) {
projectV2(number: $number) {
id
fields(first: 100) {
nodes {
... on ProjectV2SingleSelectField {
id
name
options { id name }
}
}
}
}
}
}
`;
const projectRes = await github.graphql(projectQuery, {
owner: OWNER,
number: PROJECT_NUMBER
});
const project = projectRes.organization.projectV2;
if (!project) {
throw new Error(`Could not find Main project #${PROJECT_NUMBER}.`);
}
const statusField = project.fields.nodes.find(({ name }) => name === 'Status');
const statusOption = statusField?.options.find(({ name }) => name === targetStatus);
if (!statusField || !statusOption) {
throw new Error(`Could not find Status option "${targetStatus}" in Main.`);
}
const issueProjectItemsQuery = `
query($issueId: ID!) {
node(id: $issueId) {
... on Issue {
projectItems(first: 100) {
nodes {
id
project { id }
}
}
}
}
}
`;
const issueProjectItemsRes = await github.graphql(issueProjectItemsQuery, {
issueId: contentId
});
const projectItems = issueProjectItemsRes.node?.projectItems?.nodes || [];
const projectItem = projectItems.find(
({ project: itemProject }) => itemProject.id === project.id
);
if (!projectItem) {
console.log(`Issue #${issue.number} is not in Main project #${PROJECT_NUMBER}; skipping status update.`);
return;
}
const updateStatusMutation = `
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId
itemId: $itemId
fieldId: $fieldId
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item { id }
}
}
`;
await github.graphql(updateStatusMutation, {
projectId: project.id,
itemId: projectItem.id,
fieldId: statusField.id,
optionId: statusOption.id
});
console.log(`Issue #${issue.number} closed; Main status set to "${targetStatus}".`);
return;
}
// Define your labels here
const labelToApply = 'needs triage';
@ -185,65 +88,6 @@ jobs:
}
`;
let itemId;
try {
const addRes = await github.graphql(addToProjectMutation, { projectId, contentId });
itemId = addRes.addProjectV2ItemById.item.id;
} catch (error) {
const messages = [
error && error.message,
...(Array.isArray(error && error.errors)
? error.errors.map(e => e && e.message)
: [])
].filter(Boolean).join('\n');
if (!messages.includes('Content already exists in this project')) {
throw error;
}
console.log('Item already in project, fetching existing item...');
const existing = await github.graphql(
`query($contentId: ID!) {
node(id: $contentId) {
... on Issue {
projectItems(first: 50) {
nodes { id project { id } }
}
}
... on PullRequest {
projectItems(first: 50) {
nodes { id project { id } }
}
}
}
}`,
{ contentId }
);
const nodes = (existing.node && existing.node.projectItems && existing.node.projectItems.nodes) || [];
const match = nodes.find(n => n.project.id === projectId);
if (!match) {
throw new Error(`Item already in a project, but not found in project ${projectId}`);
}
itemId = match.id;
}
// === 4. SET PROJECT STATUS ===
const statusOptionId = isPR ? STATUS_TECH_REVIEW : STATUS_BACKLOG;
const statusName = isPR ? 'Tech Review' : 'Backlog';
console.log(`Setting status to "${statusName}" for ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
await github.graphql(
`mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
updateProjectV2ItemFieldValue(input: {
projectId: $projectId,
itemId: $itemId,
fieldId: $fieldId,
value: { singleSelectOptionId: $optionId }
}) {
projectV2Item {
id
}
}
}`,
{ projectId, itemId, fieldId: STATUS_FIELD_ID, optionId: statusOptionId }
);
await github.graphql(addToProjectMutation, { projectId, contentId });
console.log("Automation successfully completed!");

View File

@ -38,13 +38,13 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry (push destination)
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -200,7 +200,7 @@ jobs:
ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }}
- name: Login to Docker Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
@ -210,7 +210,7 @@ jobs:
# images from DockerHub for unregistered users.
# https://docs.docker.com/docker-hub/usage/
- name: Login to DockerHub Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
@ -222,7 +222,7 @@ jobs:
# reuses the same PUB_DOCKER_* credentials as the DockerHub
# login above.
- name: Login to Docker Hardened Images registry (base image pull)
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -301,7 +301,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
@ -371,7 +371,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to Docker Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}

View File

@ -114,20 +114,20 @@ jobs:
persist-credentials: false
- name: Login to Docker Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to DockerHub Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -191,13 +191,13 @@ jobs:
persist-credentials: false
- name: Login to DockerHub Registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
username: ${{ secrets.PUB_DOCKER_USERNAME }}
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
- name: Login to Docker Hardened Images registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}
@ -242,7 +242,7 @@ jobs:
persist-credentials: false
- name: Login to Docker Hardened Images registry
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: dhi.io
username: ${{ secrets.PUB_DOCKER_USERNAME }}

View File

@ -34,7 +34,7 @@ permissions: {}
env:
ACTIONLINT_VERSION: '1.7.12'
ZIZMOR_VERSION: '1.32.0'
ZIZMOR_VERSION: '1.31.1'
jobs:
lint:

View File

@ -48,10 +48,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that
# other workflows can write to.
- name: Setup pnpm + Node.js
uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
uses: pnpm/setup@fa36a5ef7dcdefc3a7cb25655c122e00cf6e636e # v3.1.2
with:
working-directory: plugins
runtime: node@24.22.3
runtime: node@26.13.0
install: false
- name: Install deps

View File

@ -46,10 +46,10 @@ jobs:
# No dependency cache here: a deploy must not restore a cache that
# other workflows can write to.
- name: Setup pnpm + Node.js
uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
uses: pnpm/setup@fa36a5ef7dcdefc3a7cb25655c122e00cf6e636e # v3.1.2
with:
working-directory: plugins
runtime: node@24.22.3
runtime: node@26.13.0
install: false
- name: Install deps

View File

@ -1,90 +0,0 @@
name: Require Linked Issue
# A pull request must have at least one closing-issue reference, unless it
# carries the `no issue required` label.
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- labeled
- unlabeled
- ready_for_review
- converted_to_draft
permissions:
contents: read
pull-requests: read
issues: read
jobs:
check-linked-issue:
name: Require Linked Issue
if: ${{ !github.event.pull_request.draft }}
runs-on: ubuntu-latest
steps:
- name: Check linked issue
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
OWNER: ${{ github.repository_owner }}
REPO: ${{ github.event.repository.name }}
run: |
set -euo pipefail
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
result=$(gh api graphql \
-F owner="$OWNER" \
-F repo="$REPO" \
-F number="$PR_NUMBER" \
-f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
isDraft
labels(first: 100) {
nodes {
name
}
}
closingIssuesReferences(first: 100) {
nodes {
number
title
url
}
}
}
}
}
')
pull_request=$(echo "$result" | jq '.data.repository.pullRequest')
if [ "$(echo "$pull_request" | jq -r '.isDraft')" = "true" ]; then
echo "Draft PR: linked issue check skipped."
exit 0
fi
has_exemption=$(echo "$pull_request" | jq -r '
.labels.nodes | map(.name) | index("no issue required") != null
')
if [ "$has_exemption" = "true" ]; then
echo 'Label "no issue required" found: linked issue check skipped.'
exit 0
fi
issues=$(echo "$pull_request" | jq -r '
.closingIssuesReferences.nodes[]
| "- #\(.number): \(.title) (\(.url))"
')
if [ -z "$issues" ]; then
echo '::error::This pull request must have a linked issue, or the "no issue required" label.'
exit 1
fi
echo "Linked issue(s):"
echo "$issues"

View File

@ -1,76 +0,0 @@
name: Require Milestone on Linked Issues
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- ready_for_review
- converted_to_draft
workflow_dispatch:
permissions:
contents: read
pull-requests: read
issues: read
jobs:
check-linked-issues:
name: Require Milestones on Linked Issues
runs-on: ubuntu-latest
steps:
- name: Check linked issues
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
IS_DRAFT: ${{ github.event.pull_request.draft }}
REPOSITORY: ${{ github.repository }}
run: |
if [ "$IS_DRAFT" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
OWNER="${REPOSITORY%%/*}"
REPO="${REPOSITORY#*/}"
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
ISSUES=$(gh api graphql \
-F owner="$OWNER" \
-F repo="$REPO" \
-F number="$PR_NUMBER" \
-f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
closingIssuesReferences(first: 100) {
nodes {
number
title
url
milestone {
title
}
}
}
}
}
}
')
MISSING=$(echo "$ISSUES" | jq -r '
.data.repository.pullRequest.closingIssuesReferences.nodes[]
| select(.milestone == null)
| "- #\(.number): \(.title) (\(.url))"
')
if [ -n "$MISSING" ]; then
echo "::error::The following linked issues have no milestone:"
echo "$MISSING"
exit 1
fi
echo "All linked issues have a milestone."

View File

@ -1,45 +0,0 @@
name: Require Milestone
on:
pull_request:
types:
- opened
- reopened
- synchronize
- edited
- milestoned
- demilestoned
- ready_for_review
- converted_to_draft
permissions:
contents: read
pull-requests: read
jobs:
check-milestone:
name: Require Milestone
runs-on: ubuntu-latest
steps:
- name: Check milestone
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
pr_api="repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}"
is_draft=$(gh api "$pr_api" --jq '.draft')
milestone=$(gh api "$pr_api" --jq '.milestone.title // ""')
if [ "$is_draft" = "true" ]; then
echo "Draft PR: milestone check skipped."
exit 0
fi
if [ -z "$milestone" ]; then
echo "::error::This pull request must have a milestone."
exit 1
fi
echo "Milestone assigned: $milestone"

View File

@ -34,9 +34,6 @@ permissions:
jobs:
test-backend:
if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Backend Tests"
runs-on: penpot-extended-runner
container:
@ -94,39 +91,4 @@ jobs:
run: |
mkdir -p /tmp/penpot;
clojure -M:dev:test:coverage
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('backend/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Backend test coverage" \
--note "Namespaces under \`backend/src/app\`." \
--lcov backend/target/coverage/lcov.info \
> backend/target/coverage/summary.md
cat backend/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('backend/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Backend \
--lcov backend/target/coverage/lcov.info \
--emma-xml backend/target/coverage/coverage.xml \
--prefix backend/
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: backend-coverage
path: backend/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7
clojure -M:dev:test

View File

@ -32,9 +32,6 @@ permissions:
jobs:
test-common:
if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Common Tests"
runs-on: penpot-extended-runner
container:
@ -61,38 +58,3 @@ jobs:
working-directory: ./common
run: |
./scripts/test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('common/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Common JVM test coverage" \
--note "Namespaces under \`common/src/app\`, measured by the JVM tests." \
--lcov common/target/coverage/lcov.info \
> common/target/coverage/summary.md
cat common/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('common/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Common \
--lcov common/target/coverage/lcov.info \
--emma-xml common/target/coverage/coverage.xml \
--prefix common/
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: common-coverage
path: common/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7

View File

@ -36,9 +36,6 @@ permissions:
jobs:
test-frontend:
if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Frontend Tests"
runs-on: penpot-extended-runner
container:
@ -69,42 +66,6 @@ jobs:
run: |
./scripts/test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('frontend/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Frontend unit test coverage" \
--note "Logic namespaces under \`frontend/src/app\`, without \`app/main/ui\`." \
--lcov frontend/coverage/lcov.info \
--unloaded frontend/coverage/unloaded.txt \
> frontend/coverage/summary.md
cat frontend/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('frontend/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project Frontend \
--lcov frontend/coverage/lcov.info \
--json-summary frontend/coverage/coverage-summary.json \
--unloaded frontend/coverage/unloaded.txt
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: frontend-coverage
path: frontend/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7
- name: Component Tests
working-directory: ./frontend
env:

View File

@ -32,9 +32,6 @@ permissions:
jobs:
test-render-wasm:
if: ${{ !github.event.pull_request.draft }}
permissions:
contents: read
pull-requests: write # coverage comment on the PR
name: "Render WASM Tests"
runs-on: penpot-extended-runner
container:
@ -52,71 +49,14 @@ jobs:
- name: Format
working-directory: ./render-wasm
run: |
./fmt --check
cargo fmt --check
- name: Lint
working-directory: ./render-wasm
run: |
./lint
- name: Install cargo-llvm-cov
env:
CARGO_LLVM_COV_VERSION: "0.9.1"
run: |
case "$(uname -m)" in
aarch64|arm64)
arch="aarch64"
sha256="abf5f13c1520f8756d2192bfaaeadb0208f5b7eaa8cadc15bf847befa42b7360"
;;
*)
arch="x86_64"
sha256="b3f68e625481fed9b16444174f3fa5ebcdbde4a1878803a35eabe2dcefcdc41a"
;;
esac
archive="cargo-llvm-cov-${arch}-unknown-linux-gnu.tar.gz"
curl -fsSL --retry 3 -o "$RUNNER_TEMP/$archive" \
"https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${CARGO_LLVM_COV_VERSION}/${archive}"
echo "${sha256} $RUNNER_TEMP/$archive" | sha256sum -c -
tar -xzf "$RUNNER_TEMP/$archive" -C /usr/local/bin cargo-llvm-cov
cargo llvm-cov --version
- name: Test
working-directory: ./render-wasm
env:
RENDER_WASM_COVERAGE: "true"
run: |
./test
- name: Coverage summary
if: ${{ !cancelled() && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
run: |
node scripts/coverage-summary.mjs \
--title "Render WASM test coverage" \
--note "Rust sources under \`render-wasm/src\`, measured by cargo-llvm-cov on the native test build." \
--lcov render-wasm/target/coverage/lcov.info \
> render-wasm/target/coverage/summary.md
cat render-wasm/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
- name: Coverage comment
# Fork PRs get a read-only token, so only branches of this repo comment.
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
continue-on-error: true
env:
GITHUB_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
node scripts/coverage-comment.mjs \
--project "Render WASM" \
--lcov render-wasm/target/coverage/lcov.info \
--llvm-json render-wasm/target/coverage/summary.json
- name: Upload coverage report
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
if: ${{ !cancelled() }}
with:
name: render-wasm-coverage
path: render-wasm/target/coverage/
overwrite: true
if-no-files-found: ignore
retention-days: 7

3
.gitignore vendored
View File

@ -68,7 +68,6 @@ opencode.json
/frontend/playwright-report/
/frontend/playwright/ui/visual-specs/
/frontend/text-editor/src/wasm/
/frontend/coverage/
/frontend/dist/
/frontend/npm-debug.log
/frontend/out/
@ -122,7 +121,7 @@ opencode.json
/.opencode/reports
/.opencode/prompts
/.ci-logs
/tmp
/tmp/
/.codex/
/scripts/__pycache__/
/tools/__pycache__

2
.nvmrc
View File

@ -1 +1 @@
v24.22.3
v24.22.2

View File

@ -0,0 +1,10 @@
---
description: Create a PR for the current task branch or update an existing one — loads and follows the create-pr skill
agent: build
---
Load the **`create-pr`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -0,0 +1,10 @@
---
description: Execute a ready plan — task checklist, your confirmation, then all tasks with one commit (default) or step by step with a commit and a pause per task; creates issue + branch when on a base branch, or commits on the current branch with "direct" — loads and follows the implement-plan skill
agent: build
---
Load the **`implement-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -0,0 +1,10 @@
---
description: Investigate the chosen task, produce an implementation plan, and save it — loads and follows the make-a-plan skill
agent: build
---
Load the **`make-a-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -0,0 +1,6 @@
---
description: Resolve local git conflicts and stage the resolved files; never continues the rebase — loads and follows the resolve-git-conflicts skill
agent: build
---
Load the **`resolve-git-conflicts`** skill and follow it as your only instruction.

View File

@ -0,0 +1,10 @@
---
description: Code review — review a diff, PR, or code change — loads and follows the review-code skill
agent: build
---
Load the **`review-code`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -0,0 +1,10 @@
---
description: Plan review — evaluate an implementation plan before executing it — loads and follows the review-plan skill
agent: build
---
Load the **`review-plan`** skill and follow it as your only instruction.
## User input, overrides and additional context
$ARGUMENTS

View File

@ -11,10 +11,8 @@
- Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in.
- Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`.
- Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
- Organization SSO (Nitrate): the RPC `check-nitrate-sso` takes the route's **query string proper** (`:query-string`, e.g. `screen=workspace&team-id=…`), never a URL and never with the `?` separator: the client strips it (`rt/token->query-string`) and the server puts it back. The server builds the destination with `app.auth.oidc/build-dest-url` (`public-uri` + `?` + the params, parsed and rendered back), so the redirect can never leave the public URI; a value whose parsed keys are not param names is not a route: a URL parses into a single key holding the whole URL, and a key carrying `:`, `/`, `?` or `#` is rejected with a `:warn` and the user lands on the public URI (the RPC never fails because of it). Do not reason from `u/map->query-string`: it keeps a valueless key as `key=`, so rendering is not what filters a URL out — the key shape is. A leading `?` is still trimmed server-side, so a client bundled before the trim keeps its destination. Never require `screen`: the root of the app carries its params instead of a screen (`?template=…`), and `check-nitrate-sso` is called from that branch too (`dtm/resolve-login-team-id`, which today reads only `:authorized` and never follows the redirect, so a wrong destination there is a silent `:warn` and not a visible break); the callback re-checks the destination before redirecting, which covers state tokens minted before that. The state token also carries `:profile-id`: only that session is granted the `props :sso` entry (`session-mismatch` otherwise, and the redirect still happens).
- Organization SSO needs `app.nitrate/organization-access?` whenever the RPC receives an `:organization-id` (that id always comes from the client): the profile must be a member of the organization or its owner; otherwise the RPC answers `{:authorized true :reason :no-organization-access}`, the same short-circuit shape as `:no-team-access`.
- Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. SSO registration (OIDC and compatible providers) follows the OIDC callback gate: it requires `:registration` or `:oidc-registration`, or a team invitation token issued for the registering email (an invitation substitutes the flags only when `member-email` matches the OIDC/`prepared-register` email via `clean-email`, as in the password flow, which raises `:email-does-not-match-invitation` otherwise). The OIDC callback verifies the invitation JWT and the email before redirecting to register, and `register-profile` revalidates the invitation under a row lock when no SSO flag is set; it never requires password login. Only OIDC-typed backends (`oidc-provider-types` in `app.auth.oidc`) take the SSO registration gate; any other `:backend` in a prepared-register token is rejected as `:invalid-token`.
- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile.
- Access tokens (`app.http.access-token`): personal tokens have JWT issuer `access-token` and DB type other than `mcp`; MCP tokens have issuer `urn:penpot:mcp-token` (`actoken/mcp-token-issuer`) and DB type `mcp`. Authentication requires issuer and type to agree, and the claim `uid` to match the row's profile; other issuers and undecryptable tokens are treated as anonymous.
- Management RPC (`/api/management/methods/:name`) is shared-key only. The middleware authenticates `<id> <secret>`; the RPC wrapper `wrap-shared-key-authorization` authorizes the key id against the method's `::rpc/allowed-shared-keys` set. Absent metadata accepts any configured key; all current methods declare one (`exporter`/`media-processor` for `upload-tempfile`, `nexus` for file analysis, `exporter` for jobs, `admin-console` for Nitrate). A valid key outside the set answers 403 `:shared-key-not-allowed`; denied calls do not reach the audit wrapper.
- Personal tokens need flag `:access-tokens`. MCP tokens need `:mcp` or `:access-tokens`, and only on routes whose `actoken/authz` gets `::actoken/allow-mcp true` (the RPC routes); elsewhere (e.g. assets) they are anonymous.
@ -26,16 +24,13 @@
- `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence.
- Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read.
- File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2).
- The organization-owner fallback (`get-organization-owner-permissions` via Nitrate over HTTP) grants read-only access for a non-member organization owner. It resolves the owning team through `resolve-team-id`, which refuses soft-deleted file/project/team: the file and project lookups filter `deleted_at`, and a final `db/get*` check covers the team row itself — `nil` means no access. `db/get*` only filters soft-deleted rows when `deleted_at` is in the projection. The re-checks of open websocket subscriptions rely on this (`mem:backend/websocket-subscriptions`).
- File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics.
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. RPCs that accept `share-id` enforce the link's page scope before mutating (`share-link-pages` + `check-page-scope`): a thread or page outside the link raises `:not-found`, and that check runs before the ownership checks. `get-profiles-for-file-comments` filters its query by the same pages. `mark-all-threads-as-read` does not take `share-id`.
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries.
## Teams, projects, and invitations
- Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so.
- Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token.
- The whole invitation family is team-admin only: creating, updating and deleting invitations, minting the invitation token (`get-team-invitation-token`) and listing the pending ones (`get-team-invitations`, which carries the invited email and the offered role). The token of an invitation that grants the `owner` role is restricted to the team owner, because an admin cannot create nor promote an owner invitation (`cant-promote-to-owner`). This family is the exception to the rule above: it answers `:validation` / `:insufficient-permissions` instead of `:not-found`, so the five commands speak one language.
- With the `:admin-console` flag, the organization `send-invitations` policy (role value plus Nitrate `restrictions-met`, through `cto/allowed?`) gates every invitation command that hands out or manages access: create (and resend, which reuses it), `update-team-invitation-role`, `delete-team-invitation` and `get-team-invitation-token`. Only `get-team-invitations` stays on team admin alone, so a blocked admin still sees the list. The shared helper `check-send-invitations-policy` in `teams_invitations.clj` does a strict Nitrate lookup (fails closed; 404 means no organization) and runs outside a DB transaction, which is why the role and delete commands carry no `::db/transaction`.
- Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup.
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing.

View File

@ -9,13 +9,12 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
- Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`.
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
- WebSocket subscriptions, the connection registry (local per backend, unlike the bus), and how a lost permission gets revoked: `mem:backend/websocket-subscriptions`.
- Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí)
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
## Stable namespace map
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/main/methods/<method-name>`; `/api/rpc/command/<method-name>` remains as a backward-compatible route.
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/rpc/command/<cmd-name>`.
- `app.rpc.permissions`: permission predicate/check helper factories.
- `app.http.*`: HTTP routes and middleware.
- `app.auth.*`: provider-specific authentication helpers such as LDAP/OIDC.
@ -68,8 +67,7 @@ Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` i
- Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
- The v3 and v1 imports only materialize a library relation when both ids belong to the package and the link would neither be a self link nor close a dependency cycle (the same `bfc/library-link-status` check that `link-file-to-library` uses to raise); relations are processed in sorted `(file_id, library_file_id)` order, which `get-files-rels` also returns, so the first edge wins deterministically, and every edge it ignores is logged with a warning. The v2 import stays out of that guard on purpose: it is only reachable from the developer REPL (`app.srepl.binfile`) and keeps inserting the relations as they come. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
- A version 1 package lists its files in the metadata and the importer creates one file per entry. The v1 export (gone from the product; only tests and the developer REPL use it) unioned the requested files with their libraries without deduplicating, so a package exported from a file inside a library cycle lists that file twice. The importer reads the pair of objects of every entry to keep the stream in sync, imports the first occurrence only, warns about the repeats, and answers with one id per imported file.
- The v3 import only materializes a library relation when both ids belong to the package and differ; a relation that points outside the package or a self link is ignored with a warning. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
- The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
- Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
- The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.

View File

@ -19,12 +19,6 @@
- Renewal fires when `modified-at` is older than 6h (`default-renewal-max-age`, not configurable). It `UPDATE`s the same row (`modified-at` only) and issues a new token that keeps the original `:exp`.
- `read-session` does not check age. Idle expiration is enforced on every request by `wrap-authz` (sessions past the idle window authenticate like missing ones and are never renewed); the GC only deletes the rows afterwards.
## Revocation paths
- Profile deletion → `session/invalidate-all` (all sessions of the profile die).
- Authenticated password change → `session/invalidate-others` (every session except the caller's dies).
- Password recovery (`recover-profile`, GHSA-3j2p-qr45-q365) → `session/invalidate-all` inside the same transaction as the password `UPDATE`; there is no current session to preserve, and a captured session must not survive recovery. Sibling of GHSA-mj9f-5cwq-7p3q (logout deletes the current session row).
## GC (`::tasks/gc`, cron `session-gc`, daily)
- Deletes from `http_session_v2` where `modified_at < now - :auth-token-cookie-max-age` **or** `created_at < now - :auth-token-cookie-max-age-absolute`.

View File

@ -93,7 +93,7 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
| `organization` | Organization logos uploaded by the Nitrate management API. | Yes | Public | No reference scan. A touched object is deleted. |
| `tempfile` | Export files and temporary font downloads. | No | Authentication required | No reference scan. A touched object uses a two-hour deletion delay. |
| `upload-session` | Chunked-upload chunks. References: `upload_session_chunk.object_id` and `upload_session_chunk.session_id` (both NO ACTION DEFERRABLE: restrict semantics, procedural deletion). | No | Authentication required | Reference scan on `upload_session_chunk.object_id`; a referenced chunk freezes, an orphan is deleted after the delay. Deferred rows keep their mappings until give-up. |
| `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference: the `file_data.storage_object_id` column (FK `NO ACTION DEFERRABLE INITIALLY IMMEDIATE`); the object metadata keeps `file-id` and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
| `file-data` | Encoded file data when `file-data-backend` is `storage`. Reference metadata has `storage-ref-id`, `file-id`, and the `file_data` row ID. | Yes | Authentication required | Reference scan. |
| `file-data-fragment` | Compatibility value for file-data fragments. The current backend has no dedicated producer for this bucket. | No current write semantics | Public | No touched-object collector case. |
| `file-change` | Compatibility value for file changes. Current snapshots store data in `file_data`, not this bucket. | No current write semantics | Authentication required | No touched-object collector case. |
| `job-resource` | Storage objects owned by job rows (`job.resource_id`), written by `app.jobs.storage`. | No | Authentication required, owner-scoped | jobs-GC touch → reference scan. |
@ -118,40 +118,18 @@ Since `put-object!` uses backend-specific operations (`impl/resolve-backend` + `
## File Data
- `file-data-backend` accepts `legacy-db`, `db`, or `storage`; `legacy-db` is read-only and the default is `db`. The configuration reads `legacy-db` as `db` at startup, with a warning (`normalize-file-data-backend`), so a deployment that kept the variable saves instead of failing on every write.
- `legacy-db` reads main data from `file.data` and snapshots from `file_change.data`; it no longer writes them (`fdata/upsert!` rejects it).
- `file-data-backend` accepts `legacy-db`, `db`, or `storage`.
- `legacy-db` stores main data in `file.data` and snapshots in `file_change.data`.
- `db` stores encoded data in `file_data.data`.
- `storage` stores encoded data in storage subsystem with `file-data` bucket, keeps `data` nil in `file_data` table, and points at the object with `file_data.storage_object_id`.
- `file_data.storage_object_id` is a `NO ACTION DEFERRABLE INITIALLY IMMEDIATE` foreign key to `storage_object(id)`: the database refuses to physically delete a referenced object. The `IMMEDIATE` is load-bearing: the RI violation fires inside the `storage-gc-deleted` chunk transaction, so the chunk rolls back, is logged at error level with a generic hint plus the `constraint` and `table` the database reports (any `NO ACTION` foreign key on `storage_object` can block the delete, not only `file_data`'s) and is retried on the next run; with `INITIALLY DEFERRED` the check would fire at commit, outside the catch, and the sweep would never move past the blocked chunk.
- The column is the only source of truth. The release is a coordinated restart (no rolling deploy) when the instance uses the storage backend (`PENPOT_FILE_DATA_BACKEND=storage`) or still has rows carrying `metadata.storage-ref-id`; with the default `db` backend and no such rows the previous and the new code read the same `file_data.data` column, so a rolling deploy is safe. `metadata.storage-ref-id` is no longer written or read, and the `file-data` metadata schema does not declare it. The migration backfills the rows it can resolve (valid uuid with an existing `storage_object`) and removes the key from those rows; a corrupt or orphan pointer keeps the key, which nothing reads. A large instance runs `operations/2.20.0-file-data-storage-object-id.md`: column, index and FK online, then the backfill (only if a row still carries `metadata.storage-ref-id`) with every instance stopped, verifies, and marks the migration as applied (`INSERT INTO migrations`). The backfill must never run while the previous code serves when the storage backend is in use (or rows with the pointer remain), because that code writes and scans the metadata pointer. A rollback to the previous code cannot read the rows written by this release when the storage backend is in use.
- `fdata/upsert!` touches the `storage_object_id` it receives in the params (the pointer the persisted row had) before replacing it, so the replaced blob is reclaimed. Callers pass the column from the file map (main/restore), `nil` for a new row (fragments, snapshots, new files) or the row itself (offload); there is no query in the upsert path, which is why every caller reads the file under the row lock first (the migration-on-read path included, see `mem:backend/subtleties`). It returns `{:written <bool> :storage-object-id <uuid or nil>}`, the pointer of the object the write created (nil for `db`/`legacy-db`), and `binfile.common/update-file!` propagates it so a migrating read returns the map with the pointer of the row it persisted instead of the pre-write one. The write also compares the expected pointer with `IS NOT DISTINCT FROM` (the caller's value, not the new one) and raises `:stale-file-data-pointer` when the row changed under it, so a stale map fails instead of leaking the replaced blob; the check lives in `upsert-in-database` because the `legacy-db` branches do not use the upsert SQL.
- `binfile.common/update-file!` takes the file map read from the database plus the fields to change (`[cfg file & params]`), merges them and builds the `file_data` params from the merged map, so the pointer travels in the read map and a caller cannot lose it by building a partial map; the params cannot override `:storage-object-id`. `insert-file!` always drops it (an INSERT never replaces a persisted pointer; `duplicate-file` hands the source map there). `bfc/get-file` keeps the pointer for the write paths, and every response goes through `bfc/file->response`, which strips it and any residual `storage-ref-id` left in the metadata of the rows the migration does not backfill: each `file-data` object is authentication-only and any logged-in profile could otherwise fetch it from `/assets/by-id`.
- The in-place binfile import was removed: `import-files!` raises `:not-supported` when the cfg carries `::bfc/file-id`, so no caller can overwrite a storage-backed row without carrying its previous pointer.
- `storage` stores encoded data in storage subsystem with `file-data` bucket and keeps `data` nil in `file_data` table.
- The `file_data.metadata.storage-ref-id` value points to the storage object.
- `fdata/upsert!` touches a storage object from incoming metadata before it stores the new row.
- File snapshots use `file_data` for snapshot data and `file_change` for snapshot metadata.
- Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil and `storage_object_id` points at the new object) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer locks the `file` row first (the update RPC and the snapshot paths with `FOR UPDATE` through `lock-for-update?`, the migration-on-read path with `FOR UPDATE`). A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`.
- The offload normalizes each `SELECT fd.*` row before `upsert!`: the row builder omits NULL columns, jsonb metadata arrives as a `PGobject`, and a JSON `null` decodes to nil and must disappear instead of travelling as `:metadata nil`, which the update schema rejects.
- Tiered offload (`:tiered-file-data-storage` flag + `db` backend): `file-gc` submits an `:offload-file-data` job (priority 10, delay 1000, same params) only when it actually processed the file; the handler takes the `file` row with `FOR UPDATE SKIP LOCKED` and skips (`{:skipped 1}`, no error) when the row is locked or gone, then moves every live `file_data` row of that file with `backend = 'db'` to `backend = 'storage'` via `fdata/upsert!` (one `put-object!` in bucket `file-data` per row, `data` becomes nil) inside a single all-or-nothing transaction per file. The skip only serializes because every `file_data` writer holds at least `FOR SHARE` on the `file` row. A storage failure aborts only that file and bubbles for a runner retry; orphans go to `storage-gc-touched`. A second run is a no-op because the query only matches `backend = 'db'`.
- Offload pacing: none by deliberate decision. Sleeps only make sense when batching multiple transactions; the per-file transaction is short by construction and the scheduler run takes minutes per production metrics, so neither loop sleeps. `jobs/heartbeat` (pool-based liveness, ~1 write/s) stays on both paths.
- Offload metrics (bounded labels only, never ids or error text): `penpot_file_data_offload_rows_total{type}` and `penpot_file_data_offload_bytes_total{type}` (`type`: `main`/`snapshot`/`fragment`, successful runs only) plus `penpot_file_data_offload_timing_ms{result}` (`result`: `ok`/`skip`/`error`). Helpers live in `app.jobs.metrics`; the job-def requires `::mtx/metrics`.
- Writes always resolve the backend through `fdata/default-backend` (`cf/get :file-data-backend`), so post-offload edits land back in hot (`db`) storage without extra code; reads stay transparent through `fdata/resolve-file-data` + `decode-file-data` + `realize`.
- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/touch-storage-object` with the row's `storage_object_id`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
- `storage-gc-touched` resolves the `file-data` reference with `SELECT EXISTS (SELECT 1 FROM file_data WHERE storage_object_id = ?)`, an indexed lookup that no longer parses the object metadata.
- A row with `backend='storage'` and no readable object makes `resolve-file-data` raise `:missing-storage-object` with the file and row ids instead of failing later in the storage backend.
- The `file-id` and row id kept in the storage object metadata are informational for diagnostics; nothing parses them to resolve the reference (the reference is the column).
## File Change
- `file_change` is partitioned by `HASH (file_id)` in 16 partitions, with the same bounds as `file_data`, so a join between them is resolved partition by partition (`enable_partitionwise_join` is set on every pooled connection).
- The automatic migration (`0158-partition-file-change`) refuses to run when `file_change` is above 10 GiB total size or 2M rows (`pg_total_relation_size` + `pg_class.reltuples`, catalog reads). It analyzes the table first, because `reltuples` is -1 without statistics. It raises with the path of `operations/2.20.0-file-change-partition.md`, because the copy happens inside the migration transaction and a big instance would restart in a loop. Below the guard it raises `statement_timeout` to 30 minutes for the copy and the index builds, and analyzes the new partitions at the end, after the copy and the indexes, so the planner does not start with empty statistics.
- Its primary key is `(file_id, id)`: a partitioned table needs the partition key in the primary key, so every access filters by `file_id`. From 2.20 the four snapshot commands (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot`, `unlock-file-snapshot`) require the `file-id` param, because they use it to prune the partition; without it they answer `:params-validation`.
- It has a `kind` column: `1` is a snapshot (it has a label and its data lives in `file_data`; the read-only `legacy-db` backend keeps old payloads in `data`) and `2` is a revision (it has no label and it holds the changes). The values are in `app.features.file-changes`.
- `kind` is a deliberate reserve, not a leftover: today no query filters by it, the queries still tell a snapshot from a revision by its label (`label IS NOT NULL`), and `kind` takes over when they are rewritten. No constraint ties `kind` and `label` together on purpose: the day `kind` is the source of truth, that constraint would have to go. It stays nullable on purpose: the previous version inserts without it, and a rollback to it has to keep writing; the `NOT NULL` comes in a later migration.
- Every write of the application to a row of `file_change` moves `updated_at` to the moment of the write (`lock-by!`, `unlock!`, the snapshot rename and delete, `delete-object`, the restore of a file). The actions of the foreign keys (`ON DELETE SET NULL` on `locked_by` and `profile_id`) change the row without touching it; during the partition both tables carry the same keys, so both receive them. The runbook of the partition uses it as the watermark of its delta passes: a write of the application that leaves it behind is lost at the cutover.
- `file_change_legacy` is a frozen copy of the previous layout of the partition: it keeps the rows and the indexes of the old table. A new instance (empty table) does not keep it. On an upgraded instance the table stays until the operator drops it by hand days later (`DROP TABLE file_change_legacy;`, see `operations/2.20.0-file-change-partition.md`); nothing drops it automatically.
- A revision carries only the delta of a save (the changes). The features of the file are not repeated on it: nobody read them from there.
- A snapshot carries the features and the migrations of the file.
- It keeps the `data` column for reading only: migration `0157-migrate-legacy-file-data` moved the existing blobs to `file_data`, and `legacy-db` remains as a read-only backend for the payloads still in `data`. `fdata/upsert!` rejects `legacy-db` and the `file-data-backend` default is `db`. It has no `data_ref_id` or `data_backend` columns: nothing read them since the file storage refactor.
- It has no index for the system snapshots. The old one filtered them by `data IS NOT NULL`, and with the payload living in `file_data` the column no longer tells snapshots apart. The new one could only filter them by `deleted_at IS NULL`, but the system snapshots always carry a deletion date in the future, so it would never match a row. The query that would use it filters that date with an "or", which no partial index can serve.
- The `objects-gc` purge groups each chunk by `file_id` and locks it with `file_id = ?` and `id = ANY(?::uuid[])`: a tuple `IN ((file_id, id), ...)` gives the planner no constant partition key, so it scans all 16 partitions in every chunk. Each group is one `FOR UPDATE SKIP LOCKED` query inside the chunk transaction.
- Deleting an offloaded row touches its storage object first (`objects-gc/delete-file-data-chunk` via `fdata/process-metadata`), so `storage-gc-touched` reclaims the blob with the usual delay; restoring a file unmarks the referenced objects.
## Metrics

View File

@ -45,15 +45,12 @@
- `job` is deletion-protected at the SQL level (`deletion_protection__tgr`): a stray `DELETE FROM job` raises instead of dropping the row and its `job_event` cascade without touching the referenced `storage_object`. The only intended deleters, the jobs GC and the submit dedupe, disable the guard with `SET LOCAL rules.deletion_protection TO off` inside their transaction; tests use `th/db-force-delete`.
- A missing job-def raises (`:no-job-definition`) instead of completing. Throwing with `ex-data :type ::retry` still controls retry behavior; `:strategy ::noop` retries without incrementing retry count.
- Cron entries claim their `scheduled_task` row with `FOR UPDATE SKIP LOCKED`, disable statement/idle-in-transaction timeouts locally, submit one `job` row per entry when no active instance exists (no-overlap), and reschedule themselves in `finally` unless interrupted. Worker, dispatcher, and cron components do not start when the DB pool is read-only.
- Integrant params schemas name injected components as bare `::ns/key` entries (pool, metrics, msgbus, job defs), and a bare entry is a malli registry lookup, not `any?`: the owning namespace must `sm/register!` it at load, or the backend crashes at boot with `:malli.core/invalid-schema` instead of asserting. Unit tests never build those keys, so `backend-tests.worker-test` compiles the three worker schemas and `backend-tests.http-websocket-test` compiles the revocation-watcher one to catch it. The assertion only runs with `PENPOT_FLAGS=enable-backend-asserts`, so without it a broken schema reaches production and only crashes at boot.
- Whether a bare entry works has nothing to do with its position or with how many of them there are: it is a registry lookup, so it works only if that exact keyword was `sm/register!`'d. `[:map ::db/pool ::metrics/metrics]` is fine, `[:map :app.aaa/unregistered :app.ccc/unregistered]` throws. When a component is **not** registered as a malli type, spell it as an explicit pair instead of registering a type just to satisfy the lookup — `[:app.nitrate/client {:optional true} [:maybe :map]]` is how `auth.oidc/schema:routes-params` and `loggers.audit.archive-task/schema:audit-log-archive-params` declare it. `{:optional true}` matters because `init-key` yields `nil` when the owning feature flag (`:admin-console`) is off, and a required key would then fail the assert on a perfectly normal boot.
- `sm/valid?` catches every `Throwable` and returns `false`, so an uncompilable schema and a genuinely invalid value both surface as a bare `Assert failed:` with no cause. When an `assert-key` fires with no message, compile the schema by hand to see the real exception.
- Integrant params schemas name injected components as bare `::ns/key` entries (pool, metrics, msgbus, job defs), and a bare entry is a malli registry lookup, not `any?`: the owning namespace must `sm/register!` it at load, or the backend crashes at boot with `:malli.core/invalid-schema` instead of asserting. Unit tests never build those keys, so `backend-tests.worker-test` compiles the three worker schemas to catch it.
## Config and HTTP/session middleware
- `app.config/config` and `flags` are dynamic `defonce` vars populated from `PENPOT_*` env vars through the shared schema string transformer. Tests and tooling can bind them.
- Duration config values are parsed as ISO-8601 with the `PT` prefix, so a day is written in hours (`:jobs-retention` and `:jobs-user-ttl` default to a week, and an operator writes `168h`): `7d` does not parse, and the value fails validation at boot instead of being silently ignored.
- `app.main` wires `PENPOT_DATABASE_*` into `::db/pool`, and `app.db/create-datasource-config` forwards pgjdbc's own knobs as `dataSourceProperties`, which `Properties.getProperty` silently drops unless the value is a String (hence the `str`). A query parameter of the same name in `PENPOT_DATABASE_URI` (for example `?prepareThreshold=0`) wins over the property: pgjdbc ignores the Properties value when the URL carries it, so the URI is the place to look when a variable seems to do nothing. The prepared statement cache is three optional `PENPOT_DATABASE_PREPARED_STATEMENT_CACHE_*` keys: the threshold is the executions before a statement is server-prepared and accepts `-1` or non-negatives (`{:min -1}`; the driver clamps any other negative to `-1`, so rejecting it only turns a typo into a boot error), its `0` stops server-side prepared statements while the driver's client-side cache of parsed queries stays in use, `..._QUERIES` and `..._SIZE` reject negatives and `0` in either disables that cache, and the size is bytes rounded up to the whole MiB the driver accepts. Do not describe the threshold `0` as disabling the cache: it only stops the server-side prepares.
- `parse-flags` automatically adds `:disable-secure-session-cookies` when `public-uri` is plain HTTP and not localhost. This changes cookie defaults without an explicit env flag.
- The backend sets Clojure `*assert*` globally from the `:backend-asserts` feature flag. Assertion-dependent checks can therefore differ by runtime flags.
- Request body parsing is mostly POST-oriented and supports Transit JSON plus plain JSON. Plain JSON request keys are kebab-decoded before being merged into `:params`. A body the parser cannot read is a client error, classified at the parse site and never by exception class (`clojure.data.json` signals its syntax errors with a bare `java.lang.Exception`): 400 `:validation`/`:malformed-json`, one `warn` line (`malformed request body`) carrying path, peer address and content-type instead of a stack trace, and `penpot_http_malformed_requests_total`. `RequestTooBigException` keeps its 413.
@ -87,29 +84,27 @@
- The `media-storage-bytes-per-team` quota check on `upload-file-media-object` is TEMPORARILY DISABLED (2026-10-19, kept commented in `app.rpc.commands.media` with its `sql:get-team-id-for-file` lookup still active for the audit props): the quote performs a full `SUM` over `storage_object` per upload and was too slow in production. The `upload-sessions-per-profile` quota is unaffected and stays active. Re-enable only with a cheaper approximation.
- Storage abstraction, backend configuration, logical buckets, object lifecycle, deduplication, access rules, and garbage collection: `mem:backend/storage`.
- SVG validation strips DOCTYPE and uses secure SAX parsing. Basic SVG info falls back to 100x100 dimensions when width/height/viewBox are missing.
- The SVG sanitizer (`app.media.svg`) is a **blocklist**, not an allowlist: it drops dangerous elements and attributes by name and escapes every text and attribute value when it writes the tree back, so a payload hidden in a value cannot become live markup again. The allowlist rework is [#12190](https://github.com/penpot/penpot/issues/12190) and lands in media-processor, not here. `sanitize-svg` also tracks the namespace bindings through the recursion and drops any element that is not in the SVG namespace, so a document whose root is not an `svg` element in the SVG namespace is rejected and a child that rebinds `xmlns` to XHTML (`iframe`/`srcdoc`/`meta`) is dropped with its subtree; it compares the scheme of an URL the way the browser does (dropping tab/CR/LF/spaces), so `java&#x09;script:` does not slip through the `javascript:` filter. The paths that store an uploaded SVG as its final object go through `app.media.svg/sanitize-svg-upload` (media upload, the URL download, both thumbnail writers and the organization logo); the three binfile imports go through `app.binfile.common/sanitize-imported-svg`, which calls `sanitize-svg` directly. The profile picture is not an SVG: `import-profile-picture` ignores an `image/svg+xml` download, because the avatar is always rendered as an image. The caller checks the size before calling it: the helper reads the whole file, and the size limit applies to the upload, not to the sanitized copy, which can be up to six times bigger.
- Upload handlers run the `media.v` checks in a fixed order: `check-upload-path`, then `check-media-type`, then `check-media-size` (or `check-font-size` for fonts). The path check gates every later read: `check-media-type` sniffs the bytes at `:path` and `check-media-size` reads `fs/size`, so a path the client named outside `allowed-upload-prefixes` must be rejected before either touches the disk. `check-media-size` leans on that and answers a file it cannot read with the path code (`:params-validation`), not `:media-max-file-size-reached`. Keep the order when touching or adding a handler; two branches that reorder these lines conflict.
- Raster metadata is shell-derived with ImageMagick `identify`, verifies detected MIME against the supplied MIME, and swaps dimensions for EXIF orientations 6/8.
- Remote image download requires 2xx status, `content-length`, a known MIME, and size under the configured maximum before writing the temp file; mismatched byte count is an internal error.
- Font processing shells out to FontForge and WOFF conversion tools and can derive TTF/OTF/WOFF variants from uploaded fonts.
## File data persistence
- File data backends for writing are `db` and `storage`; `legacy-db` is a read-only alias for old payloads. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores nil data and points at the object with the `storage_object_id` column (FK `NO ACTION DEFERRABLE`).
- `fdata/upsert!` touches the storage object the persisted row pointed at before replacing it; the pointer travels in the params (`file->file-data-params` from the read file map, `nil` for a new row, the row for the offload) and the params of `update-file!` cannot override it. `binfile.common/update-file!` merges the read file map with the fields to change, so the pointer survives; `insert-file!` drops it because an INSERT never replaces a persisted object. The column is the only source of truth and the metadata no longer carries `storage-ref-id` (the migration removes it from the backfilled rows and the write path never stores it); the release is a coordinated restart when the storage backend is in use (the previous code cannot read the new rows), while the default `db` backend reads the same data column and tolerates a rolling deploy. Responses built from a file map go through `bfc/file->response`, which strips the pointer and a residual `storage-ref-id` from the metadata.
- File data backends are `legacy-db`, `db`, and `storage`. The storage backend keeps encoded file data in storage bucket `file-data`; the DB row stores metadata with `storage-ref-id` and nil data.
- `fdata/upsert!` touches any storage object referenced by incoming metadata before storing the new row/blob.
- Pointer-map fragments are persisted separately as type `fragment`, and only modified pointer maps are written. A pointer keeps its id while nothing changes; content (`assoc`/`without`) and metadata (`with-meta`) changes produce a fresh id, so a fragment row is never rewritten after it is written. Re-assoc with an identical value returns the same map in array and hash maps alike, so a save keeps the fragment id of every page it does not change.
- `enable-objects-map` skips unloaded pointer pages: their fragment was already wrapped when it was written, and loading it only to wrap it again costs one page read per untouched page on every save. A page with a plain fragment is wrapped when it is first loaded or edited; a data migration that enables `fdata/objects-map` must rewrite the data itself.
- Snapshots share fragment rows with the live file (they are stored with pointers, not materialized). `file-gc` marks every fragment referenced neither by the current main file nor by an unpurged snapshot; the file is read unrealized and unchanged pages keep their ids via reattach, so a pass over an unchanged file writes no new fragments.
- A save attaches its automatic snapshot before applying pending file migrations: the snapshot keeps the pre-migration pointers, data and migration set, and restoring it leaves the next read to migrate the file. This also holds for files without the feature. The preview (`get-file-snapshot`) applies the pending migrations in memory with the live file's libraries and never persists them: the stored version keeps its data, the payload carries the snapshot's `:migrations` (an ordered set) and `fdata/objects-map` is realized for clients that do not declare it. The historical feature set is deliberately not checked against the enabled one: a version may predate a feature change. The window is narrow: the `get-file` RPC reads with `migrate?` on and its ETag hashes `available-migrations`, so a post-deploy read cannot be a 304 from the pre-deploy state and migrates before any save in the normal flow; the pre-migration snapshot only appears when the save is the first request to touch the file after the deploy (an open tab that saves without re-reading, or a non-UI writer). `create-file-snapshot` and the restore backup read with `migrate?` on, so those versions are stored already migrated.
- `:migrations` is always a `d/ordered-set`: `file-migrations/resolve-applied-migrations` builds one, `fmg/migrate` re-wraps the union after applying, and `file-snapshots/decode-snapshot` converts the `file_change` pgarray (as a plain vector, `set/difference` and `contains?` would compare indexes instead of names). A snapshot with NULL `:migrations` and an old `:version` predates the `file_change.migrations` column (SQL migration 0140): the preview regenerates the set from the stored version and applies that full chain, while `restore!` uses a fixed 67 baseline, so preview and restore can disagree for very old snapshots.
- The migration read path (`binfile.common/get-file*`) reads the `file` row once (`sql:get-file`, with its data columns; with the storage backend the blob lives in the object, not in the column) plus an explicit `resolve-applied-migrations`, because a row read does not resolve them. A read with no pending migration is completed by `attach-file-data` on that map: one row read and one migrations resolution. Only when a migration is pending is the row re-read under `FOR UPDATE` on the `file` row, where `need-migration?` is re-checked on the fresh map and the migrated file persisted (`binfile.common/migrate-file`); that re-read resolves the migrations again, and `migrate-file` once more after persisting, to return the applied set. The migration path holds two raw copies of the blob (the first read and the locked re-read) but a single decoded one: the alternative, a row-only probe before the read, avoided the first raw copy at the cost of an extra row read on every open, and was removed. `get-file` runs the read inside `db/tx-run!` (joining the caller's transaction when the cfg carries a connection) so the persist commits with the read; a read-only read (`:read-only?`) takes no lock and writes nothing, so it runs on a plain connection (`db/run!`) instead of opening a transaction of its own, and with a cfg that already carries a connection it still joins the caller's. It accepts a cfg map only and throws an `IllegalArgumentException` otherwise (`assert` is elided without `:backend-asserts`); a data read rejects `:lock-for-share?` (only row-only reads accept it), because the migration branch re-reads with `FOR UPDATE`, and a lock option needs a read-write transaction from the caller because PostgreSQL rejects row locks in a read-only transaction. A row-only read (`load-data?` false) resolves the applied migrations by default too (`:load-migrations?`), because the row can be completed later and the caller may need them; a caller that does not want them passes `:load-migrations? false`. Persisting is an optimization of the read (the read already returns migrated data): a row that changed between the first read and the locked read is not overwritten, and a pending migration discovered on the unlocked read is applied in memory only (the next read persists it under the lock). This was the only writer of the file row outside the lock protocol (save, library absorption, restore, `file-gc` and offload already take it); without it, a save that committed in between was overwritten by the stale map (lost content, `revn` moving back so the next save fails with `:revn-conflict`, and the object the save left as a frozen orphan). `migrate-file` captures `fdata/original-pointers` before realizing, reattaches unchanged positions after applying the migrations, and persists the reattached file; it returns that reattached map when it persists (the pages the migration does not change keep their pointers and fragment ids) and the realized map when it does not, so a read is plain only when the caller asks with `:realize?`, and a migration that adds `fdata/objects-map` skips the reattach so the pages get wrapped.
- The migration read path (`binfile.common/migrate-file`) captures `fdata/original-pointers` before realizing, reattaches unchanged positions after applying the migrations, and persists the reattached file; it returns the realized file because reads expose plain data. A migration that adds `fdata/objects-map` skips the reattach so the pages get wrapped.
- `fdata/load-pointer` reads rows that a previous `file-gc` pass marked on purpose (explicit `::db/remove-deleted false`): the fragment is only gone once `objects-gc` purges it. The `get-file-fragment` RPC keeps rejecting marked rows.
- Bind the fragment loader through `fdata/with-pointers` (reads), `fdata/with-tracked-pointers` (writes with a loader) or `fdata/with-tracked` (writes with nothing to load yet, e.g. file creation), and `fdata/with-untracked` (validation passes that must track nothing) instead of hand-rolled `binding` forms. Two direct bindings remain on purpose: the fine-grained tracked toggles of `app.rpc.commands.files-update` (validation/repair turn tracking off, restore the existing atom and read it before repairing), and `binfile.common/encode-file` (a fresh tracked set inheriting the current one). Converge them only if atom-scoped macros (`with-tracking`, `with-inherited-tracking`) are added; today the extra indirection buys nothing.
- Snapshot creation takes `FOR UPDATE` on the file row: both `create-file-snapshot` and the backup read of `restore-file-snapshot` hold it from the read to the insert, so they serialize with a `file-gc` pass (`FOR UPDATE ... SKIP LOCKED`) and with a concurrent save, and never pin fragments the pass is about to mark. The lock is exclusive and not `FOR SHARE` because the migration-on-read path may re-read the same row with `FOR UPDATE`: a shared lock that asks for the exclusive one deadlocks two concurrent readers (one creating a snapshot, the other saving, both seeing a pending migration). `get-file-snapshot` realizes the snapshot pointers with the live file id (fragment rows are keyed by file, not by snapshot).
- Snapshot creation takes `FOR SHARE` on the file row: both `create-file-snapshot` and the backup read of `restore-file-snapshot` hold it from the read to the insert, so they serialize with a `file-gc` pass (`FOR UPDATE ... SKIP LOCKED`) instead of pinning fragments the pass is about to mark. `get-file-snapshot` realizes the snapshot pointers with the live file id (fragment rows are keyed by file, not by snapshot).
- `fdata/realize` combines pointer realization and object-map realization. Use it before operations that need complete in-memory file data instead of pointer placeholders.
- A `file_data` row is one of three classes, told apart by `type`: `main` is the file's current state and carries `id = file_id`; `snapshot` is a point-in-time copy whose `id` is the one its `file_change` row shares; `fragment` is a piece of the pointer-map, referenced from the file's data.
- Their lifecycles differ: `main` and `snapshot` live with the file (dated when it is deleted, undated when it is restored), while a `fragment` carries no date and is collected by `file_gc` once the file stops referencing it. Neither is a session buffer, so do not read a `fragment` as an ephemeral revision.
- File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only read by the `legacy-db` backend (it is no longer written), plus the rows written before the 2025-07 file storage refactor.
- File snapshots (versions) keep their metadata row in `file_change` and their payload in `file_data` (`type = 'snapshot'`). With the `db` backend `file_change.data` stays NULL: that column is only written by the `legacy-db` backend, plus the rows written before the 2025-07 file storage refactor.
- Tell a snapshot apart by `label IS NOT NULL` (with `created_by` / `deleted_at` as needed), never by `file_change.data IS NOT NULL`: the label holds with every file data backend, and the payload column stopped meaning "this is a snapshot" when the payload moved out. Queries that identify versions must use the label. `0157-migrate-legacy-file-data` moved the labelled leftovers to `file_data` and deleted the unlabelled payload rows, so from 2.19 on the label is the only marker there is.
- The version quota counts of `app.rpc.quotes` (`sql:get-snapshots-per-file`, `sql:get-snapshots-per-team`) are served by the partial index `file_change__user_snapshots__idx`, whose predicate mirrors their where clause; changing one without the other drops the index only scan and the count walks the whole table again.
- Revisions live only in `file_change` and have no `file_data` counterpart. They are born already scheduled for deletion one hour later (`app.rpc.commands.files-update`) and the snapshot paths cannot reach one, because they resolve their row through a `label IS NOT NULL` filter.

View File

@ -3,18 +3,14 @@
JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
- READ `mem:testing` FIRST — it defines the execution discipline (no piping, tee to file, preferred commands) that applies to all JVM test runs.
- Test DB/Redis come from the `PENPOT_TEST_*` env vars already set in the shell (`PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`, ...); per instance they point at `penpot_test` / `penpot_test_wsN` and valkey DB `6+N` (`mem:devenv/core`). Do not override them with values copied from CI logs or other sources unless the user asks (e.g. "use a temporary database"); the CI env (`redis://redis/1`, `postgresql://postgres/penpot_test`) only works inside the CI docker network.
- The test database may have migrations applied from newer versions (it is shared dev infra). Reinitializing it — `dropdb`/`createdb` of the `PENPOT_TEST_*` database only — is explicitly permitted when that blocks a test run; never drop the main `penpot` database or another instance's test database.
- All CLI commands must be executed from the `backend/` subdirectory.
- Tests are invoked directly via `clojure -M:dev:test` (kaocha) — there is no pnpm wrapper. Kaocha auto-discovers test namespaces, so no runner registration is needed.
- **Preferred way to run them: `scripts/ci <module> --test`**, which runs lint, tests and format per module and writes a log per task under `.ci-logs/`. It exports `PENPOT_FLAGS` with `enable-backend-asserts` appended, so use it rather than calling `clojure -M:dev:test` by hand.
- **Why the flag matters:** `clojure.core/assert` is elided at compile time when `*assert*` is false, and `app.config` only sets that var from the flags when it loads. The test fixture redefines the flags after that, so they never reach it: without the flag the suite runs with every assertion compiled out, and a test that depends on an assert only means something with it set. `scripts/ci` and `tests-backend.yml` export the flag (GitHub Actions sets it too). When running the suite directly — including a `--focus` run — the command is `PENPOT_FLAGS="enable-backend-asserts" clojure -M:dev:test`.
- Coverage: if code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
- Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var. Keep the `PENPOT_FLAGS="enable-backend-asserts"` prefix on these too; a focused run without it hides assert-dependent failures exactly like a full one.
- A **whole-suite** crash reported as `Execution error ... Assert failed: ...` with no `FAIL in` line is not a test failure: an `ig/assert-key` ran while the test system was being built, which happens outside `clojure.test`, so kaocha reports it as a load/build error and the summary has no counts. Fix the offending schema (see `mem:backend/subtleties`) rather than hunting for a test.
- Isolated run: `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace, or `clojure -M:dev:test --focus backend-tests.my-ns-test/my-test-var` for a specific test var.
- Multiple focus symbols are supported by REPEATING the flag (kaocha accumulates them, last-wins does not apply): `clojure -M:dev:test --focus backend-tests.ns-one --focus backend-tests.ns-two` — preferred over a shell loop of isolated runs.
- Regression run: `clojure -M:dev:test` to ensure no regressions in related functional areas.
- Line coverage: append the `:coverage` alias (`clojure -M:dev:test:coverage`, kaocha-cloverage). It instruments every `app.*` namespace under `src/` at load, before any test runs, so even a `--focus` run instruments them all; options live under `:cloverage/opts` in `tests.edn`, output in `target/coverage/` (lcov, HTML, and EMMA `coverage.xml` whose form counts the PR report shows as statements; cloverage measures no functions or branches) (`lcov.info`, HTML). About 2x slower and noisy with reflection warnings (instrumented forms lose type hints): never use it for normal runs. `tests-backend.yml` runs it and renders the step summary with `scripts/coverage-summary.mjs`.
- If you need to filter output, tee to a temp file first: `clojure -M:dev:test 2>&1 | tee /tmp/penpot-test-output.txt`.
- RPC test helpers `command!`/`management-command!` split the data map: qualified keys become server params, unqualified keys become request body params. To inject request-level context (headers, `:app.http/auth-key-id`, ip), pass a map under `:app.http/request` metadata; non-map `IRequest` stubs fall back to a dummy request.
- The test system is `main/system-config` + `main/worker-config` with the
@ -34,4 +30,4 @@ JVM `clojure.test` (kaocha runner) under `backend/test/backend_tests/`.
`worker-config` can talk to a different Redis key prefix than a cfg the test
builds itself. The symptom is a component that silently does nothing
locally and races in CI (which sets no `PENPOT_TENANT`).
- `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there). The logger swaps the record on an executor thread, after the call that logged it returns: keep the watch installed until the record arrives (poll for it) and only then remove it, or the record lands on nobody under load.
- `backend-tests.helpers` has `metrics-stub` (a metrics instance with no collectors, where every `run!` is a no-op), `init-climit` (a concurrency limiter built from an inline config map, with an optional second argument for the metrics instance to publish to) and `with-climit`, the macro that builds one, runs the body and halts it. `init-climit` starts the republisher daemon thread, so a test that builds a limiter without halting it leaks that thread for the rest of the JVM run: use `with-climit`. Log lines are capturable with `add-watch` on `app.common.logging/log-record` (it holds the last record; the watcher sees every one, but only of the levels the logger has enabled: a line the app writes at TRACE never gets there).

View File

@ -1,44 +0,0 @@
# Backend WebSocket Subscriptions
## Registry
- `app.http.websocket/state` holds `{:connections {conn-id → wsp} :by-profile {profile-id → #{conn-id}}}` in **one** atom, so a single `swap!` keeps both consistent; `register-connection`/`unregister-connection` maintain them together and drop a profile from the index once its last connection leaves.
- Each connection's own `::ws/state` atom holds its subscriptions (`::file-subscription`, `::team-subscription`). `state` is shadowed by `::ws/state` in every `handle-message` destructuring, so only `on-connect`, `on-disconnect` and the `repl-*` helpers touch the registry — reading the registry from a handler is a bug.
- `::profile-id` and `::session-id` on a connection live in the `app.http.websocket` namespace, while `::ws/id`, `::ws/state`, `::ws/output-ch`, `::ws/session-id` live in `app.util.websocket`. Mixing them up yields silently `nil` values (a `nil` `session-id` makes the channel's `:xf` drop **every** message).
- The registry is local to one backend instance. The message bus is shared (`mem:prod-infra/core`). Anything that must reach connections owned by another instance has to go through the bus — closing a connection from an RPC handler only ever works in a single-backend deployment.
## Revocation model
A subscription is authorized once, at `:subscribe-file`/`:subscribe-team`, and that decision used to be trusted for the whole connection life (GHSA-m53j-2766-6jqw, medium). Two independent mechanisms now bound it:
- **Announcement (primary).** A mutation that can change access publishes on `app.rpc.notifications/internal-revocation-topic` (`notify-permissions-changed` for one profile, `notify-team-permissions-changed` for one team id — the watcher resolves the members itself, so a large team is one event, not N). The `::revocation-watcher` Integrant component in `app.http.websocket` subscribes and calls `revalidate-profile-subscriptions`, which looks the profile up in `:by-profile` and re-checks every subscription with `file-readable?`/`team-readable?`: existence (the file gate is the same `sql:get-minimal-file` as the `:subscribe-file` gate, made non-throwing via `{:db/throw-if-not-exists false}`; the team gate is `db/get*` with `::sql/columns [:id :deleted-at]` — `deleted_at` must be in the projection or the soft-delete filter is a no-op) AND the `has-read-permissions?` predicates — and `sql:file-permissions` additionally filters `deleted_at` on file, project and team, as do `sql:get-team-id-for-file`/`-project` plus a final team-row check in `resolve-team-id`, so the organization-owner fallback grants nothing through soft-deleted file/project/team. The existence gate closes that fallback (expelled org owner, deleted resource). The event is only the trigger; the fresh check is the decision, which is what keeps a downgrade to `viewer` (still readable) and a non-member organization owner on a live resource from being mistaken for a revocation. Both publishers defer through `db/after-commit`, because the watcher checks on another connection and would otherwise see the pre-commit state. `:subscribe-team` deliberately has no existence gate of its own; only the re-checks add one, matching the file side (user decision, PR #12189 review).
- **Interval (safety net).** `start-relay` runs the relay as a `go-loop` that waits on the subscription channel and a one-shot tick channel together via `sp/alts!` with `:priority true` on the tick: when both hold something, the tick wins, so a steady stream of messages can never push the re-check away. The cadence is `subscription-revalidation-interval` (default `default-revalidation-interval`, 5 min, a code constant in `app.http.websocket` and **not** in the `config.clj` default map, same rule as session lifetimes). The deadline is fixed per cadence: a single tick channel covers a whole cadence and message turns reuse it (minting a fresh timeout per turn would both churn timers and starve under load exactly like an in-loop timer does); a new one is minted only after a tick fires. It covers what no announcement can: Nitrate transferring organization ownership (an HTTP call to another service) and rows deleted straight over SQL. Closing the channel ends the loop either way, because a take on a closed channel is always ready with nil — every way a subscription can end closes it, so a finished relay never keeps polling. Cost is three or four queries per subscription per interval (existence row + predicates), plus one Nitrate call per team per interval when the `:admin-console` flag is on (cached 30 s); one `timeout-chan` per subscription per cadence on promesa's default scheduler (one or two threads shared with every other scheduled task) — do not shorten the interval without sizing that. A tick abandoned at teardown (the subscription closed while it was pending) still fires its scheduled close harmlessly. Two places fail open, deliberately: `still-authorized?` (a transient DB error logs and keeps the subscription; `InterruptedException` is rethrown) and the `catch Throwable` in `watch-revocations`.
Which mutations announce: `delete-team-member`, `leave-team`, `update-team-member-role` (member only), `delete-team`, `delete-file`, `delete-project`, `move-files`, `move-project` (one team event each). Anything not listed here is covered only by the interval. `delete-share-link` deliberately does not: a share-link visitor never passes the websocket subscribe check, because `check-read-permissions!` is called without a `share-id` and that arity only considers team membership.
`close-file-subscription`/`close-team-subscription` are idempotent: no-op on a mismatched id, and they `dissoc` the subscription so a repeat call and `:pointer-update` do not publish to a dead topic. Closing the channel is what ends the relay (`take!` on a closed channel returns `nil`).
## Topic and layering
- `internal-revocation-topic` is a fixed uuid that is never a real resource id, so it cannot collide with a file, team or profile topic — and deliberately not the `uuid/zero` system topic, which is `sp/pipe`d straight into the client output and anything published there reaches every client. No client can subscribe to the internal topic: `:subscribe-file`/`:subscribe-team` only accept ids that already passed a permission check, and `:broadcast`/`:pointer-update` only republish to ids the server already holds.
- The topic and the publish helpers live in `app.rpc.notifications`, not in `app.http.websocket`: the command namespaces cannot require the watcher, because `app.http.websocket` already requires `app.rpc.commands.files`/`teams`. The watcher keeps the registry, which is the part it owns.
## Time and channel traps in this namespace
- The relay waits with `sp/alts!` over `[tick-ch channel]` with `:priority true`, following the `msgbus.clj` pattern (`start-io-loop` destructures the same `[val port]` shape and compares ports with `identical?`). A take on a closed channel is always ready with nil, which is what ends the loop — and what makes closing the channel from any teardown path prompt even while parked in `alts!`. `sp/timeout-chan` is one-shot: it closes once after the delay, and one abandoned at teardown still fires its scheduled close harmlessly.
- `ct/diff a b` returns **b − a** (`Duration/between`), so "time until a future instant" is `(inst-ms (ct/diff (ct/now) future))`.
- `ct/duration` unit keywords are singular: `{:millis 150}`, `{:minutes 5}`, never `:milliseconds`/`:days`.
- `still-authorized?` takes the check itself: `(boolean (check-access))`. Calling `(boolean pred)` on the closure is always true and silently disables the interval — this exact bug shipped once and no test caught it until the resubscribe-counting test existed. Naming rule here: no `?` for the action that queries and can fail (`check-access`), `?` only for the safe yes/no answer (`still-authorized?`).
## Tests
`backend/test/backend_tests/http_websocket_test.clj` drives the real Redis-backed msgbus, so the revocation tests are end-to-end: register the connection in the real registry, run the real command, wait for the subscription to drop, then assert no content arrives. A connection built without `register-connection` is invisible to the watcher, so the watcher-based tests must go through `open-registered-connection`. Interval tests shorten the config via `with-redefs` on `cf/get` rather than waiting minutes. `with-clean-registry` runs as an `:each` fixture, so no test can leak a live relay loop into the next one. `captured-events` forwards the keyword args through to the real `mbus/pub!` — rebuilding them positionally publishes to the topic `:message` with a `nil` payload and silently untests every other notice. Interval tests take access away with a direct row delete, never with a command: a command announces, so the test would pass even with the interval removed. The same rule covers the two counting/fan-out tests: `revocation-closes-subscriptions-that-are-no-longer-authorized` takes access away with `delete-team-membership!` (raw row delete, publishes nothing) because a command announcement lets the live watcher close both subscriptions before the test's own `revalidate-profile-subscriptions` call, and its `(= 2 …)` count then reads 0; `team-event-closes-every-member-subscription` soft-deletes the team row (`sql:team-members` joins `team_profile_rel` and `profile` only, so the watcher still resolves every member) and asserts nothing closed before one synthetic `notify-team-permissions-changed`, leaving the command path to `deleting-a-team-announces-once` and `deleting-a-team-closes-open-subscriptions`. The resubscribe-counting test is the one that proves finished relays stop polling. The soft-delete + org-owner re-check tests pair a direct row soft-delete with `with-redefs` on `nitrate/organization-owner-of-team?`: before the existence gate, the org-owner fallback kept the subscription open on a deleted resource. The soft-deleted-project/team-closes-org-owner-file-subscription tests pin the root fix in `app.rpc.permissions` (`resolve-team-id` refuses dead file/project/team for every caller of the fallback). `watcher-consumes-revocation-events-from-the-bus` takes access away with a direct row delete and drives the watcher with a synthetic bus event — never a command and never a second `notify-*` — so the "still subscribed" assertion cannot race an after-commit hook and a broken command announcement cannot mask a broken watcher. `moving-a-project-closes-source-team-subscriptions` pins the move-project asymmetry: the file subscription drops, the source-team subscription survives (the member is still a member).
`revocation-watcher-schema-resolves-every-ref` covers the `::revocation-watcher` params schema, which no other test compiles. It existed broken for a while: the three injected keys were written as bare keywords, which is a valid malli *property* position and not a key, so the schema raised `:malli.core/invalid-schema` — invisible unless `PENPOT_FLAGS=enable-backend-asserts` was set, and a crash of the whole system build rather than one test (see `mem:backend/testing` on the flag).
## See also
- `mem:backend/auth-permissions-product-domains` for the permission predicates and the not-found semantics that the re-checks reuse.
- `mem:prod-infra/core` for why the bus crosses backends and the registry does not.
- `mem:frontend/routing-app-shell-subtleties` for the client side: the frontend already reacts to `team-membership-change` by navigating away, which is why revocation needs no new client message.
- `.agents/plans/2026-10-07-websocket-subscription-revocation.md` for the original plan, and the sibling `.agents/plans/2026-10-05-leave-team-file-access-cleanup.md` whose hardened permission queries this re-verification depends on (without live team membership required in `sql:file-permissions`, the re-check would keep reporting access for a member who already left).

View File

@ -1,4 +1,4 @@
# Clojure Core
# Clojure Design Rules
How to shape a function in this codebase. Each rule came from a review round; do not re-derive from taste.
@ -8,8 +8,7 @@ How to shape a function in this codebase. Each rule came from a review round; do
- **Functions that obtain data start with `get-` or `fetch-`.** A reader must see at a glance that the function reads (`get-file`, `get-snapshot`, `get-file-json`); a bare noun hides it (`file-json` says nothing about reading). New helpers that fetch by id follow this, never a `current-` prefix: the id parameter already says which one it is.
- **`?` marks a predicate, not a data prop.** A var or function that answers a question takes `?` (`export-error?`, `(defn editable? …)`); a map key that names a property does not (`:export-error`, never `:export-error?`), even when its value is a boolean. Existing `?` keys are legacy to remove gradually; do not add a new one.
- **Do not shape production code for tests.** No return values added just to assert them; assert on the observable effect. When preconditions move, move the tests with them.
- **All code-facing text MUST be in English** — identifiers, comments, docstrings, test descriptions. No exceptions; see the hard rule in `AGENTS.md`.
Reference implementation: the http metrics samplers in `app.http` (`sample-worker-metrics` / `sample-connector-metrics` are guard-free; `sample-http-metrics` guards at the boundary).
See also: `mem:clojure/idioms` (language behaviors and naming exceptions), `mem:clojure/logging` (structured logging), `mem:testing` (TDD and test conventions).
See also: `mem:clojure/idioms` (language behaviors), `mem:testing` (TDD and test conventions).

View File

@ -9,7 +9,6 @@ Behaviors confirmed against the language/stdlib — do not re-derive from assump
- Valid: `(defn ^:async f)`, `(defn- ^:async f)`, `(^:async fn [] …)` (the latter is what stock `t/async` generates itself).
- `^:async` fns never throw synchronously (rejected promises instead); `try/catch/finally` supported; continuations are microtasks, timers and RxJS schedulers are macrotasks (FIFO).
- Inside an `^:async` fn, a nested call that needs a temp binding (e.g. calling a local fn `(g (f x))`, whose arg gets a `G__` binding) compiles to `(await (async function(){…})())`: `await` unwraps a returned promise, so `(settled (upload (make-ctx)))` blocks until the upload promise settles instead of passing the promise on (a test that pushes the answer later hangs, idle). Bind each step in its own `let` binding.
- `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/core`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
- `!` suffix allowed cases (only exceptions to the no-new-`!` rule in `mem:clojure/design-rules`): mutating public `app.db` API (`insert!`, `update!`, `delete!`, `run!`, `tx-run!`, `exec!`, ...) and fns whose contract is to work over transients (stdlib `conj!`, `assoc!`, `dissoc!`, `disj!`, `persistent!` and custom helpers over them); any further case must be registered here with justification before use.
- Penpot extends `clojure.core/Inst` in `app.common.time` ( JVM-only section), so `inst-ms` also accepts `Duration` (returns `.toMillis`, NOT epoch millis), `FileTime` and `OffsetDateTime`. It only works where that namespace is loaded; on a bare `Duration`, prefer explicit `.toMillis`, which reads as what it is (an amount, not an instant).
- The threading macros insert the threaded value at DIFFERENT positions: `->` and `some->` put it FIRST (`(-> x (f a))` → `(f x a)`), `->>` and `some->>` put it LAST (`(->> x (f a))` → `(f a x)`). `some->`/`some->>` also short-circuit on nil. With unary calls both nest the same way (`(-> x (f) (g))` and `(->> x (f) (g))` are both `(g (f x))`): what changes between them is the position inside each call, not the order of the steps. A pipeline that calls a fn whose first argument is a config/context (`app.features.fdata/resolve-file-data` is `(cfg, file)`) must use `->>`/`some->>`; with `->`/`some->` the arguments are swapped and the call silently returns the wrong value (it cost a broken `get-snapshot` in `app.features.file-snapshots`).
- `cuerdas.core/index-of` returns `nil` (not `-1`) when the substring is absent, like `clojure.string/index-of`. `(pos? nil)` **throws on the JVM** but evaluates to `false` in CLJS (JS coercion), so `(if (pos? (str/index-of s "\n")) …)` works in the browser and crashes the backend. Guard with `some?` before `pos?` (this was `app.common.exceptions/first-line`).

View File

@ -1,7 +0,0 @@
# Clojure Logging Conventions
Structured logging through `app.common.logging` (`l/trc`/`l/dbg`/`l/info`/`l/wrn`/`l/error`/`l/fatal`): `:hint` carries the human-readable message, every other datum goes in its own keyword field (`:id`, `:container`, `:removed`, ...).
- `:hint` values start lowercase: `"fix dangling children"`, not `"Fix dangling children"`. Uppercase-first hints exist as legacy (~57 occurrences, the densest in `backend/src`; also `l/trc` in migration `legacy-28` and `log/error` in `app.common.files.repair`) — do not copy them, and do not mass-rewrite them unless the surrounding code needs the change.
- A name that requires uppercase (an env var like `PENPOT_MEDIA_PROCESSING_SERVICE_URI`, an acronym like `MCP` or `LDAP`) stays uppercase inside an otherwise lowercase sentence.
- Repair work done by a file migration logs at `l/trc`, only when it changes something, and with the ids it touched: the house style of `migrations.cljc` (`legacy-28` precedes; the file has no `warn`/`info` lines). The shipped `log4j2.xml` pins the `app` logger to `info` (static file, `monitorInterval="60"`), so those trace lines appear only after the operator raises the level; the durable record of a repaired file is the migration id in its `:migrations`.

View File

@ -8,7 +8,6 @@ Each change is a map such as `{:type ... :id ... :page-id ...}`. Common families
- `:add-obj`, `:mod-obj`, `:del-obj`: shape lifecycle. `:mod-obj` contains `:operations`, commonly `{:type :set :attr ... :val ... :ignore-geometry ... :ignore-touched ...}` or `{:type :set-touched ...}`.
- `:add-component`, `:mod-component`, `:del-component`: component/library metadata.
- Shapes stored inside the component record (deleted components keep them in `:objects`) are never written as a whole map: `mod-component` carries only component metadata by default, and shapes travel as `:mod-obj`/`:add-obj`/`:del-obj` changes scoped with `:component-id`. `pcb/update-component` has an `:objects? true` option for the two repairs that move or clear that key, and `ctkl/mod-component` reads an absent `:objects` as "leave the stored shapes alone" (only an explicit `nil` drops them). Before this, one component swap resent the full object map of every deleted component it touched (~6 MB in the request); do not reintroduce it.
- `:add-children`, `:remove-children`, `:reg-objects`: tree and object-map edits.
- `:set-option`, `:add-page`, `:mov-page`, and related file/page metadata changes.
@ -22,7 +21,7 @@ High-value builder operations:
- `pcb/with-page-id`, `pcb/with-objects`, `pcb/with-library-data`: set context for following operations.
- `pcb/update-shapes ids update-fn`: emits `:mod-obj` with diff-derived `:set` ops. Options include `{:with-objects? true}`, `{:ignore-touched true}`, and `{:attrs #{...}}`.
- `pcb/add-objects`, `pcb/change-parent`, `pcb/remove-objects`, `pcb/resize-parents`: shape/tree edits.
- `pcb/add-component`, `pcb/update-component`, `pcb/mod-component`: component/library edits. Repairs that fix shapes inside a component use `pcb/with-container` + `pcb/with-objects` + `pcb/update-shapes` so the changes come out scoped with `:component-id`.
- `pcb/add-component`, `pcb/update-component`, `pcb/mod-component`: component/library edits.
- `pcb/set-translation? true`: marks the whole change set as translation-only, which lets component sync skip expensive work.
## Applying changes in tests

View File

@ -26,12 +26,6 @@ Respect the intended abstraction direction in new/refactored code:
Some legacy code violates this layering; do not copy those violations into new code when a focused refactor is practical.
## Build-mode dependent laziness
`app.common.data.macros/lazy-delay` returns a delay whose expression is evaluated eagerly by default. Only the builds that set the `app.common.data.macros/lazy-delay?` closure define to `true` defer it to the first deref; every production build does (frontend `main` and `worker`, exporter, library), so the Closure compiler replaces the constant and drops the eager branch. On the JVM it is always eager.
`app.common.schema/check-fn` resolves its schema through it: an invalid schema definition fails when the namespace loads in development and in tests, and only on the first check call in production. The other lazily built parts of `check-fn` (validator and explainer) are always deferred.
## Focused memory routing
Model, schema, and persistence shape:

View File

@ -52,8 +52,3 @@ its own drift. Comparing `:selrect.width` from shape A to
`:selrect.width` from shape B is comparing two values that
"semantically" should be equal but were computed via different
operation chains — exact equality will often be false.
## Library sync comparison
- `update-attrs` (`app.common.logic.libraries`) compares main vs copy attrs exactly first, then with tolerance for: `drifting-geometry-attrs` (`:x :y :width :height :selrect :points`, via `gsh/close-attrs?`); path and bool `:content` (`segment/content-close?`; bool shapes keep their path in `:content`); and text `:position-data` (`position-data-close?` over `mth/close-map?`: every numeric field of every entry within `mth/close?`, other fields such as fills, fonts and text exact). All other attrs (`:opacity`, radii, `:transform`, `:rotation`, text `:content`) stay exact: tolerance there hides real changes.
- `close-attrs?` pairs lists with `d/zip`, which truncates to the shorter list and treats nil vs vector as equal for `:points`/`:position-data`. Do not use it for variable-length data. That is why `:position-data` has its own helper, and why `attr-equal?` checks the `:points` count before calling it.

View File

@ -34,7 +34,3 @@
- When a file had no previous `:migrations`, `migrate-file` marks all migrations as migrated in metadata so callers persist the complete migration set, not only transformations that changed data.
- Closed schemas (`schema:stroke-attrs` and others) reject a key that was dropped from the declaration, so removing a declared attr needs a migration in the same change, for data already written with it.
- Rebase collision on a migration number: upstream keeps the id. Renumber the branch migration to the next free one, keep it last in `available-migrations`, and rename everything that spells the number — its test file and namespace (`files_migrations_00NN_test.cljc`), the test names, and comments in production code or memories that name it.
- Withdrawing a migration id: remove both the `defmethod` and the id from `available-migrations`; the replacement migration subsumes it and files that recorded the old id re-migrate through the replacement (persisted `:migrations` keep the old id via union — do not rewrite them, including playwright fixture JSONs). Idempotent/pure-cleanup migrations only; never withdraw one whose transform must not run again unless the replacement is safe for those files. Precedents: "0021-fix-shape-svg-attrs" -> "0035-fix-shape-svg-attrs-and-defs"; "legacy-36" -> "0036-clear-nil-keys-in-objects".
- A shipped migration whose behavior must change AND reach every file (most files already recorded its id, so `migrate` skips it): rename it to `<id>-v2` and swap the old id for the new one in `available-migrations` at the SAME position. Precedents: `0002-normalize-bool-content-v2`, `0003-convert-path-content-v2`, `0008-fix-library-colors-v4`, and `0035-fix-shape-svg-attrs-and-defs` withdrawing `0021-fix-shape-svg-attrs`. Files keep the old id in `:migrations`, which is harmless because `migrate` diffs against the available set. Cost: one full pass over every file on its next read (the `get-file` ETag hashes `available-migrations`, so clients refetch).
- The alternative is patch the old id plus add a new migration id for the files that already applied it. Then the position in the list decides which repair runs first: a coerce-then-drop repair must come before a drop-only one, or the strings are gone before they can be parsed.
- `0033-normalize-main-instance-names-v2` repairs the container before its naming walk: child ids stored as strings are parsed with `uuid/coerce`, ids missing from `:objects` are dropped, duplicates are removed (the one `0032` adds when a child is stored as a string). The walk only re-assocs a shape when the normalization changed it (`identical?` on the normalized shape), so a healthy container keeps its `ObjectsMap` instance: no `modified` flag, no page rewrite. `0035-fix-dangling-children` prunes whatever is left.

View File

@ -15,7 +15,6 @@ From `common/`:
- Full JS test run (always builds, build output visible): `pnpm run test`
- Focus a JVM test namespace: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test`
- Focus a JVM test var: `clojure -M:dev:test --focus common-tests.logic.variants-switch-test/test-basic-switch`
- JVM line coverage: `clojure -M:dev:test:coverage` (kaocha-cloverage, about 2x slower; `:cloverage/opts` in `tests.edn`, which excludes `app.common.test-helpers.*`; output in `target/coverage/` (lcov, HTML, and EMMA `coverage.xml` whose form counts the PR report shows as statements; cloverage measures no functions or branches)). `scripts/test` (CI) uses it; the JS side has no coverage.
- Focus a JS test namespace: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test`
- Focus a JS test var: `pnpm run test:quiet -- --focus common-tests.logic.comp-sync-test/test-sync-when-changing-attribute`
- Quiet logging during a JS run: append `--log-level warn` (or `trace|debug|info|warn|error`)

View File

@ -6,9 +6,8 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- A section's top-level memory is `<section>/core`. When a section is relevant, read the core memory before focused memories.
- Edits/stale refs/duplication cleanup: `mem:memory-maintenance`.
- Cross-cutting testing principles, TDD workflow, and anti-patterns: `mem:testing`.
- Structured logging conventions (`:hint` casing and fields, log levels): `mem:clojure/logging` — read before adding or changing any log call.
- Verified Clojure language behaviors that contradict common assumptions (e.g. `int?` covers `Long`; `integer?` is the general predicate): `mem:clojure/idioms` — read before assuming stdlib predicate semantics.
- Read `mem:clojure/core` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
- Read `mem:clojure/design-rules` (function shape, boundary checks, naming) before writing or modifying Clojure/ClojureScript code — not only when designing a solution or an API.
- Promises: which call style to use where (`fmap`/`mcat`/`merr` vs `then`/`catch`) and how they bridge to rx: `mem:clojure/promesa` — read before chaining promises or mixing them with observables.
# Development workflow
@ -24,6 +23,7 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- **Never `git push`, force-push, or modify `git origin`** (or any other remote). The user pushes from their own shell; if a push is required, say so and wait. Never amend a commit that the user has already pushed unless explicitly asked.
- **Never edit `CHANGES.md` by hand.** The changelog is generated from GitHub milestones during the release process; update it only via `mem:workflow/updating-changelog` (the `update-changelog` skill flow) or on explicit user request.
- You have access to the GitHub CLI `gh` or corresponding MCP tools.
- Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool.
- Before writing code, analyze the task in depth and describe your plan. If the task is complex, break it down into atomic steps. After making changes, run the applicable lint and format checks for the affected module before considering the work done (per example `mem:backend/core` or `mem:frontend/core`).
- Align `let` binding values: when a `let` form has multiple bindings spanning several lines, align the value forms to the same column with spaces.
- If you introduce delimiter errors (mismatched parens/brackets) in Clojure/CLJS files, fix them with `scripts/paren-repair` BEFORE running lint/format checks. See `mem:scripts/paren-repair` for usage.
@ -60,8 +60,6 @@ The memory is structured in a way that you can get the critical information abou
- `scripts/psql` — PostgreSQL client wrapper with devenv defaults. Companion: `scripts/db-schema` for DDL dumps. See `mem:scripts/psql`.
- `scripts/taiga.py` — Fetch public issues, user stories, and tasks from the Penpot Taiga project without authentication. See `mem:scripts/taiga`.
- `scripts/gh.py` — GitHub operations helper: list milestone issues, fetch PR details, compare against CHANGES.md. Requires `gh` CLI. See `mem:scripts/gh`.
- `scripts/coverage-summary.mjs` — Renders an lcov file as a markdown table grouped by source area, for `$GITHUB_STEP_SUMMARY` (`--title`, `--lcov`, optional `--note`, `--unloaded`). Used by the backend, common and frontend test workflows; module coverage commands are in `mem:backend/testing`, `mem:common/testing`, `mem:frontend/testing`.
- `scripts/coverage-comment.mjs` — Keeps one PR comment (`<!-- coverage-report -->`) with the coverage table of Backend, Common, Frontend and Render WASM, in the format of the Vitest Coverage Report Action (status, metric, %, covered/total, changed-file coverage, run/commit footer). Each test job updates its own project: the data of all three lives base64-encoded in a hidden `coverage-data` block, and the job re-reads after `SETTLE_MS` and rewrites if a concurrent job dropped its data; extra report comments from simultaneous creates are merged and deleted. Targets (`THRESHOLDS`, per project and metric, status colour only, never fail CI) are on lines only: 70 Backend, 70 Common, 50 Frontend, 40 Render WASM; other metrics get the blue dot. Statements are Clojure forms for Backend/Common (`--emma-xml`) and llvm regions for Render WASM (`--llvm-json`). Shared lcov parsing: `scripts/coverage-lcov.mjs`. The test workflows run it only for PRs from branches of this repo (job-level `pull-requests: write`, `continue-on-error`): fork PRs get a read-only token, so they only get the run-page summary. A fork-safe `workflow_run` commenter would run only from the default branch's copy of the workflow.
- `scripts/project-anomalies.py` — Main-board anomaly check for a milestone (`check` writes `tmp/<MILESTONE>-ANOMALIES.md`). Bulk resolution via `scripts/gh.py`. See `mem:scripts/project-anomalies`.
- `scripts/changelog.py` — Changelog checks for `mem:workflow/updating-changelog`: `boom` (generate the `:boom:` section from `breaking change` issues), `check-merged` (PR merge status), `cross-ref` (milestone PRs vs changelog section), `report` (anomaly report to CHANGES-ISSUES.md). Calls `scripts/gh.py` via subprocess. See `mem:scripts/changelog`.
- `scripts/error-reports.mjs` — Query error reports via RPC API with token authentication. Supports list/get operations with filtering and pagination. See `mem:scripts/error-reports`.

View File

@ -15,7 +15,6 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_TEST_DATABASE_URI`, `PENPOT_TEST_REDIS_URI`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is.
- Per-instance test databases are created by `ensure-instance-test-database` (called from `start-instance`): `CREATE DATABASE` guarded by a `pg_database` existence check, so it is idempotent. Needed because `postgresql_init.sql` only runs on first volume creation. The backend applies migrations itself on first use, so an empty database is enough.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_s3_bucket()`. Never duplicates `defaults.env`.
- Frontend admin-console is off by default in devenv: `docker-compose.main.yml` sets `PENPOT_DEVENV=true`, which makes `app.config` (`goog-define devenv-mode`) prepend `disable-admin-console` ahead of the user `config.js` flags; flags parse in order and user flags win, so `enable-admin-console` in `frontend/resources/public/js/config.js` re-enables it. Outside devenv (prod builds) the default is unset and the admin-console stays on, matching `flags/default`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants

View File

@ -12,7 +12,7 @@ Frontend: CLJS SPA; React/Rumext; Potok; RxJS; okulary refs; SCSS modules; share
- `app.render_wasm.*`: frontend bridge to Rust/WASM renderer.
- `app.util.*`: DOM, HTTP, i18n, keyboard, codegen, and general frontend utilities.
- `frontend/packages/*` and `frontend/text-editor`: JS/TS workspace packages consumed by the app.
- Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared organization permission rules (`allowed?`, `can-send-invitations?`) are in `common/src/app/common/types/organization.cljc`.
- Nitrate subscription/organization UI and flows live under `app.main.data.nitrate` and `app.main.ui.nitrate*`; backend/API behavior is covered by backend memories, and shared permission rules are in `common/src/app/common/types/nitrate_permissions.cljc`.
- `app.config` external bridges (`external-feature-flag`, `external-session-id`, `external-context-info`, `external-notify-register-success`, `initialize-external-context-info`) delegate to `globalThis` hooks injected by the SaaS host. Never delete `external-feature-flag`, even with zero call sites: it is the seam for future A/B tests.

View File

@ -5,11 +5,8 @@
- Dashboard initialization fetches projects and fonts for the team, then listens to websocket messages only for global topic `uuid/zero` or the current profile id.
- Project fetch replaces each project map completely instead of merging, so fields such as `deleted-at` can disappear cleanly.
- Dashboard file/project mutations are often optimistic local updates with fire-and-forget RPC watchers. Bulk permanent delete/restore paths use SSE progress and progress notifications.
- The v3 format carries no size per file: the size of one is the sum of its own entries plus the `objects/` its media and thumbnail metadata reference, the only link between a file and that shared storage.
- Those metadata entries are read one at a time on purpose: a container holds thousands and every read opens its own decompression stream, which makes Chromium fail the analysis — zip.js then masks the cause behind a `Cannot close a ERRORED writable stream` — while Firefox tolerates it. Do not turn this back into a parallel read. The read is best-effort: a reference that cannot be read leaves that file without size, never the whole package.
- The `:error` of an import entry is user-facing text, never an i18n key. The legacy SSE import stays in the backend for other clients.
- The import wizard runs the package as a job of the backend: it reads the manifest of the file in the main thread (`app.main.data.imports`, no web worker) and follows the job, so its header paints the milestone of the job (the stage and its counter) instead of the messages the old SSE stream sent; every file of a container reports the same milestone, so `jp/current-milestone-text` writes it once and the rows never repeat it. Before the job exists, the header paints a client-side `:upload` milestone built from the chunked upload (`uploads/upload-blob-chunked` `:on-progress`, chunks done/total), which never comes from the backend; the chunks go one at a time and it writes them as a percentage of the file, not as a counter. Once created and still waiting for a worker, no row says anything: the header says the import waits for a slot while no job of it has started. The bar of the header measures the whole import, never a file: indeterminate while checking, waiting for a slot or rebuilding, because the client cannot know a global index and total of operations. The upload is the exception — the chunked upload knows its chunks, so the bar is determinate while the containers are sent, weighted by their size (`imp/upload-pct` over `:container-size`, the only percentage the client really knows). The export modal follows the same one-job-per-file model, but every file is queued up front (even the ones whose job is created only after the previous one is over) and its queue ends only with a milestone: `:start`/`:retry` do not end it there. Neither modal paints a generic "…ing file…" line on the rows any more: the rows of the import wizard carry only what is not progress — the name, the size, `Ready` once their container finished uploading, and the error text —, and the button that drops a file only shows while the wizard still offers its import step (the confirmation screen). Above the rows, the import wizard paints a stepper (Check/Upload/Import) plus one phase line: `imp/import-step` derives the step from the dialog status and the entries (upload holds while any file is pending, either `:import-ready` or carrying its own `:upload` milestone), so Check covers analyze, Upload covers the chunked client-side upload, and Import covers queue plus job. A file of a version 3 package shows what it occupies inside its container, computed while analyzing (`imp/file-sizes`): its own `files/<id>…` entries plus the storage objects its media and thumbnails reference, whose metadata is the only link to `objects/<id>…`. The container's own size stays in `:container-size` (what the dropped file weighs) and is what the upload bar weighs, never what a row shows. A file the manifest lists but the container holds no entry of has no size at all, and its row then shows no subtitle: what the package weighs never stands in for it. The subtitle is the size alone, with no extension. While analyzing with no failures nothing renders below the phase header — no rows, no spinner (`imp/awaiting-analysis?`); the header text is the only "Checking…" message. Rows appear once the analysis resolves (one per file inside the package), and a failed analysis takes the error-summary path instead. The ready step and its Continue wait for the end of the analysis (`imp/ready-to-import?`, fed with the completion of `imp/analyze`): a ready file is not enough, because the files still to arrive would never be imported. While rebuilding, the phase title counts the files and `imp/imported-count` counts them done with the job's own `:files` counter, so the number moves while a container rebuilds its files one by one. The `:error` of an import entry is user-facing text (the backend hint, or a translated label), never an i18n key: the wizard renders it as-is, it does not translate it again. The link-later step takes the file id from the `:id` of its resolution entry, never from the key of the resolution map: the map comes from the job result, so what a key holds depends on the result schema of the job-def that wrote it, while the wizard indexes its selection by the id of the entry. The legacy SSE import stays in the backend for other clients.
- File creation/duplication strips file `:data` before putting file summaries into dashboard state.
- The pending invitations are a team-admin only list (backend `:is-admin` already includes the owner). The dashboard hides the invitations entry from the header tabs and the team-options dropdown for non-admins, `invitations-section-content` keys the section on `:is-admin`, and `fetch-invitations` answers `insufficient-permissions` by emitting `invitations-fetched` with `[]`, so a demoted admin does not keep the cached emails and the header does not paint the invite button from a stale list. The organization `send-invitations` policy (`can-invite?`) does not gate the list, but it gates every action on it: inviting (header and empty-state buttons render disabled with the `dashboard.invite-profile-disabled` tooltip), the role selector (read-only label), row selection (so resend and delete never show) and copy link (`invitation-link-copyable?`). The backend enforces the same rule.
## Viewer

View File

@ -15,9 +15,6 @@
## Runtime initialization and permissions
- The frontend initializes `@penpot/plugins-runtime` only after `features/initialize` and only when feature `plugins/runtime` is active. It also installs the runtime `isPluginError` predicate into frontend error handling.
- `app.plugins/init-plugins-runtime` returns whether `ɵloadPlugin` was installed. It calls `preg/signal-runtime-ready` even on failure: `workspace.cljs` waits on that promise before `mcp/init`, so skipping it hangs MCP initialization forever.
- When SES bootstrap fails, `app.plugins` emits one `plugins-runtime-unavailable` telemetry event (with the SES error message), and `app.main.data.plugins/load-plugin!` shows the lazy `plugins.runtime-unavailable` toast when the user opens a plugin. `start-plugin!` (MCP path) stays silent by design.
- Runtime availability is detected by the existing `(fn? (unchecked-get global "ɵloadPlugin"))` check; do not add a second mechanism.
- Manifest parsing expands write permissions to read permissions (`content:write` => `content:read`, etc.). Permission checks also allow the all-zero plugin id and the hard-coded MCP plugin id.
- Manifest URL origin differs by manifest version: v1 clears the path; v2 joins `.` to the plugin URL. Existing plugin ids are reused by matching manifest name and host.
- The MCP plugin id is defined in `app.plugins.register` to avoid a circular dependency with workspace MCP code.
@ -37,7 +34,5 @@
- Plugin listeners are watches on the global store and callbacks are debounced about 10ms. Callback exceptions are caught and logged so plugin code does not crash the app.
- `selectionchange` callbacks receive arrays of shape id strings, while `filechange`, `pagechange`, and `shapechange` return proxies.
- `contentsave` fires only when persistence status transitions to `:saved`; it calls the callback with no value.
- `penpot.waitForSave(timeout?)` (`app.plugins.persist/wait-for-save`; runtime forwarder checks `content:read`) waits on the **shared** persistence queue, not only the plugin's own commits. Flow: ~100ms intake race (buffer/queue non-empty vs deadline, capped to the timeout) → empty resolves; pending emits `::dps/force-persist` and races `dps/wait-persisted-or-error` against `::dps/rollback-rejected` / `::dps/rolled-back` (rollback rejects even if status later reads `:saved`). Default 60s. `:error` status at intake rejects at once (`earlier-failure`): `save-retrying` while `dps/save-retrying?` (slow retry cycle, recovers by itself), otherwise the stored `[:error :code]` with reopen advice (a terminal failure stays until the file is reopened, so it fails every call). Rejections are `Error`s with `code`: `save-timeout`, `rolled-back`, `save-retrying`, or the failure code (`save-failed` unless the backend gave one).
- Plugin timeouts validate through `u/valid-timeout?` (positive finite number ≤ `u/max-timeout`, signed 32-bit) and reject through `u/reject-not-valid`, which rejects an `Error` (never a bare string) so callers can read `.message`.
- `contentrollback` is not a state watch: it subscribes to `st/stream` for `::dps/rolled-back` (kept in `subscriptions`, disposed by `remove-listener`), not debounced, delivered async so the reverted state is in place. Payload `{fileIds pageIds shapeIds}` from `pcm/rollback-targets` (page shapes only, from the reverted commits' redo changes). When the rollback falls back to reloading the file, `::dps/rolled-back` waits for the reloaded data, so the callback never reads the rejected data. The runtime's `validEvents` must list every event name, or `penpot.on` rejects it; a stale runtime build rejects new events. The MCP plugin forwards these rollbacks with the next task response and the server puts a warning first in that tool response (`RollbackNotices`). Unsent ones survive a tab or plugin reload in `penpot.localStorage` (key `rollbacks/<file-id>`, `LocalStorageRollbackStore`; the MCP manifest in `app.main.data.workspace.mcp` and `mcp/packages/plugin/public/manifest.json` grant `allow:localstorage`, which the runtime checks on access); the entry goes once a response carried them.
- Plugin history `undoBlockBegin` creates a workspace undo transaction with a JS `Symbol`; `undoBlockFinish` commits that symbol. Missing finish eventually relies on the workspace transaction timeout.

View File

@ -7,8 +7,6 @@
- `set-token!` and `replace-token!` are deliberately asymmetric. `set-token!` is a total no-op when the token already is the current one (no URL write, no report), so navigating to the screen you are on cannot grow the history stack. `replace-token!` always writes and always reports, because it is how the app corrects the URL in place (legacy `#/…` translation, cleaned share links) and that correction must reach the router even when the token did not change. This matches `Html5History`, which did the same; do not "fix" the asymmetry without deciding what navigates to the current screen.
- `app.util.dom/replace-history-state!` is the deliberate bypass used by `app.main.ui.dashboard`/`app.main.ui.static` to rewrite the URL **without** routing. Keep using it there instead of `replace-token!`, which would re-enter the router.
- Route params live entirely in the query map under the reserved `screen` key; duplicate query params can become vectors, so use `rt/get-query-param` when a scalar is required.
- The organization SSO flow sends the **query string** (`rt/get-current-query-string`), not the URL and not the token: the backend builds the destination URL from `public-uri` + `?` + those params, so the client never chooses the origin of the redirect. The history token (`rt/get-current-token`, `location.search`) keeps its leading `?` because every other consumer concatenates it over the base path (`str cf/public-uri token`, `dom/append-query-param`, `dom/remove-query-param`); `rt/token->query-string` is the only bridge between the two shapes, so do not trim at a call site by hand; `rt/token->url` is the single client builder of an absolute destination from either shape (used by the SSO retry and error flow), so do not assemble `public-uri + "?" + params` at call sites. The gate is not the only caller: `dtm/resolve-login-team-id` runs from the root branch of `on-navigate` too, and there the URL carries its params instead of a screen (`?template=…`) — that call only reads `:authorized`, so it never follows the destination, which is exactly why a wrong destination there goes unnoticed. Do not add a rule that requires `screen`. See `mem:backend/auth-permissions-product-domains`.
- The router events expose their payload through `@event` (`IDeref`): `rt/navigate` returns `{:id :params :options}` and `rt/nav-raw` returns its `:href`/`:uri`. `rt/nav-raw` is the way out of the router (raw URL, outside the token scheme) and nothing in the app listens to `::rt/nav-raw`; the destination travels in the effect, so `@event` is the only way to tell where the user is being sent (tests included).
- Legacy `#/…` hash URLs translate client-side to the query format (one-version compat; see `legacy-routes` in `app.main.ui.routes`, TODO(next-version) to delete).
- Unknown/empty routes trigger an extra `get-profile`/`get-teams` check before redirecting. This avoids invitation and root-route race conditions.
- The root app has two error layers, and they reset differently. `:exception` in the store renders `static/exception-page*` directly and is cleared by `rt/navigated`; a React render error is caught by `app.main.ui.error-boundary` (`react-error-boundary`), whose error lives in internal React state that only `reset-error-boundary` clears. Because every exit from the error page (the logo, the no-permission dialogs, the SSO dialog, the feedback link) only *navigates*, the boundary is given `:resetKeys [(eb/reset-key route)]` — the screen name — so a navigation counts as a fresh attempt at rendering. Three traps in that `resetKeys` value, all learned the hard way: it must be a real JS array (`#js [...]`), never a ClojureScript vector, because `react-error-boundary` reads `.length` and `.some` and a `PersistentVector` has neither, so `componentDidUpdate` throws `some is not a function`; it must hold the screen name as a *string*, because the library compares entries with `Object.is` and ClojureScript does not intern keywords, so two equal keywords are two objects and a keyword key would read as new on every route rebuilt, resetting the boundary on a param change inside the same screen; and it must be the screen rather than the params, so a param change does not replay the screen that just crashed. Navigating to the *same* screen therefore does not reset, and a reload is the only way out of that case.
@ -24,9 +22,6 @@
- `flash-persistence` only delegates to another handler for `:authentication`, `:not-found` and `:code :vern-conflict`. Everything else it reports itself, and the report type is derived from the cause: `#{:validation :server-error :internal}` are application defects and report as `unhandled-exception` so they reach the internal reports and the alerts; the rest stay `handled-exception`. `:cause-type` wins over `:type` when classifying, because `persistence-failed` wraps the original cause under `:type :persistence`. Environment failures are deliberately kept handled: they land here after the retry budget is spent and the user's connection is not our bug. `:assertion` is also kept handled, matching its own handler.
- The report governor is global and stateful across tests: two reports with the same fingerprint inside `report-window-ms` collapse. Call `errors/reset-report-governor!` between assertions that expect an emission.
- Error messages are bounded at the source: `app.common.exceptions/get-hint` clamps to `max-message-length` (1000 chars) and the CLJS `format-throwable` clamps the message line of the stack trace and honors `:trace-length`. A message can embed a whole payload (a serialized file, for example) and reach megabytes; the bound keeps the audit `:hint` prop and the `:report` string small. The `Context: Hint:` header in `generate-report` prints `get-hint` (not the raw `:hint` from `ex-data`) for the same reason.
- A team scoped navigation carries a team whenever any of the four tiers exists, and `app.main.data.common/resolve-team-id` decides it for every team scoped `go-to-*` helper of that namespace: the UUID the caller passes, `:current-team-id`, the team of the route already on screen (`?team-id`), then the profile `:default-team-id`. `go-to-feedback` and `go-to-viewer` are not team scoped and do not go through it, and `:team-id :default` keeps meaning the profile default, resolved before the rest. The route goes before the profile default because an emitter that runs from a team screen knows which team the user is looking at, and the personal team would open their file under another team. Two `rt/nav` emitters outside that namespace still decide their own id and can still emit nil, and they are the residual: `app.main.ui.nitrate.entry/nitrate-entry*` (the profile default) and the root branch of `app.main.ui.routes/on-query-navigate` (`dtm/resolve-login-team-id`, which answers nil when the profile has no default team). When none of the tiers exists the helper still fires the navigation with a nil id, so the not-found page appears instead of the click doing nothing.
- `app.main.ui/team-container*` raises `:not-found :invalid-team-id` for a dashboard or workspace route without a valid `team-id`; the raise is what keeps a broken URL from painting a blank shell, so it stays after the hook fix of #12728. `exception-section*` skips `errors/submit-report` for `:not-found`, so this failure never reaches PostHog and cannot be measured from the error dashboard. The exit from that page is the `error-container*` header (logo plus `not-found.no-permission.go-dashboard`), which emits `go-to-dashboard-recent` with the profile default and falls back to `rt/nav-root`; every not-found and authentication page already carries it, so there is no button to add there.
- A click cannot reach a team screen before its team is resolved: `team-container*` paints its children only when the route id matches the store team, and `refs/team` derives from `:current-team-id`, so with that key empty the dashboard and the workspace paint nothing clickable. The nil-id emissions come from events that run on their own (login, invitation acceptance, membership change, the sidebar logo, the workspace shortcuts) and from typed or bookmarked URLs.
## Store and websocket

View File

@ -17,8 +17,7 @@ Frontend testing is async-first: everything essentially asynchronous is modeled
- `mock/with-mocks` (callback style, legacy compat): installs with `set!` so mocks survive async boundaries; bodies run deferred past the current tick via `asap`, so only done-chained (`t/async`) contexts are allowed; `done'` restores and completes exactly once (twice only warns; never calling it stalls the run and leaks the mocks). Prefer `mock/with-mocks*` for new tests.
- `mock/with-mocks*` (direction): body forms wrapped in a generated `^:async` fn, evaluates to a promise — `await` it, `await` nested scopes too, no `done` in test code. Rejections and non-promise returns report as `:error` via `run-mocked`.
- `mock/stub` wraps fns for arities 0-6 (the `:esm` test build dispatches multi-arity vars as `cljs$core$IFn$_invoke$arity$N`); when the mocked var is variadic-defined (or called with more than 6 args), the call compiles to variadic dispatch, so use a plain variadic `fn` — the stub does not forward variadic. A mock must not call the mocked var again (self-delegation inside a multi-arity function recurses).
- Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for`, `wait-for-store`, `observe` (stream → termination promise; asserts provided, timeout rejects).
- `wait-for` waits on a **signal**, not on turns: `:signal` (an observable) re-checks the predicate on every emission and resolves on the event that changes the state; `:timeout-ms` (default 5000) only bounds failure then. Without a signal it polls against a real-time deadline — 50 turns were ~54 ms idle, and turns are not a unit of time. `wait-for-store` passes `ptk/input-stream` as the signal (the store notifies after applying the update, so the predicate sees the new state); the persistence fixture exposes `requests$` for `(= N (count @requests))`. A wait that resolves on an event consumes no turn. The waits with nothing to listen to stay on the poll: test atoms filled outside the store (`@cycles`, `@reports`, `@done`), `@failures`, and predicates over JS objects or closures.
- Helpers in `frontend-tests.helpers.async`: `->promise` (single-value observable → promise; beicon has no `to-promise`), `await-response` (subscribe→push→await, atomic), `settle`, `wait-for` (immediate check + bounded poll, fails instead of hanging), `observe` (stream → termination promise; asserts provided, timeout rejects).
- Fixtures return promises (`with-watchdog`, `with-persistence`); `await` them from `^:async` tests. `main_errors` and `fonts` are fully migrated (no legacy `with-mocks` left).
- Valid `^:async` placements: `mem:clojure/idioms`.
@ -38,15 +37,9 @@ To assert over emitted event sequences, observe termination: subscribe through `
- `st/emit!` doubles collect heterogeneous events: audit `DataEvent`s deref, toast reify-objects do not — discriminate with `ptk/type` (total, never throws), never blind `deref`.
- Subscription order decides delivery order: never resolve settlement from a pre-subscribed branch racing the pipeline.
- Auto-answering mocks lose deadline expressiveness (can't time answers), need teardown timer-cancellation, and post-teardown deliveries hit real implementations — explicit pushes + async delivery + `wait-for` won on every axis.
- A wait that observes an effect the test asserts takes a signal. A wait that means "the side effect advanced before the next emit" has no event to listen to: keep the poll (or an explicit `settle`) and say so, because a signal would resolve inside the same `emit!` and the next step would overtake the pipeline (`persistence-rollback-test` drove its four emits this way).
- With `rp/cmd!` mocked, a store-state wait can resolve before the flow that changes it ends, and the mock scope then closes in flight: the next call hits the real HTTP layer (malformed URL from a nil `cf/public-uri`) and the error is reported on a later test. `unlink-file-from-library` dissocs the library before its async chain, so the only proof the chain ran inside the mock is its second RPC call; wait for that, through the mock.
- Teardown belongs to the terminal continuation, never to `finally`-around-triggers (it would dispose in-flight flows).
- A body that awaits must be `(^:async fn …)` even if the rest is sync; sync sequences are atomic vs the event loop.
- `app.main.errors` prints every handled cause with `ex/print-throwable` before notifying, and the app logger does not gate it. A test that runs a real handler end to end stubs that var (`main-errors-test/quiet-throwable`): without it, the 10 000-error governor test alone wrote 190 k of the 192 k lines of a run log (13,8 MB), and GitHub truncates a step log past ~4 MB, cutting the summary and any failing assertion. Same for a production `console.error` a test provokes on purpose: `code-gen-markup-svg-wasm-test` silences `(.-error js/console)` in a `try/finally` while two WASM panics go through `get-svg-markup` (2 blocks, 70 lines, and the exception dumps its own `data`/`arr`).
- The `:esm` test build cannot load the CommonJS `sax` module behind `tubax`, so SVG parsing fails in tests (`dwm/valid-svg-string?` answers false, `dwm/svg->clj` errors): mock both with the parsed data.
- A hook-order violation in a mounted component reaches neither the root's `onUncaughtError`/`onCaughtError`/`onRecoverableError` nor the markup: React logs an internal error to `console.error` and recovers, so the callbacks stay empty and the tree repaints. Assert on a captured `console.error` too. It only fires when a render *adds* hooks (nil → uuid); a render that calls none at all slips through React's bookkeeping, so the regression test needs the "hooks arrive" direction. `frontend-tests.ui.team-container-test` does both, with a jsdom document.
- `beicon.v2.core` has no `never`. A stream that must **not emit** is `(rx/empty)`: it completes at once, which is enough for `merge`, `filter` and `take` where only the emission matters. A stream that must **stay open** is a `(rx/subject)` nobody pushes to; `rx/empty` would complete the flow behind it. An undeclared `rx/never` compiles to an undefined var and the test dies as a rejected body, which `cljs.test` reports as a test with zero assertions instead of a failure.
- `set!` mocks are global and outlive the fixture, so a flow another module left running (a store's pending timer) can still reach the mocked transport mid-test; `with-persistence` records every command for that reason. Filter to the commands your layer issues — persistence only sends `update-file` — before counting or asserting emptiness, and never let a stray call release a `wait-for` on one.
From `frontend/`:
- Full unit test run (always builds, suppressed output): `pnpm run test:quiet`.
@ -57,13 +50,12 @@ From `frontend/`:
- Build test target only (no run): `pnpm run build:test`.
- After `build:test` has been run, run the compiled runner directly: `node target/tests/test.js [--focus ...] [--log-level ...]`.
- Watch tests: `pnpm run watch:test`.
- Coverage: `pnpm run test:coverage` (builds, runs the suite under `NODE_V8_COVERAGE`, keeps its exit code). `scripts/coverage.js` maps V8 data back to `.cljs`/`.cljc` through the test build's source maps (`ast-v8-to-istanbul` + acorn) and writes `coverage/` (`lcov.info`, `coverage-summary.json`, `html/`, `unloaded.txt`); `tests-frontend.yml` renders the step summary with `scripts/coverage-summary.mjs` and uploads `coverage/` as the `frontend-coverage` artifact. `scripts/test` (CI) runs `test:coverage`, not `test`. Scope: `frontend/src/app/**` minus `app/main/ui/` (`EXCLUDED_DIRS`); namespaces no test loads are listed apart, never counted. Never pass `ignoreSourceCode` to `convert`: CLJS maps point past source lines and it throws. Trust line % over branch % (branches are of the generated JS).
New frontend test namespaces must be required/listed in `frontend_tests/runner.cljs`; new vars in existing namespaces need no runner change.
## Playwright integration tests
Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, run the whole suite with a worker count sized to the machine: `pnpm run test:e2e --workers=N` from `frontend/`, about one worker per two hardware threads, because each keeps two or three busy and holds about 2 GB of browser memory. The config's default of one worker leaves it slow; four take a 16-thread host from 48 minutes to 35 and fit a 64 GB machine beside a watching devenv, while a 32 GB machine takes two, or four with a slim devenv. `--grep "pattern"` narrows the run. Ensure dependencies are installed through `./scripts/setup` if the environment is not prepared.
Do not add, modify, or run Playwright integration tests under `frontend/playwright` unless explicitly asked. When explicitly asked, use `pnpm run test:e2e` or `pnpm run test:e2e --grep "pattern"` from `frontend/`; ensure dependencies are installed through `./scripts/setup` if the environment is not prepared.
Integration tests fake backend behavior by intercepting network/websocket traffic, so every RPC or websocket the page needs must be mocked. Use existing Page Object Models:
- `BasePage.mockRPC` intercepts RPC calls and already prefixes `/api/rpc/command/`; pass command names such as `get-profile`, not full URLs.

View File

@ -46,7 +46,6 @@
- `start-undo-transaction` stamps the transaction with a fresh `:commit-group` (dropped from the history entry); `commit-changes` uses `(or undo-group (:undo-group tx) (:commit-group tx) (uuid/next))`, so all commits of one transaction are one rollback unit. Pre-setting `:undo-group` instead breaks entries whose group is set later (sync transactions).
- Undo transactions can stay open only temporarily; timed-out pending transactions are force-committed after about 20s. Undo entries are capped at 50.
- Undo/redo are ignored while a normal editor/drawing interaction is active, except grid-layout edition handles undo through this path. `undo`, `redo` and `undo-to-index` also do nothing while the save lock holds: `materialize-undo` would move the index while the commit is refused.
- Undo entries carry a required boolean `:translation` (`schema:undo-entry` rejects an entry without it; `initialize-workspace` sets it when it records a local commit) and `undo`, `redo` and `undo-to-index` pass it back to `commit-changes`, so replaying a pure move skips component sync and thumbnails like the original commit did. Merging ANDs the flags: `stack-undo-entry`, `accumulate-undo-entry` (`empty-tx` starts at true) and undo groups in `undo-to-index` (`every?`). Test harnesses that mirror the recorder (`tokens/helpers/state.cljs`, `composable_tests/interpreter.cljs`) must pass it too.
- After local commits and when render-wasm is active, text shapes get derived `:position-data` recomputed in a separate commit tagged `#{:position-data}`; that tag is excluded from the position-data watcher to avoid loops.
## Refs

View File

@ -57,7 +57,7 @@ mcp/
│ ├── src/
│ │ ├── main.ts # handles communication
│ │ └── plugin.ts # plugin implementation
│ └── package.json # Plugin deps: published plugin-styles, linked plugin-types
│ └── package.json # Includes @penpot-mcp/common dependency
└── prepare-api-docs # Python project for the generation of API docs
```
@ -107,27 +107,12 @@ From the `mcp/` directory, run
(node reports `Promise resolution is still pending but the event loop has
already resolved` for the file's root test). Inside tests the logger writes to
stdout directly. `NODE_TEST_CONTEXT` is the env var `node --test` sets.
* `packages/plugin` takes `@penpot/plugin-styles` from the registry but links
`@penpot/plugin-types` to `plugins/libs/plugin-types` (`link:` in the manifest
and in `mcp/pnpm-lock.yaml`, with no registry entry, because a linked package
has no resolution): the published types do not declare the `penpotMgmt`
global the plugin needs. On a plugins libs version bump, only the styles
version moves. After editing the manifest, `pnpm install --lockfile-only` in
`mcp/` must leave the lock untouched.
* Cross-cutting testing principles and anti-patterns: `mem:testing`.
## OAuth for remote MCP clients
- Authorization/discovery URLs, self-contained client IDs, consent and PKCE/code binding, backend Redis state, deployment wiring, and compatibility limits: `mem:mcp/oauth`.
## Host and Origin allowlist (`RequestGuard`)
- `RequestGuard` checks `Host` and `Origin` (hostname only, port ignored) on `/mcp`, the REPL (Express middleware, 403) and the plugin WebSocket (close `1008` before any message is read). It stops web pages on the user's machine from driving a local server (DNS rebinding, cross-site WebSocket).
- Allowlist: `PENPOT_MCP_ALLOWED_HOSTS` (`*` turns the check off); unset → loopback only in single-user mode (local or remote: no other barrier, `execute_code` still runs in the user's session), any host in multi-user mode (relies on user tokens; behind nginx `/mcp/ws` the server sees `Host: penpot-mcp:4402`). A missing `Origin` passes (non-browser clients).
- Entries must be `*`, a hostname or a bracketed IPv6 address; one with a scheme, port or path makes `fromEnvironment` throw, so the server does not start. Startup logs a `warn` when the check is off in single-user mode.
- The plugin iframe has `allow-same-origin`, so its WebSocket carries the real plugin origin (`http://localhost:4400` locally, the Penpot origin when integrated).
- Test mocks of `PenpotMcpServer` passed to `PluginBridge` must provide `requestGuard`.
## Backend tools (`submit_feedback`)
- Tools that call the Penpot backend directly (not through the plugin) register only when `PENPOT_MCP_BACKEND_URI` is nonblank (`PenpotMcpServer.initTools`). Currently only `SubmitFeedbackTool`.
@ -148,11 +133,10 @@ For parallel devenvs, prefer same-origin MCP routing: each Penpot instance shoul
## Plugin reconnect policy
- The plugin treats WebSocket close code `1008` (policy violation) as terminal: it stops auto-reconnecting and stays disconnected until the user explicitly reconnects. Other close codes keep the capped-backoff retry. The decision lives in `ReconnectPolicy.ts` (`shouldReconnectAfterClose`), kept as a pure module so it is unit-testable without DOM/CSS.
- The MCP server emits `1008` for a `Host`/`Origin` outside the `RequestGuard` allowlist, for a missing `userToken` in multi-user mode, for missing/invalid connection metadata, and to close a connection displaced by a newer one with the same user and session ID.
- The MCP server emits `1008` for a missing `userToken` in multi-user mode, for missing/invalid connection metadata, and to close a connection displaced by a newer one with the same user and session ID.
- Session IDs are deterministic per tab and file, so a reconnect reuses its ID while the old socket may be half-open: the newest connection wins (`PluginBridge.displaceConnection`). Across instances, each new connection publishes a session claim on a per-user Redis channel; an instance displaces its local connection only on a foreign claim that arrives after its own claim (channel order, no clocks). Discovery deduplicates by session ID.
- A tab rejected with `1008` never reaches `connected`, so the frontend's 60s reconnect watcher (`start-reconnect-watcher` in `app.main.data.workspace.mcp`, started only on `connected`) does not engage; recovery is manual via the MCP menu.
## Rollback warnings
- `ExecuteCodeTaskHandler` awaits `penpot.waitForSave()` after the code and outside its swallow catch, so a save failure, save timeout or rollback during the wait fails that `execute_code` call itself. It hard-fails as "missing Plugin API support" when `penpot.waitForSave` is absent (old Penpot or stale runtime build, or broken `api.cljs` wiring). Both waits run only when `penpotMgmt.workspace.status` is `"ready"` (or there is no `penpotMgmt`), checked at each wait: dashboard calls skip them, because persistence `:error` outlives the workspace and would fail successful `penpotMgmt` calls. Budget: layout wait 30s before the code + save wait 60s after it, under `PENPOT_MCP_TOOL_TIMEOUT_S` (120s). `RollbackNotices` below only cover rollbacks that land after the call returned (concurrent edits, a response after a timeout).
- The plugin's `RollbackLog` (`packages/plugin/src/RollbackLog.ts`) holds the `contentrollback` events not yet sent with a task response; `Task.sendResponse` attaches them and acknowledges them once sent, and the server's `unclaimed-rollbacks` message (a response that came after its task timed out) puts them back. With a `RollbackStore` (`LocalStorageRollbackStore` over `penpot.localStorage`, one entry per file) every change is written through, so a reload of the tab or plugin keeps them; storage failures fall back to memory. Plugin tests run with `node --experimental-strip-types`, which rejects TypeScript parameter properties: declare fields explicitly.

View File

@ -18,7 +18,6 @@
## Sandbox and global cleanup
- `src/index.ts` calls `bootstrapSES()` (in `src/lib/bootstrap.ts`) at module load. If `repairIntrinsics` throws (e.g. SES 2.2+ rejects a `Temporal` wrapped by a browser extension), the error is caught, `getPluginsRuntimeError()` exposes it, and `initPluginsRuntime` installs no `ɵ*` globals: the frontend app still boots, plugins and MCP are unavailable in that environment. On failure, `Function.prototype.toString` has already been replaced but no intrinsics are hardened and no compartment/sandbox is created — never run plugins on that path.
- The runtime uses SES compartments. Public API return values are passed through `ses.safeReturn` before crossing back to plugin code.
- Plugin `fetch` is sanitized: credentials are omitted and Authorization is blanked. The exposed response only includes ok/status/statusText/url/text/json.
- Timer callbacks are wrapped to mark plugin-originated errors, and timeout/interval IDs are tracked so plugin close can clear them.

View File

@ -43,13 +43,7 @@ Production runs the deployed toolbox, not the devenv one:
- **Start from the ids, not from an inventory.** The ids come from the error or the logs. Unbounded queries over tables with tens of millions of rows (`storage_object`, `file_media_object`) do not finish in a reasonable time; bounded to an id list they are index lookups.
- **One query per reference column.** Two columns in one `IN (col_a, col_b)` lose the index and scan the whole table; split them.
- The read-only replica cuts long queries: see `mem:prod-infra/grafana`.
- A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block). A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_index` and `pg_index.indisvalid`).
- A partitioned parent cannot take `CREATE INDEX CONCURRENTLY` (`cannot create index on partitioned table ... concurrently`). Build one index per partition with `CONCURRENTLY`, then `CREATE INDEX ... ON ONLY` on the parent and `ALTER INDEX ... ATTACH PARTITION` for each child; the parent index only becomes valid when every partition is attached.
- Write the migration as plain, idempotent SQL: `ADD COLUMN IF NOT EXISTS`, `CREATE INDEX IF NOT EXISTS`, and `DROP CONSTRAINT IF EXISTS` followed by `ADD CONSTRAINT`. Do not encode shape checks as `pg_catalog` introspection: they are easy to get wrong (a missing `NOT condeferred` accepts an `INITIALLY DEFERRED` foreign key) and they duplicate what the runbook verifies by hand. `IF NOT EXISTS` matches the name, not the definition, so the out-of-band procedure verifies the index shape and completeness.
- The runbook is the contract for a large instance: it builds and verifies the out-of-band work and then marks the migration step as applied (`INSERT INTO migrations (module, step) ...`), so the body does not run at startup. Verify the column, the pending rows, the index (valid, no predicate or expression, every partition attached) and the constraint shape before inserting the row; the marker is the only thing that skips the migration.
- A reference that a GC task resolves must be `DEFERRABLE INITIALLY IMMEDIATE`, never `INITIALLY DEFERRED`: the immediate check fires inside the transaction that catches the violation and retries the chunk; the deferred one fires at commit, outside the catch. A validated foreign key belongs out of band: `ADD FOREIGN KEY` runs as an `ALTER TABLE` and it scans the whole referencing table; `NOT VALID` on a partitioned table only exists since PostgreSQL 18.
- A backfill that removes a key an old reader uses (for example `metadata.storage-ref-id`, which the previous storage GC scans) must run with the old code stopped. Otherwise the old reader loses the reference and can delete a live object, and the rows it writes in the meantime miss the backfill.
- `ADD COLUMN` takes a brief `ACCESS EXCLUSIVE` even when `IF NOT EXISTS` skips it, and `ADD FOREIGN KEY` takes `ACCESS EXCLUSIVE` on the referencing table and `SHARE ROW EXCLUSIVE` on the referenced one. A migration accepts those locks; a deployment that cannot take them runs the runbook and marks the step as applied.
- A migration that builds an index runs inside the migration transaction, so it cannot use `CONCURRENTLY`, and a plain `CREATE INDEX` locks the table against writes for the whole build. On a large table (file saves write to `file_change` on every save) build the index out of band, before deploying, with `CREATE INDEX CONCURRENTLY` on its own (never inside a transaction block); the shipped migration uses `IF NOT EXISTS`, finds it and does nothing. A concurrent build interrupted midway leaves the index invalid: drop it and retry (check `pg_indexes` and `pg_index.indisvalid`).
## See also

View File

@ -34,10 +34,8 @@ From `render-wasm/`:
- Clean one target: `./clean [frontend|export]`; never run root `cargo clean` on the shared `target/`.
- Watch rebuild: `./watch [frontend|export]`; its initial and change-triggered builds use `./build`.
- Rust tests: `./test` or `cargo test <name>`.
- Rust coverage: `RENDER_WASM_COVERAGE=1 ./test` runs the same tests under `cargo llvm-cov` (needs `cargo-llvm-cov` on PATH; the script adds `llvm-tools-preview`) and writes `target/coverage/` (`lcov.info` with absolute paths, `summary.json` llvm export totals, `html/`). Native build only: code behind wasm-only `cfg` is not measured. `tests-wasm.yml` downloads a pinned, checksum-verified `cargo-llvm-cov` release because the devenv image lacks it.
- Cross-cutting testing principles and anti-patterns: `mem:testing`.
- Lint: `./lint`.
- Format: `./fmt`; check only: `./fmt --check`.
- `./build`, `./test`, `./lint` and `./fmt` all run `setup`, which selects the Rust version pinned in `_build_env` (skip with `PENPOT_WASM_PREPARED=1`).
- Format check: `cargo fmt --check`.
Do not change exported WASM function signatures without updating the corresponding frontend bridge and verifying the frontend renderer path.

View File

@ -6,10 +6,8 @@
- `check-merged <PR...>` (`--file`, `--stdin` accepted) — warn on any non-merged PR, exit 1 when found; run before writing changelog entries.
- `boom <MILESTONE> [--changes CHANGES.md --write]` — generate `### :boom: Breaking changes & Deprecations` from the closed, non-excluded milestone issues labelled `breaking change`, sorted by issue number. Each issue becomes a `#### <issue title>` block: content = body of the issue's last `## Breaking change note` comment (whole markdown, headings demoted to `#####` outside code fences; fallback = heading plus refs with a stderr warning), refs = issue link plus merged closing PRs on their own line, with `(by @author)` for `community contribution`. `--write` replaces only that subsection of the version (inserts it after the version header when missing, keeps the heading when there are no entries; idempotent). Print-only by default. Uses `gh.py comments`.
- `security <MILESTONE> [--changes CHANGES.md --write]` — generate `### :lock: Security fixes` from the advisories whose `patched_versions` include the version (via `gh.py advisories --patched-version`), sorted by GHSA id. Entry: `- Fix <description> (<advisory URL>) (PR: [#N](...))`; description = `Fix` + advisory summary, first word decapitalized unless it starts with an acronym; PRs = merged milestone PRs labelled `pr pt` whose body contains the GHSA id, comma-separated, and no PR is legal. Warns on stderr when a `pr pt` PR references an advisory with no matching `patched_versions`. `--write` replaces only that subsection, placed after `:rocket:` (or `:boom:`, or the version header when neither exists); empty entries remove the subsection; idempotent. Print-only by default.
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings. PR links inside `:lock:` entries count as referenced; advisory URLs never appear here by design.
- `dev-only-bugs <MILESTONE> [--apply]` — list closed milestone bugs that never reached production, without `no changelog` (bugs carrying `release blocker` are analyzed too). Sure: parent issue is a `Feature`/`Enhancement` of the same milestone; `--apply` adds `no changelog` to these only. Candidates (never labelled automatically): parent is an epic of the milestone, `regression` label, or the "Affected versions" / "Environment" body field names only hourly, develop, staging, PRE or the milestone version itself (`only_unreleased_envs`; any production hint or other version keeps the bug in — versions compare as full semver with missing patch = 0, so for milestone 2.19.2 the released 2.19.0 keeps the bug in). Reads `body` and `parent` from one batched `gh.py issue` lookup.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps and the `dev-only-bugs` lists (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
- `cross-ref <MILESTONE> [--changes CHANGES.md]` — fetch milestone PRs with `--state all` once, report merged PRs missing from the version section plus CLOSED (unmerged) warnings; GHSA entries never appear here by design.
- `report <MILESTONE> [--changes CHANGES.md --output CHANGES-ISSUES.md]` — overwrite the anomaly report: A/B milestone mismatches, E `:boom:` entry without the label, F `:rocket:` sync, G `:boom:` sync, plus the C/D `:rocket:` gaps (warnings). `:boom:`/`:rocket:` rule violations are fixed by regenerating or relabelling, not by editing entries in place.
## Conventions
@ -19,4 +17,4 @@
- PR/issue reference parsing covers standard `[#N](.../pull/N)` / `[#N](.../issues/N)` links plus legacy `PR:[N]` / `[Github #N]` forms.
- Shared parsing helpers (`extract_version_section`, `extract_subsection`, `collect_changelog_prs/issues`) live at module top so future subcommands reuse them instead of duplicating regexes.
- Report anomaly types: A/B milestone mismatches, E `:boom:` entry whose issue lacks the `breaking change` label (counted, entries preserved — human labels the issue or moves the entry), G `breaking change` issue missing from `:boom:` (counted, same contract as F); C/D are `:rocket:` gaps (warnings, never counted).
- `issue_excluded` (closed, non-Task, non-Rejected, no `no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift.
- `issue_excluded` (closed, non-Task, non-Rejected, no `release blocker`/`no changelog` label) is module-level and shared by `report` and `boom` so the exclusions cannot drift.

View File

@ -106,7 +106,7 @@ produce deployable artifacts.
## Notes
- `mcp` has no lint task and `media-processor` has no lint task; both show
as skipped. `render-wasm` uses `./lint`, `./test`, and `./fmt`.
as skipped. `render-wasm` uses `./lint`, `./test`, and `cargo fmt`.
- `scripts/test_ci.py` (unittest, `python3 scripts/test_ci.py`) covers the
flag wiring, one build command per module and the help text through
`--dry-run`; it executes no real build.

View File

@ -40,7 +40,7 @@ python3 scripts/gh.py issues "2.16.0" --label "bug"
python3 scripts/gh.py issues "2.16.0" --label "bug,regression"
# Exclude by label
python3 scripts/gh.py issues "2.16.0" --exclude "no changelog"
python3 scripts/gh.py issues "2.16.0" --exclude "release blocker,no changelog"
# Show only issues NOT yet in CHANGES.md
python3 scripts/gh.py issues "2.16.0" --compare CHANGES.md
@ -81,7 +81,7 @@ python3 scripts/gh.py prs --milestone "2.16.0" --state all
Fetch issue details by number in batches of 50 (same input styles as `prs`: numbers, `--file`, `--stdin`). No filters. A dead number fails its whole batch (unlike PRs, issues error instead of resolving to null), so the batch falls back to one-by-one lookups and reports misses as `{"number", "error": "not_found"}`. Any other failure aborts.
**Output**: JSON array to stdout; progress to stderr. Each issue carries the shared issue fields plus `body` and `parent` (`number`, `issue_type`, `milestone`, or null); `changelog.py dev-only-bugs` reads both.
**Output**: JSON array to stdout; progress to stderr.
### `comments`
@ -118,9 +118,6 @@ python3 scripts/gh.py advisories --severity critical
# Filter by state
python3 scripts/gh.py advisories --state triage
# Filter by patched version declared in the advisory
python3 scripts/gh.py advisories --patched-version 2.19.1
# Get full detail for a single advisory
python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
```
@ -129,8 +126,6 @@ python3 scripts/gh.py advisories GHSA-xvj6-fh9w-gjw7
**Detail output** (single advisory) adds: description, vulnerabilities (package, version ranges), credits, timestamps.
`--patched-version` filters the raw advisories before formatting: the match is an exact token of `vulnerabilities[*].patched_versions` (split on `,`/`;`), so `2.19.1` never matches `2.19.10`. Every state is included (`triage`, `draft`, `published`, `closed`, `withdrawn`). The raw `vulnerabilities` stay internal to the filter: the list output keeps the summary schema.
**Output**: JSON to stdout; progress to stderr.
## Key principles

View File

@ -73,8 +73,6 @@ a test that reproduces it:
- **One Assertion Per Concept** — each test verifies one behavior; split
compound assertions
- **Descriptive Test Names** — names read like specifications
- **English Only** — test names, `testing` strings, comments, and every
code-facing text MUST be in English; see the hard rule in `AGENTS.md`.
## Prefer Real Implementations Over Mocks
@ -144,10 +142,10 @@ E2E tests should not be added unless explicitly requested.
When running ANY test command (CLJS/JS or JVM):
1. **NEVER pipe test output directly to `| head`, `| tail`, `| grep`, or similar filters** — this can hide failures and cause you to miss critical errors.
2. **ALWAYS redirect to a file first, then read the file:**
2. **ALWAYS pipe to a file first, then read the file:**
```bash
# CORRECT:
pnpm run test > /tmp/test-output.txt 2>&1
pnpm run test 2>&1 > /tmp/test-output.txt
grep -A 5 "failures" /tmp/test-output.txt
# WRONG:

View File

@ -43,9 +43,10 @@ that breaks them is wrong, even if the code is right.
- For another commit: `./scripts/check-commit -c <ref>`
- **NEVER** hand-wave the body as "one long line". If a line exceeds 80,
break it at a space.
- **`Co-authored-by:` is for human co-authors only** (`Co-authored-by: Name <email>`). AI providers, agents and bots must not be credited that way; use `AI-assisted-by:` instead. `scripts/check-commit` rejects a trailer value that names an AI provider or agent (`Claude`, `Copilot`, `Cursor`, `Codex`, `Gemini`, `Aider`, `Cline`, `Windsurf`, `DeepSeek`, `OpenCode`, `devin-ai`, `amazon q`, and similar) or carries a bot identity (`[bot]`, `bot@`). `NON_HUMAN_COAUTHOR_PATTERN` is the list; extend it when a new tool starts adding trailers. A person whose own name or email matches one of those tokens (a surname like `Cline`, an address at a provider) is rejected too; amend the trailer or credit them with `Reviewed-by:`.
- **NEVER add a `Co-authored-by:` trailer.** It is forbidden; `scripts/check-commit` fails the commit when any line matches it (case-insensitive).
- CI runs `scripts/check-commit` on every new commit (`mem:workflow/creating-commits` is enforced by `.github/workflows/commit-checker.yml`), so a forbidden trailer fails the build.
- Exceptions inside the body (do not wrap these): `Signed-off-by:`, `Co-authored-by:`, `AI-assisted-by:` trailers, and lines carrying a URL.
- Exceptions inside the body (do not wrap these): `Signed-off-by:`,
`AI-assisted-by:` trailers, and lines carrying a URL.
**AI-assisted-by trailer rules:**
- Use only the model name, e.g. `mimo-v2.5`, `deepseek-v4-flash`
@ -55,14 +56,6 @@ that breaks them is wrong, even if the code is right.
`:bug:` bug fix · `:sparkles:` enhancement · `:tada:` new feature · `:recycle:` refactor · `:lipstick:` cosmetic · `:ambulance:` critical fix · `:books:` docs · `:construction:` WIP · `:boom:` breaking · `:wrench:` config · `:zap:` perf · `:whale:` docker · `:paperclip:` other · `:arrow_up:` dep upgrade · `:arrow_down:` dep downgrade · `:fire:` removal · `:globe_with_meridians:` translations · `:rocket:` epic/highlight
## Atomic Commits
A PR squash-merges into one commit by default, so the split inside the branch
does not reach `main` and nothing depends on it. When the PR asks for a
regular merge (`mem:workflow/creating-prs` → Merge Strategy), every commit
must stand alone: it builds, passes the linter, and needs no later commit on
the same branch to make sense.
## Referencing Issues
Use `Closes #NNNN` (not `Fixes #NNNN`) in commit and PR body text for readable context. The explicit issue-to-PR link is assigned with `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` as described in `mem:workflow/creating-prs`; do not rely on the keyword to create that link.

View File

@ -60,7 +60,6 @@ the issue (`scripts/changelog.py boom`), so the issue must carry both:
Post the note as a new comment. Do not add a breaking-change section to the
issue body and do not hand-write the changelog block. When the issue comes
from a PR, reuse the text of the PR's breaking-change section.
Do not hard-wrap the note: one line per paragraph or bullet.
The full changelog rules (sections, exclusions, `:rocket:`, pre-flight and
anomaly report) live in `mem:workflow/updating-changelog`.

View File

@ -32,19 +32,6 @@ PR titles follow commit title conventions:
See `mem:workflow/creating-commits` for emoji codes. Squash merge uses the PR title as the final commit subject, so title format matters.
## Merge Strategy
Policy: `CONTRIBUTING.md` → "Merging", under "Pull Requests".
- Squash merge is the default, so the PR's own commits never reach `main`
and the PR title becomes the landed subject.
- A regular merge is possible on request only. The request is the **first
line** of the PR body, above the `**Note:**` line:
`> **Merge:** do not squash; each commit is self-contained.`
- Ask for it only when every commit on the branch is self-contained (builds
and passes lint on its own). Default to squash unless the user asks for
the opposite.
## Description Body
Include concise sections covering:
@ -73,8 +60,6 @@ PR descriptions follow this structure:
```
The "Note:" line is required at the top. Adjust if this is a manual (non-AI) PR.
When the PR requests a regular merge instead of a squash merge, its
`**Merge:**` line goes above the `**Note:**` line (see Merge Strategy).
## Writing Principles

View File

@ -2,6 +2,5 @@
Node and pnpm pins: `mem:workflow/updating-node` and `mem:workflow/updating-pnpm`. Only the parts the Dockerfiles cannot show:
- JDK: two pins that move together — `docker/devenv/Dockerfile` (`setup-jvm` stage, full JDK) and `docker/images/Dockerfile.backend` (JDK + `jlink` runtime). Same build, same sums in both. They compile and run the same backend: `manage.sh` builds the bundle inside the devenv image (`build-backend-bundle` → `build "backend"`) and `backend/scripts/run.template.sh` starts the runtime with `--enable-preview`, which requires the runtime major to match the compiler major. A runtime line behind the compiler line is a silent trap.
- JDK line: tracks a non-LTS line, so there is no minor to pick — the last build of the line is the ceiling; moving line is a major and needs a real build to validate the `jlink` module list (`--add-modules` in `Dockerfile.backend`). JDK 27 keeps all of them; check with `bin/jlink --add-modules <list>` against the downloaded JDK before bumping. Source of the URL and both sums: the Azul metadata API (`api.azul.com/metadata/v1/zulu/packages/?java_version=<n>&os=linux&arch=<x64|aarch64>&java_package_type=jdk&archive_type=tar.gz&latest=true&release_status=ga&availability_types=CA`), then confirm the sums by downloading the tarball and running `sha256sum`.
- JDK: the image tracks a non-LTS line, so there is no minor to pick — the last build of the line is the ceiling; moving line is a major and needs a real build to validate the `jlink` module list.
- ImageMagick: publish `penpotapp/imagemagick:<ver>` **before** bumping the `COPY --from` refs in the backend, exporter and devenv Dockerfiles; otherwise both builds break on the missing tag.

View File

@ -2,7 +2,7 @@
> Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human release managers who need full context. That is why it uses plain language and examples. It is the canonical home of the changelog flow; the `update-changelog` skill is only an entry point that routes here.
`CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory whose `patched_versions` include the version. It is never written from scratch by hand; the only manual parts are the wording of a description when the title is not enough and the advisory summaries (polished on GitHub, the source of the generated security entries).
`CHANGES.md` (repository root) is the release notes file. Every entry comes from GitHub: a milestone issue, its merged PR, or a security advisory. It is never written from scratch by hand; the only manual parts are the security advisory entries and the wording of a description when the title is not enough.
The commands live in `scripts/gh.py` (GitHub queries) and `scripts/changelog.py` (changelog checks). Their flags and internals are documented in `mem:scripts/gh` and `mem:scripts/changelog`.
@ -19,17 +19,14 @@ Editing `CHANGES.md` happens only on explicit request or as part of the release
| Question | Source |
|---|---|
| Which issues belong to the version? | GitHub milestone |
| Which security fixes? | Advisories whose `patched_versions` include the version |
| Which section? | Issue Type |
| Is it a breaking change? | `breaking change` label |
| Is it a highlight? | `release highlight` label |
- **Issue Type decides the section**: `Feature` → `:tada:`, `Enhancement` → `:sparkles:`, `Bug` → `:bug:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
- **Security fixes come from advisories**: an advisory whose `patched_versions` include the version becomes a `:lock:` entry (section 7). `triage`, `draft` and `published` are all valid states; the advisory may still be unpublished when the changelog is written. A security fix may also carry a regular bug issue; both entries are legal and independent (the issue lives in `:bug:` as any other bug).
- **Issue Type decides the section**: `Bug` → `:bug:`, `Feature`/`Enhancement` → `:sparkles:`, `Task` is excluded (internal chores). Labels `bug`/`enhancement` and title emoji prefixes are often wrong or missing; do not use them.
- **Two labels override the Issue Type**: `breaking change` → an entry only under `:boom:`, never in its regular section; `release highlight` → an extra entry under `:rocket:` on top of its regular entry.
- **Only closed issues**. An open issue in the milestone is not generated, even if someone added a provisional entry by hand before the merge.
- **Excluded issues**: `no changelog`, `Task` issue type, `Rejected` project status. `release blocker` is a workflow label (the issue blocks the release), not a changelog exclusion: bugs carrying it are analyzed by `dev-only-bugs` (section 8, step 3), and closed non-bug issues carrying it are reviewed by hand (section 8, step 3).
- **Bugs that never reached production** stay out: a bug found and fixed while building a feature of the same, unreleased version never affected users. The team rarely labels them, so the flow finds them with `dev-only-bugs` and labels them `no changelog` (section 8, step 3).
- **Excluded issues**: `no changelog`, `release blocker`, `Task` issue type, `Rejected` project status.
- **Excluded PRs**: a PR with `release blocker` or `no issue required` does not bring an entry of its own.
- **An issue with no milestone** belongs to another (probably private) project: it is neither changelog candidate nor anomaly.
@ -40,7 +37,7 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
```markdown
# CHANGELOG
## 2.20.0 (Unreleased)
## 2.19.0 (Unreleased)
### :boom: Breaking changes & Deprecations
@ -54,31 +51,21 @@ The newest version goes at the top, right after the `# CHANGELOG` header. Inside
- Add configurable keyboard shortcuts [#9924](https://github.com/penpot/penpot/issues/9924) (PR: [#10237](https://github.com/penpot/penpot/pull/10237))
### :lock: Security fixes
- Fix <user-facing description of the vulnerability> (<advisory URL>) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :tada: Features
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :sparkles: Enhancements
- Improve <user-facing description of the existing behaviour> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :bug: Bugs
### :bug: Bugs fixed
- Fix <user-facing description of the bug> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
### :sparkles: New features & Enhancements
- Add <user-facing description of the feature> [#ISSUE](https://github.com/penpot/penpot/issues/ISSUE) (PR: [#PR](https://github.com/penpot/penpot/pull/PR))
```
- `:boom:` first, then `:rocket:`, `:lock:`, `:tada:`, `:sparkles:`, `:bug:`. Features come before enhancements so they stand out to readers deciding whether to update.
- **This layout applies from 2.20.0.** Versions up to 2.19.x use the old one: `:bug: Bugs fixed` before `:sparkles: New features & Enhancements`, which mixes features and enhancements. Keep the old layout in those sections, including new 2.19.x patches, so a minor line reads the same. The in-app update dialog only reads `:rocket:`, so the change does not affect it.
- Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries. `:lock:` is omitted (and removed on `--write`) when the version has no security advisories.
- `:boom:` first, then `:rocket:`, `:bug:`, `:sparkles:`. Other headings exist in old versions (`:heart:` community contributions); do not add them to new versions.
- Include a section only when it has entries. Historical versions may have an empty `:boom:` or `:rocket:` heading; `boom --write` keeps the `:boom:` heading even when there are no entries.
- One blank line between the last entry of a section and the next heading.
- No manual line wrapping: one line per bullet or paragraph, however long.
## 4. Regular entries (`:tada:`, `:sparkles:`, `:bug:`)
## 4. Regular entries (`:bug:`, `:sparkles:`)
The entry shape is:
@ -129,12 +116,11 @@ Rules for the note:
- **Write it for admins**: what changes, what they must do, and what breaks if they do nothing. Do not paste the full runbook; explain the implications and link the runbook or docs.
- **Link a long-lived branch**, `staging` or `main`, never a feature branch (`issue-1234`). The link may 404 until the PR merges; that is expected.
- **Do not add issue or PR links**: the generator appends them.
- **Do not hard-wrap the text**: one line per paragraph or bullet. The generator copies lines verbatim, so wrapped lines end up wrapped in `CHANGES.md`.
- **One note per issue**. If several comments carry the marker, the generator uses the last one and warns.
- **No note**: the generator warns on stderr and emits only the heading and references; fix the issue, not `CHANGES.md`.
- **`####`, not `###`**: `###` is the level of the changelog sections. An entry at `###` would render as a sibling section and the scripts that read and rewrite `:boom:` would cut the section there. The note author never writes the `####`: the generator adds it.
- **Only merged closing PRs** are referenced; unmerged or superseded PRs are skipped.
- **Never hand-edit the section**. Generate it with `boom` (section 8). The note lives on GitHub; edit it there.
- **Never hand-edit the section**. Generate it with `boom` (section 7). The note lives on GitHub; edit it there.
## 6. `:rocket:`: epics and highlights
@ -143,52 +129,35 @@ Rules for the note:
- If no issue is labelled, leave the section out; do not pick highlights yourself.
- Never edit `:rocket:` by hand: add or remove the label and regenerate.
## 7. `:lock:`: security fixes
Generated from the advisories that declare the version in `patched_versions`; never written by hand.
- **Source and filter**: `python3 scripts/gh.py advisories --patched-version <VERSION>` keeps the advisories whose `vulnerabilities[*].patched_versions` contain the version as an exact token (so `2.19.1` never matches `2.19.10`). Any state is valid — `triage`, `draft`, `published` — because the advisory may still be unpublished when the changelog is written.
- **Fill the advisories first**: `patched_versions` must be filled before generating the changelog. The release process publishes the advisories with (or after) the release; the changelog does not wait for publication.
- **PR reference**: a merged PR in the milestone labelled `pr pt` whose body contains the GHSA id contributes its number inline; several PRs go comma-separated. An advisory with no PR is legal: the entry links only the advisory.
- **Missing `patched_versions`**: when a `pr pt` PR references an advisory that declares no matching `patched_versions`, `security` prints a warning on stderr and leaves it out. Fix the advisory, not `CHANGES.md`.
- **Entry format**: `- Fix <description> (<advisory URL>) (PR: [#PR](...))`. The description is `Fix` plus the advisory summary, with the first word decapitalized unless it starts with an acronym (`SVG`, `CVE-...`).
- **Wording**: the flow may polish the description. Re-running `security --write` rewrites it from the summary, so the durable place for wording is the advisory summary on GitHub; keep it a problem statement, because the generator adds `Fix`.
- **Empty version**: no matching advisories → the subsection is not inserted (and is removed by `--write`).
- **Never hand-edit the section**: generate it with `security` (section 8).
## 8. Flow
## 7. Flow
1. **Confirm the version** (a semver string such as `2.19.0`) with the user when it is not obvious.
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
3. **Keep out bugs that never reached production**: `python3 scripts/changelog.py dev-only-bugs "<VERSION>"` lists them; add `--apply` to label the sure ones `no changelog` on GitHub, then re-fetch step 2. The label puts the decision on the issue, so it survives a regeneration. Bugs carrying `release blocker` are analyzed like any other bug.
- **Sure**: a bug whose parent issue is a `Feature` or `Enhancement` of the same milestone. `--apply` labels these.
- **Candidates**: a bug whose parent is an epic of the same milestone, that carries the `regression` label, or whose "Affected versions" / "Environment" field names only hourly, develop, staging, PRE or the milestone version itself. `regression` also marks regressions users hit in production, and the field says where the bug was seen, not since when it exists, so never label these yourself: show them to the user, who labels the ones they confirm. The anomaly report lists them too (section 10).
- **Closed non-bug issues carrying `release blocker`** have no automatic check: `python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog"` lists them (closed, non-Task). Show them to the user: the internal ones get the `no changelog` label, the rest keep their entry.
4. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
5. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
6. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
7. **Reconcile the existing text** (section 9).
8. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
9. **Generate `:rocket:`** from the `release highlight` label (section 6).
10. **Generate `:lock:`**: `python3 scripts/changelog.py security "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The contract lives in section 7.
11. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. PRs linked from `:lock:` count as referenced; an advisory with no PR is invisible here by design.
12. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 10).
2. **Fetch the milestone issues**: `python3 scripts/gh.py issues "<VERSION>" --exclude "release blocker,no changelog"` (closed by default; add `--state all` only to inspect open issues). `gh.py` already drops `Task` and `Rejected`; pass `--include-tasks` / `--include-rejected` only to inspect them. To find what is missing from an existing section, add `--compare CHANGES.md`.
3. **Fetch PR details when needed**: `python3 scripts/gh.py prs <numbers...>` (also `--milestone`). Use it for the PR author, labels and merge state.
4. **Categorize** by Issue Type and the two labels (section 2), and **build the descriptions** (section 4).
5. **Verify PR merge status** before writing: `python3 scripts/changelog.py check-merged <PR numbers...>`. Only merged PRs go in the changelog; if a closing PR was closed unmerged, look for the PR that superseded it.
6. **Reconcile the existing text** (section 8).
7. **Generate `:boom:`**: `python3 scripts/changelog.py boom "<VERSION>"` prints the section; add `--write` to replace it in `CHANGES.md`. The write replaces only that subsection, inserts it after the version header when missing, keeps the heading when there are no entries, and is idempotent.
8. **Generate `:rocket:`** from the `release highlight` label (section 6).
9. **Cross-check PRs**: `python3 scripts/changelog.py cross-ref "<VERSION>"`. It lists merged milestone PRs missing from the section and warns about closed (unmerged) PRs. GHSA entries never appear; that absence is expected.
10. **Generate the anomaly report**: `python3 scripts/changelog.py report "<VERSION>"`. It overwrites `CHANGES-ISSUES.md` with the current state (section 9).
11. **Security advisories (GHSA)**: add them by hand under `:bug:` as `- Fix <user-facing description> (<advisory URL>)`, with no issue or PR link. Do not fetch or verify the URL: it may be draft and return 404. Write the description in imperative mood from the advisory title. These entries are invisible to the automation; apply only the duplicate/backport check to them.
## 9. Pre-flight: reconcile before writing
## 8. Pre-flight: reconcile before writing
Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md` before adding the new section:
1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports. The same holds after merging a patch branch into `staging`: the fix keeps its released patch entry and leaves the newer, unreleased section (the merge conflict in `CHANGES.md` is resolved by keeping both sections, newest first, minus the duplicate).
1. **Duplicate across versions** → remove it from the current section; the earlier version wins for backports.
2. **Stale milestone assignment** (the issue moved out of this milestone) → remove the entry.
3. **Newly applied exclusion labels** (`no changelog`) → remove the entry. (`release blocker` is not an exclusion label; it never removes an entry by itself.)
3. **Newly applied exclusion labels** (`no changelog`, `release blocker`) → remove the entry.
4. **Issue no longer closed, deleted, or now `Rejected`** → remove the entry.
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 10); do not remove it silently.
6. **Issue Type changed** → to `Task`: remove the entry; to another type: move the entry to that type's section.
5. **Unmerged or moved PR reference** → fix the reference or remove the entry. A PR merged in a different milestone is an anomaly (section 9); do not remove it silently.
6. **Issue Type changed to `Task`** → remove the entry.
7. **Breaking change out of sync** (a `breaking change` issue outside `:boom:`, or a `:boom:` entry without the label) → regenerate the section; a labelled issue has no regular entry.
8. **Highlight out of sync** (`:rocket:` entry without the label, or a labelled issue missing) → remove it or add it.
9. **Missing valid issues** (closed, non-excluded, referenced nowhere) → add them to their section.
## 10. Anomaly report
## 9. Anomaly report
`report` classifies problems it cannot decide alone. The total counts A, B, E, F and G; C and D are warnings.
@ -200,22 +169,19 @@ Re-fetch the milestone (do not trust cached data) and fix these in `CHANGES.md`
- **C — released X.Y.0 version without a `:rocket:` section** (gap; patches never carry highlights).
- **D — `:rocket:` entry without issue AND PR references** (gap).
**Bugs that may never have reached production** are warnings too (section 8, step 3): sub-issues of a milestone feature still unlabelled (run `dev-only-bugs --apply`) and candidates for the user to confirm. They never count toward the total.
Anything else is a rule violation, not a report item: fix it in the pre-flight and re-run.
## 11. Pitfalls
## 10. Pitfalls
- **Open issues with provisional entries**: a PR author may add the entry before the merge. `boom` will not generate it until the issue closes; keep the hand-written block identical to what the generator would produce.
- **Labels, not emoji**: title prefixes are cosmetic and often missing; the section comes from Issue Type and the two labels.
- **Taiga references**: old entries link to Taiga. Resolve them to the GitHub issue via the description text or the PRs that mention the Taiga URL, then link issue and PR.
- **Long-lived links only**: runbooks and docs referenced from a note must point to `staging` or `main`, never to a feature branch that will be deleted.
- **Generated sections**: `:boom:`, `:rocket:` and `:lock:` are generated. Never patch them by hand; edit the label, the note comment or the advisory and re-run.
- **Deleted or transferred issues**: an issue referenced in the changelog that returns 410/404 (deleted or moved out of the repo) must lose its entry in the pre-flight. Its security fixes, if any, stay in `:lock:` through the advisory.
- **Generated sections**: `:boom:` and `:rocket:` are the only label-generated parts. Never patch them by hand; edit the label or the note comment and re-run.
## 12. Where the code is
## 11. Where the code is
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories` (with `--patched-version`). Details: `mem:scripts/gh`.
- `scripts/changelog.py`: `check-merged`, `boom`, `security`, `cross-ref`, `dev-only-bugs`, `report`. Details: `mem:scripts/changelog`.
- `scripts/gh.py`: `issues`, `issue`, `comments`, `prs`, `link-issue`, `advisories`. Details: `mem:scripts/gh`.
- `scripts/changelog.py`: `check-merged`, `boom`, `cross-ref`, `report`. Details: `mem:scripts/changelog`.
- Tests: `scripts/test_gh.py`, `scripts/test_changelog.py`.
- The note contract for issue authors: `mem:workflow/creating-issues`, section **Breaking changes**.

View File

@ -19,7 +19,6 @@ No single source of truth: nothing reads `.nvmrc` at build time, and the version
- Renovate also opens these bumps, on `renovate-*` branches and with its config living outside this repo; a manual bump and Renovate's must not diverge on any of the six sites.
- A backport merge can deliver a newer Node into `docker/` and the plugins-deploy workflows while main's `.nvmrc` stays behind, because main receives partial backports of `.github/` and `docker/` from develop. Merging main into staging therefore conflicts on the pin lines: take main's set and finish the bump (`.nvmrc` plus the `packageManager` fields via `mem:workflow/updating-pnpm`), or the six sites drift.
- The newer version wins in both directions. A maintenance→main merge (`main-2.18.x` into `main`) can carry the newer Node while `.nvmrc` and the `packageManager` fields stay on the older pair; then adopt the newer set and finish the bump on all six sites plus the 35 `packageManager` fields.
- Node stays on the 24.x line even when the rest of `.github/` moves on: `99db795966` (`update GitHub Actions`) took the two `plugins-deploy-*` `runtime:` pins to `node@26.13.1` on develop while `.nvmrc` stayed on `v24.22.3`. The project keeps every site on the `.nvmrc` value; resolve that conflict back to the 24.x pin instead of taking develop's.
- Both workflows must stay valid YAML. The `python3` in the devenv has no `yaml` module, so validate with `uv run --with pyyaml python3 -c ...` instead of a bare import.
## Verification

View File

@ -28,12 +28,6 @@
- Before `gh pr create` / `gh pr edit` → `mem:workflow/creating-prs` (title format, body structure, AI note)
Don't infer format from the title of a previous commit/issue/PR — the memory
is the source of truth.
- **Everything in code MUST be in English.** Identifiers (functions,
variables, types, namespaces), code comments, docstrings, test
descriptions, log hints — always English, regardless of the language the
author works in. Tests are not an exception. The only place another
language may appear in the codebase is product copy rendered through the
i18n system.
## CRITICAL: Read module memories BEFORE writing any code
@ -45,8 +39,6 @@ Do this **before planning, before coding, before touching any file**:
3. Read each affected module's **core memory** — the name is `<module>/core`
(e.g. `frontend/core`, `backend/core`, `common/core`).
4. If the core memory references deeper `mem:` memories relevant to your task, read those too.
5. Before writing or modifying Clojure or ClojureScript, read `mem:clojure/core`
and the relevant focused Clojure memories it references.
**STOP: Do not proceed until you have read the core memory of every affected module.**
Skipping this step is the #1 cause of incorrect or incomplete work.
@ -88,6 +80,11 @@ Review every prose output against these rules before delivering.
Memories are the **primary project guidance** — not docs or readme files.
They are dense, agent-oriented notes: terse bullets, invariants, no prose.
## Entry point
Start at `critical-info` (the graph root). It describes the project structure,
module dependency graph, and references section-level core memories.
## Progressive discovery model
Memories form a **reference graph**, not a flat list:
@ -99,8 +96,11 @@ critical-info ← read first (graph root)
└─ ... ← deeper memories as needed
```
Follow the mandatory reading sequence above. Cross-cutting topics can also be
referenced directly from `critical-info` or other memories.
When working on a task:
1. Read `critical-info` to identify which sections are affected.
2. Read the affected section's `core` memory for an overview.
3. Follow `mem:` references in the core memory to focused memories relevant to your task.
4. Continue following references deeper as needed.
## Accessing memories
@ -120,8 +120,9 @@ When you encounter a `mem:` reference relevant to your task, read that memory ne
## Topic/folder organization
Memories are grouped into folders for modules or topics. Project modules have
a `<module>/core` memory; other topic folders may contain focused memories only.
Memories are grouped into folders that mirror project modules or topics:
`backend/`, `common/`, `frontend/`, `render-wasm/`, `exporter/`, `workflow/`, etc.
Each folder's top-level memory is `<folder>/core`.
---
@ -157,3 +158,4 @@ precision while maintaining a strong focus on maintainability and performance.
- `scripts/check-fmt-clj` — Check Clojure formatting without modifying files.
- `scripts/ci` — CI orchestration script for running lint, tests, format checks and, on request (`--build`), production builds across modules. See `mem:scripts/ci`.
- `scripts/gh.py` — Multi-purpose GitHub CLI helper. Subcommands: `issues` (list issues in a milestone), `prs` (fetch PR details), `advisories` (list/inspect security advisories). See `python3 scripts/gh.py --help`.

View File

@ -1,167 +1,5 @@
# CHANGELOG
## 2.21.0
### :boom: Breaking changes & Deprecations
### :lock: Security fixes
- Fix real-time collaboration WebSocket never re-validates file access after `:subscribe-file`, letting revoked users keep receiving live edit content (https://github.com/penpot/penpot/security/advisories/GHSA-m53j-2766-6jqw) (PR: [#12189](https://github.com/penpot/penpot/pull/12189))
### :tada: Features
- Auto-link libraries during import based on slugified name [#9263](https://github.com/penpot/penpot/issues/9263) (PR: [#9958](https://github.com/penpot/penpot/pull/9958))
- Implement basic bullet lists [#12630](https://github.com/penpot/penpot/issues/12630) (PR: [#12720](https://github.com/penpot/penpot/pull/12720))
### :sparkles: Enhancements
- Improve performance of payload decoding in the workspace (by @lukel97) [#12916](https://github.com/penpot/penpot/issues/12916) (PR: [#12704](https://github.com/penpot/penpot/pull/12704))
- Keep the search when changing library in the Swap component menu (by @desty) [#12335](https://github.com/penpot/penpot/issues/12335) (PR: [#12873](https://github.com/penpot/penpot/pull/12873))
### :bug: Bugs
- Fix workspace stuttering when switching pages in a large file (by @lukel97) [#12917](https://github.com/penpot/penpot/issues/12917) (PR: [#12811](https://github.com/penpot/penpot/pull/12811))
- Fix workspace freezes when undoing or redoing a move of a board with many components (by @lukel97) [#12913](https://github.com/penpot/penpot/issues/12913) (PR: [#12870](https://github.com/penpot/penpot/pull/12870))
- Fix dashboard freezes for about a second after duplicating a large file (by @lukel97) [#12914](https://github.com/penpot/penpot/issues/12914) (PR: [#12911](https://github.com/penpot/penpot/pull/12911))
- Fix not being able to collapse a token set group after reordering it [#12625](https://github.com/penpot/penpot/issues/12625) (PR: [#12671](https://github.com/penpot/penpot/pull/12671))
- Fix slow variant switch with the WebGL renderer [#12968](https://github.com/penpot/penpot/issues/12968) (PR: [#12985](https://github.com/penpot/penpot/pull/12985))
- Fix login page GitHub logo showing with low contrast in the light theme [#12938](https://github.com/penpot/penpot/issues/12938) (PR: [#12943](https://github.com/penpot/penpot/pull/12943))
- Fix leak and black screen on wasm-critical errors [#12977](https://github.com/penpot/penpot/issues/12977) (PR: [#12978](https://github.com/penpot/penpot/pull/12978))
## 2.20.0
### :boom: Breaking changes & Deprecations
#### Add table partitioning for file-change
The file history table (`file_change`) is now partitioned by file, with the same 16 partitions as the file payload table (`file_data`). Queries that filter by file, and the join between the two tables, resolve against a single partition, so they stay fast as the table grows. The rewrite also removes the dead columns and stores the table in its final layout; the space the old table held comes back when it is dropped. A save no longer writes the file features on every revision, and the team snapshot quota is a single lookup.
The four snapshot RPC methods (`update-file-snapshot`, `delete-file-snapshot`, `lock-file-snapshot` and `unlock-file-snapshot`) now require `file-id`. The frontend already sends it; API clients that call them must add it.
On small and medium instances the table is rewritten automatically on boot; there is nothing to do. On large instances (hundreds of GB in `file_change`) it does not fit in the startup transaction: run it by hand **before upgrading to 2.20.0** following the [large-instance runbook](https://github.com/penpot/penpot/blob/staging/operations/2.20.0-file-change-partition.md). The copy runs while the instance is online, followed by a cutover of a few minutes with the backend and the workers stopped. The old table is kept for rollback; check the disk headroom before starting, and drop the old table days later to get the space back.
[#12416](https://github.com/penpot/penpot/issues/12416) (PR: [#12417](https://github.com/penpot/penpot/pull/12417))
#### Protect the file data storage objects with a foreign key in the backend
This release moves the file data storage pointer out of `file_data.metadata->>'storage-ref-id'` and into a new `file_data.storage_object_id` column with a foreign key to `storage_object(id)`. The metadata key is gone: nothing writes or reads it.
What it means for self-hosted instances:
- The release is a coordinated restart. The new code reads the column; the migration drops `storage-ref-id` from the rows it backfills. Rolling back to the previous binary is not supported after this release runs.
- A fresh database or a small instance needs nothing special: the startup migration does the whole job.
- A large instance should not let the startup migration run. Build the column, the index and the foreign key online beforehand, stop the instances, run the backfill only if a row still carries `metadata.storage-ref-id`, verify the end state, and mark the migration as applied. The full procedure is documented in [`operations/2.20.0-file-data-storage-object-id.md`](https://github.com/penpot/penpot/blob/main/operations/2.20.0-file-data-storage-object-id.md).
- If the storage backend was never enabled, there is nothing to backfill and the stopped window is just the restart.
If you upgrade blindly: on fresh databases and small instances the startup migration completes the change. On a large instance, the foreign key validation scan can block writes for a long time; follow the runbook instead.
[#12768](https://github.com/penpot/penpot/issues/12768) (PR: [#12769](https://github.com/penpot/penpot/pull/12769))
### :rocket: Epics and highlights
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
### :tada: Features
- Add a feedback channel to the MCP server [#12490](https://github.com/penpot/penpot/issues/12490) (PR: [#12551](https://github.com/penpot/penpot/pull/12551))
- Allow MCP to work with multiple open Penpot files [#11759](https://github.com/penpot/penpot/issues/11759)
- Implement full text format paste within Penpot [#12047](https://github.com/penpot/penpot/issues/12047) (PR: [#12137](https://github.com/penpot/penpot/pull/12137))
- Add team, project and file management to the Plugin API for global plugins [#12170](https://github.com/penpot/penpot/issues/12170) (PR: [#12169](https://github.com/penpot/penpot/pull/12169))
- Serve Penpot under a subpath [#3194](https://github.com/penpot/penpot/issues/3194) (PR: [#12358](https://github.com/penpot/penpot/pull/12358))
- Allow MCP to access and manage Penpot Dashboard resources [#11761](https://github.com/penpot/penpot/issues/11761)
- Add OAuth support for MCP client connections [#12034](https://github.com/penpot/penpot/issues/12034) (PR: [#12480](https://github.com/penpot/penpot/pull/12480))
### :sparkles: Enhancements
- Auto-scroll sortable lists when dragging an item near their top or bottom edge (by @ShreyashAgare26) [#10478](https://github.com/penpot/penpot/issues/10478) (PR: [#11595](https://github.com/penpot/penpot/pull/11595))
- Make WebGL the default renderer for new accounts [#11855](https://github.com/penpot/penpot/issues/11855) (PR: [#12880](https://github.com/penpot/penpot/pull/12880))
- Prevent the hosted MCP server from running execute_code twice when the plugin is open in two tabs [#12008](https://github.com/penpot/penpot/issues/12008) (PR: [#11927](https://github.com/penpot/penpot/pull/11927))
- Allow detaching typography tokens [#11887](https://github.com/penpot/penpot/issues/11887) (PR: [#11969](https://github.com/penpot/penpot/pull/11969))
- Complete the Russian translations [#11955](https://github.com/penpot/penpot/issues/11955) (PR: [#11956](https://github.com/penpot/penpot/pull/11956))
- Run .penpot file import and export as durable background jobs [#12194](https://github.com/penpot/penpot/issues/12194) (PR: [#12056](https://github.com/penpot/penpot/pull/12056), [#12192](https://github.com/penpot/penpot/pull/12192))
- Open files and projects in a new tab with Ctrl/Cmd+click in the dashboard [#12228](https://github.com/penpot/penpot/issues/12228) (PR: [#12230](https://github.com/penpot/penpot/pull/12230))
- Add an RPC method to read file-level plugin data [#12258](https://github.com/penpot/penpot/issues/12258) (PR: [#10171](https://github.com/penpot/penpot/pull/10171))
- Add a get-file-tokens RPC method to read a file's design tokens [#12260](https://github.com/penpot/penpot/issues/12260) (PR: [#12172](https://github.com/penpot/penpot/pull/12172))
- Show the progress and the size of each file in the import and export dialogs [#12340](https://github.com/penpot/penpot/issues/12340) (PR: [#12192](https://github.com/penpot/penpot/pull/12192))
- Focus the search field when opening the color tokens tab of the color picker [#12344](https://github.com/penpot/penpot/issues/12344) (PR: [#12345](https://github.com/penpot/penpot/pull/12345))
- Harden the file data offload job with a row lock, per-file transactions, throttle and metrics [#12398](https://github.com/penpot/penpot/issues/12398) (PR: [#12403](https://github.com/penpot/penpot/pull/12403))
- Render email templates inside the mail job in the backend [#12445](https://github.com/penpot/penpot/issues/12445) (PR: [#12448](https://github.com/penpot/penpot/pull/12448))
- Run local WASM exports for shapes and boards without freezing the UI [#12512](https://github.com/penpot/penpot/issues/12512) (PR: [#12515](https://github.com/penpot/penpot/pull/12515))
- Use WASM to generate SVG markup in the code inspector when render-wasm is active [#12567](https://github.com/penpot/penpot/issues/12567) (PR: [#12608](https://github.com/penpot/penpot/pull/12608))
- Close the workspace search with the Escape key (by @filipsajdak) [#9540](https://github.com/penpot/penpot/issues/9540) (PR: [#12374](https://github.com/penpot/penpot/pull/12374))
- Restrict management RPC methods to their declared shared keys [#12730](https://github.com/penpot/penpot/issues/12730) (PR: [#12736](https://github.com/penpot/penpot/pull/12736))
- Reduce memory used by duplicate shape ids in the workspace [#12791](https://github.com/penpot/penpot/issues/12791) (PR: [#12792](https://github.com/penpot/penpot/pull/12792))
- Reduce memory consumption on shape decoding [#12789](https://github.com/penpot/penpot/issues/12789) (PR: [#12790](https://github.com/penpot/penpot/pull/12790))
- Notify other users when a snapshot is restored [#12425](https://github.com/penpot/penpot/issues/12425) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Normalize the configured tenant to lowercase [#12842](https://github.com/penpot/penpot/issues/12842) (PR: [#12843](https://github.com/penpot/penpot/pull/12843))
- Speed up saves on large files by checking only the pages a save changes [#12602](https://github.com/penpot/penpot/issues/12602) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Retry saves after short server failures and keep the workspace open when a save fails [#12601](https://github.com/penpot/penpot/issues/12601) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Switch the workspace to read-only mode when the file can't be saved [#12507](https://github.com/penpot/penpot/issues/12507) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Validate upload content against real file bytes in media uploads [#12796](https://github.com/penpot/penpot/issues/12796) (PR: [#12801](https://github.com/penpot/penpot/pull/12801))
- Add the file and MCP session to audit events sent from plugins and the MCP [#12902](https://github.com/penpot/penpot/issues/12902) (PR: [#12972](https://github.com/penpot/penpot/pull/12972))
- Add skip-if-active option to job submission in the backend jobs substrate [#12265](https://github.com/penpot/penpot/issues/12265) (PR: [#12266](https://github.com/penpot/penpot/pull/12266))
- Add HTML paste from clipboard in text editor v3 [#10465](https://github.com/penpot/penpot/issues/10465)
- Skip loading the thumbnail rasterizer in WebGL renderer sessions to cut workspace memory [#13029](https://github.com/penpot/penpot/issues/13029) (PR: [#12835](https://github.com/penpot/penpot/pull/12835))
- Speed up uploading page shapes to WASM [#13030](https://github.com/penpot/penpot/issues/13030) (PR: [#12854](https://github.com/penpot/penpot/pull/12854))
### :bug: Bugs
- Fix font upload failing with a regex error [#10436](https://github.com/penpot/penpot/issues/10436) (PR: [#11951](https://github.com/penpot/penpot/pull/11951))
- Fix dashboard file thumbnails loading slow for large multi-page files [#12053](https://github.com/penpot/penpot/issues/12053) (PR: [#12054](https://github.com/penpot/penpot/pull/12054))
- Fix Plugin API resolving a font family token to its separate words when the family name has several words [#11748](https://github.com/penpot/penpot/issues/11748) (PR: [#11750](https://github.com/penpot/penpot/pull/11750))
- Fix file import recreating circular library references when the exported libraries use each other [#12052](https://github.com/penpot/penpot/issues/12052) (PR: [#12062](https://github.com/penpot/penpot/pull/12062))
- Fix asset redirect responses advertising a cache lifetime of about 1000 days instead of 24 hours (by @ners) [#12761](https://github.com/penpot/penpot/issues/12761) (PR: [#12758](https://github.com/penpot/penpot/pull/12758))
- Fix an internal error on typography tokens due to circular dependencies on other token types [#9580](https://github.com/penpot/penpot/issues/9580)
- Fix the MCP server rejecting request bodies over 100kb (express.json default) with PayloadTooLargeError (by @GitNimay) [#12080](https://github.com/penpot/penpot/issues/12080) (PR: [#12450](https://github.com/penpot/penpot/pull/12450))
- Fix library component, color and typography counters showing zero when opening files from the dashboard (by @lukel97) [#12826](https://github.com/penpot/penpot/issues/12826) (PR: [#12787](https://github.com/penpot/penpot/pull/12787))
- Fix workspace crash with Maximum call stack size exceeded when applying a library update [#12576](https://github.com/penpot/penpot/issues/12576) (PR: [#12661](https://github.com/penpot/penpot/pull/12661))
- Fix canvas rendering an imported shape without its hole when the SVG fill rule is evenodd [#11947](https://github.com/penpot/penpot/issues/11947) (PR: [#12006](https://github.com/penpot/penpot/pull/12006))
- Fix Plugin API component.instance() and createShapeFromSvg() nesting the new shape into whatever board is at (0, 0) of the current page (by @makesomethingshit) [#12086](https://github.com/penpot/penpot/issues/12086) (PR: [#12499](https://github.com/penpot/penpot/pull/12499))
- Fix view mode breaking zoom, board navigation, and comments when the URL has no page id [#12684](https://github.com/penpot/penpot/issues/12684) (PR: [#12687](https://github.com/penpot/penpot/pull/12687))
- Fix autosave stopping and valid edits being lost when part of a save fails validation [#12553](https://github.com/penpot/penpot/issues/12553) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix dashboard duplicate copying a file without the edits that are still saving [#12603](https://github.com/penpot/penpot/issues/12603) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Alt measurements still showing a shape that was deleted [#12605](https://github.com/penpot/penpot/issues/12605) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix Plugin API ignoring changes to a deleted shape without an error [#12606](https://github.com/penpot/penpot/issues/12606) (PR: [#12555](https://github.com/penpot/penpot/pull/12555))
- Fix typography sample overflowing its row in locales with a wider sample (ko, ar) (by @desty) [#11486](https://github.com/penpot/penpot/issues/11486) (PR: [#11487](https://github.com/penpot/penpot/pull/11487))
- Fix token not showing when selecting two text layers (by @ShreyashAgare26) [#11924](https://github.com/penpot/penpot/issues/11924) (PR: [#11944](https://github.com/penpot/penpot/pull/11944))
- Fix inner shadow fill not working in the SVG render while it works in WebGL [#11967](https://github.com/penpot/penpot/issues/11967) (PR: [#12141](https://github.com/penpot/penpot/pull/12141))
- Fix @penpot/plugin-styles loading Work Sans from a third-party CDN (by @ump45nose) [#12023](https://github.com/penpot/penpot/issues/12023) (PR: [#12087](https://github.com/penpot/penpot/pull/12087))
- Fix schema error on swatch component for image fills [#12135](https://github.com/penpot/penpot/issues/12135) (PR: [#12136](https://github.com/penpot/penpot/pull/12136))
- Fix worker storing more shape data than needed [#12740](https://github.com/penpot/penpot/issues/12740) (PR: [#12840](https://github.com/penpot/penpot/pull/12840))
- Fix worker not freeing pages [#12738](https://github.com/penpot/penpot/issues/12738) (PR: [#12751](https://github.com/penpot/penpot/pull/12751))
- Fix editing the L value in HSL mode resetting H and S to 0 (by @makesomethingshit) [#12615](https://github.com/penpot/penpot/issues/12615) (PR: [#12629](https://github.com/penpot/penpot/pull/12629))
- Restrict dashboard plugins to the MCP plugin and localhost plugins [#12948](https://github.com/penpot/penpot/issues/12948) (PR: [#12949](https://github.com/penpot/penpot/pull/12949))
- Fix workspace memory growing on every page switch with the WebGL renderer [#13028](https://github.com/penpot/penpot/issues/13028) (PR: [#12877](https://github.com/penpot/penpot/pull/12877))
- Fix workspace showing React error #310 when the team container renders without a team id (by @filipsajdak) [#12728](https://github.com/penpot/penpot/issues/12728) (PR: [#12920](https://github.com/penpot/penpot/pull/12920))
## 2.19.2
### :bug: Bugs fixed
- Fix referential integrity error when copying component variants with nested components across libraries and teams [#10704](https://github.com/penpot/penpot/issues/10704) (PR: [#12928](https://github.com/penpot/penpot/pull/12928))
- Fix migration 0033 inserting nil shape entries for dangling child references that blocked file opening [#12936](https://github.com/penpot/penpot/issues/12936) (PR: [#12979](https://github.com/penpot/penpot/pull/12979))
- Fix switching a variant and adding a component instance taking over 10 seconds in files with many token-styled shapes [#13019](https://github.com/penpot/penpot/issues/13019) (PR: [#13021](https://github.com/penpot/penpot/pull/13021))
- Fix autosave failing after uploading an SVG as a component in the assets panel, and on other mains named unlike their component [#12773](https://github.com/penpot/penpot/issues/12773) (PR: [#12794](https://github.com/penpot/penpot/pull/12794))
- Fix `enable-oidc-registration` not working when password login is disabled [#12964](https://github.com/penpot/penpot/issues/12964) (PR: [#12974](https://github.com/penpot/penpot/pull/12974))
- Fix self-hosted backend opening 60 database connections at startup and keeping them open while idle (by @UsamaSadiq) [#12622](https://github.com/penpot/penpot/issues/12622) (PR: [#13052](https://github.com/penpot/penpot/pull/13052))
## 2.19.1
### :lock: Security fixes
- Fix editor-role team members retrieving the signed invitation token of any pending invitation (https://github.com/penpot/penpot/security/advisories/GHSA-4mmc-j8rh-74pf) (PR: [#12892](https://github.com/penpot/penpot/pull/12892))
- Fix share-link page-scope escape in `get-profiles-for-file-comments` leaking the identity of comment authors outside the link's scope (https://github.com/penpot/penpot/security/advisories/GHSA-89cq-3wv7-987r) (PR: [#12888](https://github.com/penpot/penpot/pull/12888))
- Fix missing authorization and unbounded array in mark-all-threads-as-read that let any authenticated user write the status of inaccessible threads (https://github.com/penpot/penpot/security/advisories/GHSA-j4vc-cmf6-jhr7) (PR: [#12894](https://github.com/penpot/penpot/pull/12894))
- Fix incomplete SVG sanitizer escaping that let a "sanitized" SVG carry live script (CVE-2026-105694) (https://github.com/penpot/penpot/security/advisories/GHSA-qwhj-rchj-58mg) (PR: [#12887](https://github.com/penpot/penpot/pull/12887))
### :bug: Bugs fixed
- Fix component instance (variant) inside a layout resetting to its original size after a shared library update of an unrelated component [#12670](https://github.com/penpot/penpot/issues/12670) (PR: [#12851](https://github.com/penpot/penpot/pull/12851))
## 2.19.0
### :boom: Breaking changes & Deprecations

View File

@ -20,7 +20,6 @@ Center](https://help.penpot.app/).
- [Title format](#title)
- [Description](#description)
- [Review process](#review-process)
- [Merging](#merging)
- [What we won't accept](#what-we-wont-accept)
- [Good first issues](#good-first-issues)
- [Commit Guidelines](#commit-guidelines)
@ -112,8 +111,8 @@ Branch names are not enforced, but we recommend the following:
- Otherwise, use a short, descriptive name with words separated by hyphens
and no slashes (e.g. `fix-ellipse-icon-typo`, `feat-auto-link-libraries`).
Since PRs are squash-merged by default, the branch name does not survive
into the commit history — what matters is the [PR title](#title).
Since PRs are squash-merged, the branch name does not survive into the
commit history — what matters is the [PR title](#title).
### Format
@ -157,27 +156,9 @@ for more concrete information.
- Address review feedback by **pushing new commits** — do not
force-push during review, as it breaks comment threads.
- PRs require at least **one approval** before merge.
- We **squash-merge** by default; see [Merging](#merging) for the reason
and for how to ask for a regular merge. The PR title becomes the final
- We use **squash-merge** by default. The PR title becomes the final
commit message, so follow the [title format](#title) above.
### Merging
PRs are **squash-merged** by default. One PR lands as one commit, which keeps
the change atomic: the history stays readable, and `git bisect` never stops
on an intermediate commit that fails the tests or the linter.
This is the default, not a hard rule. If you keep your commits atomic and
self-contained — each one builds, passes the linter and makes sense on its
own — you may ask for a regular merge instead. Put the request in the first
line of the PR description, above everything else:
> **Merge:** do not squash; each commit is self-contained.
Maintainers honor the request when the commits hold up, and squash-merge
otherwise. After a squash merge the PR title becomes the commit subject on
the main branch; see [Title format](#title).
### What we won't accept
To save time on both sides, please avoid submitting PRs that:
@ -244,11 +225,8 @@ Commit messages must follow this format:
those prefixes to an existing subject)
- **Wrap body lines at 80 characters or fewer** (trailers and URLs excepted)
- Separate the subject from the body with a **blank line**
- Use `Co-authored-by:` only for a human co-author
(`Co-authored-by: Name <email>`). AI providers, agents and bots are not
allowed there; for AI participation add an `AI-assisted-by: <model-name>`
trailer instead. `./scripts/check-commit` rejects the trailer when its value
names one of them.
- Never add a `Co-authored-by` trailer — it is forbidden and fails
`./scripts/check-commit`
You can check a commit against these rules with `./scripts/check-commit`.
CI runs the same script on every new commit in

View File

@ -13,6 +13,9 @@
<a href="https://tree.taiga.io/project/penpot/" rel="nofollow">
<img alt="Managed with Taiga.io" src="https://img.shields.io/badge/managed%20with-TAIGA.io-709f14.svg">
</a>
<a href="https://gitpod.io/#https://github.com/penpot/penpot" rel="nofollow">
<img alt="Gitpod ready-to-code" src="https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod">
</a>
</p>
<p align="center">

View File

@ -40,6 +40,7 @@
nrepl/nrepl {:mvn/version "1.8.0"}
org.postgresql/postgresql {:mvn/version "42.7.13"}
org.xerial/sqlite-jdbc {:mvn/version "3.53.4.0"}
com.zaxxer/HikariCP {:mvn/version "7.1.0"}
@ -53,8 +54,6 @@
org.jsoup/jsoup {:mvn/version "1.23.2"}
com.j256.simplemagic/simplemagic {:mvn/version "1.17"}
at.yawk.lz4/lz4-java
{:mvn/version "1.12.0"}
@ -97,11 +96,6 @@
:jvm-opts ["-Dlog4j2.configurationFile=log4j2-devenv-repl.xml"]
:extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}}}
;; Use after :test (`-M:dev:test:coverage`) to run the suite with cloverage.
:coverage
{:main-opts ["-m" "kaocha.runner" "--plugin" "cloverage"]
:extra-deps {lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}}}
:outdated
{:extra-deps {com.github.liquidz/antq {:mvn/version "2.11.1276"}}
:main-opts ["-m" "antq.core"]}

View File

@ -4,7 +4,7 @@
"license": "MPL-2.0",
"author": "Kaleidos INC Sucursal en España SL",
"private": true,
"packageManager": "pnpm@12.21.0+sha512.66fd8278ee28935ce951792d34367ff3af5c432a2756445823029533835b8da529bbe0be5130afd56ed0713fb8789e47bad82d93bd5ae2fe54b15562d6276ef0",
"packageManager": "pnpm@12.18.0+sha512.9d7882d5dc82147ecfc4e062aa34834e0ad0296710a1837a1115ae012127d36b7109e321ac1598a1dd0a65597681b1e99917e12ed0d749eae91187943363beb3",
"repository": {
"type": "git",
"url": "https://github.com/penpot/penpot"

131
backend/pnpm-lock.yaml generated
View File

@ -7,162 +7,153 @@ importers:
configDependencies: {}
packageManagerDependencies:
pnpm:
specifier: 12.21.0
version: 12.21.0
specifier: 12.18.0
version: 12.18.0
packages:
'@pnpm/exe.android-arm64@12.21.0':
resolution: {integrity: sha512-iaWidxR5B9NUBpwMznCQu1yUpKB6BjTY5BYX2lORj76VsfINdj8Z8XWSzSNrGI8qlxa28Jzdp5fiaspzOaJZlA==}
'@pnpm/exe.android-arm64@12.18.0':
resolution: {integrity: sha512-fOF6IwgvnUKEmVedGqiAsNPfN2Ry8GvaqzSQOiF3RJFbpBfPKO/6YT54s1GjToyYinRxw9preHHPe4y2N3SzhA==}
cpu: [arm64]
os: [android]
'@pnpm/exe.android-x64@12.21.0':
resolution: {integrity: sha512-35e5v3wlwjIYxDywq+R6/z8CE4gLOEgRxt18yqz1tMRLmx2mnW+CFJI2Q8+srTNjn7gykXUC0x1k195JNikC5A==}
'@pnpm/exe.android-x64@12.18.0':
resolution: {integrity: sha512-6OiolnaCsB3/GcJ9XNrenbUH3zFHRJ+5QynrMUUEyThbgnnCzWxa199hpQlOmxN4rAXuYQ0IYgWFNUZxyzg31Q==}
cpu: [x64]
os: [android]
'@pnpm/exe.darwin-arm64@12.21.0':
resolution: {integrity: sha512-F5OyMX/6zLSowpTZEQRojbCmPS1Pq+fyRaIinslsIe6Glh3qbyQ52u+Fe0eEzSaRuIbWZkXmAAXSWh6Q5Wrlgg==}
'@pnpm/exe.darwin-arm64@12.18.0':
resolution: {integrity: sha512-zkq+4YzVIX39ETYLyC3W7LY7MpHEU/J3nRhHl4i2yPld2Cf2+rEKRh3ejJ4AmaQ77HHJLCktY4Z3jq0fpcxFXg==}
cpu: [arm64]
os: [darwin]
'@pnpm/exe.darwin-x64@12.21.0':
resolution: {integrity: sha512-EzpvcsaE6hrrgYeEiAFYd2rSgFybYyxXhkAY7K3y6Hg3loVfcyJ81hHczpFzi8lW5RXtUlFECRPZ8ZAMRFQYYg==}
'@pnpm/exe.darwin-x64@12.18.0':
resolution: {integrity: sha512-qMsL9qx92unOTa6hgkfiTJv8ceZaTx0IQg7InhYDVX2ZrTECujaVEEVypnJK5m3eGUsP1KD7Y4qIgleCwfRsYw==}
cpu: [x64]
os: [darwin]
'@pnpm/exe.freebsd-x64@12.21.0':
resolution: {integrity: sha512-wNjuEnmdLks4aayPu/QZWQ5tQM9Jn+odUp/vwQM8BXRYoulkTZhtZTHoD60N6mUdLWef1hgUeUh8llNGEUsimw==}
'@pnpm/exe.freebsd-x64@12.18.0':
resolution: {integrity: sha512-/9vy/4ms52cU4MjZ5NhHd6N2d9L9W0kARf5/ff2tweAHu0VeUzs/0qp/7ZHFuddm2Wa/m0AKQoPum9AVUCqKsw==}
cpu: [x64]
os: [freebsd]
'@pnpm/exe.linux-arm64-musl@12.21.0':
resolution: {integrity: sha512-Ceit9zn4g5ZqCJtx1fV8eJE7p8aeXMrGZ+Dc01vRqZLIUNgNeI+Ur0ruyqxtsnJx6rGADyXzohumxPLHjaJ5AQ==}
'@pnpm/exe.linux-arm64-musl@12.18.0':
resolution: {integrity: sha512-Vp+TIMPPyackVCXkOdEKH/RmAn/r07EcIyrgZYQK7VrkksqZOie0GnWIx+85mPqeFEYCRtXIZAFJ9L4ceTndDg==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@pnpm/exe.linux-arm64@12.21.0':
resolution: {integrity: sha512-4Oj/iF2TDCe2lcH0n4TupPR5sSdrNLclJB6xBElLMPCUOAvBhdf2rRlSaO5M4erJ6g4Fyy7YTXTpcmfFsKyxEA==}
'@pnpm/exe.linux-arm64@12.18.0':
resolution: {integrity: sha512-2dpyVBCHT+FfpuQh9i+7MXAqNWYLCvD5vYr5p6Eimxoga8USLnjbUgyj72AOrO0hjc5ri3TEFP3jiyBHjkSZpQ==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@pnpm/exe.linux-ppc64@12.21.0':
resolution: {integrity: sha512-s2o7rdSDeUTBFsNqxptpZXlTgT6cojmMVUOvTQvFNSv6Easian02ha4tR32fwwD97Au4YzIIjMCwywCF4D82Sw==}
'@pnpm/exe.linux-ppc64@12.18.0':
resolution: {integrity: sha512-BBOOOfNJ2QwIIya3eKqOjiN4WNTszfRz7HCvw9XggcT7+ftMjNP8XJ/GfW+ltCBeUFR9d8WBVsr2m/NLjX2CZQ==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@pnpm/exe.linux-riscv64@12.21.0':
resolution: {integrity: sha512-EYv3GzxeBTITCuM3yGHHbpRFpUvyulkXpQ6LsjKAQNL3iaba1E6h9iDmlouLb3Io4AMMsfdk0/uG6poQ41uewA==}
'@pnpm/exe.linux-riscv64@12.18.0':
resolution: {integrity: sha512-7kMN77xSOYJX8GPSH5RbNAc5oM61LMmcSUukDDUwllnZPXm8GRPskCNsTWy8SCw01Kr1FB+5SwGNMiTdiziLNQ==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@pnpm/exe.linux-s390x@12.21.0':
resolution: {integrity: sha512-8VXGNGVNEfs7pL9GTZOzFxjFEnMt4WL3rLOADR5/kPQ7ROTV9GfpVI2DPO/zArXFSkvfc9Zad/WypHLbvzoaTQ==}
'@pnpm/exe.linux-s390x@12.18.0':
resolution: {integrity: sha512-htuSCpEcmDzhKoJqXbPkvfr176dBHU0QIF5mWcRiqxRXAB/1JoYiMYotfjWpk+ztgpB9qlx4XnWlLFAf7gzc6g==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@pnpm/exe.linux-x64-musl@12.21.0':
resolution: {integrity: sha512-MnC5vpIXYubhDZCgCwlfJMg7Cg6HupN2IuouT9gD/tUmOfVdQYqeyYccWbthyo/Jeto8de1oQlxeRN1+1Hl2DQ==}
'@pnpm/exe.linux-x64-musl@12.18.0':
resolution: {integrity: sha512-Uj+vzfIQ62kDP10x36yHZIXVs5ohdNtiPYwn5FhCLlNScHPo0OAI+ShGDtLofiBXKE7umu71Jmjm7N8a8zVcDw==}
cpu: [x64]
os: [linux]
libc: [musl]
'@pnpm/exe.linux-x64@12.21.0':
resolution: {integrity: sha512-8eufCjHXspKbNS/kh8q4Q0gqJoOuvC4Q5pgos6XS8By/nAMtP+jf8F1Cw9er840kfAX+PJ/ts+A9qnLE91rvXQ==}
'@pnpm/exe.linux-x64@12.18.0':
resolution: {integrity: sha512-OJ+X2wcraldCwj75pCIerWARQSRpdfj0eBTs/2ODyhfeW46ylOnjTIE/GKra6Mn6E72/gBmXS8igrUioa+F9fQ==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@pnpm/exe.netbsd-x64@12.21.0':
resolution: {integrity: sha512-eyh1M8hQZCkLWxuva7S/fc1SbFbgqg0/5di0dgWmyiuc2Io3dbWVihDun8X+XM8zsR5uhIDvvZpbS9aZQbCovg==}
cpu: [x64]
os: [netbsd]
'@pnpm/exe.win32-arm64@12.21.0':
resolution: {integrity: sha512-7xTxxUXo/grRLivSeIl6B21gnsz7ooe5EMPcJmQqwxc8WLJVReN8MzrlHru8bqXfVr4X3+6ZGZscntKenlCBqQ==}
'@pnpm/exe.win32-arm64@12.18.0':
resolution: {integrity: sha512-wEY6jiACGcbhhi5yD59doHqNTndS7AaujaPmtWAu1DKkHru2hy5fJ9yUGecW72Ypaw2/I3BEprfMOopap2NQtw==}
cpu: [arm64]
os: [win32]
'@pnpm/exe.win32-x64@12.21.0':
resolution: {integrity: sha512-9TpkJemVApxJ1Z1w0ZFGN+AuSx4nJOqXSJUqMqDvaXTYwGOzvBiiWdFP9GN0ketiWCQS1Q0/pza2E7I3pP0OBQ==}
'@pnpm/exe.win32-x64@12.18.0':
resolution: {integrity: sha512-5FJkghO7RLRkQpWxeVEygW9XsmnNuxzPfXbgwFWCHsFyFhpDn/vOdL88ToQEM0Kt5igdW5s6U0/IDH8Dvkt3pg==}
cpu: [x64]
os: [win32]
pnpm@12.21.0:
resolution: {integrity: sha512-Zv2CeO4ok1zpUXktNDZ/869cQyonVkRYIwKVM4NbjaUpu+C+UTCv1W7QcT+4eJ5Hutgtk71a4v5UsVVi1idu8A==}
pnpm@12.18.0:
resolution: {integrity: sha512-nXiC1dyCFH7PxOBiqjSDTgrQKWcQoYN6ERWuASEn02txCeMhrBWYod0KZVl2gbHpmRfhLtDXSerpEYeUM2O+sw==}
engines: {node: '>=18'}
hasBin: true
snapshots:
'@pnpm/exe.android-arm64@12.21.0':
'@pnpm/exe.android-arm64@12.18.0':
optional: true
'@pnpm/exe.android-x64@12.21.0':
'@pnpm/exe.android-x64@12.18.0':
optional: true
'@pnpm/exe.darwin-arm64@12.21.0':
'@pnpm/exe.darwin-arm64@12.18.0':
optional: true
'@pnpm/exe.darwin-x64@12.21.0':
'@pnpm/exe.darwin-x64@12.18.0':
optional: true
'@pnpm/exe.freebsd-x64@12.21.0':
'@pnpm/exe.freebsd-x64@12.18.0':
optional: true
'@pnpm/exe.linux-arm64-musl@12.21.0':
'@pnpm/exe.linux-arm64-musl@12.18.0':
optional: true
'@pnpm/exe.linux-arm64@12.21.0':
'@pnpm/exe.linux-arm64@12.18.0':
optional: true
'@pnpm/exe.linux-ppc64@12.21.0':
'@pnpm/exe.linux-ppc64@12.18.0':
optional: true
'@pnpm/exe.linux-riscv64@12.21.0':
'@pnpm/exe.linux-riscv64@12.18.0':
optional: true
'@pnpm/exe.linux-s390x@12.21.0':
'@pnpm/exe.linux-s390x@12.18.0':
optional: true
'@pnpm/exe.linux-x64-musl@12.21.0':
'@pnpm/exe.linux-x64-musl@12.18.0':
optional: true
'@pnpm/exe.linux-x64@12.21.0':
'@pnpm/exe.linux-x64@12.18.0':
optional: true
'@pnpm/exe.netbsd-x64@12.21.0':
'@pnpm/exe.win32-arm64@12.18.0':
optional: true
'@pnpm/exe.win32-arm64@12.21.0':
'@pnpm/exe.win32-x64@12.18.0':
optional: true
'@pnpm/exe.win32-x64@12.21.0':
optional: true
pnpm@12.21.0:
pnpm@12.18.0:
optionalDependencies:
'@pnpm/exe.android-arm64': 12.21.0
'@pnpm/exe.android-x64': 12.21.0
'@pnpm/exe.darwin-arm64': 12.21.0
'@pnpm/exe.darwin-x64': 12.21.0
'@pnpm/exe.freebsd-x64': 12.21.0
'@pnpm/exe.linux-arm64': 12.21.0
'@pnpm/exe.linux-arm64-musl': 12.21.0
'@pnpm/exe.linux-ppc64': 12.21.0
'@pnpm/exe.linux-riscv64': 12.21.0
'@pnpm/exe.linux-s390x': 12.21.0
'@pnpm/exe.linux-x64': 12.21.0
'@pnpm/exe.linux-x64-musl': 12.21.0
'@pnpm/exe.netbsd-x64': 12.21.0
'@pnpm/exe.win32-arm64': 12.21.0
'@pnpm/exe.win32-x64': 12.21.0
'@pnpm/exe.android-arm64': 12.18.0
'@pnpm/exe.android-x64': 12.18.0
'@pnpm/exe.darwin-arm64': 12.18.0
'@pnpm/exe.darwin-x64': 12.18.0
'@pnpm/exe.freebsd-x64': 12.18.0
'@pnpm/exe.linux-arm64': 12.18.0
'@pnpm/exe.linux-arm64-musl': 12.18.0
'@pnpm/exe.linux-ppc64': 12.18.0
'@pnpm/exe.linux-riscv64': 12.18.0
'@pnpm/exe.linux-s390x': 12.18.0
'@pnpm/exe.linux-x64': 12.18.0
'@pnpm/exe.linux-x64-musl': 12.18.0
'@pnpm/exe.win32-arm64': 12.18.0
'@pnpm/exe.win32-x64': 12.18.0
---
lockfileVersion: '9.0'

View File

@ -690,113 +690,15 @@
(assoc :query (u/map->query-string params)))]
(redirect-response uri))))
(defn- public-uri
"The configured public URI, parsed, with the app base path normalized with
a trailing slash."
[]
(u/ensure-path-slash (u/uri (cf/get :public-uri))))
(defn- public-uri-base
"The public URI as a string, with the app base path normalized with a
trailing slash. Every URL the server builds starts here, so the app base
path is never lost."
[]
(str (public-uri)))
(defn- redirect-with-organization-sso-error
[{:keys [dest-url organization-id organization-name]}]
(-> (str (or dest-url (public-uri-base)))
(-> (str (or dest-url (cf/get :public-uri)))
(u/append-query-param :sso-error true)
(u/append-query-param :organization-id organization-id)
(cond-> organization-name
(u/append-query-param :organization-name organization-name))
(redirect-response)))
(defn- routable-params?
"True when `params` are params of a route: at least one, and no key carries
a scheme, a path or a fragment. `screen=workspace&team-id=…` and the root
params of the application (`template=…`) qualify; `https://evil.example/`
does not."
[params]
(and (seq params)
(every? #(nil? (re-find #"[/:?#]" (name %))) (keys params))))
(defn build-dest-url
"Return the absolute URL for `query-string` under the current public URI.
The query string is the route of the client (`screen=…&params`, and the root
of the application may carry only its params, like `template=…`): it does not
carry the `?` that separates it from the path, which is put back here, so the
destination is always built by the server and the client only picks the route.
A value that still comes with the separator is trimmed, so a client bundled
before that change keeps its destination.
The params are parsed and rendered back, so a value that is not a route never
becomes a destination: it is ignored with a warning and the user lands on the
public URI, which is also where a blank one lands."
[query-string]
(try
(let [base (public-uri-base)
query (str query-string)
route (if (str/starts-with? query "?")
(subs query 1)
query)
params (u/query-string->map route)]
(cond
(str/blank? query)
base
(not (routable-params? params))
(do (l/warn :hint "ignoring a redirect target that is not a route"
:query-string (subs route 0 (min 200 (count route))))
base)
:else
(str base "?" (u/map->query-string params))))
(catch Exception e
;; The destination must never break the SSO flow: a query string the
;; server cannot parse lands on the public URI instead.
(l/warn :hint "could not build the redirect target" :cause e)
(public-uri-base))))
(defn- same-origin?
"True when `uri` has the same origin as `base`: scheme, host (case
insensitive) and port, with the default port made explicit."
[uri base]
(let [default-port (fn [scheme] (if (= "https" (str/lower (str scheme))) 443 80))
port (fn [u] (or (:port u) (default-port (:scheme u))))]
(and (= (str/lower (str (:scheme uri)))
(str/lower (str (:scheme base))))
(= (str/lower (str (:host uri)))
(str/lower (str (:host base))))
(= (port uri) (port base)))))
(defn- under-base-path?
"True when the path of `uri` is under the base path of `base`."
[uri base]
(str/starts-with? (str (:path uri)) (:path base)))
(defn- trusted-dest-url
"Return `dest-url` when it points inside the current public URI, nil
otherwise.
The destination is built by the server (see `build-dest-url`), but state
tokens minted before that was the case may carry any URL, so the callback
checks the destination before redirecting to it. The check compares the
parsed origin, so neither a userinfo prefix
(`https://penpot.example.com@evil.example/`) nor a look-alike host
(`https://penpot.example.com.evil.example/`) can pass, and it requires
the path to be under the base path of the public URI, so the destination
stays inside the application."
[dest-url]
(when (some? dest-url)
(let [dest (try (u/uri (str dest-url)) (catch Exception _ nil))
base (public-uri)]
(when (and (some? dest)
(same-origin? dest base)
(under-base-path? dest base))
(str dest-url)))))
(defn- redirect-to-register
[cfg info provider]
(let [info (assoc info
@ -1013,18 +915,11 @@
(defn build-organization-sso-auth-redirect-uri
"Build the OIDC authorization redirect URI for an organization SSO config.
Raises if the config is incomplete or OIDC discovery fails.
`query-string` is the route of the client (`screen=…&params`, the query
string proper, without its separator): the destination is built here, so it
can never point outside the public URI.
`profile-id` is the profile that started the flow; it travels in the
state token so the callback only grants the organization SSO entry to that
very session."
[cfg sso & {:keys [query-string organization-id provider profile-id]}]
Raises if the config is incomplete or OIDC discovery fails."
[cfg sso & {:keys [dest-url organization-id provider]}]
(let [organization-id (or organization-id (:organization-id sso))
issuer (organization-sso-discovery-uri sso)
dest-url (build-dest-url query-string)]
dest-url (or dest-url (str (cf/get :public-uri)))]
(when-not issuer
(raise-invalid-sso-config
:hint "missing issuer"
@ -1034,7 +929,6 @@
state-token (tokens/generate cfg {:iss "oidc"
:dest-url dest-url
:organization-id organization-id
:profile-id profile-id
:issuer issuer
:exp (ct/in-future "4h")})]
(build-auth-redirect-uri oidc-provider state-token))
@ -1130,30 +1024,16 @@
(defn- organization-sso-callback-handler
"Handle the organization-SSO branch of the OIDC callback: state carries
:dest-url — exchange the authorization code with the OIDC provider to
verify authentication actually occurred, then redirect back to dest-url.
The state also carries the :profile-id that started the flow: only that
session is granted the organization SSO entry, so a state prepared by
someone else cannot mark the session of the one who completes it."
verify authentication actually occurred, then redirect back to dest-url."
[cfg request state code]
(let [dest-url (trusted-dest-url (:dest-url state))]
(let [dest-url (:dest-url state)]
(try
(let [organization-id (:organization-id state)
sso (nitrate/call cfg :get-organization-sso {:organization-id organization-id})
provider (prepare-organization-sso-provider cfg sso)
_info (get-info cfg provider state code)
session (session/get-session request)
exp (ct/in-future {:hours 4})
;; A state minted before the profile claim existed never
;; matches, and neither does the session of another profile.
same-profile? (and (some? session)
(some? (:profile-id state))
(= (:profile-id state) (:profile-id session)))]
(if (and (some? session) (not same-profile?))
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-failed"
:failure-reason "session-mismatch")
(do
exp (ct/in-future {:hours 4})]
(when (and session organization-id)
(let [props (-> (or (:props session) {})
(update :sso assoc organization-id exp))]
@ -1161,8 +1041,8 @@
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-token-issued")))
(submit-organization-sso-auth-event
cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded")))
(redirect-response (or dest-url (public-uri-base))))
cfg request (:profile-id session) organization-id "organization-sso-auth-succeeded")
(redirect-response dest-url))
(catch Throwable cause
(let [{:keys [code]} (ex-data cause)]
(binding [l/*context* (errors/request->context request)]
@ -1180,36 +1060,10 @@
(let [organization-id (:organization-id state)
organization-name (:name (nitrate/call cfg :get-organization-summary {:organization-id organization-id}))]
(redirect-with-organization-sso-error
{:dest-url (or dest-url (public-uri-base))
{:dest-url dest-url
:organization-id organization-id
:organization-name organization-name}))))))
(def oidc-provider-types
"Backends authenticating through an OIDC-compatible provider.
Only these backends take the SSO registration gate."
#{"oidc" "google" "github" "gitlab"})
(defn sso-registration-allowed?
"SSO registration is allowed when public registration or explicit
OIDC registration is enabled. Shared by the callback gate and the
`register-profile` token gate — keep one definition."
[]
(or (contains? cf/flags :registration)
(contains? cf/flags :oidc-registration)))
(defn- valid-invitation-for?
"Returns true when `invitation-token` is a verifiable
`:team-invitation` JWT issued for `email`. Never throws: a garbage
token means there is no valid invitation."
[cfg invitation-token email]
(try
(boolean
(when (and (some? invitation-token) (some? email))
(let [invitation (tokens/verify cfg {:token invitation-token :iss :team-invitation})]
(= (profile/clean-email (:member-email invitation))
(profile/clean-email email)))))
(catch Throwable _ false)))
(defn- callback-handler
[cfg {:keys [params] :as request}]
(if-let [error (get params :error)]
@ -1242,8 +1096,8 @@
(redirect-with-error "email-domain-not-allowed")
:else
(if (or (sso-registration-allowed?)
(valid-invitation-for? cfg (:invitation-token info) (:email info)))
(if (or (contains? cf/flags :registration)
(contains? cf/flags :oidc-registration))
(redirect-to-register cfg info provider)
(redirect-with-error "registration-disabled")))

View File

@ -24,6 +24,7 @@
[app.common.weak :as weak]
[app.config :as cf]
[app.db :as db]
[app.db.sql :as sql]
[app.features.fdata :as fdata]
[app.features.file-migrations :as fmigr]
[app.loggers.audit :as-alias audit]
@ -42,6 +43,7 @@
(def ^:dynamic *state* nil)
(def ^:dynamic *options* nil)
(def ^:dynamic *reference-file* nil)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; DEFAULTS
@ -128,12 +130,18 @@
(defn initial-state
[]
{:storage-objects #{}
:files #{}})
:files #{}
:teams #{}
:projects #{}})
(defn collect-storage-objects
[state items]
(update state :storage-objects into xf-map-media-id items))
(defn collect-summary
[state key items]
(update state key into xf-map-media-id items))
(defn lookup-index
[id]
(when id
@ -151,7 +159,7 @@
[index obj & attrs]
(reduce (fn [index attr-fn]
(let [old-id (attr-fn obj)
new-id (uuid/next)]
new-id (if (::overwrite *options*) old-id (uuid/next))]
(assoc index old-id new-id)))
index
attrs))
@ -201,7 +209,6 @@
p.team_id,
coalesce(fd.backend, 'legacy-db') AS backend,
fd.metadata AS metadata,
fd.storage_object_id AS storage_object_id,
fd.data AS data
FROM file AS f
LEFT JOIN file_data AS fd ON (fd.file_id = f.id AND fd.id = f.id)
@ -211,6 +218,7 @@
(str sql:files-with-data " WHERE f.id = ?"))
(def sql:get-file-without-data
(str "WITH files AS (" sql:files-with-data ")"
"SELECT f.id,
f.project_id,
f.created_at,
@ -225,23 +233,12 @@
f.features,
f.version,
f.vern,
p.team_id
FROM file AS f
INNER JOIN project AS p ON (p.id = f.project_id)
WHERE f.id = ?")
f.team_id
FROM files AS f
WHERE f.id = ?"))
(defn- migrate-file
"Apply the pending migrations to the file.
With `persist-after-migration?` the migrated map is also persisted, which is
only safe from a map read under the `file` row lock (see `get-file*`). The
returned map is then the reattached one: decoded data where the pages the
migration did not change keep their pointers and their fragment ids, plus
the `:storage-object-id` of the row the write left, so a caller that
persists again touches the current object. Without `persist-after-migration?`
the realized map is returned, which is what a read-only context can do.
`get-file*` realizes the result when the caller asks for it."
[cfg {:keys [id] :as file} {:keys [persist-after-migration?]}]
[{:keys [::db/conn] :as cfg} {:keys [read-only?]} {:keys [id] :as file}]
(fdata/with-tracked-pointers cfg id
(let [libs (delay (get-resolved-file-libraries cfg file))
;; Capture the pointer layout before realizing: the migration
@ -255,38 +252,51 @@
objects-map-added? (and (not objects-map-before?)
(contains? (:features realized) "fdata/objects-map"))]
(if persist-after-migration?
(if (or read-only? (db/read-only? conn))
realized
(let [file (if objects-map-added?
realized
(fdata/reattach-unchanged original realized))
written (update-file! cfg file)]
(fdata/reattach-unchanged original realized))]
;; When the file is migrated, we break the rule of not
;; performing mutations on get operations and update the file
;; with all migrations applied. The reattached map is also what
;; the read returns: the caller realizes it only when it asks.
(assoc (fmigr/resolve-applied-migrations cfg file)
:storage-object-id (:storage-object-id written)))
realized))))
;; with all migrations applied. The caller still receives the
;; realized file: the read contract is plain data.
(update-file! cfg file)
(fmigr/resolve-applied-migrations cfg realized))))))
(defn- fetch-row
"Read the file row. With `load-data?` the file data columns come along, but
they are not resolved nor decoded (see `attach-file-data`). With
`load-migrations?` the applied migrations are resolved, which is what
`need-migration?` expects. A skip-locked read answers nil instead of raising
when the row is gone."
(defn- get-file*
[{:keys [::db/conn] :as cfg} id
{:keys [include-deleted?
{:keys [migrate?
realize?
decode?
skip-locked?
lock-for-update?
lock-for-share?
include-deleted?
load-data?
throw-if-not-exists?
load-migrations?
load-data?]}]
lock-for-update?
lock-for-share?]
:or {lock-for-update? false
lock-for-share? false
load-data? true
migrate? true
decode? true
include-deleted? false
throw-if-not-exists? true
realize? false}
:as options}]
(let [sql (if load-data?
(assert (db/connection? conn) "expected cfg with valid connection")
(when (and (not load-data?)
(or lock-for-share? lock-for-update? skip-locked?))
(throw (IllegalArgumentException. "locking is incompatible when `load-data?` is false")))
(let [sql
(if load-data?
sql:get-file
sql:get-file-without-data)
sql (cond
sql
(cond
lock-for-update?
(str sql " FOR UPDATE of f")
@ -296,20 +306,42 @@
:else
sql)
sql (if (and skip-locked? (or lock-for-share? lock-for-update?))
sql
(if skip-locked?
(str sql " SKIP LOCKED")
sql)
file (db/get-with-sql conn [sql id] {::db/throw-if-not-exists false
file
(db/get-with-sql conn [sql id]
{::db/throw-if-not-exists false
::db/remove-deleted (not include-deleted?)})
file (some-> file
(d/update-when :features db/decode-pgarray #{})
(d/update-when :metadata fdata/decode-metadata))
file (if (and file load-migrations?)
(fmigr/resolve-applied-migrations cfg file)
file)]
(or file
file
(-> file
(d/update-when :features db/decode-pgarray #{})
(d/update-when :metadata fdata/decode-metadata))]
(if file
(if load-data?
(let [file
(->> file
(fmigr/resolve-applied-migrations cfg)
(fdata/resolve-file-data cfg))
will-migrate?
(and migrate? (fmg/need-migration? file))]
(if decode?
(cond->> (fdata/decode-file-data cfg file)
(and realize? (not will-migrate?))
(fdata/realize cfg)
will-migrate?
(migrate-file cfg options))
file))
file)
(when-not (or skip-locked? (not throw-if-not-exists?))
(ex/raise :type :not-found
:code :object-not-found
@ -317,124 +349,14 @@
:table :file
:file-id id)))))
(defn- attach-file-data
"Resolve the data of an already read row and decode it when `decode?`. It
completes the map it receives instead of reading the row again. It does not
realize: `get-file*` owns that step, once per path."
[cfg file {:keys [decode?]}]
(let [file (fdata/resolve-file-data cfg file)]
(if decode?
(fdata/decode-file-data cfg file)
file)))
(defn- fetch-file
"Read the row with its data resolved and decoded, under the requested lock."
[cfg id options]
(when-let [file (fetch-row cfg id options)]
(attach-file-data cfg file options)))
(def ^:private
default-get-file-options
{:load-migrations? true
:throw-if-not-exists? true
:persist-after-migration? true
:load-data? true
:migrate? true
:decode? true})
(defn- get-file*
[{:keys [::db/conn] :as cfg} id options]
(assert (db/connection? conn) "expected cfg with valid connection")
(let [{:keys [migrate?
realize?
decode?
load-data?
lock-for-share?
read-only?]
:as options}
(merge default-get-file-options options)
read-only?
(or read-only? (db/read-only? conn))]
(when (and realize? (or (not load-data?) (not decode?)))
(throw (IllegalArgumentException. "invalid options, you only can realize when loading data and decoding")))
(when (and lock-for-share? load-data?)
(throw (IllegalArgumentException. "invalid options, `:lock-for-share?` is only allowed on row-only reads")))
(cond
;; The caller only wants the row: that read is the answer.
(not load-data?)
(fetch-row cfg id options)
(and migrate? decode? (not read-only?))
(when-let [file (fetch-row cfg id options)]
(if (fmg/need-migration? file)
(when-let [file (fetch-file cfg id (assoc options :lock-for-update? true))]
(cond-> file
(fmg/need-migration? file)
(as-> $ (migrate-file cfg $ options))
realize?
(->> (fdata/realize cfg))))
(let [file (attach-file-data cfg file options)]
(cond-> file
realize?
(->> (fdata/realize cfg))))))
:else
(when-let [file (fetch-file cfg id options)]
(cond-> file
(and migrate? decode? (fmg/need-migration? file))
(as-> $ (migrate-file cfg $ (assoc options :persist-after-migration? false)))
realize?
(->> (fdata/realize cfg)))))))
(defn get-file
"Get file, resolve all features and apply migrations.
Runs inside a transaction: it joins the caller's when the cfg already
carries a connection and opens one otherwise, so the migration write
commits with the read. A read-only read (`:read-only?`) takes no lock and
writes nothing, so it runs on a plain connection instead of opening a
transaction of its own; with a cfg that already carries a connection it
still joins the caller's. Accepts a cfg map only: a bare pool or a
connection raises `IllegalArgumentException`.
Useful when you have plan to apply massive or not surgical
operations on file, because it removes the overhead of lazy fetching
and decoding.
Options:
- `:realize?` returns plain data instead of fragment pointers; it
requires `:load-data?` and `:decode?`.
- `:decode?` decodes the stored blob (default true).
- `:load-data?` reads the data columns (default true). A row-only read
(`false`) is the only mode that accepts `:lock-for-share?`.
- `:migrate?` applies the pending migrations on read (default true).
- `:load-migrations?` resolves the applied migrations on a row read
(default true).
- `:read-only?` applies the pending migrations in memory only and skips
the transaction. It does not take a lock.
- `:lock-for-update?`, `:skip-locked?`, `:include-deleted?` and
`:throw-if-not-exists?` are the usual read knobs. A lock needs a
read-write transaction from the caller: PostgreSQL rejects row locks in
a read-only transaction.
- `:persist-after-migration?` is internal: `get-file*` turns it off
for the in-memory migration of a read-only context. Do not pass it."
and decoding."
[cfg file-id & {:as opts}]
(when-not (map? cfg)
(throw (IllegalArgumentException.
"expected cfg map, not a pool or a connection")))
(if (:read-only? opts)
(db/run! cfg get-file* file-id opts)
(db/tx-run! cfg get-file* file-id opts)))
(db/run! cfg get-file* file-id opts))
(defn clean-file-features
[file]
@ -462,27 +384,19 @@
inner join project as p on (p.id = f.project_id)
where fpr.file_id = ?
and fpr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1
from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id
and tpr.profile_id = fpr.profile_id
and t.deleted_at is null)
and tpr.profile_id = fpr.profile_id)
union all
select tpr.is_owner,
tpr.is_admin,
tpr.can_edit
from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
inner join project as p on (p.team_id = tpr.team_id)
inner join file as f on (p.id = f.project_id)
where f.id = ?
and tpr.profile_id = ?
and t.deleted_at is null
and p.deleted_at is null
and f.deleted_at is null
union all
select ppr.is_owner,
ppr.is_admin,
@ -492,14 +406,10 @@
inner join project as p on (p.id = ppr.project_id)
where f.id = ?
and ppr.profile_id = ?
and f.deleted_at is null
and p.deleted_at is null
and exists (select 1
from team_profile_rel as tpr
inner join team as t on (t.id = tpr.team_id)
where tpr.team_id = p.team_id
and tpr.profile_id = ppr.profile_id
and t.deleted_at is null)")
and tpr.profile_id = ppr.profile_id)")
(defn- get-file-permissions*
[conn profile-id file-id]
@ -547,6 +457,27 @@
[cfg project-id]
(db/get cfg :project {:id project-id}))
(def ^:private sql:get-teams
"SELECT t.* FROM team WHERE id = ANY(?)")
(defn get-teams
[cfg ids]
(let [conn (db/get-connection cfg)
ids (db/create-array conn "uuid" ids)]
(->> (db/exec! conn [sql:get-teams ids])
(map decode-row-features))))
(defn get-team
[cfg team-id]
(-> (db/get cfg :team {:id team-id})
(decode-row-features)))
(defn get-fonts
[cfg team-id]
(db/query cfg :team-font-variant
{:team-id team-id
:deleted-at nil}))
(def ^:private sql:get-files-rels
"SELECT flr.*,
fls.synced_at
@ -557,13 +488,10 @@
ON fls.file_id = flr.file_id
AND fls.library_file_id = flr.library_file_id
WHERE flr.file_id = ANY(?)
AND l.deleted_at IS NULL
ORDER BY flr.file_id, flr.library_file_id;")
AND l.deleted_at IS NULL;")
(defn get-files-rels
"Given a set of file-id's, return all matching relations with the libraries.
Rows come back in stable `(file-id, library-file-id)` order so export
and import resolve cycles deterministically."
"Given a set of file-id's, return all matching relations with the libraries"
[cfg ids]
(assert (set? ids) "expected a set of uuids")
@ -606,20 +534,6 @@
(->> (db/exec! conn [sql:get-libraries ids'])
(into #{} xf-map-id))))))
(defn library-link-status
"Check whether linking `file-id` to `library-id` is safe. Returns
`:ok` when the link can be created, `:self-link` when both ids are
the same, and `:circular` when the link would close a dependency
cycle (i.e. `file-id` is already among the transitive libraries of
`library-id`). Shared by the UI command (`link-file-to-library`,
which raises) and the v3 importer (`import-file-relations`, which
skips and logs), so both stay consistent."
[cfg file-id library-id]
(cond
(= file-id library-id) :self-link
(contains? (get-libraries cfg [library-id]) file-id) :circular
:else :ok))
(defn get-file-object-thumbnails
"Return all file object thumbnails for a given file."
[cfg file-id]
@ -628,6 +542,15 @@
:deleted-at nil})
(not-empty)))
(defn get-file-thumbnail
"Return the thumbnail for the specified file-id"
[cfg {:keys [id revn]}]
(db/get* cfg :file-thumbnail
{:file-id id
:revn revn
:data nil}
{::sql/columns [:media-id :file-id :revn]}))
(def ^:private sql:get-missing-media-references
"SELECT fmo.*
FROM file_media_object AS fmo
@ -786,12 +709,7 @@
(let [library-ids (get-libraries cfg [file-id])
data (reduce (fn [data library-id]
(if-let [library (get-file cfg library-id :include-deleted? true)]
;; The library is read without `:realize?`, so its
;; `:components` may be a pointer map. Bind the
;; library's own loader: the absorb walks the
;; components and would raise without one.
(cond-> (fdata/with-pointers cfg library-id
(ctf/absorb-assets data (:data library)))
(cond-> (ctf/absorb-assets data (:data library))
(= library-id (cfo/get-tokens-source data))
(cfo/absorb-tokens-lib (:data library)))
data))
@ -874,33 +792,18 @@
(dissoc :metadata)
(dissoc :migrations)))
(defn file->response
"Strip the internal keys of a file map before it leaves the backend in a
response. The file data storage pointer is bookkeeping for the write path:
a `file-data` object is authentication-only, so a leaked id would let any
logged-in profile fetch the blob from `/assets/by-id`. The metadata key of
the old pointer is stripped too: a row the migration does not backfill keeps
it, and the metadata schema is open, so it would travel in the response."
[file]
(-> file
(dissoc :storage-object-id)
(d/update-when :metadata #(dissoc % :storage-ref-id))))
(defn- file->file-data-params
[{:keys [id] :as file}]
[{:keys [id] :as file} & {:as opts}]
(let [created-at (or (:created-at file) (ct/now))
modified-at (or (:modified-at file) created-at)]
(-> (d/without-nils
(d/without-nils
{:id id
:type "main"
:file-id id
:data (:data file)
:metadata (:metadata file)
:created-at created-at
:modified-at modified-at})
;; The storage object the persisted row pointed at (nil for a new or
;; imported file). `upsert!` touches it before replacing the row.
(assoc :storage-object-id (:storage-object-id file)))))
:modified-at modified-at})))
(defn insert-file!
"Insert a new file into the database table. Expectes a not-encoded file.
@ -924,63 +827,34 @@
:cause cause)
(throw cause))))
;; An INSERT never replaces a persisted pointer: `duplicate-file` hands
;; the source file map here, and touching the source object would be a
;; write on a live object.
(->> (file->file-data-params (dissoc file :storage-object-id))
(->> (file->file-data-params file)
(fdata/upsert! cfg))
nil))
(defn update-file!
"Update an existing file on the database. Expects a not-encoded file map,
as returned by `get-file`, and optionally the fields to change as keyword
args; they are merged into the file map before persisting.
"Update an existing file on the database. Expects not encoded file."
[{:keys [::db/conn] :as cfg} {:keys [id] :as file} & {:as opts}]
The storage object the persisted row pointed at travels in the map
(`:storage-object-id`), and `upsert!` touches it before replacing the row.
That is why the caller must pass the map it read instead of building a
partial one: nothing is queried here. The params only change mutable fields
of the file: the pointer and the identity (`:id`, `:file-id`) always come
from the read map.
Returns the map `fdata/upsert!` answers with: `:written` and the
`:storage-object-id` of the object this write created, so a caller that
persists again touches the current object instead of the replaced one."
[{:keys [::db/conn] :as cfg} file & {:as params}]
;; A map read with `:load-data? false` has no `:data`; persisting it would
;; wipe the file content. `ex/raise` instead of `assert`: the guard must
;; exist in the deployed binary, where `clojure.core/assert` is compiled
;; out.
(when-not (contains? file :data)
(ex/raise :type :internal
:code :missing-file-data
:hint "expected a file map read with data"
:file-id (:id file)))
(let [reset? (::reset-migrations? params false)
;; The pointer and the identity are bookkeeping of the persisted row,
;; not editable fields: they can only come from the read map. A pointer
;; in the params would skip the touch and leak the replaced blob; an
;; `:id` would retarget the UPDATE.
file (merge file (dissoc params ::reset-migrations?
:storage-object-id
:id
:file-id))]
(if reset?
(if (::reset-migrations? opts false)
(fmigr/reset-migrations! conn file)
(fmigr/upsert-migrations! conn file))
(let [file (encode-file cfg file)
file-params (file->params (dissoc file :id))]
(let [file
(encode-file cfg file)
file-params
(file->params (dissoc file :id))
file-data-params
(file->file-data-params file)]
(db/update! conn :file file-params
{:id (:id file)}
{:id id}
{::db/return-keys false})
(fdata/upsert! cfg (file->file-data-params file)))))
(fdata/upsert! cfg file-data-params)
nil))
(defn save-file!
"Applies all the final validations and perist the file, binfile
@ -993,7 +867,8 @@
(let [file (-> file
(assoc :created-at timestamp)
(assoc :modified-at timestamp)
(assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5})))
(cond-> (not (::overwrite cfg))
(assoc :ignore-sync-until (ct/plus timestamp (ct/duration {:seconds 5}))))
(update :features
(fn [features]
(-> (::features cfg #{})
@ -1010,7 +885,9 @@
(when (ex/exception? result)
(l/error :hint "file schema validation error" :cause result))))
(insert-file! cfg file opts)))
(if (::overwrite cfg)
(update-file! cfg file (assoc opts ::reset-migrations? true))
(insert-file! cfg file opts))))
(def ^:private sql:get-file-libraries
"WITH RECURSIVE libs AS (

View File

@ -482,11 +482,9 @@
(bfm/apply-pending-migrations! cfg)
;; Knowing that the ids of the created files are in index,
;; just lookup them and return them in the same shape as v3. The
;; metadata of an old package may repeat a file id, so the answer
;; carries one id per imported file.
;; just lookup them and return them in the same shape as v3.
(let [files (-> bfc/*state* deref :files)]
{:file-ids (into [] (comp (keep #(get-in @bfc/*state* [:index %])) (distinct)) files)})))))
{:file-ids (into [] (keep #(get-in @bfc/*state* [:index %])) files)})))))
(defmethod read-import :v1
[options]
@ -512,11 +510,15 @@
(defmethod read-section :v1/files
[{:keys [::bfc/input ::bfc/project-id ::bfc/name] :as system}]
(let [imported (volatile! #{})]
(doseq [[idx expected-file-id] (d/enumerate (-> bfc/*state* deref :files))]
(let [file (read-obj! input)
media (read-obj! input)
file-id (:id file)]
file-id (:id file)
file-id' (bfc/lookup-index file-id)
file (bfc/clean-file-features file)
thumbnails (:thumbnails file)]
(when (not= file-id expected-file-id)
(ex/raise :type :validation
@ -525,20 +527,6 @@
:expected-id expected-file-id
:hint "the penpot file seems corrupt, found unexpected uuid (file-id)"))
(if (contains? @imported file-id)
;; Old exports could repeat a file id in the metadata (the file was
;; also one of its own libraries). The entry is read above to keep
;; the stream in sync, but the file is imported only once.
(l/warn :hint "ignoring repeated file entry"
:file-id (str file-id)
::l/sync? true)
(let [file-id' (bfc/lookup-index file-id)
file (bfc/clean-file-features file)
thumbnails (:thumbnails file)]
(vswap! imported conj file-id)
(l/dbg :hint "processing file"
:id (str file-id)
:features (:features file)
@ -571,8 +559,8 @@
(dissoc :thumbnails))
file (bfc/process-file system file)]
;; All features that are enabled and requires explicit migration
;; are added to the state for a posterior migration step.
;; All features that are enabled and requires explicit migration are
;; added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features system)
(set/difference cfeat/no-migration-features)
(set/difference (:features file)))]
@ -581,34 +569,19 @@
(l/dbg :hint "create file" :id (str file-id') ::l/sync? true)
(bfc/save-file! system file ::db/return-keys false)
file-id')))))))
file-id'))))
(defmethod read-section :v1/rels
[{:keys [::db/conn ::bfc/input ::bfc/timestamp] :as cfg}]
(let [rels (->> (read-obj! input)
(sort-by (fn [{:keys [file-id library-file-id]}]
[(str file-id) (str library-file-id)])))
[{:keys [::db/conn ::bfc/input ::bfc/timestamp]}]
(let [rels (read-obj! input)
ids (into #{} (-> bfc/*state* deref :files))]
;; Insert all file relations
(doseq [{:keys [library-file-id] :as rel} rels]
(let [rel (-> rel
(update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))]
(if (contains? ids library-file-id)
(case (bfc/library-link-status cfg (:file-id rel) (:library-file-id rel))
:self-link
(l/warn :hint "ignoring self-referencing library link"
:file-id (:file-id rel)
:lib-id (:library-file-id rel)
::l/sync? true)
:circular
(l/warn :hint "ignoring circular library link"
:file-id (:file-id rel)
:lib-id (:library-file-id rel)
::l/sync? true)
:ok
(do
(l/dbg :hint "create file library link"
:file-id (:file-id rel)
@ -619,7 +592,7 @@
(bfc/upsert-file-library-sync! conn {:file-id (:file-id rel-params)
:library-file-id (:library-file-id rel-params)
:synced-at (or (:synced-at rel)
timestamp)}))))
timestamp)})))
(l/warn :hint "ignoring file library link"
:file-id (:file-id rel)

View File

@ -0,0 +1,445 @@
;; This Source Code Form is subject to the terms of the Mozilla Public
;; License, v. 2.0. If a copy of the MPL was not distributed with this
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
;;
;; Copyright (c) KALEIDOS SUBSIDIARY SL
(ns app.binfile.v2
"A sqlite3 based binary file exportation with support for exportation
of entire team (or multiple teams) at once."
(:refer-clojure :exclude [read])
(:require
[app.binfile.common :as bfc]
[app.common.data :as d]
[app.common.features :as cfeat]
[app.common.logging :as l]
[app.common.time :as ct]
[app.common.transit :as t]
[app.common.uuid :as uuid]
[app.config :as cf]
[app.db :as db]
[app.db.sql :as sql]
[app.loggers.audit :as-alias audit]
[app.loggers.webhooks :as-alias webhooks]
[app.storage :as sto]
[app.storage.tmp :as tmp]
[app.util.events :as events]
[app.worker :as-alias wrk]
[clojure.set :as set]
[cuerdas.core :as str]
[datoteka.io :as io]
[promesa.util :as pu])
(:import
java.sql.DriverManager))
(set! *warn-on-reflection* true)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; LOW LEVEL API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn- create-database
([cfg]
(let [path (tmp/tempfile :prefix "penpot.binfile." :suffix ".sqlite")]
(create-database cfg path)))
([cfg path]
(let [db (DriverManager/getConnection (str "jdbc:sqlite:" path))]
(assoc cfg ::db db ::path path))))
(def ^:private
sql:create-kvdata-table
"CREATE TABLE kvdata (
tag text NOT NULL,
key text NOT NULL,
val text NOT NULL,
dat blob NULL
)")
(def ^:private
sql:create-kvdata-index
"CREATE INDEX kvdata__tag_key__idx
ON kvdata (tag, key)")
(defn- setup-schema!
[{:keys [::db]}]
(db/exec-one! db [sql:create-kvdata-table])
(db/exec-one! db [sql:create-kvdata-index]))
(defn- write!
[{:keys [::db]} tag k v & [data]]
(db/insert! db :kvdata
{:tag (d/name tag)
:key (str k)
:val (t/encode-str v {:type :json-verbose})
:dat data}
{::db/return-keys false}))
(defn- read-blob
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:dat]})]
(:dat obj)))
(defn- read-seq
([{:keys [::db]} tag]
(->> (db/query db :kvdata
{:tag (d/name tag)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str)))
([{:keys [::db]} tag k]
(->> (db/query db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [::val]})
(map :val)
(map t/decode-str))))
(defn- read-obj
[{:keys [::db]} tag k]
(let [obj (db/get db :kvdata
{:tag (d/name tag)
:key (str k)}
{::sql/columns [:val]})]
(-> obj :val t/decode-str)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; IMPORT/EXPORT IMPL
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(declare ^:private write-project!)
(declare ^:private write-file!)
(defn- write-team!
[cfg team-id]
(let [team (bfc/get-team cfg team-id)
fonts (bfc/get-fonts cfg team-id)]
(events/tap :progress
{:op :export
:section :write-team
:id team-id
:name (:name team)})
(l/trc :hint "write" :obj "team"
:id (str team-id)
:fonts (count fonts))
(when-let [photo-id (:photo-id team)]
(vswap! bfc/*state* update :storage-objects conj photo-id))
(vswap! bfc/*state* update :teams conj team-id)
(vswap! bfc/*state* bfc/collect-storage-objects fonts)
(write! cfg :team team-id team)
(doseq [{:keys [id] :as font} fonts]
(vswap! bfc/*state* update :team-font-variants conj id)
(write! cfg :team-font-variant id font))))
(defn- write-project!
[cfg project]
(events/tap :progress
{:op :export
:section :write-project
:id (:id project)
:name (:name project)})
(l/trc :hint "write" :obj "project" :id (str (:id project)))
(write! cfg :project (str (:id project)) project)
(vswap! bfc/*state* update :projects conj (:id project)))
(defn- write-file!
[cfg file-id]
(let [file (bfc/get-file cfg file-id :realize? true)
thumbs (bfc/get-file-object-thumbnails cfg file-id)
media (bfc/get-file-media cfg file)
rels (bfc/get-files-rels cfg #{file-id})]
(events/tap :progress
{:op :export
:section :write-file
:id file-id
:name (:name file)})
(vswap! bfc/*state* (fn [state]
(-> state
(update :files conj file-id)
(update :file-media-objects into bfc/xf-map-id media)
(bfc/collect-storage-objects thumbs)
(bfc/collect-storage-objects media))))
(write! cfg :file file-id file)
(write! cfg :file-rels file-id rels)
(run! (partial write! cfg :file-media-object file-id) media)
(run! (partial write! cfg :file-object-thumbnail file-id) thumbs)
(when-let [thumb (bfc/get-file-thumbnail cfg file)]
(vswap! bfc/*state* bfc/collect-storage-objects [thumb])
(write! cfg :file-thumbnail file-id thumb))
(l/trc :hint "write" :obj "file"
:thumbnails (count thumbs)
:rels (count rels)
:media (count media))))
(defn- write-storage-object!
[{:keys [::sto/storage] :as cfg} id]
(let [sobj (sto/get-object storage id)
data (with-open [input (sto/get-object-data storage sobj)]
(io/read input))]
(l/trc :hint "write" :obj "storage-object" :id (str id) :size (:size sobj))
(write! cfg :storage-object id (meta sobj) data)))
(defn- read-storage-object!
[{:keys [::sto/storage ::bfc/timestamp] :as cfg} id]
(let [mdata (d/update-when (read-obj cfg :storage-object id)
:content-type bfc/normalize-content-type)
data (read-blob cfg :storage-object id)
;; NOTE: v2 bundles are legacy and no longer produced; the
;; sanitizer call stays as defense in depth and is covered by
;; the shared helper unit tests, with no dedicated v2
;; import test.
clean (when (bfc/svg-object? mdata)
(bfc/sanitize-imported-svg mdata data))
data (or (:bytes clean) data)
hash (or (:hash clean) (sto/calculate-hash data))
content (-> (sto/content data)
(sto/wrap-with-hash hash))
params (-> mdata
(assoc ::sto/content content)
(assoc ::sto/deduplicate? true)
(assoc ::sto/touched-at timestamp))
sobject (sto/put-object! storage params)]
(vswap! bfc/*state* update :index assoc id (:id sobject))
(l/trc :hint "read" :obj "storage-object"
:id (str id)
:new-id (str (:id sobject))
:size (:size sobject))))
(defn read-team!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} team-id]
(l/trc :hint "read" :obj "team" :id (str team-id))
(let [team (read-obj cfg :team team-id)
team (-> team
(update :id bfc/lookup-index)
(update :photo-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-team
:id team-id
:name (:name team)})
(db/insert! conn :team
(update team :features db/encode-pgarray conn "text")
::db/return-keys false)
(doseq [font (->> (read-seq cfg :team-font-variant)
(filter #(= team-id (:team-id %))))]
(let [font (-> font
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(update :woff1-file-id bfc/lookup-index)
(update :woff2-file-id bfc/lookup-index)
(update :ttf-file-id bfc/lookup-index)
(update :otf-file-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(db/insert! conn :team-font-variant font
::db/return-keys false)))
team))
(defn read-project!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} project-id]
(l/trc :hint "read" :obj "project" :id (str project-id))
(let [project (read-obj cfg :project project-id)
project (-> project
(update :id bfc/lookup-index)
(update :team-id bfc/lookup-index)
(assoc :created-at timestamp)
(assoc :modified-at timestamp))]
(events/tap :progress
{:op :import
:section :read-project
:id project-id
:name (:name project)})
(db/insert! conn :project project
::db/return-keys false)))
(defn read-file!
[{:keys [::db/conn ::bfc/timestamp] :as cfg} file-id]
(l/trc :hint "read" :obj "file" :id (str file-id))
(let [file (-> (read-obj cfg :file file-id)
(update :id bfc/lookup-index)
(update :project-id bfc/lookup-index))
file (bfc/process-file cfg file)]
(events/tap :progress
{:op :import
:section :read-file
:id file-id
:name (:name file)})
;; All features that are enabled and requires explicit migration are
;; added to the state for a posterior migration step.
(doseq [feature (-> (::bfc/features cfg)
(set/difference cfeat/no-migration-features)
(set/difference (:features file)))]
(vswap! bfc/*state* update :pending-to-migrate (fnil conj []) [feature (:id file)]))
(bfc/save-file! cfg file ::db/return-keys false))
(doseq [thumbnail (read-seq cfg :file-object-thumbnail file-id)]
(let [thumbnail (-> thumbnail
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index))
file-id (:file-id thumbnail)
thumbnail (update thumbnail :object-id
#(str/replace-first % #"^(.*?)/" (str file-id "/")))]
(db/insert! conn :file-tagged-object-thumbnail thumbnail
{::db/return-keys false})))
(doseq [rel (read-obj cfg :file-rels file-id)]
(let [rel (-> rel
(update :file-id bfc/lookup-index)
(update :library-file-id bfc/lookup-index))]
(db/insert! conn :file-library-rel rel
::db/return-keys false)
(bfc/upsert-file-library-sync! conn (assoc rel :synced-at timestamp))))
(doseq [media (read-seq cfg :file-media-object file-id)]
(let [media (-> media
(update :id bfc/lookup-index)
(update :file-id bfc/lookup-index)
(update :media-id bfc/lookup-index)
(update :thumbnail-id bfc/lookup-index))]
(db/insert! conn :file-media-object media
::db/return-keys false
::sql/on-conflict-do-nothing true))))
(def ^:private empty-summary
{:teams #{}
:files #{}
:projects #{}
:file-media-objects #{}
:team-font-variants #{}
:storage-objects #{}})
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; PUBLIC API
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defn export-team!
[cfg team-id]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (create-database cfg)]
(l/inf :hint "start"
:operation "export"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [cfg]
(setup-schema! cfg)
(binding [bfc/*state* (volatile! empty-summary)]
(write-team! cfg team-id)
(run! (partial write-project! cfg)
(bfc/get-team-projects cfg team-id))
(run! (partial write-file! cfg)
(bfc/get-team-files-ids cfg team-id))
(run! (partial write-storage-object! cfg)
(-> bfc/*state* deref :storage-objects))
(write! cfg :manifest "team-id" team-id)
(write! cfg :manifest "objects" (deref bfc/*state*))
(::path cfg))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "export"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))
(defn import-team!
[cfg path]
(let [id (uuid/next)
tp (ct/tpoint)
cfg (-> (create-database cfg path)
(assoc ::bfc/timestamp (ct/now)))]
(l/inf :hint "start"
:operation "import"
:id (str id)
:path (str (::path cfg)))
(try
(db/tx-run! cfg (fn [{:keys [::db/conn] :as cfg}]
(db/exec-one! conn ["SET idle_in_transaction_session_timeout = 0"])
(db/exec-one! conn ["SET CONSTRAINTS ALL DEFERRED"])
(binding [bfc/*state* (volatile! {:index {}})]
(let [objects (read-obj cfg :manifest "objects")]
;; We first process all storage objects, they have
;; deduplication so we can't rely on simple reindex. This
;; operation populates the index for all storage objects.
(run! (partial read-storage-object! cfg) (:storage-objects objects))
;; Populate index with all the incoming objects
(vswap! bfc/*state* update :index
(fn [index]
(-> index
(bfc/update-index (:teams objects))
(bfc/update-index (:projects objects))
(bfc/update-index (:files objects))
(bfc/update-index (:file-media-objects objects))
(bfc/update-index (:team-font-variants objects)))))
(let [team-id (read-obj cfg :manifest "team-id")
team (read-team! cfg team-id)
features (cfeat/get-team-enabled-features cf/flags team)
cfg (assoc cfg ::bfc/features features)]
(run! (partial read-project! cfg) (:projects objects))
(run! (partial read-file! cfg) (:files objects))
;; (run-pending-migrations! cfg)
team)))))
(finally
(pu/close! (::db cfg))
(let [elapsed (tp)]
(l/inf :hint "end"
:operation "import"
:id (str id)
:elapsed (ct/format-duration elapsed)))))))

View File

@ -988,7 +988,8 @@
:old-id (str (:id item))
::l/sync? true)
(db/insert! conn :file-media-object params)))
(db/insert! conn :file-media-object params
::db/on-conflict-do-nothing? (::bfc/overwrite cfg))))
(doseq [[index item] (d/enumerate thumbnails)]
(let [media-id (bfc/lookup-index (:media-id item))
@ -1037,41 +1038,18 @@
file-id')))
(defn- import-file-relations
"Import library relations from the manifest, ignoring relations whose
endpoints are not both part of the package. Of the rest, skip edges that
would close a dependency cycle or link a file to itself (same check as
`link-file-to-library`, see `bfc/library-link-status`). Relations are
processed in sorted manifest order and the first edge wins: an edge
that would close a cycle with previously imported relations is skipped
and logged."
[{:keys [::db/conn ::manifest ::bfc/timestamp] :as cfg}]
(let [relations (sort-by (fn [[file-id libr-id]] [(str file-id) (str libr-id)])
(:relations manifest))
(let [relations (:relations manifest)
file-ids (into #{} (map :id) (:files manifest))]
(doseq [[index [file-id libr-id]] (d/enumerate relations)]
(tap-progress cfg {:section :relations
:current (inc index)
:total (count relations)})
:current (inc index) :total (count relations)})
(if (and (contains? file-ids file-id)
(if (and (not= file-id libr-id)
(contains? file-ids file-id)
(contains? file-ids libr-id))
(let [file-id (bfc/lookup-index file-id)
libr-id (bfc/lookup-index libr-id)]
(case (bfc/library-link-status cfg file-id libr-id)
:self-link
(l/warn :hint "ignoring self-referencing library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
:circular
(l/warn :hint "ignoring circular library link"
:file-id (str file-id)
:lib-id (str libr-id)
::l/sync? true)
:ok
(do
(l/dbg :hint "create file library link"
:file-id (str file-id)
:lib-id (str libr-id)
@ -1079,7 +1057,7 @@
(let [rel-params {:file-id file-id
:library-file-id libr-id}]
(db/insert! conn :file-library-rel rel-params)
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))))
(bfc/upsert-file-library-sync! conn (assoc rel-params :synced-at timestamp))))
(l/warn :hint "ignoring invalid file library relation"
:file-id (str file-id)
:lib-id (str libr-id))))))
@ -1304,6 +1282,56 @@
{:file-ids file-ids
:resolution resolution})))
(defn- invalidate-thumbnails
[cfg file-id]
(let [storage (sto/resolve cfg ::db/reuse-conn true)
sql-1
(str "update file_tagged_object_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")
sql-2
(str "update file_thumbnail "
" set deleted_at = now() "
" where file_id=? returning media_id")]
(run! #(sto/touch-object! storage %)
(sequence
(keep :media-id)
(concat
(db/exec! cfg [sql-1 file-id])
(db/exec! cfg [sql-2 file-id]))))))
(defn- import-file-and-overwrite*
[{:keys [::manifest ::bfc/file-id] :as cfg}]
(when (not= 1 (count (:files manifest)))
(ex/raise :type :validation
:code :invalid-condition
:hint "unable to perform in-place update with binfile containing more than 1 file"
:manifest manifest))
(bfc/configure-import-transaction cfg)
(let [ref-file (bfc/get-minimal-file cfg file-id ::db/for-update true)
file (first (get manifest :files))
cfg (assoc cfg ::bfc/overwrite true)]
(vswap! bfc/*state* update :index assoc (:id file) file-id)
(binding [bfc/*options* cfg
bfc/*reference-file* ref-file]
(import-storage-objects cfg)
(import-file cfg file {:current 1 :total 1})
(invalidate-thumbnails cfg file-id)
(bfm/apply-pending-migrations! cfg)
{:file-ids [file-id]
:resolution {}})))
(defn- import-files
[{:keys [::bfc/timestamp ::bfc/input] :or {timestamp (ct/now)} :as cfg}]
@ -1354,7 +1382,9 @@
(tap-progress cfg {:section :manifest})
(binding [bfc/*state* (volatile! {:media [] :index {}})]
(db/tx-run! cfg import-files*))))
(if (::bfc/file-id cfg)
(db/tx-run! cfg import-file-and-overwrite*)
(db/tx-run! cfg import-files*)))))
;; --- PUBLIC API
@ -1423,12 +1453,6 @@
(defn import-files!
[{:keys [::bfc/input] :as cfg}]
(when (::bfc/file-id cfg)
(ex/raise :type :validation
:code :not-supported
:hint "in-place import is not supported"
:file-id (::bfc/file-id cfg)))
(assert
(and (uuid? (::bfc/profile-id cfg))
(uuid? (::bfc/project-id cfg)))

View File

@ -10,7 +10,6 @@
[app.common.data :as d]
[app.common.exceptions :as ex]
[app.common.flags :as flags]
[app.common.logging :as l]
[app.common.schema :as sm]
[app.common.time :as ct]
[app.common.uri :as u]
@ -159,10 +158,6 @@
[:http-server-io-threads {:optional true} ::sm/int]
[:http-server-max-worker-threads {:optional true} ::sm/int]
;; How often a websocket subscription re-checks that its access still
;; holds. The default lives in `app.http.websocket`, not here.
[:subscription-revalidation-interval {:optional true} ::ct/duration]
;; Explicit CORS allowlist used when the :cors flag is enabled.
;; Configured via PENPOT_ALLOWED_ORIGINS as a comma/whitespace
;; separated list of origins (e.g. "https://plugins.example.com").
@ -245,12 +240,6 @@
[:database-readonly {:optional true} ::sm/boolean]
[:database-min-pool-size {:optional true} ::sm/int]
[:database-max-pool-size {:optional true} ::sm/int]
;; pgjdbc prepared statement cache (see app.db). Size is in bytes and is
;; converted to the whole MiB the driver accepts. Absent keys mean the
;; driver defaults; see docs/technical-guide/configuration.md.
[:database-prepared-statement-cache-threshold {:optional true} [::sm/int {:min -1}]]
[:database-prepared-statement-cache-queries {:optional true} [::sm/int {:min 0}]]
[:database-prepared-statement-cache-size {:optional true} [::sm/int {:min 0}]]
[:quotes-teams-per-profile {:optional true} ::sm/int]
[:quotes-access-tokens-per-profile {:optional true} ::sm/int]
@ -419,22 +408,6 @@
(def explain-config
(sm/explainer schema:config))
(defn- normalize-file-data-backend
"The file data moved to `file_data` (migration `0157-migrate-legacy-file-data`), so
`file.data` and `file_change.data` are read-only leftovers and the `legacy-db`
back end is read-only: a write against it fails on every save. Accepting it would
start a healthy instance that cannot save a single file, so it reads as `db`,
which serves the legacy rows too. Warn instead of aborting: the operator may
still be halfway through the migration."
[config]
(if (= "legacy-db" (c/get config :file-data-backend))
(do
(l/warn :hint "file-data-backend=legacy-db is read-only, using db"
:file-data-backend "legacy-db"
:replacement "db")
(assoc config :file-data-backend "db"))
config))
(defn read-config
"Reads the configuration from environment variables and decodes all
known values."
@ -442,8 +415,7 @@
(->> (read-env prefix)
(merge default)
(normalize-tenant)
(decode-config)
(normalize-file-data-backend)))
(decode-config)))
(def version
(v/parse (or (some-> (io/resource "version.txt")

View File

@ -57,9 +57,7 @@
(def ^:private schema:pool-options
[:map {:title "pool-options"}
[::connection-timeout {:optional true} ::sm/int]
[::idle-timeout {:optional true} ::sm/int]
[::max-lifetime {:optional true} ::sm/int]
[::connect-timeout {:optional true} ::sm/int]
[::max-size {:optional true} ::sm/int]
[::min-size {:optional true} ::sm/int]
[::name {:optional true} :keyword]
@ -68,22 +66,10 @@
[::username {:optional true} :string]
[::validation-timeout {:optional true} ::sm/int]
[::read-only {:optional true} ::sm/boolean]
;; pgjdbc prepared statement cache. Absent keys mean the driver
;; defaults; the size is in bytes and is converted to whole MiB for
;; the driver (see create-datasource-config). The threshold is the
;; executions before a statement is server-prepared: 0 stops
;; server-side prepared statements (the cache of parsed queries stays
;; in use) and -1 prepares from the first execution. Any other
;; negative is a typo the driver would clamp to -1, so the schema
;; rejects it.
[::prepared-statement-cache-threshold {:optional true} [::sm/int {:min -1}]]
[::prepared-statement-cache-queries {:optional true} [::sm/int {:min 0}]]
[::prepared-statement-cache-size {:optional true} [::sm/int {:min 0}]]
[::mtx/metrics ::mtx/metrics]])
(def defaults
{::name :main
::min-size 0
::max-size 60
::connection-timeout 30000
::validation-timeout 10000
@ -106,11 +92,9 @@
:read-only read-only
:credentials (and (contains? cfg ::username)
(contains? cfg ::password))
:min-size (::min-size cfg)
:max-size (::max-size cfg)
:prepared-statement-cache-threshold (::prepared-statement-cache-threshold cfg)
:prepared-statement-cache-queries (::prepared-statement-cache-queries cfg)
:prepared-statement-cache-size (::prepared-statement-cache-size cfg))
:min-size (or (::min-size cfg)
(::max-size cfg))
:max-size (::max-size cfg))
(create-pool cfg))))
(defmethod ig/halt-key! ::pool
@ -124,27 +108,15 @@
(def initsql
(str "SET statement_timeout = 300000;\n"
"SET idle_in_transaction_session_timeout = 300000;\n"
;; lets the planner resolve a join between two partitioned tables
;; pairwise, when both are partitioned with the same bounds
"SET enable_partitionwise_join = on;"))
;; pgjdbc reads dataSourceProperties through Properties.getProperty, which
;; drops non-String values silently, so everything is passed as a string.
;; preparedStatementCacheSizeMiB only accepts a whole-MiB integer: the
;; configured size (bytes) is converted up, so any positive value keeps the
;; cache enabled and 0 disables the cache. A query parameter of the same name
;; in the URI wins over these properties: pgjdbc ignores the Properties value
;; when the URL carries it.
(defn- bytes->mib-ceil
[bytes]
(Math/ceilDiv (long bytes) (* 1024 1024)))
"SET idle_in_transaction_session_timeout = 300000;"))
(defn- create-datasource-config
[{:keys [::uri] :as cfg}]
;; (app.common.pprint/pprint cfg)
(let [config (HikariConfig.)]
(let [config (HikariConfig.)
max-size (::max-size cfg)
min-size (or (::min-size cfg) max-size)]
(doto config
(.setJdbcUrl (str "jdbc:" uri))
(.setPoolName (d/name (::name cfg)))
@ -154,18 +126,11 @@
(.setValidationTimeout (::validation-timeout cfg))
(.setIdleTimeout (::idle-timeout cfg))
(.setMaxLifetime (::max-lifetime cfg))
(.setMinimumIdle (::min-size cfg))
(.setMaximumPoolSize (::max-size cfg))
(.setMinimumIdle min-size)
(.setMaximumPoolSize max-size)
(.setConnectionInitSql initsql)
(.setInitializationFailTimeout -1))
(some->> (::prepared-statement-cache-threshold cfg) (str)
(.addDataSourceProperty config "prepareThreshold"))
(some->> (::prepared-statement-cache-queries cfg) (str)
(.addDataSourceProperty config "preparedStatementCacheQueries"))
(some->> (::prepared-statement-cache-size cfg) (bytes->mib-ceil) (str)
(.addDataSourceProperty config "preparedStatementCacheSizeMiB"))
(->> (mtx/get-registry (::mtx/metrics cfg))
(PrometheusMetricsTrackerFactory.)
(.setMetricsTrackerFactory config))

View File

@ -475,7 +475,7 @@
(def ^:private schema:invite-to-organization
[:map
[:invited-by ::sm/text]
[:user-name {:optional true} [:maybe ::sm/text]]
[:user-name [:maybe ::sm/text]]
[:token ::sm/text]
[:organization schema:organization-data]])
@ -497,7 +497,7 @@
(def ^:private schema:renewal-notice
[:map
[:user-name {:optional true} [:maybe :string]]
[:user-name [:maybe :string]]
[:renewal-date ::sm/text]
[:estimated-amount ::sm/text]
[:organizations [:vector schema:organization-data]]])

Some files were not shown because too many files have changed in this diff Show More