mirror of
https://github.com/penpot/penpot.git
synced 2026-11-19 08:30:42 +00:00
Compare commits
No commits in common. "develop" and "2.17.0-RC1" have entirely different histories.
develop
...
2.17.0-RC1
@ -1,536 +0,0 @@
|
||||
# Agentic development with opencode inside devenv
|
||||
|
||||
This doc shows how to run AI-assisted development for Penpot inside the
|
||||
devenv with [opencode](https://opencode.ai). It covers the setup once,
|
||||
then points to the skills that drive daily work.
|
||||
|
||||
Full reference lives in the technical guide:
|
||||
|
||||
- [Dev environment](../docs/technical-guide/developer/devenv.md)
|
||||
|
||||
This file does not repeat those guides. It gives the short path and
|
||||
leaves room for notes we add step by step.
|
||||
|
||||
## TL;DR
|
||||
|
||||
```bash
|
||||
./manage.sh pull-devenv
|
||||
./manage.sh run-devenv --ws 0 --attach
|
||||
```
|
||||
|
||||
Then open a shell in the container tmux session, run `opencode` inside
|
||||
~/penpot directory.
|
||||
|
||||
## 1. Introduction
|
||||
|
||||
The LLM client — opencode, Claude Code, or Codex — runs in a shell
|
||||
inside the plain devenv container, with the repo mounted and the
|
||||
skills in this folder driving the work. One client session per
|
||||
workspace (`ws0` is the live repo, `ws1+` are sibling clones).
|
||||
|
||||
This doc is written around opencode, but Claude Code runs the same
|
||||
way inside devenv and follows exactly the same flows. This is not
|
||||
the "agentic devenv" (`--agentic`) from the technical guide, which
|
||||
runs the client outside devenv and wires it in over MCP — here the
|
||||
client lives inside the sandboxed devenv docker.
|
||||
|
||||
The content lives at the paths the ecosystem agreed on: `AGENTS.md` at the
|
||||
repository root and the skills in this folder, `.agents/skills/`. Codex,
|
||||
opencode, Cursor, Zed, Amp, omp and pi read both without any setup.
|
||||
|
||||
Claude Code reads `AGENTS.md` from 2.1.277, through the built-in
|
||||
`agents-md` plugin, but only in a project that has no instruction file of
|
||||
its own. A `CLAUDE.md`, `.claude/CLAUDE.md` or `CLAUDE.local.md` anywhere
|
||||
from the repository root down to your working directory hands the whole
|
||||
project back to `CLAUDE.md`, and this file's guidance stops arriving. Keep
|
||||
none of them here. Personal steering goes in `AGENTS.local.md`, or in
|
||||
`.claude/rules/*.md` and `~/.claude/CLAUDE.md`, which the plugin does not
|
||||
count and which therefore leave the project guidance in place.
|
||||
|
||||
Skills are untouched by that plugin and still load from `.claude/skills`
|
||||
alone, which is not committed, so link it once per clone:
|
||||
|
||||
```bash
|
||||
mkdir -p .claude && ln -s ../.agents/skills .claude/skills
|
||||
# or, with Node:
|
||||
npx skills add ./.agents/skills --agent claude-code
|
||||
```
|
||||
|
||||
Below 2.1.277, and on Bedrock, Vertex and Foundry where the fallback has
|
||||
not arrived, add `ln -s AGENTS.md CLAUDE.md` and drop it once your client
|
||||
has the feature. The auto-update channel decides when that is: `latest`,
|
||||
the default, has it; `stable` runs about a week behind; Homebrew and the
|
||||
Linux packages update by hand. Personal skills go in
|
||||
`.agents/local/skills/<name>/`.
|
||||
|
||||
Every one of those paths is gitignored, so your own files keep working and
|
||||
no checkout overwrites them. Inside the devenv, `ws0` sees the links from
|
||||
the live checkout, and `ws1` and above are seeded on their first sync.
|
||||
|
||||
Unlike the agentic devenv, running the client inside the devenv
|
||||
docker gives it full access to the live environment: every
|
||||
dependency already resolved by the image, so the agent can write
|
||||
and run tests directly, query the running PostgreSQL, and reach
|
||||
the backend and frontend through nREPL — no proxies, no round
|
||||
trips outside the container.
|
||||
|
||||
And if you later want vision, it is one MCP entry away — a
|
||||
headless Playwright server in your `opencode.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"mcp": {
|
||||
"playwright": {
|
||||
"type": "local",
|
||||
"command": ["npx", "-y", "@playwright/mcp", "--headless"],
|
||||
"enabled": true,
|
||||
"env": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 2. Quickstart: bring up devenv and run opencode inside
|
||||
|
||||
Pull once, then bring up the workspace you want (add `--ws 1`,
|
||||
`--ws 2`, … for more):
|
||||
|
||||
```bash
|
||||
./manage.sh pull-devenv
|
||||
./manage.sh run-devenv --ws 0 --attach # ws0 (the live repo)
|
||||
```
|
||||
|
||||
This attaches to the container tmux session. Open a new shell there
|
||||
(`Ctrl+b c`), `cd` to the repo, and run opencode directly:
|
||||
|
||||
```bash
|
||||
cd penpot;
|
||||
opencode
|
||||
```
|
||||
|
||||
One session drives exactly one workspace — for N parallel workspaces,
|
||||
open one container shell and one opencode session per workspace.
|
||||
|
||||
Stop with `./manage.sh stop-devenv [--ws N | --all]`. Shared infra
|
||||
stops when the last workspace stops.
|
||||
|
||||
|
||||
## 3. Connecting providers
|
||||
|
||||
### Starting point: Zen free models, no login needed
|
||||
|
||||
The easiest way to try the setup is Zen's free models — they need no
|
||||
login and no key. Just run opencode, pick a free model, and work.
|
||||
|
||||
That said, creating a Zen account and connecting with your API key is
|
||||
still worth it from day one: it unlocks the full model list, spend
|
||||
limits, auto-reload, and the Go-overflow fallback below. Connect
|
||||
through the TUI:
|
||||
|
||||
- Run `/connect`, pick a provider, paste the key.
|
||||
- Run `/models` to see what that provider offers.
|
||||
|
||||
### OpenCode Go (subscription, best value for daily use)
|
||||
|
||||
$10/month subscription with generous included usage — up to $60/month
|
||||
of model consumption at published per-token rates, depending on the
|
||||
model. Best value if you work mostly with open coding models
|
||||
(GLM, Kimi, Qwen, DeepSeek, MiniMax, LongCat…).
|
||||
|
||||
The strong part: when you hit a model's monthly limit, Go can fall
|
||||
back to your Zen balance instead of blocking (enable *Use balance* in
|
||||
the console). So the setup many of us use is Go + Zen credit on top —
|
||||
subscription first, pay-as-you-go overflow after.
|
||||
|
||||
### OpenCode Zen (pay per use)
|
||||
|
||||
Zen is the opencode team's gateway: curated models tested for coding
|
||||
agents, fair prices, no markups, stable latency. You top up credit
|
||||
and pay per request, with monthly spend limits and auto-reload.
|
||||
|
||||
Free usage is generous: the free models carry limits good enough for
|
||||
real work, not just a quick taste. Worth knowing: brand-new,
|
||||
unannounced models often show up on Zen first with a very generous
|
||||
free quota so people try them — e.g. `0x Alpha`, which later turned
|
||||
out to be GLM-5.3-Flash. Keep an eye on the free list; the newest
|
||||
entry is often the best deal.
|
||||
|
||||
Two reasons to have a Zen account even with Go:
|
||||
|
||||
- It absorbs Go overflow (see above).
|
||||
- Its free models let you try the whole setup before paying.
|
||||
|
||||
### OpenRouter (widest catalog)
|
||||
|
||||
If you already have an account, connect it: the widest model range in
|
||||
one place. Trade-off is latency and occasional instability versus Zen,
|
||||
which is tuned for coding agents.
|
||||
|
||||
Beyond code: OpenRouter also serves image, video, and audio models.
|
||||
opencode itself cannot call those directly — it is built for code —
|
||||
but a cheap model can quickly build you a small tool or script that
|
||||
talks to them through the OpenRouter API. So if you also generate
|
||||
content other than code and text, having OpenRouter connected is
|
||||
worth it: the agent wires the plumbing for you.
|
||||
|
||||
### OpenAI (subscription or API key)
|
||||
|
||||
If you have an OpenAI subscription or API access, connect it — it
|
||||
works very well as a daily driver alongside (or instead of) Go/Zen.
|
||||
|
||||
### Suggested combos
|
||||
|
||||
| Profile | Connect |
|
||||
|---|---|
|
||||
| Try it out | Nothing (Zen free models, no login) |
|
||||
| Try it out, properly | Zen account + API key (free models + limits) |
|
||||
| Daily use, best value | Go + Zen credit (overflow) |
|
||||
| Widest model choice | Add OpenRouter |
|
||||
| Already pay OpenAI | Add OpenAI account |
|
||||
|
||||
## 4. Recommended models
|
||||
|
||||
Personal picks from Andrey, current as of September 2026. Models come
|
||||
and go, so treat this as a snapshot — the shape (one cheap solver,
|
||||
one reviewer/planner, one explorer) matters more than the names.
|
||||
|
||||
| Model | Role | How often |
|
||||
|---|---|---|
|
||||
| Muse Spark 1.3 (`high`) | Main solver: plan, review, develop. Sharp and cheap — covers ~70% of coding tasks. | Daily |
|
||||
| GLM-5.3-Flash | Reasoning all-rounder, now mostly code/plan reviewer and planner. | Daily |
|
||||
| DeepSeek V4.1 Flash (`high`) | Explorer: code and idea exploration, sometimes development. Especially good at small bash/node utilities for repo chores and changelog updates. | Daily |
|
||||
| LongCat 2.0 | Backup solver, occasional stand-in for Muse Spark 1.3. | Weekly |
|
||||
| GPT-5.6 Luna | Alternative to DeepSeek Flash; pricier, unclear the extra cost pays off. | Rarely |
|
||||
| Qwen3.8 Flash | As strong as the top three; used in rotation to avoid hammering one model. Less Go subsidy than the top picks, so mostly in overflow mode. | Overflow |
|
||||
| MiMo-V2.5-Pro | Former main model; slightly pricier now next to Muse Spark / GLM-Flash / LongCat, and less Go subsidy — used in overflow. | Overflow |
|
||||
| Kimi K3 | Heavy reasoning for hard reviews and plans. Expensive, ~1% of tasks. | Rarely |
|
||||
| GLM-5.3 | Same slot as Kimi K3: hard reviews and plans only. | Rarely |
|
||||
|
||||
**TL;DR:** the first three (Muse Spark 1.3, GLM-5.3-Flash, DeepSeek
|
||||
V4.1 Flash) are a good starting point.
|
||||
|
||||
## 5. Customizing your `opencode.json`
|
||||
|
||||
opencode merges config in this order (later wins):
|
||||
|
||||
1. Global: `~/.config/opencode/opencode.json` (on host, or the dir
|
||||
mounted with `--opencode-config-dir` inside devenv —
|
||||
see §9 Advanced usage).
|
||||
2. Project: `opencode.json` at the repo root (gitignored on purpose —
|
||||
use it to override the global entries for one workspace).
|
||||
|
||||
Below is a full working example of my personal config at the date of
|
||||
writing this. It is only an example: define whatever subagents you
|
||||
need, with whatever models you like or work with.
|
||||
|
||||
Copy it to `opencode.json` on the root of the repo:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"disabled_providers": ["amazon-bedrock"],
|
||||
"subagent_depth": 2,
|
||||
"agent": {
|
||||
"compaction": {
|
||||
"model": "opencode-go/deepseek-flash",
|
||||
"variant": "high"
|
||||
},
|
||||
"title": {
|
||||
"model": "opencode-go/deepseek-flash",
|
||||
"variant": "low"
|
||||
},
|
||||
"explore": {
|
||||
"model": "opencode-go/deepseek-flash",
|
||||
"variant": "high"
|
||||
},
|
||||
"build": {
|
||||
"prompt": "{file:.agents/prompts/engineer-agent-prompt.md}",
|
||||
"permission": {
|
||||
"external_directory": {
|
||||
"/tmp/**": "allow"
|
||||
}
|
||||
}
|
||||
},
|
||||
"general": {
|
||||
"prompt": "{file:.agents/prompts/engineer-agent-prompt.md}",
|
||||
"permission": {
|
||||
"external_directory": {
|
||||
"/tmp/**": "allow"
|
||||
}
|
||||
}
|
||||
},
|
||||
"engineer-glm": {
|
||||
"mode": "subagent",
|
||||
"model": "opencode-go/glm-5.3-flash",
|
||||
"variant": "high",
|
||||
"prompt": "{file:.agents/prompts/engineer-agent-prompt.md}",
|
||||
"permission": {
|
||||
"*": "allow",
|
||||
"task": {
|
||||
"*": "allow"
|
||||
}
|
||||
}
|
||||
},
|
||||
"engineer-kimi": {
|
||||
"mode": "subagent",
|
||||
"model": "opencode/kimi-k3",
|
||||
"variant": "high",
|
||||
"prompt": "{file:.agents/prompts/engineer-agent-prompt.md}",
|
||||
"permission": {
|
||||
"*": "allow",
|
||||
"task": {
|
||||
"*": "allow"
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
"engineer-qwen": {
|
||||
"mode": "subagent",
|
||||
"model": "opencode-go/qwen3.7-plus",
|
||||
"variant": "high",
|
||||
"prompt": "{file:.agents/prompts/engineer-agent-prompt.md}",
|
||||
"permission": {
|
||||
"*": "allow",
|
||||
"task": {
|
||||
"*": "allow"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
What the blocks mean:
|
||||
|
||||
- `compaction` / `title` / `explore`: cheap background agents. Keep
|
||||
them on a fast model; `title` uses the `low` variant on purpose.
|
||||
- `build` / `general`: the main agents. They load the shared prompt
|
||||
`{file:.agents/prompts/engineer-agent-prompt.md}` and may only touch
|
||||
`/tmp/**` outside the repo without asking for explicit permision.
|
||||
- `engineer-*`: one subagent per model family, all with the same
|
||||
prompt and full permissions (`"*": "allow"`). They purpose are
|
||||
specially for delegate work to them because are defined to be used
|
||||
only as subagents.
|
||||
- `disabled_providers` / `subagent_depth`: global guards. Keep
|
||||
`"$schema"` — opencode refuses to start if any field is wrong.
|
||||
|
||||
How the `engineer-*` subagents are actually used — delegating work to
|
||||
them to keep the main context clean — is covered in §6 Common agentic
|
||||
flows.
|
||||
|
||||
Note this is opencode-only: other clients have their own way of
|
||||
defining subagents or helpers — or none at all.
|
||||
|
||||
## 6. Common agentic flows
|
||||
|
||||
Work happens two ways: directly in your session, or delegated to a
|
||||
subagent. Besides the `engineer-*` subagents from §5 there is a
|
||||
builtin `general` subagent. Delegating planning and review to a
|
||||
subagent starts a fresh, clean context with a clean prompt instead
|
||||
of growing the main session — the main lever for keeping context
|
||||
small. To delegate without switching models, delegate to `general`.
|
||||
|
||||
### Issue / error report flow
|
||||
|
||||
1. **Frame the problem.** Enter Plan mode (TAB in opencode) and paste the
|
||||
report with your intent: "investigate this and find the possible cause",
|
||||
"investigate and tell me where this points", or "does this still apply?".
|
||||
Explore until you and the agent roughly agree on the problem.
|
||||
2. **Write the plan.** Invoke `@make-a-plan`. Skills use the current agent;
|
||||
select Build mode first when the skill needs to write files. Use Plan mode
|
||||
only when you want a hard guarantee that the agent modifies no file under
|
||||
any circumstance. If you
|
||||
explored with a weaker model but want a stronger one to write the
|
||||
plan, switch models first or delegate:
|
||||
`@make-a-plan delegate to @engineer-glm`.
|
||||
3. **Iterate on the plan.** The plan is saved to `.agents/plans/`, so you
|
||||
never depend on LLM memory: read the file directly, or invoke `@review-plan`
|
||||
for a second opinion (delegation works here too). It applies clear plan
|
||||
improvements and discusses consequential decisions one by one before
|
||||
applying them. Complex plans deserve a review; simple ones can skip it.
|
||||
4. **Execute.** Invoke `@implement-plan`. It first prints the full picture —
|
||||
whether it will create an issue and a branch, the execution style, and a
|
||||
task checklist — and waits for your go-ahead. Say "step by step" to stop
|
||||
after each task (one commit per task) so you can verify as it goes;
|
||||
the default runs all tasks with one final commit. For a plan based on a
|
||||
code review, it then asks whether to have the original reviewer verify the
|
||||
corrections. If that session is unavailable, it tells you and uses a new
|
||||
reviewer with the review and implementation context.
|
||||
5. **Land the work.** When it finishes, either push yourself and invoke
|
||||
`@create-pr`, or loop `@review-code` → `@make-a-plan` →
|
||||
`@implement-plan` until the findings are addressed, then push and invoke
|
||||
`@create-pr`. Nothing pushes for you — you always push from your shell.
|
||||
|
||||
> Note: `@implement-plan` checks the current branch. On a base branch
|
||||
> (`main`, `develop`, `staging`) it creates a GitHub issue and a branch
|
||||
> `issue-NNNN`; on an existing feature branch it continues there and
|
||||
> creates nothing. The pre-run summary tells you which applies. Read the
|
||||
> skill at `.agents/skills/implement-plan/SKILL.md` — it is
|
||||
> self-explanatory.
|
||||
|
||||
### Big feature with multiple plans
|
||||
|
||||
When the work is too large for a single plan, tell `@make-a-plan`
|
||||
up front: produce a high-level roadmap where each task will get its
|
||||
own execution plan, and the roadmap doubles as the progress tracker.
|
||||
|
||||
From there the flow mirrors the issue flow above, one level down:
|
||||
take each roadmap task in turn, write its own plan (`@make-a-plan`,
|
||||
delegating when it helps), review it when the task is complex
|
||||
(`@review-plan`), implement it (`@implement-plan`), and mark progress
|
||||
on the roadmap as you land each piece.
|
||||
|
||||
### Plan file naming (base + derivatives)
|
||||
|
||||
Base plans live in `.agents/plans/` as `YYYY-MM-DD-<slug>.md`.
|
||||
Derived plans reuse the parent basename verbatim and append one
|
||||
suffix per level with `--`, with no new date — the parent prefix
|
||||
keeps everything adjacent in `ls`:
|
||||
|
||||
- Review followup on implemented work:
|
||||
`2026-09-14-paste-before-init-crash.md` →
|
||||
`2026-09-14-paste-before-init-crash--review-01.md`
|
||||
- Roadmap sub-plan (task number from the roadmap):
|
||||
`2026-09-20-upload-pipeline-roadmap.md` →
|
||||
`2026-09-20-upload-pipeline-roadmap--task-01-chunk-upload.md`
|
||||
- Chained: `...--task-02-gc--review-01.md`
|
||||
|
||||
While a plan is still unimplemented, `@make-a-plan` revises it in
|
||||
place. Once implemented and reviewed, it writes a new followup file.
|
||||
Full rules live in the `planner` skill.
|
||||
|
||||
### Plan status lifecycle
|
||||
|
||||
Every plan carries `Status: draft | reviewed | done` plus an
|
||||
append-only `Review Log` (UTC ISO 8601, one line per entry):
|
||||
|
||||
- `draft` — fresh from `@make-a-plan`.
|
||||
- `reviewed` — review feedback incorporated. `@review-plan` applies clear
|
||||
improvements and agreed decisions, then updates the status and log.
|
||||
- `done` — `@implement-plan` closes it on completion, logging the
|
||||
issue URL when one exists (never commit hashes).
|
||||
|
||||
A plan approved with no changes goes `draft` → `done` directly.
|
||||
|
||||
A plan is never staged or committed: the `done` status and its
|
||||
`Review Log` are session bookkeeping, not part of the change. Their
|
||||
absence from the commit is expected.
|
||||
|
||||
## 7. Connecting `gh` CLI with a token
|
||||
|
||||
The `create-issue` and `create-pr` flows need an authenticated `gh`
|
||||
so they can run on their own. Create a fine-grained token with the
|
||||
minimum scopes:
|
||||
|
||||
1. GitHub → Settings → Developer settings → Personal access tokens →
|
||||
Fine-grained tokens → Generate new token.
|
||||
2. Under Organization permissions, grant access to **Projects**.
|
||||
3. Under Repository permissions, grant at least **Issues** and
|
||||
**Pull requests**.
|
||||
|
||||
Then authenticate the CLI and follow the prompts:
|
||||
|
||||
```bash
|
||||
gh auth login
|
||||
```
|
||||
|
||||
Verify with `gh auth status` (token lives in
|
||||
`~/.config/gh/hosts.yml`). You still push from your own shell — the
|
||||
agents only read and open issues and PRs.
|
||||
|
||||
## 8. Troubleshooting / FAQ
|
||||
|
||||
> TBD — filled in step by step as issues come up.
|
||||
|
||||
## 9. Advanced usage
|
||||
|
||||
### Personal agents and prompts without committing them here
|
||||
|
||||
Bind-mount a host dir over the container's `~/.config/opencode`:
|
||||
|
||||
```bash
|
||||
./manage.sh run-devenv --ws 0 --opencode-config-dir ../penpot-opencode
|
||||
```
|
||||
|
||||
It applies at container creation, so changing it needs a stop + rerun
|
||||
of that instance.
|
||||
|
||||
## Summary of available skills
|
||||
|
||||
### How the skills are organized
|
||||
|
||||
**Flows** are the six skills you invoke by name. Each one covers one step
|
||||
in the life of a change: plan it, review the plan, implement it, review
|
||||
the code, open the pull request.
|
||||
|
||||
**References** hold the quality standards. A flow's reviewer loads them;
|
||||
you rarely touch them directly.
|
||||
|
||||
**Procedures** define how one concrete step is done — a plan document, an
|
||||
issue, a commit. Flows call them, but they also work on their own.
|
||||
|
||||
**Utilities** are small helpers for everyday work: search, file lookup,
|
||||
JSON, REPL access, and so on.
|
||||
|
||||
### Flows
|
||||
|
||||
| Skill | What it does | When you would say |
|
||||
|---|---|---|
|
||||
| [`make-a-plan`](skills/make-a-plan/SKILL.md) | Researches the task, writes an implementation plan, asks you the open questions in plain language, and saves the plan to `.agents/plans/`. | "make a plan for the token refresh bug" |
|
||||
| [`review-plan`](skills/review-plan/SKILL.md) | Evaluates a plan before anyone writes code: completeness, ordering, risks. Approves it or asks for changes. | "review this plan before we start" |
|
||||
| [`implement-plan`](skills/implement-plan/SKILL.md) | Shows you the full flow first — the issue and branch it will create (or the branch it continues on), the execution style, and the task checklist — and, after your go-ahead, executes a ready plan. Default: every task, one commit. On request ("step by step"): one task, one commit, your confirmation between tasks. On request ("direct"): no issue and no branch, commits on the current branch. | "implement the plan" · "step by step, one commit per task" · "direct, no branch" |
|
||||
| [`review-code`](skills/review-code/SKILL.md) | Reviews a diff, branch, or PR and returns findings ranked by impact. | "review my changes before I push" |
|
||||
| [`create-pr`](skills/create-pr/SKILL.md) | Opens a pull request for the current branch — with checks on base branch, commits, issue, and push state — or updates an existing PR's title and description. | "open a PR for this branch" |
|
||||
| [`resolve-git-conflicts`](skills/resolve-git-conflicts/SKILL.md) | Untangles merge or rebase conflicts: explains both sides, proposes a resolution, applies it after you approve. Never runs `git rebase --continue`. | "resolve these conflicts" |
|
||||
|
||||
### References
|
||||
|
||||
| Skill | What it holds |
|
||||
|---|---|
|
||||
| [`plan-review-criteria`](skills/plan-review-criteria/SKILL.md) | The plan review rubric: six axes, severity levels, approval standard, output format. The `review-plan` reviewer loads it. |
|
||||
| [`code-review-criteria`](skills/code-review-criteria/SKILL.md) | The code review rubric: five axes, core principles (DRY, KISS, YAGNI), severity format, verdict. The `review-code` reviewer loads it. |
|
||||
|
||||
### Procedures
|
||||
|
||||
| Skill | What it does |
|
||||
|---|---|
|
||||
| [`planner`](skills/planner/SKILL.md) | The spec of a good plan: context, architecture decisions, tasks with acceptance criteria, checkpoints. Used by `make-a-plan`. |
|
||||
| [`create-issue`](skills/create-issue/SKILL.md) | Creates a GitHub issue that follows Penpot conventions. Used by `implement-plan`; also works on its own. |
|
||||
| [`create-commit`](skills/create-commit/SKILL.md) | Makes a commit the Penpot way: emoji subject, clear body, `AI-assisted-by` trailer. Used by `implement-plan`; also works alone when you say "commit this". |
|
||||
|
||||
### Utilities
|
||||
|
||||
| Skill | What it does |
|
||||
|---|---|
|
||||
| [`bat-cat`](skills/bat-cat/SKILL.md) | Read files in the terminal with syntax highlighting and line numbers. |
|
||||
| [`fd-find`](skills/fd-find/SKILL.md) | Find files by name or pattern, respecting `.gitignore`. |
|
||||
| [`ripgrep`](skills/ripgrep/SKILL.md) | Fast content search with regular expressions. |
|
||||
| [`jq-json-processor`](skills/jq-json-processor/SKILL.md) | Slice, filter, and reshape JSON output. |
|
||||
| [`nrepl-eval`](skills/nrepl-eval/SKILL.md) | Run Clojure or ClojureScript code in the live REPL sessions (backend and frontend). |
|
||||
| [`testing`](skills/testing/SKILL.md) | The repo's testing rules and TDD workflow, loaded before writing tests. |
|
||||
| [`local-ci`](skills/local-ci/SKILL.md) | Run CI-style lint, test, and format checks for the modules you touched with `scripts/ci`, and read the logs when they fail. |
|
||||
| [`security-and-hardening`](skills/security-and-hardening/SKILL.md) | Security checks for code that handles user input, auth, or external services. |
|
||||
| [`ste`](skills/ste/SKILL.md) | Rewrites prose in Simplified Technical English. Loads only when you name it. |
|
||||
| [`refine-prompt`](skills/refine-prompt/SKILL.md) | Rewrites a rough prompt into a clearer one. Never runs the prompt. |
|
||||
| [`update-changelog`](skills/update-changelog/SKILL.md) | Regenerates `CHANGES.md` from a GitHub milestone. |
|
||||
| [`find-project-anomalies`](skills/find-project-anomalies/SKILL.md) | Checks a milestone against the Main project board and reports anomalies to `tmp/<MILESTONE>-ANOMALIES.md`. |
|
||||
|
||||
### A typical round
|
||||
|
||||
1. `@make-a-plan` — you get a plan and a saved file in `.agents/plans/`.
|
||||
2. `@review-plan` — a second opinion; approve or request changes.
|
||||
3. `@implement-plan` — the code gets written and committed. Starting from a base branch, it also opens the GitHub issue and the `issue-NNNN` branch; the plans that follow continue on that same branch.
|
||||
4. `@review-code` — a reviewer checks the commit.
|
||||
5. `@create-pr` — the branch goes up as a pull request.
|
||||
|
||||
Every step also works on its own, and you can always say what you want
|
||||
in plain words — the agents pick the right skill from what you say.
|
||||
|
||||
### Adding or changing a skill
|
||||
|
||||
Create a folder here with a `SKILL.md` inside. The file needs `name` and
|
||||
`description` in its frontmatter, and a clear "When to use" section so
|
||||
agents know when to reach for it. Keep one job per skill, and keep the
|
||||
two families apart: flows are named with a verb first; reference skills
|
||||
end in `-criteria`.
|
||||
@ -1,15 +0,0 @@
|
||||
Act as a senior full-stack software engineer for this project.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. Read `AGENTS.md` first and follow its memory-reading rules: read `mem:critical-info`, then the core memory of every module your work touches, plus any deeper memories they reference.
|
||||
2. Work autonomously: explore the codebase first, follow existing patterns and conventions, apply DRY/KISS.
|
||||
3. Verify before finishing: run tests, lint and fm, fix anything you broke. Never report done with failing checks.
|
||||
4. Before finishing, review the affected memories and documentation against the implementation. If the change introduces behavior, contracts, decisions, or constraints that are not documented, or makes existing documentation inaccurate, update the relevant memories and docs in the same change.
|
||||
|
||||
## Strong Rules
|
||||
|
||||
1. All new functionality ships with tests. No exceptions.
|
||||
2. Do not touch unrelated modules.
|
||||
3. Never `git push`, force-push, or modify remotes. Only create commits when the
|
||||
command or the user explicitly instructs it.
|
||||
@ -1,95 +0,0 @@
|
||||
---
|
||||
name: code-review-criteria
|
||||
description: Code review criteria for correctness, clarity, architecture, security and performance. Loaded by the review-code subagent.
|
||||
---
|
||||
|
||||
# Code Review Criteria
|
||||
|
||||
Assess whether the change improves the codebase and is safe to merge. Review
|
||||
the diff in context, its intent and relevant tests. Follow project conventions
|
||||
and module memories. Do not block on personal preferences or invent problems.
|
||||
Apply each axis in proportion to the change and its risks.
|
||||
|
||||
Follow `review-code` for delegation and its read-only contract. Never run tests,
|
||||
builds, linters or other state-changing checks during a review or follow-up.
|
||||
Inspect prior test results or CI evidence when available. Identify their source
|
||||
and scope; if evidence is missing, stale or inaccessible, say what is
|
||||
unverified. Never claim a suite passed without evidence.
|
||||
|
||||
## Review axes
|
||||
|
||||
### Correctness and tests
|
||||
|
||||
Check requirements, relevant boundaries, error paths, concurrency and state.
|
||||
Use the module's contracts to decide where validation belongs; do not demand
|
||||
redundant guards inside functions with established preconditions. Check that
|
||||
new functionality has tests covering its behavior and important failure
|
||||
cases, using the appropriate layer and project testing guidance. Missing
|
||||
required tests are findings; documentation-only changes need not add tests.
|
||||
Do not infer that a test failed before a fix without evidence.
|
||||
|
||||
### Readability and simplicity
|
||||
|
||||
Check names, control flow and maintainability against existing conventions.
|
||||
Flag duplication when it creates a concrete risk of drift or maintenance
|
||||
cost, not merely because code appears more than once. In tests, prefer
|
||||
self-contained, readable cases even when they repeat setup. Suggest the
|
||||
simplest remedy that preserves behavior.
|
||||
|
||||
### Architecture and compatibility
|
||||
|
||||
Check module boundaries, existing abstractions, contracts and unnecessary
|
||||
complexity. Review surrounding code when needed to understand the change.
|
||||
Do not label a compatibility shim dead code without checking its purpose.
|
||||
For file formats or data model changes, consult the relevant compatibility,
|
||||
migration and feature memories; for Clojure, logging and translations, read
|
||||
the corresponding guidance when relevant.
|
||||
|
||||
### Security
|
||||
|
||||
Evaluate relevant input boundaries, authorization, secrets, data handling and
|
||||
external integrations. Consult project security guidance for the affected
|
||||
technology; do not assume a generic framework or tool applies to this module.
|
||||
|
||||
### Performance
|
||||
|
||||
Assess relevant hot paths, resource bounds, data access and scaling. Explain
|
||||
the workload or conditions in which a problem matters; do not flag speculative
|
||||
optimizations without a concrete risk.
|
||||
|
||||
## Scope and dependencies
|
||||
|
||||
Review relevant generated files or lockfile changes when they affect the
|
||||
contract, dependencies or security; do not skip a lockfile-only dependency
|
||||
change. Check dependency changes against available changelogs, project tools
|
||||
and relevant lockfile diffs. Treat size as a signal to assess reviewability,
|
||||
not an automatic finding. Identify unrelated work in the target diff when it
|
||||
materially impedes review; do not review unrelated working-tree changes.
|
||||
|
||||
## Findings and verdict
|
||||
|
||||
Assign each finding a stable ID (`F1`, `F2`, …), independent of severity.
|
||||
Never renumber it within the review or follow-up; severity may change as
|
||||
evidence changes. New findings receive unused IDs.
|
||||
|
||||
For each finding state severity, file and location, concrete circumstances,
|
||||
impact, remedy and the tests needed to verify it; explain when no new test is
|
||||
needed. If a consequential decision requires user input, give the real options,
|
||||
consequences and recommendation within that finding, without pausing to ask.
|
||||
|
||||
- **Critical:** security incident, data loss or comparably severe breakage;
|
||||
blocks merge.
|
||||
- **High:** a significant defect or missing required test; must be addressed.
|
||||
- **Medium:** a substantive issue worth fixing but not a merge blocker.
|
||||
- **Low / Suggestion:** an optional improvement with a concrete benefit.
|
||||
|
||||
Base severity on impact and project requirements. Missing tests required for
|
||||
new functionality must be addressed; do not invent a test requirement for
|
||||
changes where none applies.
|
||||
|
||||
Return a short summary, findings ordered by impact and a verdict with the
|
||||
relevant IDs: **Approve**, **Approve with minor changes**, or **Request
|
||||
changes**. State verification evidence and limitations. Do not duplicate
|
||||
remedies or tests in separate sections. If no findings exist, say so. In
|
||||
follow-up, retain original IDs and
|
||||
report resolved, pending and not-addressed findings as `review-code` requires.
|
||||
@ -1,64 +0,0 @@
|
||||
---
|
||||
name: create-commit
|
||||
description: Stage, review, and commit files following Penpot commit conventions.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Skill: create-commit
|
||||
|
||||
Produce a git commit that follows Penpot's commit message conventions. This
|
||||
skill handles staging review and safety checks — it does not
|
||||
implement features or push.
|
||||
|
||||
## When to Use
|
||||
|
||||
- After code changes are complete and files need to be committed
|
||||
- When delegated by a workflow step (e.g. implement-plan) to handle the commit
|
||||
|
||||
## Required Reading
|
||||
|
||||
Before drafting any commit, read `mem:workflow/creating-commits` end-to-end. It
|
||||
is the authoritative source for the commit message format, the emoji menu,
|
||||
subject/body limits, and the `AI-assisted-by` trailer. Follow it exactly.
|
||||
|
||||
## Iron Rules (non-negotiable)
|
||||
|
||||
Follow the message rules and exceptions in `mem:workflow/creating-commits`.
|
||||
**Run `./scripts/check-commit` and require exit code 0.** A non-zero exit is a
|
||||
hard blocker: fix the message and re-check it before reporting success.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Stage the files** specified by the calling context. Do not ask for
|
||||
confirmation.
|
||||
2. Run `git diff --staged` to review the content. If you see secrets (API keys,
|
||||
tokens, passwords, private keys, `.env` values), debug prints, or anything
|
||||
that does not match the stated intent, **STOP** and tell the user before
|
||||
committing.
|
||||
3. Draft the message following the format in the memory doc, wrapping the body
|
||||
at 80 characters per line, and run:
|
||||
```bash
|
||||
git commit -m "<subject>" -m "<body>"
|
||||
```
|
||||
(or `git commit -F -` if the body has unusual characters).
|
||||
4. **Verify the message with the checker**:
|
||||
```bash
|
||||
./scripts/check-commit
|
||||
```
|
||||
If it fails, amend the message (`git commit --amend`) until it passes. Do
|
||||
not finish with a failing checker.
|
||||
5. The `AI-assisted-by` trailer value is provided by the calling context — use
|
||||
it verbatim.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do not push. Pushing is a separate workflow handled by the user.
|
||||
- Do not run `git reset`, `git checkout`, `git restore`, `git clean`, or `rm`.
|
||||
- Do not pass `--author`. Author identity comes from the local git config.
|
||||
- Do not amend a commit you did not create in this session, unless explicitly asked.
|
||||
- Do not bypass pre-commit hooks (`--no-verify`) unless explicitly asked.
|
||||
- Add `Co-authored-by:` only for a human co-author. AI providers, agents and
|
||||
bots are rejected by `./scripts/check-commit`; for AI participation use the
|
||||
`AI-assisted-by:` trailer.
|
||||
- Do not add untracked files that were not created in this session.
|
||||
- Do not skip the `scripts/check-commit` verification step.
|
||||
@ -1,31 +0,0 @@
|
||||
---
|
||||
name: create-issue
|
||||
description: Create or update GitHub issues (from PR, from draft body, retitle existing). Routes to the canonical flow in `mem:workflow/creating-issues`.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Skill: create-issue
|
||||
|
||||
Entry point for all GitHub issue work. All rules (title derivation, metadata,
|
||||
body templates, Issue Type IDs), all flows, and all `gh` / GraphQL commands
|
||||
live in `mem:workflow/creating-issues` (file:
|
||||
`.serena/memories/workflow/creating-issues.md`). This skill routes to the
|
||||
right flow.
|
||||
|
||||
## When to Use
|
||||
|
||||
- **Create from PR** — PR exists; the issue is the changelog/release unit,
|
||||
the PR is the implementation. Issue = WHAT, PR = HOW.
|
||||
→ memory section **Creating Issues from PRs**
|
||||
- **Create from draft body** — Taiga story, user report, discussion; no PR
|
||||
yet.
|
||||
→ memory section **Creating Issues from Draft Body**
|
||||
- **Create as sub-issue** — the issue must be grouped under an umbrella/EPIC
|
||||
issue; create it first, then link it to its parent.
|
||||
→ memory section **Adding an Issue as a Sub-issue**
|
||||
- **Retitle existing issue** — current title is vague, prefixed, or stale.
|
||||
→ memory section **Retitling an Existing Issue**
|
||||
|
||||
Everything else (title derivation, metadata policy, body templates, Issue
|
||||
Type IDs, create/verify/cleanup commands) lives in the memory — go to the
|
||||
matching section there.
|
||||
@ -1,133 +0,0 @@
|
||||
---
|
||||
name: create-pr
|
||||
description: PR flow — open a new PR for the current task branch (validates base branch, commits, issue and push state) or update an existing PR's title or description to match Penpot conventions. Use it when the user asks to open or create a PR, in any phrasing.
|
||||
---
|
||||
|
||||
# Create PR
|
||||
|
||||
Two modes. **Open mode** takes the current task branch to a new, validated
|
||||
PR. **Update mode** rewrites an existing PR's title or description. Gather
|
||||
information, validate, and act in one pass. If validation fails, STOP with a
|
||||
single coherent message that lists every problem and states exactly what
|
||||
information is missing — never fix or work around problems silently.
|
||||
|
||||
Both modes require an authenticated `gh` CLI (`gh auth status`) and never
|
||||
push — the user pushes from their own shell.
|
||||
|
||||
## When to use
|
||||
|
||||
- The user asks to open or create a NEW PR for the current task branch, in
|
||||
any phrasing ("open a PR", "create the pull request", "put this up for
|
||||
review") — or invokes `@create-pr`. → **Open mode**.
|
||||
- The user asks to fix or update an EXISTING PR's title or description to
|
||||
match conventions. → **Update mode**.
|
||||
|
||||
If the running agent cannot write (for example, the plan agent), say so and
|
||||
stop — this skill needs the build agent.
|
||||
|
||||
## Open mode
|
||||
|
||||
### 1. Gather context (read-only)
|
||||
|
||||
- Current branch: `git rev-parse --abbrev-ref HEAD`.
|
||||
- Target base branch: run `./scripts/detect-target-branch` from the repo root.
|
||||
It prints the nearest ancestor branch of HEAD (exit 0) or fails (exit 1).
|
||||
- Commits: `git log --oneline <base>..HEAD`.
|
||||
- Push state (local): `git rev-parse --verify origin/<branch>` and compare
|
||||
with HEAD. It reads the local remote-tracking ref — no network, no SSH. It
|
||||
reflects the last push or fetch this clone knows about.
|
||||
- Issue: from the session context, or from the branch name — `issue-NNNN`
|
||||
maps to issue NNNN; recover its title and body with `gh issue view NNNN`.
|
||||
|
||||
### 2. Validate — stop with one message if anything fails
|
||||
|
||||
Run all checks before reporting, then report every failure together:
|
||||
|
||||
1. **Base branch not usable.** If the script fails (exit 1), or its output —
|
||||
after stripping an optional `remotes/origin/` prefix — is not one of the
|
||||
canonical branches (`develop`, `staging`, `main`), stop and ask the user
|
||||
to re-run with more context — for example, passing the base branch
|
||||
explicitly in their invocation. An explicit base given by the user
|
||||
overrides the script's output.
|
||||
2. **On a base branch.** There is no task branch to merge — say so and stop.
|
||||
3. **No commits.** The branch has no commits ahead of the base — say so and
|
||||
stop.
|
||||
4. **No clear issue.** There is no issue in the session context, and the
|
||||
branch name has no `issue-NNNN` pattern (or `gh issue view` finds nothing)
|
||||
— say so and stop. Exception: the user's invocation says `no issue` /
|
||||
`without issue` — then continue without an issue reference.
|
||||
5. **Branch not pushed.** The remote-tracking ref `origin/<branch>` is
|
||||
missing, or `git rev-parse origin/<branch>` differs from HEAD — the
|
||||
branch was never pushed, or has commits the remote does not have. Never
|
||||
push yourself; ask the user to push and to invoke `@create-pr` again
|
||||
afterwards, then stop.
|
||||
|
||||
### 3. Already-open PR
|
||||
|
||||
Check whether a PR already exists for this branch (`gh pr list --head
|
||||
<branch>`). If one exists, report its URL and stop — do not create a second
|
||||
one. Title or description fixes belong to Update mode.
|
||||
|
||||
### 4. Write and create the PR
|
||||
|
||||
Write the title and body following `mem:workflow/creating-prs` (title format,
|
||||
description structure, writing principles) and `mem:workflow/creating-commits`
|
||||
(commit type emojis). Derive the title and body from the commits and, when
|
||||
there is one, from the issue body. Include `Closes #NNNN` for readable context,
|
||||
but do not treat it as the link; use `python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>` after creating the PR.
|
||||
|
||||
Repeat the `AI-assisted-by:` trailer in the body, once per model that worked
|
||||
on the branch, so the PR states the assistance where a reviewer reads it. The
|
||||
branch commits keep their own trailers, and a squash merge carries every one
|
||||
of them into the landed message.
|
||||
|
||||
Before offering or accepting a draft PR, warn that CI doesn't run on them. Add
|
||||
`--draft` only when the user agrees to that.
|
||||
|
||||
```bash
|
||||
gh pr create --repo penpot/penpot --base "<BASE>" --title "<TITLE>" \
|
||||
--project "Main" --body-file /tmp/pr-body.md
|
||||
```
|
||||
|
||||
`--base` is the branch resolved in step 1: without it the PR opens against the
|
||||
repository default, which is wrong for a branch cut from `staging`. `--project
|
||||
"Main"` is required by `mem:workflow/creating-prs`.
|
||||
|
||||
If an issue is present, run the explicit assignment command from
|
||||
`mem:workflow/creating-prs` before reporting success:
|
||||
|
||||
```bash
|
||||
python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
|
||||
```
|
||||
|
||||
### 5. Report
|
||||
|
||||
Report the PR URL and stop.
|
||||
|
||||
## Update mode
|
||||
|
||||
1. Identify the PR: the number given by the user, or `gh pr list --head
|
||||
<branch>`.
|
||||
2. Write the new title and/or body following `mem:workflow/creating-prs`.
|
||||
3. Apply and verify:
|
||||
|
||||
```bash
|
||||
gh pr edit <NUMBER> --repo penpot/penpot --title "<TITLE>" --body-file /tmp/pr-body.md
|
||||
gh pr view <NUMBER> --repo penpot/penpot --json title,body
|
||||
```
|
||||
|
||||
If the updated body contains `Closes #NNNN`, run the explicit assignment
|
||||
command from `mem:workflow/creating-prs`:
|
||||
|
||||
```bash
|
||||
python3 scripts/gh.py link-issue <ISSUE_NUMBER> <PR_NUMBER>
|
||||
```
|
||||
|
||||
4. Report and stop.
|
||||
|
||||
## User context
|
||||
|
||||
Extra context in the user's invocation (the message that triggered this skill)
|
||||
plays the role command arguments play elsewhere: overrides such as `no issue` /
|
||||
`without issue`, an explicit base branch (`from origin/staging`), a PR number
|
||||
for Update mode, and so on.
|
||||
@ -1,116 +0,0 @@
|
||||
---
|
||||
name: find-project-anomalies
|
||||
description: Check a GitHub milestone against the Main project board, report the five anomaly types to tmp/<MILESTONE>-ANOMALIES.md, and fix missing milestone assignments on request.
|
||||
---
|
||||
|
||||
# Skill: find-project-anomalies
|
||||
|
||||
Check every issue and PR in a given GitHub milestone against the Main
|
||||
project board. Report the five anomaly types below, then fix the ones a
|
||||
human confirms.
|
||||
|
||||
## Getting the version ($ARGUMENTS)
|
||||
|
||||
The milestone version comes from the skill arguments (`$ARGUMENTS`), the
|
||||
same way other skills receive theirs. Accept `2.17.0`, `v2.17.0`,
|
||||
`milestone 2.17.0`, or any text clearly containing one `X.Y.Z` version.
|
||||
|
||||
If no version is given and none can be deduced from the arguments or the
|
||||
conversation context, **stop and ask the user** which milestone to check.
|
||||
Do not guess, do not default to the latest release.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Before a release, to catch board/milestone drift while there is still time
|
||||
to fix it
|
||||
- After triage sessions, to verify every milestone item is tracked, owned,
|
||||
and correctly assigned
|
||||
- Whenever someone asks "is milestone X.Y.Z clean on Main?"
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
- Python 3.8+
|
||||
- `scripts/gh.py` for GitHub queries, `scripts/project-anomalies.py` for
|
||||
detection and reporting (`python3 scripts/project-anomalies.py check --help`)
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Resolve the version (see above), then run the check
|
||||
|
||||
```bash
|
||||
python3 scripts/project-anomalies.py check "2.17.0"
|
||||
# explicit path:
|
||||
python3 scripts/project-anomalies.py check "2.17.0" --output tmp/2.17.0-ANOMALIES.md
|
||||
```
|
||||
|
||||
This fetches milestone issues and PRs in bulk (one GraphQL call per 50
|
||||
items), resolves outsiders the same way, and overwrites
|
||||
`tmp/<MILESTONE>-ANOMALIES.md`. Every number renders as a full
|
||||
`[#N](https://github.com/penpot/penpot/issues/N)` or
|
||||
`[#N](https://github.com/penpot/penpot/pull/N)` link.
|
||||
|
||||
### 2. Read the report and judge each anomaly
|
||||
|
||||
The five types, in report order:
|
||||
|
||||
1. **OPEN issue with a MERGED PR** — the fix landed but the issue never
|
||||
closed (or it reopened). Fix: close the issue, or move it out of the
|
||||
milestone if the fix did not actually land here.
|
||||
2. **Milestone issue, PR elsewhere or issue off Main** — a *merged* closing
|
||||
PR in another milestone (or none), or the issue itself not on the Main
|
||||
board. An unmerged PR's milestone means nothing (it landed nowhere), so
|
||||
it never triggers this type. Fix: align the milestones (see step 3), or
|
||||
add the issue to Main.
|
||||
3. **Milestone PR, issue elsewhere or off Main** — the mirror view over
|
||||
*merged* PRs only: a PR released here closes an issue tracked elsewhere
|
||||
(or nowhere on Main). Unmerged PRs never trigger this type — their
|
||||
milestone means nothing yet. Fix: align the milestones (see step 3), or
|
||||
add the issue to Main.
|
||||
4. **MERGED PR on a `needs triage` issue** — landed without triage. Only a
|
||||
human can triage; never auto-remove the label.
|
||||
5. **MERGED PR on an unassigned, non-community issue** — no owner and no
|
||||
`community contribution` label (checked on both issue and PR). Either
|
||||
assign an owner or confirm it is community work. Never invent an owner.
|
||||
|
||||
Scope notes (deliberate, not bugs):
|
||||
- Project membership is checked **on the issue side only**. PRs are not the
|
||||
unit tracked on Main, so an unprojected PR alone is never an anomaly.
|
||||
- An issue with *no* milestone closed by a milestone PR **is** reported
|
||||
here (type 3): unlike the changelog flow, this pairing always needs a
|
||||
human look.
|
||||
|
||||
### 3. Fix missing milestone assignments (only these, only on confirmation)
|
||||
|
||||
The only mechanical fix in this skill is assigning a missing milestone.
|
||||
The report already prints the exact command next to each such case:
|
||||
|
||||
```bash
|
||||
gh pr edit <PR> --milestone "<MILESTONE>"
|
||||
gh issue edit <ISSUE> --milestone "<MILESTONE>"
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Apply **only** when one side lacks a milestone. When both sides have
|
||||
*different* milestones, a human decides which one moves — never pick a
|
||||
side yourself.
|
||||
- Confirm **each fix (or each small batch)** with the user before running
|
||||
it. Read back what the command will change.
|
||||
- Everything else (closing issues, triaging, assigning owners, adding
|
||||
items to the Main board) is human work: point at it, do not do it.
|
||||
|
||||
### 4. Re-run until clean
|
||||
|
||||
After fixes land, re-run step 1 and confirm the report shows zero
|
||||
anomalies. The `tmp/` report is scratch output (gitignored): quote or
|
||||
paste entries into chat when reporting back, do not commit it.
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **No version, no run.** Stop and ask when the milestone is unknown.
|
||||
- **Report first, touch nothing.** No board or milestone changes without
|
||||
explicit per-item confirmation.
|
||||
- **Milestone gaps auto-fix; judgment calls do not.** Only a missing
|
||||
milestone is mechanical. triage, ownership, and close/reopen decisions
|
||||
belong to humans.
|
||||
- **Every number clickable.** The report is useless without full GitHub links.
|
||||
@ -1,175 +0,0 @@
|
||||
---
|
||||
name: implement-plan
|
||||
description: Implementation flow — execute a ready plan from the session context
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Implement Plan
|
||||
|
||||
This flow is run once a plan is ready (for example, from plan mode). Execute
|
||||
the plan already prepared in the current session context. It never pushes —
|
||||
the user pushes.
|
||||
|
||||
By default it ends with exactly one commit. When the user asks for it
|
||||
("step by step"), it commits once per task instead and waits for the
|
||||
user's confirmation after each one (see *Execution modes*).
|
||||
|
||||
## Plan file handling
|
||||
|
||||
Plan files are local workflow artifacts. They must never be staged or
|
||||
committed, even when `.gitignore` already excludes them. Never use
|
||||
`git add -f`, change ignore rules, or otherwise force a plan file into a
|
||||
commit. Close and update the plan when this flow requires it, but leave it
|
||||
in the local working tree.
|
||||
|
||||
## When to use
|
||||
|
||||
- The user asks to implement or execute a plan, in any phrasing:
|
||||
"implement the plan", "execute it", "go build it" — or runs
|
||||
`@implement-plan`.
|
||||
- A ready, reviewed plan is in the session context or a plan file path
|
||||
was given (typically after `@make-a-plan` or `@review-plan`).
|
||||
|
||||
Do not use it to produce plans — that is the `make-a-plan` flow.
|
||||
|
||||
For a plan derived from a code review, retain its review target, reviewer
|
||||
session ID, original findings and decisions. Record the starting revision and
|
||||
the exact commits or diff produced by this implementation, plus test evidence,
|
||||
in session context for follow-up. Do not put commit hashes in the plan file.
|
||||
|
||||
## 1. Read the plan first
|
||||
|
||||
Identify the plan to execute — from the file path the user gave, the
|
||||
arguments, or the session context. Read it completely. Read the required
|
||||
memories before writing any code: `mem:critical-info` and the core memory
|
||||
of every module the plan touches, plus the deeper memories they reference
|
||||
(AGENTS.md governs this).
|
||||
|
||||
## 2. Detect the flow (no questions)
|
||||
|
||||
Inspect the current branch with `git rev-parse --abbrev-ref HEAD`, pick the
|
||||
mode, and announce it in one line before presenting anything. Detection is
|
||||
read-only: nothing is created until the user confirms (step 3).
|
||||
|
||||
- **On a base branch** (`main`, `develop`, `staging`) → **standalone mode**:
|
||||
a new GitHub issue and a branch `issue-NNNN` will be created after the
|
||||
user's confirmation.
|
||||
- **On any other branch** (a feature branch, typically `issue-NNNN`) →
|
||||
**continue mode**: the implementation continues on the current branch.
|
||||
No issue or branch is created. The branch name provides the issue
|
||||
reference when it follows the `issue-NNNN` pattern.
|
||||
|
||||
Arguments override detection: `standalone`, `continue`, `direct`
|
||||
(`no branch` / `direct commit`), `no issue` / `without issue`, or an
|
||||
explicit base such as `from origin/develop`.
|
||||
|
||||
**Direct mode** (`direct`, `no branch`, `direct commit`): no issue and
|
||||
no branch — the implementation and the commit land on the current branch
|
||||
as it is, even when it is a base branch. Best for small or tooling-only
|
||||
changes the user wants committed in place.
|
||||
|
||||
**Standalone while already on a feature branch:** stop and explain that this
|
||||
would stack branches. Ask the user to re-run with an explicit base, for
|
||||
example `from origin/develop` — then branch from that base instead of HEAD.
|
||||
|
||||
## 3. Present the checklist and wait
|
||||
|
||||
Before touching the repository, show the user the full picture:
|
||||
|
||||
- **The flow**: whether the GitHub issue and the branch will be created
|
||||
(standalone mode — give the planned branch name, `issue-NNNN` or
|
||||
`plan-<slug>`), whether you continue on the current branch
|
||||
(continue mode — name it), or whether everything lands on the current
|
||||
branch as it is (direct mode — name it, and say so when it is a base
|
||||
branch).
|
||||
- **The execution style**: batch or step-by-step (see *Execution modes*).
|
||||
- A checklist (todolist) of the plan's tasks, in order.
|
||||
|
||||
Then WAIT for the user's explicit confirmation. Do not start until you
|
||||
have it. If the plan has no discrete tasks, ask the user how to split
|
||||
it, or propose running it as a single change.
|
||||
|
||||
## 4. Execute the plan
|
||||
|
||||
**Standalone setup, after the confirmation:** create the issue with the
|
||||
**`create-issue`** skill, following the *Creating Issues from Draft Body*
|
||||
flow in `mem:workflow/creating-issues`. Derive the issue title and body
|
||||
from the plan, capture the new issue's number — call it **NNNN** — and
|
||||
create the branch from the current HEAD:
|
||||
|
||||
```
|
||||
git checkout -b issue-NNNN
|
||||
```
|
||||
|
||||
If the arguments say `no issue` / `without issue`, skip the issue and
|
||||
create a branch named `plan-<slug>` instead, where `<slug>` is the plan
|
||||
title, lowercase and hyphen-separated.
|
||||
|
||||
If the arguments say `direct` / `no branch` / `direct commit`, skip the
|
||||
issue and the branch: implement and commit on the current branch as it
|
||||
is. If it is a base branch, the checklist presentation already said so —
|
||||
no further confirmation is needed.
|
||||
|
||||
### Batch mode (default)
|
||||
|
||||
Implement every task in one go. Work methodically, keeping changes
|
||||
focused on what the issue requires. Respect the plan's proposed
|
||||
parallelization when it applies.
|
||||
|
||||
When the implementation is complete, load the **`create-commit`** skill and
|
||||
follow its workflow to commit
|
||||
only the implementation, tests, memory, and documentation changes. Before
|
||||
committing, verify that the staged file list does not contain the plan
|
||||
file. Provide a brief summary of what was implemented and why, the issue
|
||||
reference (`issue-NNNN`) when there is one, and the model name you are
|
||||
running as so the `AI-assisted-by` trailer is set correctly.
|
||||
|
||||
Only after the commit exists and passes its checks, close the plan: flip
|
||||
`Status` to `done` and append one UTC ISO 8601 line to `Review Log` with the
|
||||
issue URL when one exists; otherwise record `done` without inventing an
|
||||
identifier. Never record commit hashes. Leave the plan local and uncommitted.
|
||||
If the commit or its checks fail, do not close the plan.
|
||||
|
||||
### Step-by-step mode (on request)
|
||||
|
||||
When the user asks for it — "step by step", "task by task", "one commit
|
||||
per task" — loop one task at a time:
|
||||
|
||||
- Execute exactly ONE task.
|
||||
- Commit it now: load the **`create-commit`** skill and follow it —
|
||||
one commit per task, never two tasks in one commit. Same inputs as
|
||||
always: what and why, the issue reference, your model name.
|
||||
- After the final task commit, close the plan file (`Status: done`, one
|
||||
`Review Log` line with the issue URL when one exists). Leave it local
|
||||
and uncommitted; never amend the task commit to include it.
|
||||
- Show the user the result (what changed, files touched, how it was
|
||||
verified).
|
||||
- WAIT for the user's confirmation before starting the next task.
|
||||
|
||||
Never batch in this mode: no two tasks in one commit, and no new task
|
||||
before the user confirms. If a task turns out much bigger than planned,
|
||||
stop and ask the user before splitting it.
|
||||
|
||||
## When you are done
|
||||
|
||||
After the entire plan is implemented and verified, including the final task in
|
||||
step-by-step mode, if it addresses code review findings, ask whether the user
|
||||
wants the original reviewer to verify the corrections. Use the `question` tool
|
||||
if available; otherwise ask in chat. Wait for the answer. Do not start the
|
||||
follow-up without consent, and do not ask after each intermediate task.
|
||||
|
||||
If the user accepts, follow `@review-code`'s follow-up procedure with the
|
||||
review context, decisions, exact correction range or diff and test evidence.
|
||||
If the original reviewer cannot be resumed, say so and use a new reviewer with
|
||||
the same context. If the user declines, stop without starting a review. The
|
||||
follow-up is read-only; further fixes require the user's direction.
|
||||
|
||||
End by suggesting the next steps (suggestions, not a required pipeline — any
|
||||
instruction from me overrides them):
|
||||
|
||||
- `@review-code` — for a new independent review of the changes.
|
||||
- `@create-pr` — when the task is done and the branch is ready to merge.
|
||||
|
||||
## User input, overrides and additional context
|
||||
|
||||
$ARGUMENTS
|
||||
@ -1,109 +0,0 @@
|
||||
---
|
||||
name: local-ci
|
||||
description: Run local CI-style checks with ./scripts/ci (lint, tests, format, and on request production builds) per monorepo module. Use when verifying changes before declaring work done, running lint or tests locally, fixing formatting, repairing Clojure delimiter errors, or checking that a module still compiles.
|
||||
---
|
||||
|
||||
# Local CI
|
||||
|
||||
Run the same checks CI runs, locally, for the modules you touched, with
|
||||
`scripts/ci`. Each task writes a log file; the final summary says what
|
||||
passed and what failed.
|
||||
|
||||
Full details: `mem:scripts/ci` (file: `.serena/memories/scripts/ci.md`)
|
||||
|
||||
## When to use
|
||||
|
||||
- After implementing or fixing code — verify every module you touched
|
||||
before declaring the work done.
|
||||
- When the user asks to run CI, lint, tests, or format checks locally.
|
||||
- When you changed `common/` — validate its consumers too.
|
||||
|
||||
**Skip:** while exploring, planning, or reading code.
|
||||
|
||||
## Command reference
|
||||
|
||||
Run from the repo root:
|
||||
|
||||
```bash
|
||||
./scripts/ci [OPTIONS] [MODULES...]
|
||||
```
|
||||
|
||||
Modules: `frontend` `backend` `common` `render-wasm` `exporter` `mcp`
|
||||
`plugins` `library` `media-processor`, or `--all` for every module.
|
||||
|
||||
With no task flags it runs three tasks per module, in order: **lint**,
|
||||
**test**, **fmt** (format check; `--fix` formats files instead). **build**
|
||||
is opt-in: it compiles the module the way production does and is the slow
|
||||
one, so it never runs unless you ask for it.
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `--all` | Run every module |
|
||||
| `--exclude MOD` | Skip one module (repeatable) |
|
||||
| `--lint` / `--no-lint` | Run only lint / drop lint |
|
||||
| `--test` / `--no-test` | Run only tests / drop tests |
|
||||
| `--fmt` / `--no-fmt` | Run only format check / drop it |
|
||||
| `--build` / `--no-build` | Run only the production build / drop it |
|
||||
| `--fix` | Format files instead of checking (other tasks unaffected) |
|
||||
| `--paren-repair` | Fix delimiter errors in Clojure/CLJS files |
|
||||
| `--fail-fast` | Stop at the first failure |
|
||||
| `--quiet` | Suppress failure output |
|
||||
| `--dry-run` | Show what would run, execute nothing |
|
||||
| `--clean` | Delete the `.ci-logs/` directory |
|
||||
|
||||
## Reading failures
|
||||
|
||||
Every task writes its full output to `.ci-logs/<module>-<task>.log`. On
|
||||
failure the script prints only the last 30 lines. To diagnose a failure,
|
||||
**read the log file** — never re-run the command piped through filters
|
||||
(repo rule: redirect to a file first, then read it). The exit code is 1
|
||||
when any task failed; the summary lists each failed `module:task` and its
|
||||
log path.
|
||||
|
||||
## Typical workflows
|
||||
|
||||
```bash
|
||||
# Verify a module you changed: lint + tests + format check
|
||||
./scripts/ci frontend
|
||||
|
||||
# Fast pass while iterating: lint only
|
||||
./scripts/ci --lint frontend
|
||||
|
||||
# Lint + format check, skip the long test suite
|
||||
./scripts/ci --no-test frontend
|
||||
|
||||
# Format the module without running the test suite
|
||||
./scripts/ci --fix --no-test frontend
|
||||
|
||||
# Check that the module still compiles (slow: minutes)
|
||||
./scripts/ci --build frontend
|
||||
|
||||
# Compile every module (8 builds, ~4 min with warm caches)
|
||||
./scripts/ci --build --all
|
||||
|
||||
# Broke delimiters in Clojure/CLJS files: repair first, then lint
|
||||
./scripts/ci --paren-repair frontend
|
||||
./scripts/ci --lint frontend
|
||||
|
||||
# Changed common/ — validate its consumers too
|
||||
./scripts/ci common frontend backend exporter
|
||||
|
||||
# Preview what would run, without running it
|
||||
./scripts/ci --dry-run --all
|
||||
```
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Run from the repo root.
|
||||
- Test tasks are long-running (backend runs `clojure -M:dev:test`); give
|
||||
the bash call a generous timeout (10–20 minutes) instead of letting it
|
||||
time out mid-run.
|
||||
- `mcp` and `media-processor` have no lint task — they show as skipped,
|
||||
not failed. `common` has no build task for the same reason.
|
||||
- Build runs are the slow ones (frontend ~2 min even with warm caches;
|
||||
cold caches roughly double it). Give the shell a generous timeout and
|
||||
do not run two builds at once in the same checkout: `render-wasm` takes
|
||||
a build lock.
|
||||
- `--paren-repair` only fixes delimiters; run lint afterwards to catch
|
||||
what remains. See `mem:scripts/paren-repair`.
|
||||
- What to run and how to read test results: `mem:testing`.
|
||||
@ -1,116 +0,0 @@
|
||||
---
|
||||
name: make-a-plan
|
||||
description: Planning flow — research the subject of this session, produce an implementation plan with the planner skill, resolve open questions with the user in plain language, and save the final plan to .agents/plans/.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Make a Plan
|
||||
|
||||
Act as a senior software engineer: research the subject of this session in depth and
|
||||
produce a well-grounded, actionable implementation plan.
|
||||
|
||||
If the running agent cannot write (for example, the plan agent), say so and
|
||||
stop — this skill needs the build agent to save the plan.
|
||||
|
||||
## When to use
|
||||
|
||||
- The user asks to plan, design, or break down a task, in any phrasing:
|
||||
"make a plan", "how would we build X", "design an approach for Y" —
|
||||
or invokes `@make-a-plan`.
|
||||
- The user asks to rework or extend an existing plan. While the plan is
|
||||
still unimplemented (pre-`implement-plan` iteration, e.g. after
|
||||
`@review-plan` findings or user feedback), revise the saved plan file
|
||||
in place. Once the plan has been implemented and reviewed (post
|
||||
`@review-code` findings on committed work), write a new derived plan
|
||||
file instead — never rewrite the executed plan. The `planner` skill
|
||||
defines the derived naming (`--review-NN`, `--task-NN`).
|
||||
- The user says to apply additional review feedback to a still-unimplemented
|
||||
plan: apply the agreed changes, flip `Status` to `reviewed`, and append one
|
||||
UTC ISO 8601 line to `Review Log`. `review-plan` can also update the plan
|
||||
and its metadata as part of its own review flow.
|
||||
|
||||
Do not use it to execute a plan — that is the `implement-plan` flow.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Produce the plan** with the `planner` skill. By default, research the
|
||||
subject of this session and draft the plan yourself. If I ask for it (for
|
||||
example, `delegated` in the user context), delegate to the `general` subagent
|
||||
instead — the delegate must also follow the `planner` skill and receive all
|
||||
the relevant session context (a review, user feedback, and so on).
|
||||
When the plan addresses a code review, preserve the review target, reviewer
|
||||
session ID if available, original findings and IDs, and decisions including
|
||||
deferred or rejected findings and their reasons in the local plan. Map
|
||||
accepted findings to tasks. If context is unavailable, mark it unknown; do
|
||||
not invent it. Pass this context to a delegated planner too.
|
||||
2. Before asking me to decide anything, explain the plan and every open question in
|
||||
plain language. Assume I know only the high-level project goal, not the codebase,
|
||||
architecture, implementation terms, or the problem this task solves.
|
||||
3. Once all decisions are answered and the plan is final, save it verbatim to the
|
||||
announced path under `.agents/plans/` (create the directory if it does not
|
||||
exist). This step is the flow's explicit authorization to write the plan
|
||||
file — the only write allowed here. A fresh plan uses
|
||||
`.agents/plans/YYYY-MM-DD-<slug>.md` with `Status: draft` and an
|
||||
empty `Review Log`; a derived plan uses the parent
|
||||
basename plus the `planner` suffix (`--review-NN`, `--task-NN`) in its
|
||||
own new file, also starting as `draft`. If I later ask for changes to a still-unimplemented plan,
|
||||
update the saved file directly.
|
||||
4. Present me with a clear, self-contained summary of the plan's most relevant points
|
||||
only after all required decisions have been answered. Write it for someone who knows
|
||||
only the project's high-level goal and may not know the plan's low-level context.
|
||||
Explain necessary technical language in plain terms, include the problem being
|
||||
solved and the proposed outcome, and do not assume that listing technical task names
|
||||
is enough.
|
||||
|
||||
### Hard rule — read-only while planning
|
||||
|
||||
While this flow runs, act read-only: research with read-only tools only.
|
||||
Never edit source files, never run builds, tests, linters, or any command that
|
||||
modifies state, and never commit. The single allowed write is the plan file in
|
||||
step 3. This rule expires when I approve the plan or move on to another task;
|
||||
then you act as a normal build agent again.
|
||||
|
||||
When the plan contains open questions, do not show them as bare technical questions or
|
||||
assume that I understand the technical language or technical words used in the plan.
|
||||
For each question, first explain:
|
||||
|
||||
- What part of the user problem the decision affects.
|
||||
- The relevant concept from the beginning, with a small concrete example.
|
||||
- What each available option would make the system do.
|
||||
- The practical benefits, costs, risks, and user-visible consequences of each option.
|
||||
- Which option the planner recommends and why.
|
||||
|
||||
Only after that explanation, use the `question` tool to ask the decision with clear,
|
||||
non-technical option labels. Put the recommended option first and mark it as
|
||||
`(Recommended)`. Group related questions when their context is shared, but do not ask a
|
||||
question whose meaning has not already been explained.
|
||||
|
||||
If I say that I do not understand a question or its choices, do not treat my previous
|
||||
answer as valid. Explain the concepts again from the high-level project goal, use a more
|
||||
concrete example, explain the implications, and ask the question again with the
|
||||
`question` tool. Repeat this until I can make an informed choice. If one answer creates
|
||||
new design consequences or additional decisions, explain those consequences before
|
||||
asking any new question.
|
||||
|
||||
Distinguish clearly between requirements already fixed by the roadmap or existing
|
||||
architecture and choices that actually require my input. Do not ask me to choose an
|
||||
implementation detail when the plan can resolve it safely without changing the public
|
||||
behavior. If there are no decisions that require my input, say so and present the
|
||||
summary.
|
||||
|
||||
IMPORTANT: **Under no circumstances execute the plan. Wait for the user to review it
|
||||
after all possible questions have been answered.** The final summary must explain the
|
||||
problem being solved, the proposed behavior, the main user-visible workflow, important
|
||||
constraints and risks, what is deliberately out of scope, and the path where the plan
|
||||
is saved. Never assume that a short list of task names is enough context. End
|
||||
the final response by suggesting the next steps, in this order:
|
||||
|
||||
1. `@review-plan` — to get a second opinion on the plan before executing it.
|
||||
2. `@implement-plan` — to execute the plan from the current session context.
|
||||
|
||||
These are suggestions, not a required pipeline — any instruction from me
|
||||
overrides them (for example, asking you to implement the plan directly).
|
||||
|
||||
## User input, overrides and additional context
|
||||
|
||||
$ARGUMENTS
|
||||
@ -1,330 +0,0 @@
|
||||
---
|
||||
name: migrate-ds-area
|
||||
description: Migrate one Penpot UI area to the Design System components for one group of the DS migration epic (#12375): buttons and icons (icon-button*, button*, icon*), selects (select*) and later groups. Moves the area's SCSS off the deprecated styles, proves every change with a zero-tolerance visual test, then opens the draft PR and updates the group issue (PRs table row, QA comment). Use when the user asks to migrate an area or module to the DS, continue a DS migration group, or names an area of a group issue.
|
||||
---
|
||||
|
||||
# Skill: migrate-ds-area
|
||||
|
||||
Move one UI area (one `.cljs` + `.scss` pair, or a few related modules)
|
||||
from legacy components and raw HTML to the Design System components,
|
||||
prove with pixels what changed, and ship it as a draft PR that QA can
|
||||
test.
|
||||
|
||||
The epic is #12375. Each control type is a group with its own issue
|
||||
(buttons and icons #12376, selects #12377, radios #12378, …). The group
|
||||
issue holds the plan (order, waiting areas), the changes QA should
|
||||
expect, and the PRs table where people claim areas. Read it first.
|
||||
|
||||
## Input
|
||||
|
||||
- The group issue (e.g. `#12376`) and the area: a module path (e.g.
|
||||
`dashboard/fonts`) or an area named in the group issue.
|
||||
- No area given: take the next unclaimed area in the group issue's plan
|
||||
order, and say which one.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- Never `git push`. The user pushes; then you open the PR.
|
||||
- PRs go to `develop`; branches start from `origin/develop`.
|
||||
- **Stay in the group.** Only the controls of this group change. In the
|
||||
buttons/icons group, selects, radios, numeric/text inputs, checkboxes,
|
||||
menus (`%menu-*`, `%dropdown-*`) and modal structure (`%modal-*`) stay;
|
||||
a checkbox tick or an input label icon stays legacy.
|
||||
- **Accept the DS look.** Do not add CSS to imitate legacy sizes or
|
||||
colors. Do keep positions: icons and labels must not move unless the
|
||||
DS size forces it, and then say so.
|
||||
- No negative margins to compensate; use positive DS spacing or
|
||||
`px2rem()`.
|
||||
- **Gaps in the DS wait.** An area that needs an icon or a feature the
|
||||
DS lacks keeps its legacy control; say so in the PR.
|
||||
- Skip legacy shared components that another group will replace
|
||||
(`context_menu_a11y`, `forms`, `select`, `dropdown`, `radio_buttons`,
|
||||
`search_bar`, `title_bar`, …): they wait for that group's cleanup,
|
||||
where they are updated if still used or deleted if not.
|
||||
- Debug-only UI gets no QA comment and no compare image.
|
||||
|
||||
## Devenv
|
||||
|
||||
Run every test, lint and `pnpm` command inside the workspace container,
|
||||
never on the host (a host `pnpm` relinks `node_modules` and breaks the
|
||||
container's watcher). The container is `penpot-devenv-ws<N>-main`, where
|
||||
`<N>` is the workspace number (`ws0` is the main repo; `wsN` lives in
|
||||
`~/.penpot/penpot_workspaces/wsN`). Write output to a file, then read it:
|
||||
|
||||
```bash
|
||||
C=penpot-devenv-ws1-main # your workspace
|
||||
docker exec -u penpot -w /home/penpot/penpot/frontend $C \
|
||||
bash -lc '<command>' > <scratch>/out.txt 2>&1
|
||||
```
|
||||
|
||||
When the app does not behave as the code says, check the build first:
|
||||
|
||||
- **Stale build after a branch switch** ("is not a function", or a page
|
||||
with empty `stl/css` classes): `touch` the stale `.cljs`/`.scss`, wait
|
||||
for the compiled file under `frontend/resources/public/` to update,
|
||||
hard-reload.
|
||||
- **A whole area unstyled:** a new `.scss` added after the watcher
|
||||
started is missing from `main.css` (grep `main.css` for the module
|
||||
prefix), or `main.css` was written corrupted after many `.scss`
|
||||
changed at once. Touch a `.scss`; if the module is still missing,
|
||||
restart the watcher in the workspace's tmux session (`penpot:0`,
|
||||
`./scripts/watch app`).
|
||||
- **Blank app after switching between `develop` and `staging`:**
|
||||
dependencies differ. Run `pnpm install --frozen-lockfile` in the
|
||||
container and restart the watcher.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Claim the area and branch
|
||||
|
||||
1. Read the group issue: plan, waiting areas, expected changes, PRs table.
|
||||
2. Add a row to the PRs table with no PR number, the user's @login as
|
||||
Owner and the state `Claimed` (`gh issue view/edit <n> --body-file`).
|
||||
Read the body right before editing, and read it again after saving:
|
||||
when two people edit at the same time, GitHub keeps only the last
|
||||
edit, so a row can vanish. Put yours back if it is gone.
|
||||
Skip an area someone else already claimed, unless the claim has had
|
||||
no PR for 7 days and its owner did not answer a comment asking.
|
||||
3. `git switch -c <user>-ds-<group>-<area> origin/develop`.
|
||||
4. Survey the area: every control of the group in the `.cljs`, the
|
||||
legacy icons, the placeholders and mixins the `.scss` uses, the
|
||||
`div`/`span` with `:on-click`.
|
||||
|
||||
### 2. Visual baseline
|
||||
|
||||
Add a `test.describe("<Area> controls", …)` block to the matching spec
|
||||
in `frontend/playwright/ui/visual-specs/` (project `ds`, dark theme).
|
||||
Put it next to the blocks of the same screen, not always at the end of
|
||||
the file: two PRs that both append at the end conflict.
|
||||
|
||||
- Every shot: `toHaveScreenshot({ maxDiffPixelRatio: 0, threshold: 0 })`.
|
||||
- Show every changed control at rest, on hover and open or pressed.
|
||||
Capture the element or the smallest stable container.
|
||||
- Locators must work before and after the change: text, role, testid,
|
||||
`[class*='…']`, or `a.or(b)` when the element type changes (raw
|
||||
`[class*='menu-btn']` before, `getByRole("button", { name: "Options" })`
|
||||
after). Scope `.or()` to the area so it cannot match elsewhere.
|
||||
- Missing data? Add a fixture under `frontend/playwright/data/` and mock
|
||||
it (`dashboardPage.mockRPC(...)`, `DashboardPage.mockConfigFlags` for
|
||||
flags). Watch for expired or inconsistent fixtures (dates in the past,
|
||||
wrong file for a mock).
|
||||
- Capture with `--update-snapshots=all`:
|
||||
```bash
|
||||
npx playwright test --project=ds <spec> -g "<describe>" --update-snapshots=all
|
||||
```
|
||||
Plain `--update-snapshots` keeps a snapshot within the project's 0.5%
|
||||
tolerance, so a real diff looks like 0.
|
||||
- Copy the snapshots (`<spec>.spec.js-snapshots/<Describe>-*`) to
|
||||
`<scratch>/diffs/<area>/before/` and look at them: the state you
|
||||
wanted must really be on screen.
|
||||
|
||||
### 3. Controls commit
|
||||
|
||||
#### Buttons and icons
|
||||
|
||||
| Legacy | DS |
|
||||
|---|---|
|
||||
| icon-only raw button | `[:> icon-button* {:variant "ghost" :aria-label (tr …) :icon i/x :on-click f}]` (always pass `:variant`; `"action"` for 24px icon toggles) |
|
||||
| pressed/selected | `:aria-pressed (boolean x)` |
|
||||
| text button | `[:> button* {:variant "primary"/"secondary"/"ghost"/"destructive" :type "button"} label]` |
|
||||
| `[:input {:type "button"}]` | `button*` |
|
||||
| modal close | `icon-button*` ghost, `:aria-label (tr "labels.close")`, `:tooltip-class` carrying the corner position |
|
||||
| legacy icon outside a button | `[:> icon* {:icon-id i/x :class …}]` |
|
||||
|
||||
- Not every raw button changes: rows, tabs, swatches, avatar widgets,
|
||||
full-height segments, round custom buttons and link-style text buttons
|
||||
stay raw (their legacy icon still changes). List the kept ones in the
|
||||
PR.
|
||||
- A `div`/`span` with `:on-click` in the area becomes a `[:button]`
|
||||
(with `aria-expanded`/`aria-haspopup` when it opens something). An
|
||||
`[:a]` without `href` stays; list it in the PR.
|
||||
- **Tooltip wrapper.** `icon-button*` renders inside a tooltip trigger
|
||||
wrapper. Position (`position: absolute`, grid placement, margins
|
||||
against siblings) goes on `:tooltip-class`; look (`--button-height`,
|
||||
colors) goes on `:class`.
|
||||
- **Tooltip at the right edge.** Small icon buttons at the right end of
|
||||
a sidebar row or title bar get `:tooltip-placement "top-left"`; list
|
||||
it as an expected change.
|
||||
- **Sizes through custom properties:** `--button-height: #{px2rem(64)}` /
|
||||
`--button-width`. Sass does not evaluate functions inside custom
|
||||
properties without `#{}`.
|
||||
- **A grown `button*` keeps its label at the start**: add
|
||||
`justify-content: center` when it is wider than its content.
|
||||
- **Icon colors:** DS icons use `stroke: currentColor`. Set `color` on
|
||||
the icon class, not `stroke`/`fill`. A legacy `color: transparent` left
|
||||
on the button or wrapper makes the DS icon invisible: drop it. A class
|
||||
that redefines the color variable on the icon breaks the hover that
|
||||
changes it on the button: define the variable on the button.
|
||||
- **Every DS icon needs a color of its own** (on its class or a parent):
|
||||
one that inherits none shows in the devenv debug yellow.
|
||||
- **Icon sizes:** `"s"` draws 12px inside a 16px box (compensate the
|
||||
margin by 2px per side to keep the drawing in place); `"m"` 16px; `"l"`
|
||||
24px; `"xl"` 32px.
|
||||
- **Stroke width:** DS icons keep the DS stroke width. Never set
|
||||
`stroke-width` on a DS icon to match the legacy one; a thinner or
|
||||
thicker line is part of the DS look.
|
||||
- **Legacy placeholders that style `svg` descendants** (e.g.
|
||||
`%dropdown-element-base` → `span svg { 12px box; stroke }`) override a
|
||||
DS icon placed inside them: give the icon class its own box
|
||||
(`inline-size`/`block-size` 16px) and `stroke: currentcolor` with more
|
||||
specificity.
|
||||
- **Flex shrink:** a DS svg inside a small flex button can shrink and be
|
||||
cut; add `flex-shrink: 0` to the icon class.
|
||||
- **Rotated legacy arrows:** prefer `i/arrow-left`/`-right`/`-up`/`-down`
|
||||
over rotating `i/arrow` (same path), unless a rotation class drives the
|
||||
open/closed state.
|
||||
- **Menus anchored by the button size** move when the button grows; open
|
||||
them from the top (`inset-block-start: calc(100% + var(--sp-xs))`) or
|
||||
fix the context menu offset, and say so.
|
||||
|
||||
#### Selects
|
||||
|
||||
`[:& select {:default-value v :options [{:value :label}] :on-change f}]`
|
||||
→ `[:> select* {:default-selected <id> :options <memoized vector of {:id :label}> :on-change f}]`
|
||||
(`app.main.ui.ds.controls.select`).
|
||||
|
||||
- **Ids are non-blank strings.** A blank id shows a dimmed "--", and an
|
||||
id that matches no option falls back to the first one. Give mixed,
|
||||
none and auto their own ids (`"multiple"`, `"none"`, `"auto"`) and map
|
||||
them back in the handler. Use `""` only where "--" is the wanted look.
|
||||
- **Convert the value.** The legacy select ran `d/read-string`, so
|
||||
handlers got keywords, numbers or uuids; `select*` passes the `:id`
|
||||
string. Convert with `keyword`, `d/parse-integer` or `uuid/parse`.
|
||||
- **Memoized options, live selection.** `select*` resets its selection
|
||||
when `:default-selected` or the `:options` identity changes. Build
|
||||
options with `mf/with-memo` and `mapv` (the schema wants a non-empty
|
||||
vector) and pass `:default-selected` from the live value, never a
|
||||
constant.
|
||||
- **Never pass nil.** In dev builds a present prop with value nil
|
||||
(`:class`, `:wrapper-class`, `:disabled`, `:default-selected`) throws
|
||||
`invalid props on component select*`. Use `(boolean x)`, `d/nilv`, or
|
||||
add the prop only when it has a value.
|
||||
- **Props the legacy select ignored stay out**: `select*` passes rest
|
||||
props to its button, so they would reach the DOM.
|
||||
- **Classes:** `:wrapper-class` for layout, `:class` for look.
|
||||
- **Portal:** add `:has-portal true` only where the visual test shows the
|
||||
list cut at the bottom of a panel, and say so in the QA comment.
|
||||
- **Locators:** the button is a `combobox` with no accessible name; use
|
||||
`getByRole("combobox").filter({ hasText: "Linear" })` (it matches the
|
||||
legacy select too) and `getByRole("option", { name: … })`.
|
||||
- **Keyboard:** the arrows no longer change the value while the list is
|
||||
closed. It is an expected change, already listed in #12377.
|
||||
|
||||
#### All groups
|
||||
|
||||
- New labels: add the key to `en.po` and `es.po` by hand, in sorted
|
||||
position, with a `#:` reference to the cljs line. Never run the
|
||||
translations sync.
|
||||
- The `.scss` only gets what the new controls need in this commit.
|
||||
- Some e2e specs are not Prettier-clean on develop: never run
|
||||
`prettier --write` on a whole spec; format only your block.
|
||||
|
||||
Then wait for the watcher (grep the compiled JS/CSS under
|
||||
`frontend/resources/public/` for the change), recapture with `=all`,
|
||||
copy to `after/`, diff:
|
||||
|
||||
```bash
|
||||
SK=.agents/skills/migrate-ds-area/scripts
|
||||
python3 -I $SK/diff.py <before> <after> # per-file pixel count + bbox
|
||||
python3 -I $SK/trip.py <b> <a> <out.png> <scale> # before | after | red diff
|
||||
```
|
||||
|
||||
`trip.py` pads images of different sizes, so a taller "after" still
|
||||
renders.
|
||||
Explain every diff. Fix what is not a DS-look change (moved icons, cut
|
||||
icons, lost hover color). Rerun a shot that changes on unchanged code
|
||||
before trusting it; flaky spots seen so far: the invite modal textarea
|
||||
border, the search input caret, the user account page, the path handler
|
||||
menu trigger.
|
||||
|
||||
Lint, then commit (read `mem:workflow/creating-commits` first):
|
||||
|
||||
```bash
|
||||
npx prettier --check <spec> <fixtures>; npx stylelint <scss>; clj-kondo --lint <cljs>; cljfmt check <cljs>
|
||||
git add -f <cljs> <scss> <po> <spec> <fixtures> # -f: visual-specs/ is gitignored
|
||||
./scripts/check-commit
|
||||
```
|
||||
|
||||
A plain `git add` with an ignored path in the list adds nothing; always
|
||||
`-f`, and check `git show --stat`. Never commit the screenshots.
|
||||
Subject: `:recycle: Use DS <controls> in <area>`. A pre-existing bug
|
||||
found and fixed on the way goes in its own `:bug:` commit.
|
||||
|
||||
### 4. SCSS commit
|
||||
|
||||
Run the `refactor-scss` skill on every `.scss` the PR touched (only
|
||||
those), in its own commit. Keep `refactor/` only for component-family
|
||||
placeholders (`%modal-*`, `%menu-*`, `%input-*`, `%checkbox-*`) and say
|
||||
why in a comment above the `@use`. Unused selectors are deleted, not
|
||||
commented out (list them in the commit body). Typography without an
|
||||
exact match uses the closest DS typography. Run the skill's
|
||||
**Self-review** before committing: no bare element selectors (add
|
||||
classes in the `.cljs`), no copied properties that do nothing, z-index
|
||||
tokens by purpose. Compare against the controls commit: it should be
|
||||
0 px; explain anything else (typically a typography or border change).
|
||||
For files that several modules share or that are huge, stop and propose
|
||||
a separate PR instead.
|
||||
|
||||
### 5. Hand off, PR and tracking
|
||||
|
||||
1. Report to the user: commits, visible changes, kept raw controls,
|
||||
pre-existing bugs, and the compare image path
|
||||
(`<scratch>/diffs/<area>/compare.png`, before | after | diff). Give
|
||||
the push command: `git push -u origin <branch>` (the branch tracks
|
||||
`origin/develop`, so a bare `git push` is refused).
|
||||
2. After the user pushes, check `git rev-parse HEAD origin/<branch>`,
|
||||
then (read `mem:workflow/creating-prs` first):
|
||||
```bash
|
||||
gh pr create --repo penpot/penpot --draft --base develop --head <branch> \
|
||||
--title "<commit subject>" --body-file <body> \
|
||||
--label "WG adoption" --label "no issue required" \
|
||||
--milestone <group issue milestone> --project Main
|
||||
gh pr edit <n> --repo penpot/penpot --remove-label "needs triage"
|
||||
```
|
||||
Never add `no changelog` to a PR: that label goes on issues.
|
||||
Body: AI note, `Part of #<group> (DS migration epic #12375).`, a
|
||||
short What / How, the kept raw controls, an empty `## Screenshots`
|
||||
when something is visible (the user attaches the image). Never
|
||||
`Closes #<group>`. Keep it short: reviewers find long PRs hard to read.
|
||||
3. In the group issue's PRs table, fill your row:
|
||||
`| #<n> | <area>: <controls> (<visible changes>) | @<owner> | Draft |`.
|
||||
4. Post a QA comment on the group issue: `**#<n>: <area>**`, *Where* (how
|
||||
to reach it), *Expected changes* (every visible change; QA treats
|
||||
anything not listed as a bug), *No other change* (what still works
|
||||
the same).
|
||||
|
||||
Do not mark the draft ready or ask for review: the user decides when.
|
||||
|
||||
### 6. Review fixes
|
||||
|
||||
When the reviewer comments on a PR:
|
||||
|
||||
1. Fix each comment in a new commit on the branch (never amend pushed
|
||||
commits). A comment you do not apply gets a short reason the user can
|
||||
post, with an image when it is visual.
|
||||
2. Rerun the area's visual spec before and after the fix with `=all`.
|
||||
Build a compare image for every visible change and give the user its
|
||||
full path, saying which PR it goes to.
|
||||
3. Update everything the fix makes stale, in the same turn: the QA
|
||||
comment (*Expected changes* must list the new visible changes), the
|
||||
PRs table row, and the PR body.
|
||||
4. When working on several PRs in one go, report per PR: branch, new
|
||||
commit, what changed, image path. Never mix two branches in one list.
|
||||
5. Give the push command per branch. If a rebase is coming soon, say
|
||||
that pushing now costs one extra CI run; the user decides.
|
||||
6. After a round of review fixes, check whether the same issue exists in
|
||||
the user's other open PRs of the group, and fix it there too.
|
||||
|
||||
### 7. After merges
|
||||
|
||||
Whenever PRs merge (or go from draft to ready), sync the state column of
|
||||
the group's PRs table with GitHub for **every** row, not only your own:
|
||||
read each row's PR (`gh pr view <n> --json state,isDraft`) and set
|
||||
Merged / In review / Draft. The table drifts when other people merge.
|
||||
|
||||
## Output
|
||||
|
||||
- Branch with 2 commits (controls, SCSS) plus any `:bug:` commit.
|
||||
- Visual spec block with zero-tolerance shots.
|
||||
- Compare image in the scratch folder.
|
||||
- After the push: draft PR, row and QA comment in the group issue.
|
||||
@ -1,25 +0,0 @@
|
||||
"""Pixel diff of two folders of PNG snapshots with the same file names.
|
||||
|
||||
Usage: python3 -I diff.py <before-dir> <after-dir>
|
||||
Prints, per file, the number of differing pixels and their bounding box,
|
||||
or the two sizes when they differ.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PIL import Image, ImageChops
|
||||
|
||||
before, after = sys.argv[1], sys.argv[2]
|
||||
for name in sorted(os.listdir(before)):
|
||||
other = os.path.join(after, name)
|
||||
if not os.path.exists(other):
|
||||
print(name, "missing in", after)
|
||||
continue
|
||||
a = Image.open(os.path.join(before, name)).convert("RGB")
|
||||
b = Image.open(other).convert("RGB")
|
||||
if a.size != b.size:
|
||||
print(name, "size", a.size, b.size)
|
||||
continue
|
||||
diff = ImageChops.difference(a, b)
|
||||
count = sum(1 for p in diff.getdata() if p != (0, 0, 0))
|
||||
print(name, count, diff.getbbox())
|
||||
@ -1,33 +0,0 @@
|
||||
"""Before | after | red diff image, one row per snapshot.
|
||||
|
||||
Usage: python3 -I trip.py <before-dir> <after-dir> <out.png> <scale>
|
||||
Both folders hold PNGs with the same names (crop them first if needed).
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PIL import Image, ImageChops
|
||||
|
||||
before, after, out, scale = sys.argv[1], sys.argv[2], sys.argv[3], int(sys.argv[4])
|
||||
rows = []
|
||||
for name in sorted(os.listdir(before)):
|
||||
a = Image.open(os.path.join(before, name)).convert("RGB")
|
||||
b = Image.open(os.path.join(after, name)).convert("RGB")
|
||||
w, h = max(a.width, b.width), max(a.height, b.height)
|
||||
pad_a = Image.new("RGB", (w, h), (200, 200, 200)); pad_a.paste(a, (0, 0))
|
||||
pad_b = Image.new("RGB", (w, h), (200, 200, 200)); pad_b.paste(b, (0, 0))
|
||||
mask = ImageChops.difference(pad_a, pad_b).convert("L").point(lambda v: 255 if v else 0)
|
||||
red = Image.new("RGB", (w, h), (255, 255, 255))
|
||||
red.paste((255, 0, 0), mask=mask)
|
||||
row = Image.new("RGB", (w * 3 + 20, h), (200, 200, 200))
|
||||
for i, im in enumerate([pad_a, pad_b, red]):
|
||||
row.paste(im, (i * (w + 10), 0))
|
||||
rows.append(row.resize((row.width * scale, row.height * scale), Image.NEAREST))
|
||||
width = max(r.width for r in rows)
|
||||
height = sum(r.height + 10 for r in rows)
|
||||
canvas = Image.new("RGB", (width, height), (255, 255, 255))
|
||||
y = 0
|
||||
for r in rows:
|
||||
canvas.paste(r, (0, y))
|
||||
y += r.height + 10
|
||||
canvas.save(out)
|
||||
@ -1,43 +0,0 @@
|
||||
---
|
||||
name: nrepl-eval
|
||||
description: Evaluate Clojure code via nREPL using the standalone scripts/nrepl-eval.mjs CLI tool.
|
||||
---
|
||||
|
||||
# nREPL Eval
|
||||
|
||||
Evaluate Clojure (or ClojureScript) code via a running nREPL server using
|
||||
`scripts/nrepl-eval.mjs`.
|
||||
|
||||
Full documentation: `mem:scripts/nrepl-eval` (file: `.serena/memories/scripts/nrepl-eval.md`)
|
||||
|
||||
## When to use
|
||||
|
||||
- Evaluating Clojure or ClojureScript code against the running nREPL
|
||||
sessions (backend 6064, frontend 3447) — live inspection, patching, or
|
||||
debugging.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs [options] [<code>]
|
||||
```
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--backend` | Connect to backend nREPL (port 6064) | — |
|
||||
| `--frontend` | Connect to frontend nREPL (port 3447) | — |
|
||||
| `-p, --port PORT` | nREPL server port | `6064` |
|
||||
| `-H, --host HOST` | nREPL server host | `127.0.0.1` |
|
||||
| `-t, --timeout MS` | Timeout in milliseconds | `120000` |
|
||||
| `--reset-session` | Discard stored session and start fresh | — |
|
||||
| `-e, --last-error` | Evaluate `*e` to retrieve the last exception | — |
|
||||
| `-h, --help` | Show help message | — |
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs '(+ 1 2 3)'
|
||||
./scripts/nrepl-eval.mjs --backend '(+ 1 2 3)'
|
||||
./scripts/nrepl-eval.mjs --frontend '(js/alert "hi")'
|
||||
./scripts/nrepl-eval.mjs -e
|
||||
```
|
||||
@ -1,105 +0,0 @@
|
||||
---
|
||||
name: plan-review-criteria
|
||||
description: Criteria for reviewing implementation plans, including evidence, severity and verdicts. Loaded by the review-plan subagent.
|
||||
---
|
||||
|
||||
# Plan Review Criteria
|
||||
|
||||
Evaluate whether the plan can deliver its stated goal safely and whether a
|
||||
skilled implementer can execute and verify it without guessing. Perfect plans
|
||||
do not exist: do not block on personal preferences or missing sections that
|
||||
add no useful information. Apply each criterion in proportion to the plan's
|
||||
scope and risks; explain when a criterion is not applicable.
|
||||
|
||||
Follow `review-plan` for delegation, permitted edits, decisions and handover.
|
||||
Do not run tests, builds or other state-changing checks during the review.
|
||||
|
||||
## Evidence and scope
|
||||
|
||||
- Read the plan, relevant code, project guidance and module memories. Check
|
||||
that cited paths, symbols and proposed commands exist and are appropriate;
|
||||
distinguish verified facts from assumptions and unknowns. Do not invent
|
||||
findings or claim checks were run when they were not.
|
||||
- Check that tasks serve the stated goal and respect existing requirements and
|
||||
decisions. Identify scope creep, contradictions and dependencies on external
|
||||
work. A consequential change in scope, behavior or tradeoffs needs the user's
|
||||
decision.
|
||||
- Evaluate the proposed approach even without code snippets: does it solve the
|
||||
problem, fit existing conventions and avoid unnecessary complexity? If the
|
||||
plan includes code, also check its types, signatures and examples.
|
||||
|
||||
## Six review axes
|
||||
|
||||
### 1. Completeness
|
||||
|
||||
Are the problem, intended outcome, affected areas and relevant constraints
|
||||
clear? Does the plan explain important decisions and include an appropriate
|
||||
testing strategy? Identify missing information that would change the outcome
|
||||
or force an implementer to guess. Require risk, rollback, migration or
|
||||
parallelization details only when relevant.
|
||||
|
||||
### 2. Task quality
|
||||
|
||||
Are tasks scoped, ordered and specific enough to execute? Check likely files,
|
||||
dependencies, rationale, acceptance criteria and verification where relevant.
|
||||
Acceptance criteria should describe observable behavior, including important
|
||||
failure cases. Split work when it is too large to understand or verify safely,
|
||||
not merely because it crosses an arbitrary task count.
|
||||
|
||||
### 3. Architecture and sequencing
|
||||
|
||||
Does the approach fit the existing architecture and reuse appropriate
|
||||
facilities? Do dependencies precede their consumers, and can the work be
|
||||
verified as it progresses? Prefer independently testable slices where useful;
|
||||
use checkpoints where risk or coordination warrants them, not at a fixed
|
||||
interval. Flag unrelated work or abstractions that add complexity without
|
||||
helping the goal.
|
||||
|
||||
### 4. Risk coverage
|
||||
|
||||
Consider relevant edge cases, security, performance, compatibility, data
|
||||
integrity, migrations and external dependencies. Where failure has meaningful
|
||||
consequences, check detection, mitigation and rollback or recovery. Do not
|
||||
invent risks or demand a rollback section for changes that do not need one.
|
||||
|
||||
### 5. Actionability and verification
|
||||
|
||||
Can the implementer identify the affected code, follow project conventions and
|
||||
run the stated project-specific checks? Are commands and prerequisites correct?
|
||||
Do tests and other checks demonstrate the intended behavior and important
|
||||
failure cases? Surface unknowns honestly rather than presenting an unverified
|
||||
command or assumption as fact.
|
||||
|
||||
### 6. Proposed solution quality
|
||||
|
||||
Assess correctness, clarity, architecture, security and performance of the
|
||||
proposed solution, even when the plan contains no code. For any code sketches,
|
||||
also check boundary cases, names, types and contracts. Preserve intended
|
||||
behavior unless a change is explicitly agreed; prefer the simplest solution
|
||||
that meets the requirements.
|
||||
|
||||
## Findings and verdict
|
||||
|
||||
Give each reported item a stable ID (`P1`, `P2`, …); retain it throughout the
|
||||
review and any decision walkthrough. For each item state the evidence, impact,
|
||||
concrete remedy and verification needed. For a consequential user decision,
|
||||
include real options, consequences and a recommendation.
|
||||
|
||||
- **Blocking:** implementation cannot proceed safely or an essential user
|
||||
decision remains open. Explain what blocks it.
|
||||
- **Improvement:** a clear correction that preserves the agreed goal, scope
|
||||
and behavior. Apply it under `review-plan`'s permitted-edit rules.
|
||||
- **Optional:** a suggestion that does not prevent implementation. Do not
|
||||
silently expand scope or override the user's decisions.
|
||||
|
||||
Report applied improvements and any outstanding items with their IDs. Give a
|
||||
provisional verdict before any walkthrough and a final verdict after applying
|
||||
the decisions:
|
||||
|
||||
- **Ready:** no blocking issue or essential decision remains.
|
||||
- **Awaiting decisions:** user decisions are needed before the plan is ready.
|
||||
- **Blocked:** required work or evidence is missing and cannot be resolved by
|
||||
applying clear improvements or obtaining the current decisions alone.
|
||||
|
||||
Do not approve an unresolved blocker. A plan may be ready with optional
|
||||
suggestions; identify them without treating them as required.
|
||||
@ -1,266 +0,0 @@
|
||||
---
|
||||
name: planner
|
||||
description: Read-only planning and architecture analysis — produce a structured implementation plan with task breakdown, acceptance criteria, sizing, and checkpoints. Always output to the user with the plan, suggested save path and the next steps.
|
||||
---
|
||||
|
||||
# Planner
|
||||
|
||||
Produce a plan that another engineer or agent can execute without guessing.
|
||||
|
||||
## When to Use
|
||||
|
||||
- The user asks for a plan, design, or analysis of a feature or bug.
|
||||
- The user wants to understand which parts of the codebase a task will touch.
|
||||
- The user needs a step-by-step implementation plan with file paths, function
|
||||
names, and test strategy.
|
||||
- The user asks "how would I implement X?" or "what's involved in fixing Y?".
|
||||
- The user is about to start non-trivial work and wants a bite-sized task
|
||||
breakdown.
|
||||
- A task feels too large or vague to start.
|
||||
- Work needs to be parallelized across multiple agents or sessions.
|
||||
|
||||
Do not use for a small change with obvious scope or an existing executable plan.
|
||||
|
||||
## CRITICAL: Required Reading Before Planning
|
||||
|
||||
Before drafting any plan, work through the project's own guidance:
|
||||
|
||||
1. Read `critical-info` (`.serena/memories/critical-info.md`) — the entry point
|
||||
that describes the monorepo structure and module dependency graph.
|
||||
2. From `critical-info`, identify which modules your task affects.
|
||||
3. Read each affected module's core memory, e.g. `mem:frontend/core`,
|
||||
`mem:backend/core`, `mem:common/core`, `mem:exporter/core`,
|
||||
`mem:render-wasm/core`. Follow `mem:` references deeper as needed.
|
||||
4. For each affected module, note its lint, format, and test commands so the
|
||||
plan can include concrete verification steps.
|
||||
|
||||
Skipping this step is the #1 cause of incorrect or incomplete plans.
|
||||
|
||||
## Constraints
|
||||
|
||||
- You are **analysis-only** — never create, edit, or delete source code. The
|
||||
only file you may write is the plan itself, and only when the command or
|
||||
user explicitly instructs you to save it.
|
||||
- You do **not** run builds, tests, linters, or any commands that modify state.
|
||||
- You do **not** create commits or run commands that modify version control.
|
||||
Read-only inspection such as `git status`, `git log`, `git show` and
|
||||
`git diff` is allowed.
|
||||
- Shell commands must be read-only.
|
||||
- Your output is a structured plan or analysis, ready for handoff to an
|
||||
engineer agent or developer.
|
||||
|
||||
## Planning Process
|
||||
|
||||
1. Define the problem, desired outcome, constraints, and exclusions.
|
||||
2. Trace the current behavior through the affected modules.
|
||||
3. Map dependencies and choose an implementation order that builds foundations
|
||||
before their consumers.
|
||||
4. Identify open product or architecture decisions. Resolve implementation
|
||||
details from existing conventions when they do not affect public behavior.
|
||||
5. Identify edge cases, security and data risks, performance bounds, breaking
|
||||
changes, and external dependencies.
|
||||
6. Split the work into small, ordered tasks. Prefer complete testable slices
|
||||
over unrelated layer-wide batches. Apply DRY and KISS to the proposed
|
||||
implementation.
|
||||
7. Define exact acceptance criteria and verification for every task.
|
||||
8. Add a checkpoint after every two or three tasks in a longer plan.
|
||||
9. State which tasks can run in parallel and which must remain sequential.
|
||||
|
||||
## Task Format
|
||||
|
||||
Each task follows this structure:
|
||||
|
||||
```markdown
|
||||
### Task [N]: [Short descriptive title]
|
||||
|
||||
**Description:** One or two paragraphs explaining what this task accomplishes.
|
||||
Should be clear and concise.
|
||||
|
||||
**Rationale:** Why this task exists and why this approach over the obvious
|
||||
alternatives — design decisions, trade-offs, constraints discovered during
|
||||
analysis. One or two sentences; skip only if genuinely trivial.
|
||||
|
||||
**Code sketch (optional):** Signature-, type-, or shape-level example when the
|
||||
intended interface is non-obvious. Keep it short — a skeleton that fixes the
|
||||
contract (function signature, model fields, error shape), never a full
|
||||
implementation. Omit when the task is mechanical.
|
||||
|
||||
**Acceptance criteria:**
|
||||
- [ ] [Specific, testable condition]
|
||||
- [ ] [Specific, testable condition]
|
||||
|
||||
**Verification:**
|
||||
- [ ] Relevant tests pass (module-specific test command).
|
||||
- [ ] Lint/formatter passes (module-specific check command), if applicable.
|
||||
- [ ] The core flow works end-to-end, if applicable.
|
||||
|
||||
**Dependencies:** [Task numbers this depends on, or "None"]
|
||||
|
||||
**Files likely touched:**
|
||||
- `path/to/file.clj`
|
||||
- `path/to/file_test.clj`
|
||||
|
||||
**Estimated scope:** [XS: 1 file | S: 1-2 files | M: 3-5 files | L: 5+ files]
|
||||
```
|
||||
|
||||
Use commands from `mem:testing` and affected module memories. Never substitute
|
||||
generic text such as "run the tests" when the project documents an exact
|
||||
command.
|
||||
|
||||
When possible, design each task with TDD in mind: acceptance criteria double as a test
|
||||
list, and the natural first step of the task is writing those tests before the
|
||||
implementation. Some tasks resist this (config, migrations, pure wiring) — for those, keep
|
||||
the usual verification steps.
|
||||
|
||||
## Task Sizing
|
||||
|
||||
| Size | Files | Scope | Example |
|
||||
|------|-------|-------|---------|
|
||||
| **XS** | 1 | Single function, config change, or schema tweak | Add a validation rule |
|
||||
| **S** | 1-2 | One handler or component method | Add a new RPC endpoint |
|
||||
| **M** | 3-5 | One vertical feature slice | Bookmark CRUD with tests |
|
||||
| **L** | 5-8 | Multi-component feature | Search with filtering and pagination |
|
||||
| **XL** | 8+ | **Too large — break it down further** | — |
|
||||
|
||||
Split a task when it contains independent outcomes, spans unrelated systems, or cannot be
|
||||
completed and verified in one focused session (if a task is XL, it should be broken into
|
||||
smaller tasks; agents perform best on S and M tasks).
|
||||
|
||||
## Task order and checkpoints
|
||||
|
||||
Arrange tasks so that:
|
||||
|
||||
1. Dependencies are satisfied (build foundation first)
|
||||
2. Each task leaves the system in a working state
|
||||
3. Verification checkpoints occur after every 2-3 tasks
|
||||
4. High-risk tasks are early (fail fast)
|
||||
|
||||
Add explicit checkpoints with the relevant module commands:
|
||||
|
||||
```markdown
|
||||
### Checkpoint: After Tasks 1-3
|
||||
- [ ] Relevant tests pass (module-specific command).
|
||||
- [ ] The relevant build or compilation passes, if applicable.
|
||||
- [ ] The core flow works end-to-end.
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
The plan is always delivered in the response so the user sees it regardless
|
||||
of which agent is running the skill. File writes follow `Constraints` —
|
||||
by default announce the path instead of writing.
|
||||
|
||||
Announce the save path `.agents/plans/YYYY-MM-DD-<slug>.md` (today's date,
|
||||
lowercase hyphen-separated slug, e.g. `2026-09-10-add-batch-get-profiles`;
|
||||
an explicit user path wins).
|
||||
|
||||
### Derived plans
|
||||
|
||||
Never invent a fresh slug when the plan derives from an existing one.
|
||||
The derived name is `<parent-basename>` plus one suffix per level,
|
||||
joined with `--` (double hyphen; single hyphens already separate
|
||||
slug words, so `--` marks where the derivation starts). The parent
|
||||
name is never edited, and no new date is added — the parent prefix
|
||||
already carries its date, which keeps parent and derivatives adjacent
|
||||
in `ls`. Record the real creation date inside the plan (`Created:`).
|
||||
|
||||
Valid names match:
|
||||
|
||||
```
|
||||
^\d{4}-\d{2}-\d{2}-[a-z0-9-]+(--(review-\d{2}|task-\d{2})(-[a-z0-9-]+)?)*\.md$
|
||||
```
|
||||
|
||||
- `review-NN` — a new plan addressing findings of a `review-code` or
|
||||
`review-plan` on already-implemented work. `NN` counts reviews of
|
||||
that parent from `01`. Example: parent
|
||||
`2026-09-14-paste-before-init-crash.md` →
|
||||
`2026-09-14-paste-before-init-crash--review-01.md`,
|
||||
then `--review-02.md`.
|
||||
- `task-NN-<short-slug>` — sub-plan for task `NN` of a high-level
|
||||
roadmap plan. `NN` is the roadmap task number. Example: parent
|
||||
`2026-09-20-upload-pipeline-roadmap.md` →
|
||||
`2026-09-20-upload-pipeline-roadmap--task-01-chunk-upload.md`.
|
||||
Levels chain: `...--task-02-gc--review-01.md`.
|
||||
|
||||
Never use `v2`, `final`, `new`, or `fix2` as suffixes. Keep the
|
||||
optional short slug to 3-4 lowercase hyphen-separated words.
|
||||
|
||||
Every derived plan opens its `Context` with:
|
||||
|
||||
```markdown
|
||||
Parent: `<parent-basename>.md`
|
||||
Source: review-code over `<commit>` (branch `<branch>`) | task `NN` of roadmap `<parent-basename>.md`
|
||||
Created: YYYY-MM-DD
|
||||
```
|
||||
|
||||
End the response by suggesting the next steps: `@review-plan` to get a second
|
||||
opinion on the plan and `@implement-plan` to execute it.
|
||||
|
||||
### Plan Structure
|
||||
|
||||
Use this document shape:
|
||||
|
||||
```markdown
|
||||
# Plan: Title
|
||||
|
||||
Status: draft | reviewed | done
|
||||
Review Log:
|
||||
- YYYY-MM-DDTHH:MM:SSZ — <what changed and why, one line per entry>
|
||||
|
||||
## Context
|
||||
## Affected Modules
|
||||
## Architecture Decisions
|
||||
## Risks and Considerations
|
||||
## Approach
|
||||
## Task List
|
||||
## Verification and Testing
|
||||
## Parallelization
|
||||
## Open Questions
|
||||
```
|
||||
|
||||
`Status` and `Review Log` are write-restricted metadata, not free text:
|
||||
|
||||
- `make-a-plan` creates every plan with `Status: draft` and an empty
|
||||
log. `reviewed` never means "a review was emitted" — it means
|
||||
"review feedback was incorporated". A plan approved with no changes
|
||||
goes `draft` → `done` without passing through `reviewed`.
|
||||
- Before implementation, `review-plan` applies clear review feedback and
|
||||
agreed decisions, flips to `reviewed` and appends one UTC ISO 8601 entry
|
||||
when the plan changed. `make-a-plan` can also do this for additional
|
||||
feedback when the user asks.
|
||||
- `implement-plan` flips to `done` on completion and appends one line
|
||||
with the issue URL when one exists (standalone mode); otherwise
|
||||
just `done`. Never record commit hashes — they rot on amend/rebase
|
||||
and git already links the commit.
|
||||
- The log is append-only: never rewrite or delete lines.
|
||||
- `review-code` never writes these fields.
|
||||
|
||||
Omit empty sections only when they do not apply. Every implementation task
|
||||
still requires acceptance criteria, verification, dependencies, likely files,
|
||||
and scope.
|
||||
|
||||
When the plan is purely analytical (e.g. a code review or feasibility study
|
||||
with no implementation), skip the **Approach** and **Task List** sections and
|
||||
lead with **Findings** instead, keeping the rest of the structure.
|
||||
|
||||
## Common Rationalizations
|
||||
|
||||
| Rationalization | Reality |
|
||||
|---|---|
|
||||
| "I'll figure it out as I go" | That's how you end up with a tangled mess and rework. 10 minutes of planning saves hours. |
|
||||
| "The tasks are obvious" | Write them down anyway. Explicit tasks surface hidden dependencies and forgotten edge cases. |
|
||||
| "Planning is overhead" | Planning is the task. Implementation without a plan is just typing. |
|
||||
| "I can hold it all in my head" | Context windows are finite. Written plans survive session boundaries and compaction. |
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Before delivering the plan, confirm:
|
||||
|
||||
- [ ] Every task has acceptance criteria
|
||||
- [ ] Every task has a verification step
|
||||
- [ ] Task dependencies are identified and ordered correctly
|
||||
- [ ] No task is XL or larger — break it down instead
|
||||
- [ ] Checkpoints exist after every 2-3 tasks
|
||||
- [ ] The response states the plan's path (saved or suggested) and suggests
|
||||
`@review-plan` and `@implement-plan`
|
||||
- [ ] The plan is ready for human review
|
||||
@ -1,421 +0,0 @@
|
||||
---
|
||||
name: refactor-scss
|
||||
description: Migrate one Penpot SCSS module off the deprecated styles under frontend/resources/styles/common/refactor/ to the current Design System (ds/ tokens, mixins and px2rem). Pure refactor, no visual change. Use when the user asks to refactor, migrate or clean up a .scss file to the DS.
|
||||
---
|
||||
|
||||
# Skill: refactor-scss
|
||||
|
||||
Refactor a single SCSS file so that it no longer depends on the deprecated
|
||||
styling system, using the current Design System instead.
|
||||
|
||||
This is a **pure refactor**. Do NOT introduce visual or behavioral changes.
|
||||
Preserve the generated CSS behavior exactly unless a rule below requires
|
||||
otherwise. The rules that allow a small visible change (closest DS
|
||||
typography, DS border width) want it shown in the visual check and listed
|
||||
in the report and the commit body.
|
||||
|
||||
Reviewers read every line of the result. A line that is
|
||||
copied but does nothing, a bare element selector or a raw z-index comes
|
||||
back as a review comment, so the rules below are strict about them.
|
||||
|
||||
## Input
|
||||
|
||||
- The path to one `.scss` file (e.g. `frontend/src/app/main/ui/foo/bar.scss`).
|
||||
- If no path is given, ask for it. Do not pick a file yourself.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Read the target `.scss` file and the `.cljs` file with the same name next
|
||||
to it.
|
||||
2. Read every deprecated source the file uses, so you inline real values,
|
||||
never remembered ones:
|
||||
- `frontend/resources/styles/common/refactor/mixins.scss`
|
||||
- `frontend/resources/styles/common/refactor/design-tokens.scss`
|
||||
- `frontend/resources/styles/common/refactor/spacing.scss`
|
||||
- any other file under `frontend/resources/styles/common/refactor/` that
|
||||
the stylesheet pulls in (placeholders, borders, z-index, fonts...).
|
||||
3. Read the DS modules you will map to:
|
||||
- `frontend/src/app/main/ui/ds/_utils.scss` (`px2rem()`)
|
||||
- `frontend/src/app/main/ui/ds/spacing.scss`
|
||||
- `frontend/src/app/main/ui/ds/_sizes.scss`
|
||||
- `frontend/src/app/main/ui/ds/typography.scss` (`use-typography()`)
|
||||
- `frontend/src/app/main/ui/ds/_borders.scss`
|
||||
- `frontend/src/app/main/ui/ds/z-index.scss`
|
||||
4. Apply the rules below and edit the file in place.
|
||||
5. Verify (see **Verification** and **Visual check**).
|
||||
6. Report (see **Output**). Do not commit. When the refactor goes with other
|
||||
changes, it belongs in its own commit, after the task's commits.
|
||||
|
||||
## Primary goal
|
||||
|
||||
After the refactor, the SCSS file must not depend on anything under
|
||||
`frontend/resources/styles/common/refactor/`. This includes imports, mixins,
|
||||
placeholder selectors, functions, deprecated variables and design-token
|
||||
aliases.
|
||||
|
||||
### Partial refactor
|
||||
|
||||
Some placeholders stand for a whole component family that will move to the DS
|
||||
in its own PR: `%input-element`, `%input-with-label`, `%input-checkbox`,
|
||||
`%modal-*`, `%menu-dropdown`, `%menu-item-base`, `%shortcut-*` and similar.
|
||||
Inlining them in a small task produces big, risky diffs. In that case:
|
||||
|
||||
- migrate everything else (tokens, sizes, spacing, small mixins);
|
||||
- keep the `refactor/` import only for those placeholders;
|
||||
- list each remaining dependency and the selectors that use it in the report,
|
||||
so the user can plan it as a separate PR.
|
||||
|
||||
## General rules
|
||||
|
||||
- Keep the resulting SCSS as small and readable as possible.
|
||||
- Avoid duplicated declarations and duplicated selectors.
|
||||
- Do not change class names.
|
||||
- Do not change DOM structure.
|
||||
- Do not change specificity unless required.
|
||||
- Follow the project's SCSS formatting (Prettier, see **Verification**).
|
||||
|
||||
## Imports
|
||||
|
||||
Use the DS modules. Import them with `as *` when they expose mixins,
|
||||
variables or functions:
|
||||
|
||||
```scss
|
||||
@use "ds/_utils.scss" as *;
|
||||
```
|
||||
|
||||
not
|
||||
|
||||
```scss
|
||||
@use "ds/_utils.scss" as utils;
|
||||
```
|
||||
|
||||
Do not introduce namespaces such as `utils.`, `typography.`, `borders.`,
|
||||
`spacing.` or `sizes.` unless a module explicitly requires one.
|
||||
|
||||
Remove every unused import.
|
||||
|
||||
## Deprecated mixins
|
||||
|
||||
If the stylesheet uses mixins from
|
||||
`frontend/resources/styles/common/refactor/mixins.scss`:
|
||||
|
||||
- locate the mixin;
|
||||
- inline its generated CSS (then apply the rest of these rules to it);
|
||||
- remove the `@include`;
|
||||
- remove the import.
|
||||
|
||||
Do not leave deprecated mixins.
|
||||
|
||||
## Placeholders
|
||||
|
||||
Replace every `@extend %placeholder` with the generated CSS. Remove every
|
||||
deprecated placeholder, except the component-family ones described in
|
||||
**Partial refactor**.
|
||||
|
||||
Inline only what does something on that element. Do not paste a
|
||||
placeholder (or mixin) body as it is. For each property, check:
|
||||
|
||||
- **It applies to the element.** `fill`, `stroke` and `stroke-width` only
|
||||
act on SVG shapes: drop them from a `span`, `div` or `button`.
|
||||
`color: transparent` on a wrapper that only holds an icon does nothing
|
||||
useful: drop it and set the real color.
|
||||
- **It is not the default or already given.** `stroke-width: 1px` is the
|
||||
SVG default; `fill: none` and `stroke: currentColor` are already set by
|
||||
the DS `icon*` (`.icon`). Drop them.
|
||||
- **A later rule does not override it.**
|
||||
- **It is not a stroke width on a DS icon.** DS icons keep the DS
|
||||
stroke width; drop any `stroke-width` that only imitates the legacy
|
||||
icon (1.33px, 2.25px...).
|
||||
|
||||
`%button-icon` is the usual case: on a wrapper it leaves `color:
|
||||
transparent; fill: none; stroke-width: 1px`, and none of the three is
|
||||
needed.
|
||||
|
||||
## Element selectors
|
||||
|
||||
Do not style HTML elements directly (`svg {}`, `h3 {}`, `button {}`,
|
||||
`input.x {}`, or `.component { svg {} }`).
|
||||
|
||||
When no class exists, **add one in the `.cljs`**: a DS component takes
|
||||
`:class (stl/css :x)` (`icon*`, `heading*`, `button*`…), and a plain
|
||||
element takes `{:class (stl/css :x)}`. Touching the `.cljs` for this is
|
||||
part of the refactor. A comment like "the icon has no class" is not an
|
||||
answer.
|
||||
|
||||
Prefer the DS component when one fits: `heading*` for `h1`–`h6`
|
||||
(`:level`, `:typography`), `text*` for text, `icon*` for icons.
|
||||
|
||||
`element.class` (e.g. `input.element-title`) is not needed to beat the
|
||||
input defaults: try the class alone and check it in the visual check.
|
||||
|
||||
An element selector stays only for markup you cannot reach (injected
|
||||
HTML, third-party widgets), with a comment saying so.
|
||||
|
||||
**Icon colors.** A DS icon draws with `stroke: currentColor`. Set
|
||||
`color` on the icon class or its parent, never `stroke`/`fill` on an
|
||||
`svg` selector.
|
||||
|
||||
## Color variables
|
||||
|
||||
Whenever a color changes across states, introduce a component-local CSS
|
||||
variable.
|
||||
|
||||
Instead of
|
||||
|
||||
```scss
|
||||
.button {
|
||||
color: var(--color-foreground-primary);
|
||||
|
||||
&:hover {
|
||||
color: var(--color-accent-primary);
|
||||
}
|
||||
|
||||
&:disabled {
|
||||
color: var(--color-foreground-disabled);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
write
|
||||
|
||||
```scss
|
||||
.button {
|
||||
--button-foreground-color: var(--color-foreground-primary);
|
||||
|
||||
color: var(--button-foreground-color);
|
||||
|
||||
&:hover {
|
||||
--button-foreground-color: var(--color-accent-primary);
|
||||
}
|
||||
|
||||
&:disabled {
|
||||
--button-foreground-color: var(--color-foreground-disabled);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use this pattern for `color`, `background-color`, `border-color`,
|
||||
`outline-color`, `fill`, `stroke` and any similar property.
|
||||
|
||||
## Design token aliases
|
||||
|
||||
Remove every alias defined in
|
||||
`frontend/resources/styles/common/refactor/design-tokens.scss`. Resolve each
|
||||
alias to its final DS token. Do not keep intermediate aliases.
|
||||
|
||||
Instead of `color: var(--button-foreground-hover);` write
|
||||
`color: var(--color-accent-primary);`.
|
||||
|
||||
## Spacing
|
||||
|
||||
Replace deprecated spacing variables using
|
||||
`frontend/src/app/main/ui/ds/spacing.scss` for `margin`, `padding`, `gap` and
|
||||
`inset`.
|
||||
|
||||
If no exact spacing token exists, use `px2rem()`. Do not approximate with the
|
||||
closest token.
|
||||
|
||||
## Sizes
|
||||
|
||||
Replace deprecated size variables using
|
||||
`frontend/src/app/main/ui/ds/_sizes.scss` for `inline-size`, `block-size`,
|
||||
`width`, `height` and min/max sizes.
|
||||
|
||||
If no exact size exists, use `px2rem()` from
|
||||
`frontend/src/app/main/ui/ds/_utils.scss`. Do not leave hardcoded pixel values
|
||||
unless absolutely necessary.
|
||||
|
||||
## Typography
|
||||
|
||||
When any of `font-size`, `font-weight`, `line-height`, `font-family` or
|
||||
`letter-spacing` appear, check whether together they match one of the
|
||||
definitions in `frontend/src/app/main/ui/ds/typography.scss`.
|
||||
|
||||
If they match, replace all the individual declarations with:
|
||||
|
||||
```scss
|
||||
@include use-typography("<typography-name>");
|
||||
```
|
||||
|
||||
If there is no exact match, use the **closest DS typography** (same role:
|
||||
a panel title in uppercase → `headline-small`, body text → `body-*`),
|
||||
even when more than one detail differs. Show the change in the visual
|
||||
check and state it in the report and in the commit body (e.g. "11px
|
||||
regular → 12px medium").
|
||||
|
||||
Keep the original declarations only when no DS typography fits the role
|
||||
(e.g. a 32px display number in a 14px-only area); then leave a TODO
|
||||
naming the closest one and ask the user.
|
||||
|
||||
## Borders
|
||||
|
||||
Replace deprecated border radius and border width values using
|
||||
`frontend/src/app/main/ui/ds/_borders.scss`. A raw width close to a token
|
||||
(`1.75px`) takes the token (`$b-2`); say so in the report. If nothing is
|
||||
close, use `px2rem()`.
|
||||
|
||||
## Logical properties
|
||||
|
||||
Prefer logical properties whenever they preserve the existing behavior:
|
||||
|
||||
- `width` → `inline-size`
|
||||
- `height` → `block-size`
|
||||
- `max-width` → `max-inline-size`
|
||||
- `margin-left` → `margin-inline-start`
|
||||
- `padding-right` → `padding-inline-end`
|
||||
- `top` → `inset-block-start`
|
||||
- `left` → `inset-inline-start`
|
||||
|
||||
## Z-index
|
||||
|
||||
Never leave magic z-index values. Use tokens from
|
||||
`frontend/src/app/main/ui/ds/z-index.scss`, and pick the token **by what
|
||||
the element is, not by its number**: a dropdown, menu or select →
|
||||
`--z-index-dropdown`; a toolbar, palette, sidebar or panel →
|
||||
`--z-index-panels`; and so on (read the comments in that file). A raw `4`
|
||||
on a dropdown becomes `var(--z-index-dropdown)` even though the number
|
||||
changes. Check the visual check for overlap.
|
||||
|
||||
Leave a TODO only when the element fits no token's purpose.
|
||||
|
||||
Legacy z-index values between sibling elements that do not overlap can be
|
||||
removed, but only when the **Visual check** shows no pixel change in every
|
||||
state that matters (open menus, collapsed, hidden). Say so in the report.
|
||||
|
||||
## Duplicated properties
|
||||
|
||||
Within the same selector, avoid duplicated properties, duplicated CSS
|
||||
variables and duplicated declarations introduced by the refactor. Allow
|
||||
duplicates only when they are intentional browser fallbacks.
|
||||
|
||||
## Unused classes
|
||||
|
||||
Open the `.cljs` file with the same name as the SCSS file and check whether
|
||||
every CSS class is referenced (as `(stl/css :class-name)`,
|
||||
`(stl/css-case :class-name ...)` or similar).
|
||||
|
||||
CSS modules prefix every class with its module
|
||||
(`main_ui_<path>__<class>`), so only the `.cljs` of the same module can use
|
||||
it. Before calling a selector unused, also search the whole `frontend/src`
|
||||
for the class name as a string, a `:global` use, or a dynamic class built
|
||||
at runtime.
|
||||
|
||||
If a selector is unused after that check:
|
||||
|
||||
- delete it (do not leave it commented out, do not add a TODO);
|
||||
- list every deleted selector in the report and in the commit body.
|
||||
|
||||
A rule that matches nothing because the markup changed (e.g. `.icon svg`
|
||||
when the DS icon is the `svg` itself) is deleted the same way. Before
|
||||
deleting a descendant rule, check every element it matched, not only
|
||||
the one you changed: `.placeholder .icon svg` can also color a second
|
||||
icon, which then has no color (in the devenv it shows in the debug
|
||||
yellow).
|
||||
|
||||
Every DS icon needs a color of its own (`color` on its class or on a
|
||||
parent with a defined color): an icon that inherits nothing shows in
|
||||
the debug yellow in the devenv.
|
||||
|
||||
Classes used by markup injected from outside (e.g. an external widget)
|
||||
stay, with a comment saying so.
|
||||
|
||||
Also check the other direction: every class the `.cljs` passes to
|
||||
`stl/css` / `stl/css-case` must exist in the `.scss`. A missing one silently
|
||||
does nothing; report it.
|
||||
|
||||
## Sass and CSS pitfalls
|
||||
|
||||
- Sass does not evaluate functions inside CSS custom properties. Write
|
||||
`--button-height: calc(var(--height) - #{px2rem(16)});`, not
|
||||
`... - px2rem(16))`, or the browser gets the literal `px2rem(16)`.
|
||||
- A `/` between Sass size variables is read as a division
|
||||
(`grid-template: $sz-32 / auto` breaks). Split it into
|
||||
`grid-template-rows` / `grid-template-columns` (or areas).
|
||||
- A CSS-wide keyword cannot be part of a shorthand: `inset-inline: unset 0`
|
||||
is invalid and stylelint does not catch it. Use the longhands.
|
||||
- After editing, check the compiled CSS for any of these
|
||||
(`frontend/resources/public/css/main.css`, search the module class).
|
||||
|
||||
## Uncertainty
|
||||
|
||||
If any transformation is uncertain, never guess. Leave a TODO in the code
|
||||
and explain why in the report.
|
||||
|
||||
## Self-review
|
||||
|
||||
Before the report, read the final diff as the reviewer would and search
|
||||
the file for these; each hit must be fixed or have a real reason:
|
||||
|
||||
```bash
|
||||
rg -n 'svg \{|^\s*(h[1-6]|p|hr|input|button|span|a)\b[^-_]|[a-z]\.[a-z-]+ \{' <file>
|
||||
rg -n 'stroke-width|color: transparent|fill: none|z-index: [0-9]|TODO' <file>
|
||||
rg -n '[0-9]\.[0-9]+px|[^(]\b[0-9]+px' <file>
|
||||
```
|
||||
|
||||
- an element selector → add a class (see **Element selectors**);
|
||||
- a property that does not apply or is a default → drop it (see
|
||||
**Placeholders**);
|
||||
- a raw z-index → token by purpose;
|
||||
- a typography TODO → closest DS typography;
|
||||
- a raw px value → token or `px2rem()`.
|
||||
|
||||
## Verification
|
||||
|
||||
Run the project scripts from `frontend/` inside the devenv container
|
||||
(`penpot-devenv-ws<N>-main` for workspace `wsN`; never run `pnpm` on the
|
||||
host), writing output to a file first (never pipe lint output into
|
||||
filters):
|
||||
|
||||
```bash
|
||||
docker exec -u penpot -w /home/penpot/penpot/frontend penpot-devenv-ws<N>-main \
|
||||
bash -lc 'pnpm run lint:scss; echo LINT=$?; pnpm run check-fmt:scss; echo FMT=$?' \
|
||||
> <scratch>/scss-check.txt 2>&1
|
||||
```
|
||||
|
||||
If the format check fails, run `pnpm run fmt:scss` and check again.
|
||||
|
||||
**Dependency check.** A grep for `refactor/` is not enough. For the final
|
||||
file, check:
|
||||
|
||||
- `@use` / `@import` sources: all must be `ds/...` (or the sidebar common
|
||||
module, which is DS-only);
|
||||
- every `@include`, `@extend` and `%placeholder`;
|
||||
- every `$variable`: which imported module defines it (`$br-*` exists in both
|
||||
`ds/_borders.scss` and `refactor/borders.scss`; the `@use` decides);
|
||||
- every `var(--token)`: defined in the DS (colors, spacing, sizes) or only as a
|
||||
`refactor/` alias.
|
||||
|
||||
Report any failure as it is; do not hide it.
|
||||
|
||||
## Visual check
|
||||
|
||||
"No visual change" must be shown, not assumed. When a visual spec covers the
|
||||
component (`frontend/playwright/ui/visual-specs/`, project `ds`), or one can be
|
||||
added:
|
||||
|
||||
1. Capture before (base branch styles) and after with
|
||||
`pnpm exec playwright test --project=ds -g "<test>" --update-snapshots=all`
|
||||
in the container. `=all` is required: plain `--update-snapshots` only
|
||||
rewrites past the 0.5% tolerance, so a real diff can look like 0.
|
||||
2. Before each capture, wait until the compiled CSS/JS shows the expected
|
||||
version (search `main.css` for the changed rule).
|
||||
3. Compare with zero tolerance. If pixels change, build the before | after |
|
||||
diff (red) image and explain every change in the report.
|
||||
|
||||
## Output
|
||||
|
||||
1. The diff of the file (`git diff -- <file>`).
|
||||
2. A summary with:
|
||||
- removed imports;
|
||||
- removed deprecated mixins;
|
||||
- removed placeholders;
|
||||
- typography replacements;
|
||||
- design-token aliases removed;
|
||||
- logical properties introduced;
|
||||
- deleted unused selectors;
|
||||
- TODOs requiring manual review, each with its reason;
|
||||
- remaining `refactor/` dependencies (partial refactor) and where they are
|
||||
used;
|
||||
- classes missing from the `.scss`, if any.
|
||||
3. The verification results, including the visual check (pixel count, or why
|
||||
it was not possible).
|
||||
@ -1,134 +0,0 @@
|
||||
---
|
||||
name: refine-prompt
|
||||
description: Refine and improve a user-supplied prompt for maximum clarity and effectiveness using prompt-engineering best practices and Penpot project context. Outputs a rewritten prompt (and brief rationale); never executes the prompt.
|
||||
---
|
||||
|
||||
# Refine Prompt
|
||||
|
||||
Expert prompt-engineering pass on a user-supplied prompt. Takes a draft prompt
|
||||
and returns a clearer, more effective, well-structured version — ready to be
|
||||
used with any AI model. Never executes the prompt itself.
|
||||
|
||||
## When to Use
|
||||
|
||||
- The user shares a prompt and asks to improve, refine, polish, or rewrite it.
|
||||
- The user asks "make this prompt better" or "can you clean this up?".
|
||||
- The user wants to add structure, constraints, examples, or output format to
|
||||
a vague prompt.
|
||||
- The user wants a prompt adapted for a specific target model, audience, or
|
||||
task type.
|
||||
|
||||
Do **not** use this skill to actually answer the prompt or do the task — it
|
||||
only rewrites the prompt.
|
||||
|
||||
## Role
|
||||
|
||||
You are an expert Prompt Engineer with strong knowledge of Penpot. Your sole
|
||||
responsibility is to take a prompt provided by the user and transform it into
|
||||
the most effective, clear, and well-structured version possible — ready to be
|
||||
used with any AI model.
|
||||
|
||||
You do **not** execute tasks. You do **not** write code. You only design and
|
||||
refine prompts.
|
||||
|
||||
## Required Reading Before Refining
|
||||
|
||||
Before rewriting, internalize the project context the prompt will likely run
|
||||
against:
|
||||
|
||||
1. Read `AGENTS.md` (root) for the project-level rules and conventions.
|
||||
2. Read `.serena/memories/critical-info.md` (or the equivalent entry point) to
|
||||
understand the module layout (`frontend`, `backend`, `common`,
|
||||
`render-wasm`, `exporter`, `mcp`, `plugins`, `library`).
|
||||
3. Skim the relevant module's core memory (`mem:frontend/core`,
|
||||
`mem:backend/core`, etc.) when the prompt targets a specific module — this
|
||||
lets you inject precise vocabulary, file conventions, and test commands
|
||||
into the refined prompt.
|
||||
|
||||
This step matters most when the user is preparing a prompt *about* the
|
||||
Penpot codebase. For generic prompts, focus on prompt-engineering principles
|
||||
and only weave in Penpot context when it is clearly relevant.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Analyze the original prompt: identify its intent, target audience,
|
||||
ambiguities, missing context, and structural weaknesses.
|
||||
- Ask clarifying questions if the intent is unclear or if critical information
|
||||
is missing (e.g. target model, expected output format, tone, constraints).
|
||||
Keep questions concise and grouped. Prefer to ask 1–4 questions at once
|
||||
rather than one at a time. **Use the `question` tool** to ask them so the
|
||||
user gets a structured multi-choice UI; reserve a plain `## Clarifying
|
||||
questions` markdown section for cases where the `question` tool is
|
||||
unavailable or the question is genuinely open-ended.
|
||||
- Rewrite the prompt using prompt-engineering best practices (see below).
|
||||
- Preserve the user's original intent — do not change the underlying task.
|
||||
- When the user provides Penpot project context, weave in the relevant
|
||||
conventions, module paths, and tooling.
|
||||
|
||||
## Prompt Engineering Principles
|
||||
|
||||
Apply these techniques when refining prompts:
|
||||
|
||||
- **Be specific and explicit**: Replace vague instructions with precise ones.
|
||||
- **Set the context**: Include background information the model needs to
|
||||
perform well.
|
||||
- **Specify the output format**: State the desired structure, length, tone,
|
||||
or format (e.g. bullet list, JSON, step-by-step).
|
||||
- **Add constraints**: Include what the model should avoid or not do.
|
||||
- **Use examples** (few-shot): When applicable, suggest adding examples to
|
||||
anchor the model's behaviour.
|
||||
- **Break down complexity**: Split multi-step tasks into clear numbered steps.
|
||||
- **Avoid ambiguity**: Remove pronouns and references that could be
|
||||
misinterpreted.
|
||||
- **Chain of thought**: For reasoning tasks, include "Think step by step."
|
||||
- **Role framing**: When helpful, give the model a clear role
|
||||
("You are a senior backend engineer...").
|
||||
- **Tool awareness**: When the prompt targets an agentic model, mention
|
||||
relevant tools (`grep`, `glob`, `read`, `bash`, etc.) so the model uses the
|
||||
right surface.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do **not** execute the prompt yourself.
|
||||
- Do **not** answer the question inside the prompt.
|
||||
- Do **not** add unnecessary verbosity — prompts should be as short as they
|
||||
can be while remaining complete.
|
||||
- Always preserve the user's original intent.
|
||||
- If the user provides Penpot project context, prefer Penpot-specific
|
||||
vocabulary over generic terms (e.g. name actual modules and `mem:`
|
||||
references instead of "the codebase").
|
||||
|
||||
## Output Format
|
||||
|
||||
Deliver the result in the response as two clearly separated blocks:
|
||||
|
||||
1. **Refined prompt** — a single fenced code block (markdown ```) containing
|
||||
the rewritten prompt, ready to copy and use.
|
||||
2. **What changed (brief)** — a short bulleted list of the most important
|
||||
changes you made and why (3–7 bullets max). Skip the rationale if the
|
||||
changes are trivial.
|
||||
|
||||
If you asked clarifying questions via the `question` tool, stop and wait for
|
||||
the answers before producing a refined prompt. If the `question` tool was not
|
||||
available and you asked the questions in chat, list them in a separate
|
||||
**Clarifying questions** section above the refined prompt and stop — do not
|
||||
produce a refined prompt until the user answers. If the user explicitly told
|
||||
you to proceed without questions (e.g. "just rewrite it"), make reasonable
|
||||
assumptions and note them under **Assumptions made** in the rationale block.
|
||||
|
||||
## File Persistence
|
||||
|
||||
Always persist the refined prompt to disk so it can be re-used later, versioned
|
||||
in git, and shared with other agents. The response still contains the prompt
|
||||
and rationale blocks; the file is an additional artifact, not a replacement.
|
||||
|
||||
- Save the refined prompt (the body inside the fenced code block, **without**
|
||||
the surrounding ``` fences) to `.opencode/prompts/<descriptive-name>.md`.
|
||||
- Use a **kebab-case** filename that summarises the task, e.g.
|
||||
`add-error-reports-management-rpc.md`, `backend-rpc-security-audit.md`. No
|
||||
spaces, no uppercase, no version numbers or dates in the filename.
|
||||
- If `.opencode/prompts/` does not exist, create it before writing.
|
||||
- If a file with the same name already exists, overwrite it (the file is the
|
||||
refined prompt, not a log).
|
||||
- Only skip the file write when the user explicitly opts out (e.g. "don't save
|
||||
this one", "just show it in the chat"). When in doubt, save it.
|
||||
@ -1,47 +0,0 @@
|
||||
---
|
||||
name: resolve-git-conflicts
|
||||
description: Conflict resolution flow — understand the local git conflicts, present a resolution plan, and resolve them after the user approves it. Never continues the rebase. Use it when the repo has unresolved conflicts (rebase, merge, cherry-pick) or the user asks to resolve them.
|
||||
---
|
||||
|
||||
# Resolve Git Conflicts
|
||||
|
||||
Resolve conflicts in the local repository. The user handles finishing the
|
||||
rebase themselves — you must **never** run `git rebase --continue`,
|
||||
`git rebase --skip`, `git merge --continue`, or anything similar.
|
||||
|
||||
## When to use
|
||||
|
||||
- The repository has unresolved conflicts — during a rebase, merge, or
|
||||
cherry-pick — whether the user asks about them or not.
|
||||
- The user asks to resolve conflicts, in any phrasing: "fix the merge
|
||||
conflicts", "resolve these", "what's conflicting here?".
|
||||
|
||||
## Phase 1 — Understand the problem (read-only)
|
||||
|
||||
1. Run `git status` to detect the conflict state (rebase, merge, cherry-pick, etc.) and list conflicted files.
|
||||
2. For each conflicted (unmerged) file, understand the situation **without modifying anything**:
|
||||
- Read the file and identify the conflict markers (`<<<<<<<`, `=======`, `>>>>>>>`).
|
||||
- Inspect both sides — `git show <ours>:<file>` and `git show <theirs>:<file>` — plus `git log`/`git show` on the commits involved to understand intent.
|
||||
- Identify what each side changed and why, and how they should be combined.
|
||||
|
||||
## Phase 2 — Present the resolution plan
|
||||
|
||||
3. **Present a clear plan to the user before touching any file.** For each conflicted file, state:
|
||||
- What each side changed and why.
|
||||
- Your proposed resolution and the reasoning behind it.
|
||||
- How the two sides are combined (both additive → merge; both modify the same code → keep the semantically correct version, merging intent from both sides when clear from code and context).
|
||||
4. **Ask the user only when genuinely unclear.** Do not ask about anything you can determine yourself from the code, commit messages, or context. Only decisions that are not determinable and change the outcome (e.g. conflicting product decisions, which side to discard) warrant a question. **Collect all such questions together in an "Open Questions" section at the end of the plan**, so the user has full context to answer them properly.
|
||||
5. **Wait for the user to accept the plan** (and answer any open questions) before editing, staging, or otherwise modifying anything.
|
||||
|
||||
## Phase 3 — Execute
|
||||
|
||||
6. Resolve each conflicted file by editing the file to the agreed merged content and removing all conflict markers.
|
||||
|
||||
## Phase 4 — Stage and verify
|
||||
|
||||
7. **Stage every resolved file** with `git add <file>`. Do not stage unrelated untracked files unless clearly part of the resolution.
|
||||
8. Verify no conflict markers remain (search for `<<<<<<<` / `>>>>>>>` in resolved files) and that `git status` shows no unmerged paths.
|
||||
|
||||
## Phase 5 — Report
|
||||
|
||||
9. Briefly report the conflict state, how each conflicted file was resolved (and any answers received to open questions), and stop — do **not** run `git rebase --continue` or any other continuation command.
|
||||
@ -1,112 +0,0 @@
|
||||
---
|
||||
name: review-code
|
||||
description: Code review flow — review a diff, PR, or code change, delegating the review to a subagent that follows the code-review-criteria skill. Use it when the user asks to review code or a PR, in any phrasing.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Review Code
|
||||
|
||||
Act as a senior software engineer and perform a thorough code review.
|
||||
|
||||
## When to use
|
||||
|
||||
- The user asks to review code, in any phrasing: "review this diff",
|
||||
"review the PR", "check my changes", "code review" — or runs
|
||||
`@review-code`.
|
||||
- A commit, branch, PR, or diff is ready and the user wants it assessed
|
||||
before merge.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Determine what is being reviewed** from the user context: a working-tree
|
||||
diff, a commit range, a branch, a PR (number or URL), or specific files. If
|
||||
the target is ambiguous, ask before reviewing.
|
||||
2. Delegate the review to the `general` subagent (via the task tool), unless the
|
||||
user specifies another agent. The prompt carries the **`code-review-criteria`**
|
||||
skill name, all user context, and these requirements:
|
||||
- Load `code-review-criteria` and follow its process and output format.
|
||||
- Read `AGENTS.md` (if present) and the module memories it points to for the
|
||||
code being reviewed.
|
||||
- Return the COMPLETE review, verbatim, exactly as the skill instructs it to
|
||||
be produced. Never summarize it.
|
||||
- Read-only: modify no file, create no commit, and run no tests, builds,
|
||||
linters or other state-changing checks. Use prior results or CI evidence;
|
||||
state explicitly when evidence is unavailable.
|
||||
- Do not invent problems: every finding must be real and actionable.
|
||||
- Missing tests are an issue, not a suggestion — a severity-tagged finding.
|
||||
- For each finding, give a concrete remedy and the tests needed to verify it.
|
||||
If a user decision is needed, include the real options and a recommendation
|
||||
in that finding. Do not pause to ask during the review.
|
||||
- Prioritize by impact: one structural issue outweighs ten nits, and "this
|
||||
could be better" is not a finding — say why and how.
|
||||
- Inspect generated files and lockfiles when relevant to the change, its
|
||||
dependencies or security; exclude unrelated working-tree modifications.
|
||||
3. When the subagent returns, output the review to the user verbatim. Do not
|
||||
summarize it and do not act on its findings. Retain the subagent session ID,
|
||||
review target and complete findings in the session context for follow-up;
|
||||
do not invent an ID if the tool does not provide one.
|
||||
4. Right after the review, suggest how to proceed based on the findings. These
|
||||
are suggestions — the user decides:
|
||||
- **Approve (no required changes):** say so — there is nothing to address.
|
||||
- **Minor findings:** suggest direct fixes after the user ends the review if
|
||||
appropriate; no plan is required.
|
||||
- **Substantive findings:** suggest `@make-a-plan` to make a plan to address
|
||||
them.
|
||||
- If the user wants to discuss findings one by one, use `@issue-walkthrough`
|
||||
on request; do not start it automatically. The walkthrough returns its
|
||||
decisions to `@make-a-plan`, not to automatic code editing.
|
||||
|
||||
### Follow-up after implementation
|
||||
|
||||
When the user agrees to verify corrections, resume the original reviewer using
|
||||
its session ID if the tool supports it. Give it the original review target and
|
||||
findings, the decisions and plan, the exact correction commit range or diff,
|
||||
and the test evidence. Require it to inspect the actual corrections and
|
||||
evidence, not rely on its earlier context alone. If the original session is
|
||||
unavailable or cannot be resumed, tell the user and delegate to a new reviewer
|
||||
with the same context; never claim it is the original reviewer. If the review,
|
||||
decisions or correction range cannot be established, ask for the missing
|
||||
information rather than inventing it.
|
||||
|
||||
The reviewer follows `code-review-criteria` and remains read-only. It reports
|
||||
each original finding with its unchanged ID as **resolved**, **pending**, or
|
||||
**not addressed** (including the agreed reason); report any new problems with
|
||||
new, non-conflicting IDs. For pending and new problems, give evidence, a remedy,
|
||||
tests and any user decision as in the original review. Summarize separately
|
||||
whether all agreed corrections are resolved and whether any findings remain,
|
||||
including deferred findings. Do not equate deferred with resolved. Present the
|
||||
result without implementing fixes; the user chooses whether to iterate, request
|
||||
an independent review, or stop.
|
||||
|
||||
### Hard rule — read-only while reviewing
|
||||
|
||||
This flow is read-only **for the duration of the review**: from the moment it
|
||||
starts until the user considers the review finished (including any feedback,
|
||||
questions, or clarifications about it). During that period, never fix,
|
||||
implement, edit files or create commits — not even "obvious" fixes derived from
|
||||
the findings. Never run tests, builds, linters or state-changing checks during
|
||||
the review; inspect existing evidence instead. Once the user explicitly states
|
||||
the review is done (or moves on to a different task), this rule no longer
|
||||
applies and you act as a normal build agent again.
|
||||
|
||||
## Posting as a GitHub comment
|
||||
|
||||
Post only when the user asks for it explicitly; otherwise keep the review in the
|
||||
session. PR comments go in English, with this minimal structure:
|
||||
|
||||
1. **Intro line** — one sentence: what was found and the overall tone. No
|
||||
greetings.
|
||||
2. **`## Main issues`** — one subsection per required finding: what breaks, the
|
||||
failure circumstances, and the minimal fix (code snippet when useful). A
|
||||
missing test belongs here, not under minor suggestions.
|
||||
3. **`## Minor suggestions`** — one bullet per minor finding: file/function,
|
||||
problem, concrete remedy.
|
||||
4. **Closing** — optional and short. Cut it when it adds nothing.
|
||||
|
||||
Reference every finding by its ID (`F1`, `F2`, … from
|
||||
`code-review-criteria`) so follow-up replies can point to findings
|
||||
unambiguously.
|
||||
|
||||
## User input, overrides and additional context
|
||||
|
||||
$ARGUMENTS
|
||||
@ -1,60 +0,0 @@
|
||||
---
|
||||
name: review-plan
|
||||
description: Review an implementation plan with a subagent, apply clear improvements and resolve consequential decisions with the user.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Review Plan
|
||||
|
||||
Review an unimplemented plan with a subagent and improve the plan without
|
||||
implementing it. Use this flow when the user asks for a plan review or invokes
|
||||
`@review-plan`.
|
||||
|
||||
## Flow
|
||||
|
||||
1. Identify and read the complete plan and its file. If the target or path is
|
||||
ambiguous, ask. If no saved file exists, ask for a path or have the user save
|
||||
it first. Do not edit a completed plan: propose a derived plan instead.
|
||||
2. Delegate to `general`, unless the user specifies a different subagent or
|
||||
model. Supply the plan path, relevant session context and these instructions:
|
||||
- Load `plan-review-criteria`, read `AGENTS.md` and the relevant module and
|
||||
testing memories, and review the plan against the criteria.
|
||||
- Apply clear, evidence-based improvements directly to that plan file only.
|
||||
Preserve the goal, scope, agreed decisions and intended behavior. Examples:
|
||||
clarify tasks, add missing tests, correct dependencies and specify actual
|
||||
verification commands. Do not guess commands or invent findings.
|
||||
- A change that affects the goal, user-visible behavior, scope, tradeoffs or
|
||||
an existing decision needs the user's decision. Leave it unresolved; report
|
||||
concrete options, consequences and a recommendation. Do not ask questions
|
||||
from the subagent.
|
||||
- Return a concise summary of applied changes, remaining decisions with
|
||||
stable IDs, evidence and recommendations, and an honest provisional
|
||||
verdict. Retain the subagent session ID for follow-up if available.
|
||||
- Do not change source files, other plans or documents, run state-changing
|
||||
checks, commit, or implement the plan. If the plan is not writable, report
|
||||
this and return the proposed edits instead.
|
||||
3. Show the summary. If decisions remain, start `@issue-walkthrough` immediately
|
||||
in the parent session with those questions and their IDs. Follow its one
|
||||
question per turn and read-only rules. Do not ask questions that the code or
|
||||
existing decisions settle. Wait for every answer.
|
||||
4. Resume the same subagent with the decision table and plan path. It applies
|
||||
the decisions to the plan, rechecks consistency and reports changes and any
|
||||
newly discovered consequential decisions. Repeat the walkthrough for new
|
||||
decisions, then apply those answers. If the original session cannot be
|
||||
resumed, tell the user and apply the changes in the parent or delegate a new
|
||||
subagent with the full context. If the reviewer cannot write, apply its
|
||||
proposed edits in the parent only if the plan file is writable; otherwise
|
||||
explain the blocker and stop.
|
||||
5. Once all decisions are resolved and the plan is coherent, update its
|
||||
metadata: when review feedback changed the plan, set `Status: reviewed` and
|
||||
append one UTC ISO 8601 `Review Log` entry for this review, summarizing the
|
||||
changes. Preserve prior log entries. If no feedback changed the plan, leave
|
||||
status and log unchanged. Never record commit hashes. If decisions or
|
||||
required changes remain unresolved, do not claim the plan is ready or mark
|
||||
it reviewed.
|
||||
6. Show the final summary, plan path and verdict. Suggest `@implement-plan` only
|
||||
when the plan is ready. Never execute the plan automatically.
|
||||
|
||||
The only permitted write in this flow is the specified local plan file,
|
||||
including its status and log. Do not stage or commit it. If a user instruction
|
||||
overrides automatic editing, respect it and report findings without editing.
|
||||
@ -1,39 +0,0 @@
|
||||
---
|
||||
name: rewrite-commit-message
|
||||
description: Rewrite HEAD's commit message (typically a squash that concatenated old messages) into one proper title plus description following Penpot commit conventions, preserving every distinct AI-assisted-by attribution. Amends in place, never pushes.
|
||||
slash: true
|
||||
---
|
||||
|
||||
# Skill: rewrite-commit-message
|
||||
|
||||
Rewrite HEAD's title and description from its own diff. This also handles a
|
||||
squash message with several subjects, bodies and repeated trailers.
|
||||
|
||||
## When to Use
|
||||
|
||||
- The user asks to rewrite, retitle or fix the last commit message
|
||||
- After a squash left HEAD with stacked messages and trailer repeats
|
||||
|
||||
## Steps
|
||||
|
||||
1. Check `git diff --cached` first. If any changes are staged, stop and ask
|
||||
the user to handle them; an amend must never include those changes. Read
|
||||
the full current message (`git log -1 --format=%B HEAD`) and HEAD's own
|
||||
diff (`git show --format= --stat HEAD` and the patch). Respect the project
|
||||
rule on amending pushed commits.
|
||||
2. Collect every DISTINCT `AI-assisted-by:` value in the current
|
||||
message. All of them survive into the new message, one line each:
|
||||
attributions are never dropped in a rewrite, even when most of
|
||||
the old text is.
|
||||
3. Draft one message per `mem:workflow/creating-commits` (read it
|
||||
end-to-end first): single subject (imperative, capitalized, no
|
||||
trailing period, ≤80 chars), blank line, body wrapped at ≤80
|
||||
chars covering HEAD's diff (not the old per-commit slices),
|
||||
then the preserved trailers.
|
||||
4. Write it to a file (a heredoc keeps real newlines; inline `-m`
|
||||
with `\n` does not) and `git commit --amend -F <file>`.
|
||||
5. Run `./scripts/check-commit` and require exit code 0. A non-zero
|
||||
exit is a hard blocker: fix the message, amend again.
|
||||
6. Report the new hash. Never push: the user pushes from their own
|
||||
shell. Warn when local history was rewritten over pushed commits
|
||||
(the push will need force — their call, not yours).
|
||||
@ -1,457 +0,0 @@
|
||||
---
|
||||
name: security-and-hardening
|
||||
description: Hardens code against vulnerabilities. Use when handling user input, authentication, data storage, or external integrations. Use when building any feature that accepts untrusted data, manages user sessions, or interacts with third-party services.
|
||||
---
|
||||
|
||||
# Security and Hardening
|
||||
|
||||
## Overview
|
||||
|
||||
Security-first development practices for web applications. Treat every external input as hostile, every secret as sacred, and every authorization check as mandatory. Security isn't a phase — it's a constraint on every line of code that touches user data, authentication, or external systems.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Building anything that accepts user input
|
||||
- Implementing authentication or authorization
|
||||
- Storing or transmitting sensitive data
|
||||
- Integrating with external APIs or services
|
||||
- Adding file uploads, webhooks, or callbacks
|
||||
- Handling payment or PII data
|
||||
|
||||
## Process: Threat Model First
|
||||
|
||||
Controls bolted on without a threat model are guesses. Before hardening, spend five minutes thinking like an attacker:
|
||||
|
||||
1. **Map the trust boundaries.** Where does untrusted data cross into your system? HTTP requests, form fields, file uploads, webhooks, third-party APIs, message queues, and **LLM output**. Every boundary is attack surface.
|
||||
2. **Name the assets.** What's worth stealing or breaking? Credentials, PII, payment data, admin actions, money movement.
|
||||
3. **Run STRIDE over each boundary** — a quick lens, not a ceremony:
|
||||
|
||||
| Threat | Ask | Typical mitigation |
|
||||
|---|---|---|
|
||||
| **S**poofing | Can someone impersonate a user/service? | Authentication, signature verification |
|
||||
| **T**ampering | Can data be altered in transit or at rest? | Integrity checks, parameterized queries, HTTPS |
|
||||
| **R**epudiation | Can an action be denied later? | Audit logging of security events |
|
||||
| **I**nformation disclosure | Can data leak? | Encryption, field allowlists, generic errors |
|
||||
| **D**enial of service | Can it be overwhelmed? | Rate limiting, input size caps, timeouts |
|
||||
| **E**levation of privilege | Can a user gain rights they shouldn't? | Authorization checks, least privilege |
|
||||
|
||||
4. **Write abuse cases next to use cases.** For each feature, ask "how would I misuse this?" — then make that your first test.
|
||||
|
||||
If you can't name the trust boundaries for a feature, you're not ready to secure it. This is OWASP **A04: Insecure Design** — most breaches begin in design, not code.
|
||||
|
||||
## The Three-Tier Boundary System
|
||||
|
||||
### Always Do (No Exceptions)
|
||||
|
||||
- **Validate all external input** at the system boundary (API routes, form handlers)
|
||||
- **Parameterize all database queries** — never concatenate user input into SQL
|
||||
- **Encode output** to prevent XSS (use framework auto-escaping, don't bypass it)
|
||||
- **Use HTTPS** for all external communication
|
||||
- **Hash passwords** with bcrypt/scrypt/argon2 (never store plaintext)
|
||||
- **Set security headers** (CSP, HSTS, X-Frame-Options, X-Content-Type-Options)
|
||||
- **Use httpOnly, secure, sameSite cookies** for sessions
|
||||
- **Run `npm audit`** (or equivalent) before every release
|
||||
|
||||
### Ask First (Requires Human Approval)
|
||||
|
||||
- Adding new authentication flows or changing auth logic
|
||||
- Storing new categories of sensitive data (PII, payment info)
|
||||
- Adding new external service integrations
|
||||
- Changing CORS configuration
|
||||
- Adding file upload handlers
|
||||
- Modifying rate limiting or throttling
|
||||
- Granting elevated permissions or roles
|
||||
|
||||
### Never Do
|
||||
|
||||
- **Never commit secrets** to version control (API keys, passwords, tokens)
|
||||
- **Never log sensitive data** (passwords, tokens, full credit card numbers)
|
||||
- **Never trust client-side validation** as a security boundary
|
||||
- **Never disable security headers** for convenience
|
||||
- **Never use `eval()` or `innerHTML`** with user-provided data
|
||||
- **Never store sessions in client-accessible storage** (localStorage for auth tokens)
|
||||
- **Never expose stack traces** or internal error details to users
|
||||
|
||||
## OWASP Top 10 Prevention Patterns
|
||||
|
||||
These are prevention patterns, not a ranking. For the 2021 ordering, see the quick-reference table in `references/security-checklist.md`.
|
||||
|
||||
### Injection (SQL, NoSQL, OS Command)
|
||||
|
||||
```typescript
|
||||
// BAD: SQL injection via string concatenation
|
||||
const query = `SELECT * FROM users WHERE id = '${userId}'`;
|
||||
|
||||
// GOOD: Parameterized query
|
||||
const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]);
|
||||
|
||||
// GOOD: ORM with parameterized input
|
||||
const user = await prisma.user.findUnique({ where: { id: userId } });
|
||||
```
|
||||
|
||||
### Broken Authentication
|
||||
|
||||
```typescript
|
||||
// Password hashing
|
||||
import { hash, compare } from 'bcrypt';
|
||||
|
||||
const SALT_ROUNDS = 12;
|
||||
const hashedPassword = await hash(plaintext, SALT_ROUNDS);
|
||||
const isValid = await compare(plaintext, hashedPassword);
|
||||
|
||||
// Session management
|
||||
app.use(session({
|
||||
secret: process.env.SESSION_SECRET, // From environment, not code
|
||||
resave: false,
|
||||
saveUninitialized: false,
|
||||
cookie: {
|
||||
httpOnly: true, // Not accessible via JavaScript
|
||||
secure: true, // HTTPS only
|
||||
sameSite: 'lax', // CSRF protection
|
||||
maxAge: 24 * 60 * 60 * 1000, // 24 hours
|
||||
},
|
||||
}));
|
||||
```
|
||||
|
||||
### Cross-Site Scripting (XSS)
|
||||
|
||||
```typescript
|
||||
// BAD: Rendering user input as HTML
|
||||
element.innerHTML = userInput;
|
||||
|
||||
// GOOD: Use framework auto-escaping (React does this by default)
|
||||
return <div>{userInput}</div>;
|
||||
|
||||
// If you MUST render HTML, sanitize first
|
||||
import DOMPurify from 'dompurify';
|
||||
const clean = DOMPurify.sanitize(userInput);
|
||||
```
|
||||
|
||||
### Broken Access Control
|
||||
|
||||
```typescript
|
||||
// Always check authorization, not just authentication
|
||||
app.patch('/api/tasks/:id', authenticate, async (req, res) => {
|
||||
const task = await taskService.findById(req.params.id);
|
||||
|
||||
// Check that the authenticated user owns this resource
|
||||
if (task.ownerId !== req.user.id) {
|
||||
return res.status(403).json({
|
||||
error: { code: 'FORBIDDEN', message: 'Not authorized to modify this task' }
|
||||
});
|
||||
}
|
||||
|
||||
// Proceed with update
|
||||
const updated = await taskService.update(req.params.id, req.body);
|
||||
return res.json(updated);
|
||||
});
|
||||
```
|
||||
|
||||
### Security Misconfiguration
|
||||
|
||||
```typescript
|
||||
// Security headers (use helmet for Express)
|
||||
import helmet from 'helmet';
|
||||
app.use(helmet());
|
||||
|
||||
// Content Security Policy
|
||||
app.use(helmet.contentSecurityPolicy({
|
||||
directives: {
|
||||
defaultSrc: ["'self'"],
|
||||
scriptSrc: ["'self'"],
|
||||
styleSrc: ["'self'", "'unsafe-inline'"], // Tighten if possible
|
||||
imgSrc: ["'self'", 'data:', 'https:'],
|
||||
connectSrc: ["'self'"],
|
||||
},
|
||||
}));
|
||||
|
||||
// CORS — restrict to known origins
|
||||
app.use(cors({
|
||||
origin: process.env.ALLOWED_ORIGINS?.split(',') || 'http://localhost:3000',
|
||||
credentials: true,
|
||||
}));
|
||||
```
|
||||
|
||||
### Sensitive Data Exposure
|
||||
|
||||
```typescript
|
||||
// Never return sensitive fields in API responses
|
||||
function sanitizeUser(user: UserRecord): PublicUser {
|
||||
const { passwordHash, resetToken, ...publicFields } = user;
|
||||
return publicFields;
|
||||
}
|
||||
|
||||
// Use environment variables for secrets
|
||||
const API_KEY = process.env.STRIPE_API_KEY;
|
||||
if (!API_KEY) throw new Error('STRIPE_API_KEY not configured');
|
||||
```
|
||||
|
||||
### Server-Side Request Forgery (SSRF)
|
||||
|
||||
Any time the server fetches a URL the user influenced — webhooks, "import from URL", image proxies, link previews — an attacker can aim it at internal services (cloud metadata, `localhost`, private IPs).
|
||||
|
||||
```typescript
|
||||
// BAD: fetch whatever the user gives you
|
||||
await fetch(req.body.webhookUrl);
|
||||
|
||||
// GOOD: allowlist scheme + host, reject if ANY resolved IP is private, forbid redirects
|
||||
import { lookup } from 'node:dns/promises';
|
||||
import ipaddr from 'ipaddr.js';
|
||||
|
||||
const ALLOWED_HOSTS = new Set(['hooks.example.com']);
|
||||
|
||||
async function assertSafeUrl(raw: string): Promise<URL> {
|
||||
const url = new URL(raw);
|
||||
if (url.protocol !== 'https:') throw new Error('https only');
|
||||
if (!ALLOWED_HOSTS.has(url.hostname)) throw new Error('host not allowed');
|
||||
// Resolve ALL records; a single private/reserved address fails the check.
|
||||
const addrs = await lookup(url.hostname, { all: true });
|
||||
if (addrs.some((a) => ipaddr.parse(a.address).range() !== 'unicast')) {
|
||||
throw new Error('private/reserved IP');
|
||||
}
|
||||
return url;
|
||||
}
|
||||
|
||||
await fetch(await assertSafeUrl(req.body.webhookUrl), { redirect: 'error' });
|
||||
```
|
||||
|
||||
The `range() !== 'unicast'` check covers loopback, link-local `169.254.169.254` (cloud metadata, the #1 SSRF target), private, and unique-local ranges across IPv4 and IPv6.
|
||||
|
||||
**Caveat — this still has a TOCTOU gap.** `fetch` resolves DNS again after the check, so an attacker using a short-TTL record can rebind to an internal IP between validation and connection. For high-risk surfaces, resolve once and connect to the pinned IP, or put a filtering agent in front (`request-filtering-agent` / `ssrf-req-filter`).
|
||||
|
||||
## Input Validation Patterns
|
||||
|
||||
### Schema Validation at Boundaries
|
||||
|
||||
```typescript
|
||||
import { z } from 'zod';
|
||||
|
||||
const CreateTaskSchema = z.object({
|
||||
title: z.string().min(1).max(200).trim(),
|
||||
description: z.string().max(2000).optional(),
|
||||
priority: z.enum(['low', 'medium', 'high']).default('medium'),
|
||||
dueDate: z.string().datetime().optional(),
|
||||
});
|
||||
|
||||
// Validate at the route handler
|
||||
app.post('/api/tasks', async (req, res) => {
|
||||
const result = CreateTaskSchema.safeParse(req.body);
|
||||
if (!result.success) {
|
||||
return res.status(422).json({
|
||||
error: {
|
||||
code: 'VALIDATION_ERROR',
|
||||
message: 'Invalid input',
|
||||
details: result.error.flatten(),
|
||||
},
|
||||
});
|
||||
}
|
||||
// result.data is now typed and validated
|
||||
const task = await taskService.create(result.data);
|
||||
return res.status(201).json(task);
|
||||
});
|
||||
```
|
||||
|
||||
### File Upload Safety
|
||||
|
||||
```typescript
|
||||
// Restrict file types and sizes
|
||||
const ALLOWED_TYPES = ['image/jpeg', 'image/png', 'image/webp'];
|
||||
const MAX_SIZE = 5 * 1024 * 1024; // 5MB
|
||||
|
||||
function validateUpload(file: UploadedFile) {
|
||||
if (!ALLOWED_TYPES.includes(file.mimetype)) {
|
||||
throw new ValidationError('File type not allowed');
|
||||
}
|
||||
if (file.size > MAX_SIZE) {
|
||||
throw new ValidationError('File too large (max 5MB)');
|
||||
}
|
||||
// Don't trust the file extension — check magic bytes if critical
|
||||
}
|
||||
```
|
||||
|
||||
## Triaging npm audit Results
|
||||
|
||||
Not all audit findings require immediate action. Use this decision tree:
|
||||
|
||||
```
|
||||
npm audit reports a vulnerability
|
||||
├── Severity: critical or high
|
||||
│ ├── Is the vulnerable code reachable in your app?
|
||||
│ │ ├── YES --> Fix immediately (update, patch, or replace the dependency)
|
||||
│ │ └── NO (dev-only dep, unused code path) --> Fix soon, but not a blocker
|
||||
│ └── Is a fix available?
|
||||
│ ├── YES --> Update to the patched version
|
||||
│ └── NO --> Check for workarounds, consider replacing the dependency, or add to allowlist with a review date
|
||||
├── Severity: moderate
|
||||
│ ├── Reachable in production? --> Fix in the next release cycle
|
||||
│ └── Dev-only? --> Fix when convenient, track in backlog
|
||||
└── Severity: low
|
||||
└── Track and fix during regular dependency updates
|
||||
```
|
||||
|
||||
**Key questions:**
|
||||
- Is the vulnerable function actually called in your code path?
|
||||
- Is the dependency a runtime dependency or dev-only?
|
||||
- Is the vulnerability exploitable given your deployment context (e.g., a server-side vulnerability in a client-only app)?
|
||||
|
||||
When you defer a fix, document the reason and set a review date.
|
||||
|
||||
### Supply-Chain Hygiene
|
||||
|
||||
`npm audit` catches known CVEs; it won't catch a malicious or typosquatted package. Also:
|
||||
|
||||
- **Commit the lockfile** and install with `npm ci` (not `npm install`) in CI — reproducible builds, no silent version drift.
|
||||
- **Review new dependencies before adding them** — maintenance, download counts, and whether they truly earn their place. Every dependency is attack surface (OWASP **A06: Vulnerable Components**, **LLM03: Supply Chain**).
|
||||
- **Be wary of `postinstall` scripts** in unfamiliar packages — they run arbitrary code at install time.
|
||||
- **Watch for typosquats** — `cross-env` vs `crossenv`, `react-dom` vs `reactdom`.
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
```typescript
|
||||
import rateLimit from 'express-rate-limit';
|
||||
|
||||
// General API rate limit
|
||||
app.use('/api/', rateLimit({
|
||||
windowMs: 15 * 60 * 1000, // 15 minutes
|
||||
max: 100, // 100 requests per window
|
||||
standardHeaders: true,
|
||||
legacyHeaders: false,
|
||||
}));
|
||||
|
||||
// Stricter limit for auth endpoints
|
||||
app.use('/api/auth/', rateLimit({
|
||||
windowMs: 15 * 60 * 1000,
|
||||
max: 10, // 10 attempts per 15 minutes
|
||||
}));
|
||||
```
|
||||
|
||||
## Secrets Management
|
||||
|
||||
```
|
||||
.env files:
|
||||
├── .env.example → Committed (template with placeholder values)
|
||||
├── .env → NOT committed (contains real secrets)
|
||||
└── .env.local → NOT committed (local overrides)
|
||||
|
||||
.gitignore must include:
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
*.pem
|
||||
*.key
|
||||
```
|
||||
|
||||
**Always check before committing:**
|
||||
```bash
|
||||
# Check for accidentally staged secrets
|
||||
git diff --cached | grep -i "password\|secret\|api_key\|token"
|
||||
```
|
||||
|
||||
**If a secret is ever committed, rotate it.** Deleting the line or rewriting history is not enough — assume it's compromised the moment it reaches a remote. Revoke and reissue the key first, then purge it from history.
|
||||
|
||||
## Securing AI / LLM Features
|
||||
|
||||
If your app calls an LLM — chatbots, summarizers, agents, RAG — it inherits a new attack surface. Map it to the [OWASP Top 10 for LLM Applications (2025)](https://genai.owasp.org/llm-top-10/):
|
||||
|
||||
- **Treat all model output as untrusted input (LLM05: Improper Output Handling).** Never pass LLM output straight into `eval`, SQL, a shell, `innerHTML`, or a file path. Validate and encode it exactly as you would raw user input.
|
||||
- **Assume prompts can be hijacked (LLM01: Prompt Injection).** Untrusted text in the context window — a user message, a fetched web page, a PDF — can carry instructions. The system prompt is not a security boundary; enforce permissions in code, not in the prompt.
|
||||
- **Keep secrets and other users' data out of prompts (LLM02 / LLM07).** Anything in the context can be echoed back. Don't put API keys, cross-tenant data, or the full system prompt where the model can repeat it.
|
||||
- **Constrain tool and agent permissions (LLM06: Excessive Agency).** Scope tools to the minimum, require confirmation for destructive or irreversible actions, and validate every tool argument.
|
||||
- **Bound consumption (LLM10: Unbounded Consumption).** Cap tokens, request rate, and loop/recursion depth so a crafted input can't run up cost or hang the system.
|
||||
- **Isolate retrieval data (LLM08: Vector and Embedding Weaknesses).** In RAG, treat the vector store as a trust boundary: partition embeddings per tenant so one user can't retrieve another's data, and validate documents before indexing so poisoned content can't steer answers.
|
||||
|
||||
```typescript
|
||||
// BAD: trusting model output as a command or as markup
|
||||
const sql = await llm.generate(`Write SQL for: ${userQuestion}`);
|
||||
await db.query(sql); // arbitrary query execution
|
||||
container.innerHTML = await llm.reply(userMessage); // stored XSS, via the model
|
||||
|
||||
// GOOD: model output is data — parse defensively, then validate, then encode
|
||||
let intent;
|
||||
try {
|
||||
intent = CommandSchema.parse(JSON.parse(await llm.replyJson(userMessage)));
|
||||
} catch {
|
||||
throw new ValidationError('unexpected model output'); // JSON.parse or schema failed
|
||||
}
|
||||
await runAllowlistedAction(intent.action, intent.params);
|
||||
container.textContent = await llm.reply(userMessage);
|
||||
```
|
||||
|
||||
## Security Review Checklist
|
||||
|
||||
```markdown
|
||||
### Authentication
|
||||
- [ ] Passwords hashed with bcrypt/scrypt/argon2 (salt rounds ≥ 12)
|
||||
- [ ] Session tokens are httpOnly, secure, sameSite
|
||||
- [ ] Login has rate limiting
|
||||
- [ ] Password reset tokens expire
|
||||
|
||||
### Authorization
|
||||
- [ ] Every endpoint checks user permissions
|
||||
- [ ] Users can only access their own resources
|
||||
- [ ] Admin actions require admin role verification
|
||||
|
||||
### Input
|
||||
- [ ] All user input validated at the boundary
|
||||
- [ ] SQL queries are parameterized
|
||||
- [ ] HTML output is encoded/escaped
|
||||
- [ ] Server-side URL fetches are allowlisted (no SSRF to internal services)
|
||||
|
||||
### Data
|
||||
- [ ] No secrets in code or version control
|
||||
- [ ] Sensitive fields excluded from API responses
|
||||
- [ ] PII encrypted at rest (if applicable)
|
||||
|
||||
### Infrastructure
|
||||
- [ ] Security headers configured (CSP, HSTS, etc.)
|
||||
- [ ] CORS restricted to known origins
|
||||
- [ ] Dependencies audited for vulnerabilities
|
||||
- [ ] Error messages don't expose internals
|
||||
|
||||
### Supply Chain
|
||||
- [ ] Lockfile committed; CI installs with `npm ci`
|
||||
- [ ] New dependencies reviewed (maintenance, downloads, postinstall scripts)
|
||||
|
||||
### AI / LLM (if used)
|
||||
- [ ] Model output treated as untrusted (no eval/SQL/innerHTML/shell)
|
||||
- [ ] Secrets and other users' data kept out of prompts
|
||||
- [ ] Tool/agent permissions scoped; destructive actions require confirmation
|
||||
```
|
||||
## Common Rationalizations
|
||||
|
||||
| Rationalization | Reality |
|
||||
|---|---|
|
||||
| "This is an internal tool, security doesn't matter" | Internal tools get compromised. Attackers target the weakest link. |
|
||||
| "We'll add security later" | Security retrofitting is 10x harder than building it in. Add it now. |
|
||||
| "No one would try to exploit this" | Automated scanners will find it. Security by obscurity is not security. |
|
||||
| "The framework handles security" | Frameworks provide tools, not guarantees. You still need to use them correctly. |
|
||||
| "It's just a prototype" | Prototypes become production. Security habits from day one. |
|
||||
| "Threat modeling is overkill here" | Five minutes of "how would I attack this?" prevents the design flaws no control can patch later. |
|
||||
| "It's just LLM output, it's only text" | That "text" can be a SQL statement, a script tag, or a shell command. Treat it like any untrusted input. |
|
||||
|
||||
## Red Flags
|
||||
|
||||
- User input passed directly to database queries, shell commands, or HTML rendering
|
||||
- Secrets in source code or commit history
|
||||
- API endpoints without authentication or authorization checks
|
||||
- Missing CORS configuration or wildcard (`*`) origins
|
||||
- No rate limiting on authentication endpoints
|
||||
- Stack traces or internal errors exposed to users
|
||||
- Dependencies with known critical vulnerabilities
|
||||
- Server fetches user-supplied URLs without an allowlist (SSRF)
|
||||
- LLM/model output passed into a query, the DOM, a shell, or `eval`
|
||||
- Secrets, PII, or the full system prompt placed inside an LLM context window
|
||||
|
||||
## Verification
|
||||
|
||||
After implementing security-relevant code:
|
||||
|
||||
- [ ] `npm audit` shows no critical or high vulnerabilities
|
||||
- [ ] No secrets in source code or git history
|
||||
- [ ] All user input validated at system boundaries
|
||||
- [ ] Authentication and authorization checked on every protected endpoint
|
||||
- [ ] Security headers present in response (check with browser DevTools)
|
||||
- [ ] Error responses don't expose internal details
|
||||
- [ ] Rate limiting active on auth endpoints
|
||||
- [ ] Server-side URL fetches validated against an allowlist (no SSRF)
|
||||
- [ ] LLM/model output validated and encoded before use (if AI features present)
|
||||
@ -1,85 +0,0 @@
|
||||
---
|
||||
name: ste
|
||||
description: Write or rewrite text in ASD-STE100 Simplified Technical English. ONLY use this skill when the user explicitly invokes it by name — i.e. they type "@ste" or literally write "use the ste skill" / "apply ASD-STE100". Do NOT trigger it on paraphrased intent such as "simplify this", "make it clearer", "write technical documentation", or "shorter sentences please" — the user has deliberately scoped this skill to explicit invocation only. For those requests, respond normally without loading this skill unless they name it.
|
||||
---
|
||||
|
||||
# ASD-STE100 Simplified Technical English
|
||||
|
||||
Apply the ASD-STE100 standard to all prose you produce in this task. Do not announce that you use STE, do not name the standard, and do not explain the style unless the user asks. If the user later asks you to "write more naturally," ask one short question to confirm they want to leave STE before you drop it.
|
||||
|
||||
Compliance note (for you, not for output): the official specification and its dictionary are copyright ASD. This skill encodes paraphrased rules and a publicly sourced word list. For certified aerospace/defense deliverables, tell the user that full compliance requires the free official specification (asd-ste100.org) and a human sign-off. Never claim certified compliance.
|
||||
|
||||
## When to use
|
||||
|
||||
Only when the user explicitly invokes it: they type `@ste`, or say "use
|
||||
the ste skill" / "apply ASD-STE100". Requests like "simplify this",
|
||||
"make it clearer", or "shorter sentences" do NOT invoke it — respond
|
||||
normally unless it is named.
|
||||
|
||||
## Step 0 — Classify the text
|
||||
|
||||
Before writing a single sentence, decide: is this **procedural** text (instructions someone follows) or **descriptive** text (explanation, background, description)? Every limit below depends on this. Mixed documents get classified section by section.
|
||||
|
||||
## Core rules
|
||||
|
||||
### Sentences
|
||||
- Procedural: maximum **20 words** per sentence.
|
||||
- Descriptive: maximum **25 words** per sentence.
|
||||
- Maximum **6 sentences** per paragraph. One topic per paragraph.
|
||||
- One instruction per sentence. Two actions in one sentence only if they occur at the same time.
|
||||
- Put a condition BEFORE its command: "If the pressure decreases, close the valve."
|
||||
- Do not omit articles, subjects, or verbs to save words. "Ensure file exists" is wrong; "Make sure that the file exists" is correct. Keep the word "that" after verbs like "make sure."
|
||||
- Numbers, units with numbers, abbreviations, quoted strings, code identifiers, and proper nouns each count as one word.
|
||||
|
||||
### Verbs
|
||||
- Allowed forms only: infinitive, imperative, simple present, simple past, simple future, and past participle used as an adjective.
|
||||
- Never use present perfect or continuous forms. "We have received" → "We received." "is being tested" → a simple form.
|
||||
- Never use an -ing form as a verb. An -ing word is allowed only inside a technical name ("the mounting bracket," "logging").
|
||||
- Active voice. Passive is allowed only in descriptive text when the agent is unknown or unimportant.
|
||||
- Instructions use the imperative: "Open the panel," not "You must open the panel" or "The panel should be opened."
|
||||
- Express actions as verbs, not nouns: "compress the file," not "perform compression of the file."
|
||||
- Modals: use **can** (possibility), **will** (future), **must** (requirement). Do not use should, would, could, may, might. A hedge becomes a fact or a "can": "an explosion can occur."
|
||||
- No phrasal verbs: "go down" → "decrease," "set up" → "install," "carry out" → "do."
|
||||
|
||||
### Words
|
||||
- One word, one meaning, one part of speech, used consistently. Never rotate synonyms: pick one name for a thing and repeat it.
|
||||
- Before drafting, replace unapproved vocabulary. Read `references/word-substitutions.md` and apply it; it is the working dictionary for this skill.
|
||||
- Domain-specific nouns (part names, tool names, product names, UI labels) and domain verbs (drill, ream, boot, compile) are your **technical nouns/verbs** — keep them as-is, use each consistently, and do not verb a noun or noun a verb.
|
||||
- Noun clusters: maximum **3 words** ("overhead panel light" is the limit). Longer clusters get decomposed with prepositions or hyphenated on first use: "main-gear-door retraction-winch handle."
|
||||
- American English spelling.
|
||||
- No Latin abbreviations: "e.g." → "for example," "i.e." → "that is," delete "etc."
|
||||
|
||||
### Punctuation
|
||||
- No semicolons — write two sentences.
|
||||
- Parentheses only for references, abbreviations, and item numbers.
|
||||
- Hyphenate words that act as one unit; a hyphenated word counts as one word.
|
||||
- No contractions.
|
||||
|
||||
### Warnings, cautions, notes
|
||||
- **WARNING** = risk of injury or death. **CAUTION** = risk of damage. **NOTE** = information only, never an instruction.
|
||||
- Start a warning or caution with the command or condition, then give the risk:
|
||||
"WARNING: Do not touch the terminal. The terminal has a dangerous voltage."
|
||||
- Notes obey the 25-word descriptive limit.
|
||||
|
||||
## Step 2 — Self-check pass
|
||||
|
||||
After drafting, scan your text once for each of these and fix every hit before you respond:
|
||||
|
||||
1. Any sentence over the 20/25-word limit for its type
|
||||
2. Contractions, semicolons
|
||||
3. "should," "would," "could," "may," "might"
|
||||
4. "has been," "have been," "had been," "is being," "was being"
|
||||
5. -ing words used as verbs
|
||||
6. Missing articles (a/an/the/this) before nouns
|
||||
7. Synonym rotation (the same object under two names)
|
||||
8. Any word in the unapproved column of `references/word-substitutions.md`
|
||||
9. Warnings that state the risk before the command
|
||||
|
||||
## Reference files
|
||||
|
||||
- `references/word-substitutions.md` — unapproved → approved word mappings and one-meaning rulings. Read it before drafting; it is short.
|
||||
- `references/examples.md` — worked before/after rewrites (procedural, descriptive, warnings, common mistakes). Read it when rewriting existing text or when unsure how a rule applies.
|
||||
|
||||
## What NOT to touch
|
||||
|
||||
Code blocks, command strings, file paths, error messages, quoted UI text, and proper nouns stay exactly as written. STE applies to the prose around them.
|
||||
@ -1,67 +0,0 @@
|
||||
# Worked before/after examples
|
||||
|
||||
## Verb forms
|
||||
|
||||
| Before | After |
|
||||
|---|---|
|
||||
| We have received the technical reports from HQ. | We received the technical reports from HQ. |
|
||||
| This device has been being used at Boeing since 2005. | Boeing started to use this device in 2005. |
|
||||
| The test is continued by the operator. | Continue the test. |
|
||||
| The screws should be replaced. | Replace the screws. |
|
||||
| The system is currently running diagnostics. | The system does diagnostic tests now. |
|
||||
|
||||
## Vocabulary and phrasing
|
||||
|
||||
| Before | After |
|
||||
|---|---|
|
||||
| Ensure file exists before running. | Make sure that the file exists before you run the command. |
|
||||
| Rotate the cover until the jacks are accessible. | Turn the cover until you can get access to the jacks. |
|
||||
| Extend the jack until the wheels are clear of the ground. | Extend the jack until the wheels do not touch the ground. |
|
||||
| Make sure that these steps are followed. | Obey these steps. |
|
||||
| Utilize approximately 3 liters of water. | Use about 3 liters of water. |
|
||||
| Perform a compression of the log files prior to upload. | Compress the log files before you upload them. |
|
||||
|
||||
## Noun clusters
|
||||
|
||||
| Before | After |
|
||||
|---|---|
|
||||
| Main gear door retraction winch handle | Main-gear-door retraction-winch handle |
|
||||
| Runway light connection resistance calibration | Calibration of the resistance of the runway light connection |
|
||||
| Remove the engine transmission housing attachment bolts. | Remove the bolts that attach the transmission housing to the engine. |
|
||||
|
||||
## Procedural rewrite (condition first, one instruction per sentence)
|
||||
|
||||
Before:
|
||||
> You should check that the reservoir has been filled with the correct fluid and, after attaching a clear tube to the bleed screw while ensuring its free end is submerged, the pedal can be pumped and held so that opening the screw releases trapped air.
|
||||
|
||||
After:
|
||||
> 1. Fill the reservoir with the correct fluid.
|
||||
> 2. Attach a clear tube to the bleed screw.
|
||||
> 3. Put the free end of the tube in a container of fluid.
|
||||
> 4. Push the pedal three times. Hold the pedal down.
|
||||
> 5. Open the bleed screw one half turn. Air and fluid flow into the tube.
|
||||
> 6. Close the bleed screw. Release the pedal.
|
||||
> 7. If air continues to come out, do steps 4 thru 6 again.
|
||||
|
||||
## Warnings and cautions (command first, then risk)
|
||||
|
||||
Before:
|
||||
> Note that serious data loss may potentially occur if the --force flag is used against production.
|
||||
|
||||
After:
|
||||
> CAUTION: Do not use the --force flag on the production database. The flag deletes the rows that do not match the source.
|
||||
|
||||
Before:
|
||||
> Touching the terminal could result in electrocution.
|
||||
|
||||
After:
|
||||
> WARNING: Do not touch the terminal. The terminal has a dangerous voltage.
|
||||
|
||||
## Common mistakes checklist
|
||||
|
||||
- Dropped articles: "Insert pin in bracket" → "Insert the pin in the bracket."
|
||||
- Synonym rotation: check/verify/confirm for the same action → one term, everywhere.
|
||||
- Hedges: "you may want to," "it is recommended that" → an imperative or "must."
|
||||
- Instruction buried in a NOTE: notes never instruct. Move the instruction to a numbered step.
|
||||
- Semicolon joining two clauses → two sentences.
|
||||
- "There are three bolts on the panel" → "The panel has three bolts."
|
||||
@ -1,68 +0,0 @@
|
||||
# Word substitutions and one-meaning rulings
|
||||
|
||||
Compiled from public secondary sources (STEMG/ASD public pages, TechScribe, Acrolinx, training materials). This is a working approximation, not the official ASD dictionary. When a word is not listed here and feels formal or Latin-derived, prefer the shortest common alternative.
|
||||
|
||||
## Unapproved → approved
|
||||
|
||||
| Do not use | Use instead |
|
||||
|---|---|
|
||||
| utilize, leverage, employ | use |
|
||||
| commence, initiate, begin, originate | start |
|
||||
| terminate, cease, conclude | stop, end |
|
||||
| ensure, verify, confirm, validate, check | make sure (that), examine |
|
||||
| perform, conduct, execute, carry out | do |
|
||||
| facilitate, assist | help |
|
||||
| obtain, acquire, procure | get |
|
||||
| sufficient, adequate | enough |
|
||||
| approximately | about |
|
||||
| prior to | before |
|
||||
| subsequent to, following (prep.) | after |
|
||||
| adjacent to | near |
|
||||
| accomplish | do |
|
||||
| additional, supplementary | more |
|
||||
| attempt | try |
|
||||
| require, necessitate | need, must |
|
||||
| mandatory | necessary |
|
||||
| indicate, signify | show |
|
||||
| observe (=watch) | look at, examine |
|
||||
| rotate | turn |
|
||||
| deactivate | turn off, set to off |
|
||||
| activate, energize (unless technical verb) | turn on, start |
|
||||
| toxic | poisonous |
|
||||
| in order to | to |
|
||||
| via, by means of | through, with |
|
||||
| due to, owing to | because of |
|
||||
| in the event of/that | if |
|
||||
| accessible | (rewrite: "you can get access to") |
|
||||
| remainder | rest |
|
||||
| demonstrate | show |
|
||||
| modify, alter | change |
|
||||
| construct, fabricate, build | assemble, make |
|
||||
| retain | keep |
|
||||
| locate (=find) | find |
|
||||
| depress (a button) | push, press |
|
||||
| proceed | continue, go |
|
||||
|
||||
## One meaning, one part of speech (canonical rulings)
|
||||
|
||||
- **close** — verb only: to move to a position that stops flow, or to operate a circuit breaker. The adjective is unapproved → use **near** ("do not go near the propeller").
|
||||
- **test** — noun only: "do a test," never "test the system."
|
||||
- **check** — do not use as a verb for verification → "make sure that" or "examine."
|
||||
- **follow** — means only "come after." For rules and steps use **obey**: "Obey the safety instructions."
|
||||
- **fall** — means only "move down by gravity." For quantities use **decrease**. Never the season.
|
||||
- **oil** — noun only. "Oil the bearing" → "Put oil on the bearing" / "Lubricate the bearing."
|
||||
- **right** — direction only, never "correct."
|
||||
- **clear** — "without blockage." "Wheels are clear of the ground" → "wheels do not touch the ground."
|
||||
- **help** — verb only; the noun is **aid** ("with the aid of a mirror").
|
||||
- **above / below** — physical position only. For quantities: **more than / less than**.
|
||||
- **about** — two approved senses: "approximately" and "on the subject of." Use carefully.
|
||||
- **turn** — the general verb for rotation; "turn on / turn off" for power state is standard.
|
||||
- **level** — approved as noun and adjective (documented exception to the one-POS rule).
|
||||
|
||||
## Frequent-offender function words
|
||||
|
||||
- **should / would / could / may / might** — never. Requirement → **must**. Possibility → **can**. Future → **will**.
|
||||
- **etc.** — delete, or write the full list.
|
||||
- **e.g. / i.e.** — "for example" / "that is."
|
||||
- **any / appropriate / applicable / relevant** as hedges — replace with the specific thing meant.
|
||||
- **there is / there are** openers — rewrite with a real subject: "There are three bolts on the panel" → "The panel has three bolts."
|
||||
@ -1,58 +0,0 @@
|
||||
---
|
||||
name: testing
|
||||
description: Enforce TDD workflow and testing best practices for Penpot. Use when implementing features, fixing bugs, or modifying behavior. Reads testing memory for full guidance.
|
||||
---
|
||||
|
||||
# Testing Skill
|
||||
|
||||
Enforces test-driven development and Penpot testing conventions.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Implementing new logic or behavior
|
||||
- Fixing any bug (reproduction test required)
|
||||
- Modifying existing functionality
|
||||
- Adding edge case handling
|
||||
|
||||
**Skip:** Pure configuration changes, documentation updates, or static content with no behavioral impact.
|
||||
|
||||
## Workflow
|
||||
|
||||
Follow TDD (Red → Green → Refactor) whenever practical:
|
||||
|
||||
1. **RED** — Write a failing test first
|
||||
2. **GREEN** — Write minimal code to pass
|
||||
3. **REFACTOR** — Clean up while tests stay green
|
||||
|
||||
For bug fixes, use the Prove-It Pattern: write a test that reproduces the bug, confirm it fails, implement the fix, confirm it passes.
|
||||
|
||||
## Required Reading
|
||||
|
||||
Before writing any test, read:
|
||||
|
||||
1. `.serena/memories/testing.md` — cross-cutting testing principles, TDD workflow, anti-patterns, execution discipline
|
||||
2. Module-specific testing memory for the affected module:
|
||||
- `mem:common/testing` — CLJC unit tests
|
||||
- `mem:frontend/testing` — CLJS unit tests, Playwright E2E
|
||||
- `mem:backend/testing` — JVM clojure.test conventions
|
||||
- `mem:exporter/testing` — exporter unit tests
|
||||
|
||||
## Key Rules
|
||||
|
||||
- Every behavior change needs a test
|
||||
- Test state, not interactions
|
||||
- DAMP over DRY — tests are specifications; duplication is OK if each test is self-contained and readable
|
||||
- Prefer Real > Fake > Stub > Mock
|
||||
- Arrange-Act-Assert structure
|
||||
- One assertion per concept
|
||||
- Never pipe test output to filters — redirect to file first
|
||||
- Register new test files in the module's runner/entrypoint
|
||||
|
||||
## Verification
|
||||
|
||||
After completing implementation:
|
||||
|
||||
- [ ] Every new behavior has a test
|
||||
- [ ] All tests pass for touched modules
|
||||
- [ ] Bug fixes include a reproduction test
|
||||
- [ ] Lint/formatter passes
|
||||
@ -1,61 +0,0 @@
|
||||
---
|
||||
name: update-changelog
|
||||
description: Update the project CHANGES.md with issues from a given GitHub milestone, with correct categorization and references.
|
||||
---
|
||||
|
||||
# Skill: update-changelog
|
||||
|
||||
Entry point for all changelog work. All rules — sections, exclusions, entry
|
||||
format, the `:boom:`, `:rocket:` and `:lock:` contracts, pre-flight, and the
|
||||
anomaly report — live in `mem:workflow/updating-changelog` (file:
|
||||
`.serena/memories/workflow/updating-changelog.md`). This skill routes to that
|
||||
memory; it does not repeat it.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Before a new release, to populate the changelog with all fixed issues
|
||||
- When new issues are added to an existing milestone and the changelog needs
|
||||
to be refreshed
|
||||
- To audit or repair an existing version section
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
- Python 3.8+
|
||||
- `scripts/gh.py` for GitHub queries, `scripts/changelog.py` for changelog
|
||||
checks (run `python3 scripts/changelog.py <command> --help` for usage)
|
||||
|
||||
## What to Read
|
||||
|
||||
1. `mem:workflow/updating-changelog` — the canonical flow: sources of truth,
|
||||
layout, entry rules, the `## Breaking change note` comment contract, the
|
||||
advisory (`:lock:`) contract, pre-flight checks, anomaly types, and pitfalls.
|
||||
2. `mem:scripts/changelog` — `boom`, `security`, `check-merged`, `cross-ref`,
|
||||
`dev-only-bugs`, `report`.
|
||||
3. `mem:scripts/gh` — `issues`, `issue`, `comments`, `prs`, `advisories`.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
python3 scripts/gh.py issues "<VERSION>" --exclude "no changelog"
|
||||
python3 scripts/gh.py issues "<VERSION>" --label "release blocker" --exclude "no changelog" # closed non-bug residue to review by hand
|
||||
python3 scripts/gh.py advisories --patched-version "<VERSION>"
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" # list bugs that never reached production
|
||||
python3 scripts/changelog.py dev-only-bugs "<VERSION>" --apply # label the sure ones `no changelog`
|
||||
python3 scripts/changelog.py check-merged <PR numbers...>
|
||||
python3 scripts/changelog.py boom "<VERSION>" # print the :boom: section
|
||||
python3 scripts/changelog.py boom "<VERSION>" --write # replace it in CHANGES.md
|
||||
python3 scripts/changelog.py security "<VERSION>" # print the :lock: section
|
||||
python3 scripts/changelog.py security "<VERSION>" --write # replace it in CHANGES.md
|
||||
python3 scripts/changelog.py cross-ref "<VERSION>"
|
||||
python3 scripts/changelog.py report "<VERSION>"
|
||||
```
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- `CHANGES.md` is edited only on explicit request or during the release process.
|
||||
- Only closed, non-excluded milestone issues are included; the issue is the changelog unit and the PR is inline.
|
||||
- `:boom:` and `:rocket:` are label-generated and `:lock:` is advisory-generated: never hand-edit them; edit the label, the `## Breaking change note` comment or the advisory and re-run the command.
|
||||
- Verify PR merge status before writing; only merged PRs are referenced.
|
||||
- Bugs that never reached production stay out: `dev-only-bugs --apply` labels the sure ones; candidates are the user's call, never label them yourself.
|
||||
- Fill `patched_versions` in the advisories before running `security`; unpublished advisories are valid.
|
||||
@ -9,8 +9,7 @@
|
||||
app.common.data.macros/get-in clojure.core/get-in
|
||||
app.common.data.macros/with-open clojure.core/with-open
|
||||
app.common.data.macros/select-keys clojure.core/select-keys
|
||||
app.common.logging/with-context clojure.core/do
|
||||
benches.render-wasm.errors/attempt clojure.core/do}
|
||||
app.common.logging/with-context clojure.core/do}
|
||||
|
||||
:hooks
|
||||
{:analyze-call
|
||||
@ -23,13 +22,7 @@
|
||||
rumext.v2/defc hooks.export/rumext-defc
|
||||
rumext.v2/lazy-component hooks.export/rumext-lazycomponent
|
||||
shadow.lazy/loadable hooks.export/rumext-lazycomponent
|
||||
app.util.i18n/tr hooks.i18n/tr-dynamic
|
||||
app.common.i18n/tr hooks.i18n/tr-dynamic
|
||||
benches.render-wasm.declarations/defscene hooks.benches/defscene
|
||||
benches.render-wasm.declarations/defcase hooks.benches/defcase
|
||||
benches.render-wasm.runtime.commands/defcommand hooks.benches/defcommand
|
||||
benches.render-wasm.runtime.protocol/resolve-or-reject hooks.benches/resolve-or-reject
|
||||
benches.render-wasm.runtime.protocol/resolve-or-reject-with hooks.benches/resolve-or-reject-with}}
|
||||
}}
|
||||
|
||||
:output
|
||||
{:exclude-files
|
||||
@ -41,7 +34,7 @@
|
||||
"^(?:backend|frontend|exporter|common)/deps.edn"
|
||||
"^(?:backend|frontend|exporter|common)/scripts/"
|
||||
"^(?:backend|frontend|exporter|common)/dev/"
|
||||
"^(?:backend|frontend|exporter|common)/test/(?!performance_tests/)"]
|
||||
"^(?:backend|frontend|exporter|common)/test/"]
|
||||
|
||||
:linter-name true}
|
||||
|
||||
@ -52,9 +45,6 @@
|
||||
:potok/reify-type
|
||||
{:level :error}
|
||||
|
||||
:penpot/tr-dynamic
|
||||
{:level :warning}
|
||||
|
||||
:redundant-primitive-coercion
|
||||
{:level :off}
|
||||
|
||||
@ -98,12 +88,11 @@
|
||||
:dynamic-var-not-earmuffed
|
||||
{:level :off}
|
||||
|
||||
:type-mismatch
|
||||
{:level :off}
|
||||
|
||||
:used-underscored-binding
|
||||
{:level :warning}
|
||||
|
||||
:unused-binding
|
||||
{:exclude-destructured-as true
|
||||
:exclude-destructured-keys-in-fn-args false}}}
|
||||
:exclude-destructured-keys-in-fn-args false
|
||||
}
|
||||
}}
|
||||
|
||||
@ -1,77 +0,0 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns hooks.benches
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
(defn defscene
|
||||
"Analyzes scene options and its compiled builder body."
|
||||
[{:keys [node]}]
|
||||
(let [[_ _ opts argv & body] (:children node)]
|
||||
(if (and argv (api/vector-node? argv))
|
||||
{:node (api/list-node [(api/token-node (quote do)) opts
|
||||
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
|
||||
{:node node})))
|
||||
|
||||
(defn defcase
|
||||
"Analyzes case options and its compiled operation body."
|
||||
[{:keys [node]}]
|
||||
(let [[_ _ opts argv & body] (:children node)]
|
||||
(if (and (seq body) argv (api/vector-node? argv))
|
||||
{:node (api/list-node [(api/token-node (quote do)) opts
|
||||
(api/list-node (into [(api/token-node (quote fn)) argv] body))])}
|
||||
{:node node})))
|
||||
|
||||
(defn defcommand
|
||||
"Analyzes command options and both generated functions."
|
||||
[{:keys [node]}]
|
||||
(let [[_ command & forms] (:children node)
|
||||
[doc forms] (if (string? (api/sexpr (first forms)))
|
||||
[(first forms) (next forms)]
|
||||
[nil forms])
|
||||
[options forms] (if (api/map-node? (first forms))
|
||||
[(first forms) (next forms)]
|
||||
[nil forms])
|
||||
[argv & body] forms
|
||||
[conditions body] (if (and (api/map-node? (first body)) (next body))
|
||||
[(first body) (next body)]
|
||||
[nil body])
|
||||
handlers (:async (some-> options api/sexpr))
|
||||
core (api/list-node
|
||||
(into [(api/token-node 'fn)
|
||||
(api/vector-node (mapv api/token-node handlers))]
|
||||
body))
|
||||
starred (api/token-node (symbol (str (api/sexpr command) "*")))
|
||||
measured (get (some-> options api/sexpr) :measured-variant true)
|
||||
definition (api/list-node
|
||||
(into [(api/token-node 'defn) command]
|
||||
(concat (when doc [doc]) [argv]
|
||||
(when conditions [conditions]) [core])))]
|
||||
{:node (api/list-node
|
||||
(into [(api/token-node 'do)]
|
||||
(concat (when options [(api/coerce (dissoc (api/sexpr options) :async))])
|
||||
[definition]
|
||||
(when measured
|
||||
[(api/list-node [(api/token-node 'def)
|
||||
starred command])]))))}))
|
||||
|
||||
(defn resolve-or-reject
|
||||
"Analyzes the body of a Promise-returning form."
|
||||
[{:keys [node]}]
|
||||
(let [[_ & body] (:children node)]
|
||||
{:node (api/list-node (into [(api/token-node (quote do))] body))}))
|
||||
|
||||
(defn resolve-or-reject-with
|
||||
"Analyzes the cause handler and body. Binds `cause` in the handler only."
|
||||
[{:keys [node]}]
|
||||
(let [[_ cause on-error & body] (:children node)]
|
||||
{:node (api/list-node
|
||||
(into [(api/token-node (quote do))
|
||||
(api/list-node
|
||||
[(api/token-node (quote let))
|
||||
(api/vector-node [cause (api/token-node nil)])
|
||||
on-error])]
|
||||
body))}))
|
||||
@ -1,9 +1,3 @@
|
||||
;; This Source Code Form is subject to the terms of the Mozilla Public
|
||||
;; License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
;; file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
;;
|
||||
;; Copyright (c) KALEIDOS SUBSIDIARY SL
|
||||
|
||||
(ns hooks.export
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
|
||||
@ -1,17 +0,0 @@
|
||||
(ns hooks.i18n
|
||||
(:require [clj-kondo.hooks-api :as api]))
|
||||
|
||||
(defn tr-dynamic
|
||||
[{:keys [:node]}]
|
||||
(let [[_ code & _] (:children node)]
|
||||
(when (and (some? code)
|
||||
(not (api/string-node? code)))
|
||||
(let [{:keys [:row :col :end-row :end-col]} (meta code)]
|
||||
(api/reg-finding! {:message "dynamic key in (tr ...) is invisible to rehash; use a string literal or declare it with ;; (tr \"key\")"
|
||||
:type :penpot/tr-dynamic
|
||||
:row row
|
||||
:col col
|
||||
;; end positions are required for #_:clj-kondo/ignore to match
|
||||
:end-row end-row
|
||||
:end-col end-col}))))
|
||||
{:node node})
|
||||
@ -10,7 +10,5 @@
|
||||
:extra-indents {rumext.v2/fnc [[:inner 0]]
|
||||
cljs.test/async [[:inner 0]]
|
||||
promesa.exec/thread [[:inner 0]]
|
||||
specify! [[:inner 0] [:inner 1]]
|
||||
benches.render-wasm.runtime.commands/defcommand [[:inner 0]]
|
||||
benches.render-wasm.runtime.protocol/resolve-or-reject-with [[:inner 0]]}
|
||||
specify! [[:inner 0] [:inner 1]]}
|
||||
}
|
||||
|
||||
@ -52,7 +52,7 @@ built fresh from `shared/codex.toml` + `templates/codex.toml` at launch.
|
||||
`manage.sh`.
|
||||
* **`mcp/`** (Claude Code, opencode) is the result of merging `shared/` with
|
||||
the port-substituted `templates/`. `manage.sh` writes these on every
|
||||
`run-devenv --agentic` pass. Gitignored, dedicated paths with no developer
|
||||
`run-devenv-agentic` pass. Gitignored, dedicated paths with no developer
|
||||
content — never edit by hand, your edits will be overwritten on the next
|
||||
reconcile.
|
||||
* **`.vscode/mcp.json`** is the same merge, but written to the path VS Code
|
||||
@ -85,10 +85,6 @@ target instance is running and its MCP config has been generated:
|
||||
# 'main', 'ws1' and similar spellings are rejected.
|
||||
./manage.sh start-coding-agent claude --ws 1
|
||||
./manage.sh start-coding-agent opencode --ws 2
|
||||
|
||||
# Run a different executable (e.g. a wrapper script) with the same MCP
|
||||
# wiring. --ws and --bin may come in any order.
|
||||
./manage.sh start-coding-agent claude --bin claude-alt
|
||||
```
|
||||
|
||||
Equivalents by hand (run from inside the workspace directory):
|
||||
|
||||
@ -1,3 +0,0 @@
|
||||
# Penpot API configuration for error-reports CLI tool
|
||||
PENPOT_API_URI=http://localhost:3450
|
||||
PENPOT_ACCESS_TOKEN=your-access-token-here
|
||||
9
.github/CODEOWNERS
vendored
9
.github/CODEOWNERS
vendored
@ -1,9 +0,0 @@
|
||||
# Changes to workflows, CI configuration and repository automation need
|
||||
# review from the ci-maintainers team. Issue and PR templates do not.
|
||||
/.github/workflows/ @penpot/ci-maintainers
|
||||
/.github/actions/ @penpot/ci-maintainers
|
||||
/.github/scripts/ @penpot/ci-maintainers
|
||||
/.github/dependabot.yml @penpot/ci-maintainers
|
||||
/.github/zizmor.yml @penpot/ci-maintainers
|
||||
/.github/actionlint.yaml @penpot/ci-maintainers
|
||||
/.github/CODEOWNERS @penpot/ci-maintainers
|
||||
@ -1,7 +1,7 @@
|
||||
description: An unexpected problem or behavior
|
||||
labels: ["needs triage"]
|
||||
name: Bug
|
||||
description: Create a report to help us improve
|
||||
name: Bug report
|
||||
type: Bug
|
||||
labels: ["needs triage"]
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
@ -15,7 +15,7 @@ body:
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Steps To Reproduce
|
||||
description: Steps to reproduce the behavior
|
||||
description: Steps to reproduce the behavior.
|
||||
placeholder: |
|
||||
Steps to reproduce the behavior:
|
||||
1. Go to '...'
|
||||
@ -26,22 +26,22 @@ body:
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
description: A clear and concise description of what you expected to happen.
|
||||
label: Expected behavior
|
||||
description: A clear and concise description of what you expected to happen
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: actual
|
||||
attributes:
|
||||
description: A clear and concise description of what happens instead; what the bug is.
|
||||
label: Actual behavior
|
||||
description: A clear and concise description of what happens instead; what the bug is
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: screenshots
|
||||
attributes:
|
||||
description: If applicable, add screenshots to help explain your problem.
|
||||
label: Screenshots or video
|
||||
description: If applicable, add screenshots to help explain your problem
|
||||
- type: textarea
|
||||
id: desktop
|
||||
attributes:
|
||||
@ -87,4 +87,4 @@ body:
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add any other context or screenshots about the bug here
|
||||
description: Any other context about the problem.
|
||||
2
.github/ISSUE_TEMPLATE/config.yml
vendored
2
.github/ISSUE_TEMPLATE/config.yml
vendored
@ -1 +1 @@
|
||||
blank_issues_enabled: false
|
||||
blank_issues_enabled: true
|
||||
|
||||
38
.github/ISSUE_TEMPLATE/enhancement.yml
vendored
38
.github/ISSUE_TEMPLATE/enhancement.yml
vendored
@ -1,38 +0,0 @@
|
||||
description: Improvement to existing behavior
|
||||
labels: ["needs triage"]
|
||||
name: "Enhancement"
|
||||
type: Enhancement
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Before you start
|
||||
|
||||
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
|
||||
|
||||
---
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Is your feature request related to a problem? Please describe
|
||||
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Describe the solution you'd like
|
||||
description: A clear and concise description of what you want to happen
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Describe alternatives you've considered
|
||||
description: A clear and concise description of any alternative solutions or features you've considered
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add any other context or screenshots about the enhancement here
|
||||
26
.github/ISSUE_TEMPLATE/epic.yml
vendored
26
.github/ISSUE_TEMPLATE/epic.yml
vendored
@ -1,26 +0,0 @@
|
||||
description: An epic is a large user story that cannot be delivered as defined within a single iteration
|
||||
labels: ["needs triage"]
|
||||
name: "EPIC"
|
||||
type: EPIC
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Before you start
|
||||
|
||||
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
|
||||
|
||||
---
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Epic description
|
||||
description: Tell us which epic feature do you want to discuss
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add any other context or screenshots about the feature request here
|
||||
@ -1,7 +1,7 @@
|
||||
description: Suggest an idea for this project
|
||||
description: Suggest an idea for this project.
|
||||
labels: ["needs triage"]
|
||||
name: "Feature request"
|
||||
type: Feature
|
||||
type: Enhancement
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
@ -15,24 +15,24 @@ body:
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Is your feature request related to a problem? Please describe
|
||||
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when (...)
|
||||
label: Is your feature request related to a problem? Please describe.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Describe the solution you'd like
|
||||
description: A clear and concise description of what you want to happen
|
||||
description: A clear and concise description of what you want to happen.
|
||||
label: Describe the solution you'd like.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Describe alternatives you've considered
|
||||
description: A clear and concise description of any alternative solutions or features you've considered
|
||||
label: Describe alternatives you've considered.
|
||||
description: A clear and concise description of any alternative solutions or features you've considered.
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add any other context or screenshots about the feature request here
|
||||
description: Add any other context or screenshots about the feature request here.
|
||||
26
.github/ISSUE_TEMPLATE/task.yml
vendored
26
.github/ISSUE_TEMPLATE/task.yml
vendored
@ -1,26 +0,0 @@
|
||||
description: A specific piece of work
|
||||
labels: []
|
||||
name: "Task"
|
||||
type: Task
|
||||
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
## Before you start
|
||||
|
||||
Please search our [existing issues](https://github.com/penpot/penpot/issues) and open [pull requests](https://github.com/penpot/penpot/pulls) to lessen the change of filing duplicate issues or feature requests. Thank you.
|
||||
|
||||
---
|
||||
- type: textarea
|
||||
id: task
|
||||
attributes:
|
||||
description: A clear and concise description of the task
|
||||
label: Please describe
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add any other context or screenshots about the task here
|
||||
14
.github/actionlint.yaml
vendored
14
.github/actionlint.yaml
vendored
@ -1,14 +0,0 @@
|
||||
# actionlint configuration — https://github.com/rhysd/actionlint/blob/main/docs/config.md
|
||||
self-hosted-runner:
|
||||
labels:
|
||||
- penpot-standar-runner
|
||||
- penpot-extended-runner
|
||||
|
||||
paths:
|
||||
.github/workflows/**/*.{yml,yaml}:
|
||||
ignore:
|
||||
# shellcheck style/info notices that are pervasive in the existing
|
||||
# scripts (unquoted `>> $GITHUB_OUTPUT`, repeated redirects). Real
|
||||
# shellcheck warnings and errors are still reported.
|
||||
- 'SC2086:info'
|
||||
- 'SC2129:style'
|
||||
41
.github/scripts/playwright-summary.jq
vendored
41
.github/scripts/playwright-summary.jq
vendored
@ -1,41 +0,0 @@
|
||||
def specs: [.. | objects | select(has("tests") and has("file"))];
|
||||
def dur: [.tests[].results[]?.duration // 0] | add;
|
||||
|
||||
specs as $s
|
||||
| ($s | map(select(any(.tests[]; .status == "unexpected")))) as $failed
|
||||
| ($s | map(select(any(.tests[]; .status == "flaky")))) as $flaky
|
||||
| ($s | map(select(any(.tests[]; .status == "skipped")))) as $skipped
|
||||
| ($s | length) as $total
|
||||
| ($s | map(dur) | add // 0 | . / 1000 | floor) as $cpu
|
||||
| (if ($failed | length) > 0 then "❌"
|
||||
elif ($flaky | length) > 0 then "⚠️"
|
||||
else "✅" end) as $icon
|
||||
|
||||
| "## \($icon) Integration tests\n\n"
|
||||
+ "| Total | Passed | Flaky | Failed | Skipped | Test time |\n"
|
||||
+ "|---|---|---|---|---|---|\n"
|
||||
+ "| \($total) | \($total - ($failed|length) - ($flaky|length) - ($skipped|length)) "
|
||||
+ "| \($flaky|length) | \($failed|length) | \($skipped|length) | \($cpu / 60 | floor)m |\n"
|
||||
|
||||
+ (if ($failed | length) > 0 then
|
||||
"\n### Failed\n\n"
|
||||
+ ($failed | map("- `\(.file):\(.line)` — \(.title)") | join("\n")) + "\n"
|
||||
else "" end)
|
||||
|
||||
+ (if ($flaky | length) > 0 then
|
||||
"\n### Flaky (passed on retry)\n\n"
|
||||
+ ($flaky
|
||||
| map({ t: "`\(.file):\(.line)` — \(.title)",
|
||||
r: ([.tests[].results[]? | select(.status == "failed")] | length) })
|
||||
| sort_by(-.r)
|
||||
| map("- \(.t) _(\(.r) \(if .r == 1 then "retry" else "retries" end))_")
|
||||
| join("\n")) + "\n"
|
||||
else "" end)
|
||||
|
||||
+ (if $total > 0 then
|
||||
"\n<details><summary>Slowest specs</summary>\n\n"
|
||||
+ ($s | map({ t: "`\(.file)` — \(.title)", d: (dur / 1000 | floor) })
|
||||
| sort_by(-.d) | .[0:5]
|
||||
| map("- \(.t) — \(.d)s") | join("\n"))
|
||||
+ "\n\n</details>\n"
|
||||
else "" end)
|
||||
249
.github/workflows/auto-label.yml
vendored
249
.github/workflows/auto-label.yml
vendored
@ -1,249 +0,0 @@
|
||||
name: Auto Label and Add to Project
|
||||
|
||||
# pull_request_target is required so the GitHub App secrets are available
|
||||
# for PRs from forks. It is safe here: the job never checks out or runs PR
|
||||
# code, it only reads ids from the event payload.
|
||||
on:
|
||||
issues:
|
||||
types: [opened, closed]
|
||||
pull_request_target:
|
||||
types: [opened]
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
triage:
|
||||
# The Renovate Dependency Dashboard and Renovate's GitHub Actions PRs do
|
||||
# not need triage. Other Renovate PRs (base images, Node.js, pnpm) do.
|
||||
if: >-
|
||||
!(github.event_name == 'issues'
|
||||
&& github.event.issue.user.login == 'penpot-renovate[bot]')
|
||||
&& !(github.event_name == 'pull_request_target'
|
||||
&& github.event.pull_request.user.login == 'penpot-renovate[bot]'
|
||||
&& github.event.pull_request.head.ref == 'renovate-github-actions')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Generate GitHub App token
|
||||
id: triage-app-token
|
||||
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
|
||||
with:
|
||||
client-id: ${{ secrets.TRIAGE_APP_CLIENT_ID }}
|
||||
private-key: ${{ secrets.TRIAGE_APP_PRIVATE_KEY }}
|
||||
# No owner/repositories: the token is scoped to this repository.
|
||||
# Organization permissions (the project board) still apply.
|
||||
permission-issues: write
|
||||
permission-pull-requests: write
|
||||
permission-organization-projects: write
|
||||
|
||||
- name: Process Issue or PR
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
github-token: ${{ steps.triage-app-token.outputs.token }}
|
||||
script: |
|
||||
// === 1. CONFIGURATION ===
|
||||
const PROJECT_NUMBER = 8; // <--- Replace with your project board number
|
||||
const IS_ORG = true; // <--- Set to false if this is a personal project, true if an organization
|
||||
const STATUS_FIELD_ID = 'PVTSSF_lADOAcyBPM4BV2pqzhRPY48';
|
||||
const STATUS_BACKLOG = 'f75ad846';
|
||||
const STATUS_TECH_REVIEW = 'b0f45db9';
|
||||
const OWNER = context.repo.owner;
|
||||
const REPO = context.repo.repo;
|
||||
|
||||
const issueNumber = context.issue.number;
|
||||
const isPR = !!context.payload.pull_request;
|
||||
const contentId = isPR ? context.payload.pull_request.node_id : context.payload.issue.node_id;
|
||||
|
||||
// A completed issue is already closed by GitHub. Route its Main
|
||||
// project status according to whether QA needs to verify it.
|
||||
if (context.eventName === 'issues' && context.payload.action === 'closed') {
|
||||
const issue = context.payload.issue;
|
||||
if (issue.state_reason !== 'completed') {
|
||||
console.log(`Issue #${issue.number} was closed as ${issue.state_reason}; leaving project status unchanged.`);
|
||||
return;
|
||||
}
|
||||
|
||||
const targetStatus = issue.labels.some(({ name }) => name === 'team-qa')
|
||||
? 'Ready for QA'
|
||||
: 'Product Review';
|
||||
|
||||
const projectQuery = `
|
||||
query($owner: String!, $number: Int!) {
|
||||
organization(login: $owner) {
|
||||
projectV2(number: $number) {
|
||||
id
|
||||
fields(first: 100) {
|
||||
nodes {
|
||||
... on ProjectV2SingleSelectField {
|
||||
id
|
||||
name
|
||||
options { id name }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
const projectRes = await github.graphql(projectQuery, {
|
||||
owner: OWNER,
|
||||
number: PROJECT_NUMBER
|
||||
});
|
||||
const project = projectRes.organization.projectV2;
|
||||
if (!project) {
|
||||
throw new Error(`Could not find Main project #${PROJECT_NUMBER}.`);
|
||||
}
|
||||
|
||||
const statusField = project.fields.nodes.find(({ name }) => name === 'Status');
|
||||
const statusOption = statusField?.options.find(({ name }) => name === targetStatus);
|
||||
if (!statusField || !statusOption) {
|
||||
throw new Error(`Could not find Status option "${targetStatus}" in Main.`);
|
||||
}
|
||||
|
||||
const issueProjectItemsQuery = `
|
||||
query($issueId: ID!) {
|
||||
node(id: $issueId) {
|
||||
... on Issue {
|
||||
projectItems(first: 100) {
|
||||
nodes {
|
||||
id
|
||||
project { id }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
const issueProjectItemsRes = await github.graphql(issueProjectItemsQuery, {
|
||||
issueId: contentId
|
||||
});
|
||||
const projectItems = issueProjectItemsRes.node?.projectItems?.nodes || [];
|
||||
const projectItem = projectItems.find(
|
||||
({ project: itemProject }) => itemProject.id === project.id
|
||||
);
|
||||
if (!projectItem) {
|
||||
console.log(`Issue #${issue.number} is not in Main project #${PROJECT_NUMBER}; skipping status update.`);
|
||||
return;
|
||||
}
|
||||
|
||||
const updateStatusMutation = `
|
||||
mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
|
||||
updateProjectV2ItemFieldValue(input: {
|
||||
projectId: $projectId
|
||||
itemId: $itemId
|
||||
fieldId: $fieldId
|
||||
value: { singleSelectOptionId: $optionId }
|
||||
}) {
|
||||
projectV2Item { id }
|
||||
}
|
||||
}
|
||||
`;
|
||||
await github.graphql(updateStatusMutation, {
|
||||
projectId: project.id,
|
||||
itemId: projectItem.id,
|
||||
fieldId: statusField.id,
|
||||
optionId: statusOption.id
|
||||
});
|
||||
console.log(`Issue #${issue.number} closed; Main status set to "${targetStatus}".`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Define your labels here
|
||||
const labelToApply = 'needs triage';
|
||||
|
||||
// === 2. APPLY THE LABEL ===
|
||||
console.log(`Applying label "${labelToApply}" to ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
|
||||
await github.rest.issues.addLabels({
|
||||
issue_number: issueNumber,
|
||||
owner: OWNER,
|
||||
repo: REPO,
|
||||
labels: [labelToApply]
|
||||
});
|
||||
|
||||
// === 3. ADD TO PROJECT BOARD ===
|
||||
console.log(`Fetching Project #${PROJECT_NUMBER} ID...`);
|
||||
const projectQuery = `
|
||||
query($owner: String!, $number: Int!) {
|
||||
${IS_ORG ? 'organization' : 'user'}(login: $owner) {
|
||||
projectV2(number: $number) {
|
||||
id
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
const projectRes = await github.graphql(projectQuery, { owner: OWNER, number: PROJECT_NUMBER });
|
||||
const projectId = IS_ORG ? projectRes.organization.projectV2.id : projectRes.user.projectV2.id;
|
||||
|
||||
console.log(`Adding item to project board...`);
|
||||
const addToProjectMutation = `
|
||||
mutation($projectId: ID!, $contentId: ID!) {
|
||||
addProjectV2ItemById(input: {projectId: $projectId, contentId: $contentId}) {
|
||||
item {
|
||||
id
|
||||
}
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
||||
let itemId;
|
||||
try {
|
||||
const addRes = await github.graphql(addToProjectMutation, { projectId, contentId });
|
||||
itemId = addRes.addProjectV2ItemById.item.id;
|
||||
} catch (error) {
|
||||
const messages = [
|
||||
error && error.message,
|
||||
...(Array.isArray(error && error.errors)
|
||||
? error.errors.map(e => e && e.message)
|
||||
: [])
|
||||
].filter(Boolean).join('\n');
|
||||
if (!messages.includes('Content already exists in this project')) {
|
||||
throw error;
|
||||
}
|
||||
console.log('Item already in project, fetching existing item...');
|
||||
const existing = await github.graphql(
|
||||
`query($contentId: ID!) {
|
||||
node(id: $contentId) {
|
||||
... on Issue {
|
||||
projectItems(first: 50) {
|
||||
nodes { id project { id } }
|
||||
}
|
||||
}
|
||||
... on PullRequest {
|
||||
projectItems(first: 50) {
|
||||
nodes { id project { id } }
|
||||
}
|
||||
}
|
||||
}
|
||||
}`,
|
||||
{ contentId }
|
||||
);
|
||||
const nodes = (existing.node && existing.node.projectItems && existing.node.projectItems.nodes) || [];
|
||||
const match = nodes.find(n => n.project.id === projectId);
|
||||
if (!match) {
|
||||
throw new Error(`Item already in a project, but not found in project ${projectId}`);
|
||||
}
|
||||
itemId = match.id;
|
||||
}
|
||||
|
||||
// === 4. SET PROJECT STATUS ===
|
||||
const statusOptionId = isPR ? STATUS_TECH_REVIEW : STATUS_BACKLOG;
|
||||
const statusName = isPR ? 'Tech Review' : 'Backlog';
|
||||
console.log(`Setting status to "${statusName}" for ${isPR ? 'PR' : 'Issue'} #${issueNumber}...`);
|
||||
await github.graphql(
|
||||
`mutation($projectId: ID!, $itemId: ID!, $fieldId: ID!, $optionId: String!) {
|
||||
updateProjectV2ItemFieldValue(input: {
|
||||
projectId: $projectId,
|
||||
itemId: $itemId,
|
||||
fieldId: $fieldId,
|
||||
value: { singleSelectOptionId: $optionId }
|
||||
}) {
|
||||
projectV2Item {
|
||||
id
|
||||
}
|
||||
}
|
||||
}`,
|
||||
{ projectId, itemId, fieldId: STATUS_FIELD_ID, optionId: statusOptionId }
|
||||
);
|
||||
|
||||
console.log("Automation successfully completed!");
|
||||
|
||||
69
.github/workflows/build-adhoc.yml
vendored
69
.github/workflows/build-adhoc.yml
vendored
@ -1,69 +0,0 @@
|
||||
name: _ADHOC
|
||||
|
||||
run-name: >-
|
||||
_ADHOC (${{ inputs.gh_ref }}${{ inputs.nitrate_ref != '' && format(' / nitrate:{0}', inputs.nitrate_ref) || '' }})
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Branch/ref to build in penpot/penpot'
|
||||
type: string
|
||||
required: true
|
||||
nitrate_ref:
|
||||
description: 'Branch/ref to build admin-console in penpot/penpot-nitrate (defaults to gh_ref)'
|
||||
type: string
|
||||
required: false
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built/promoted'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-bundle:
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-bundle.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
with:
|
||||
gh_ref: ${{ inputs.gh_ref }}
|
||||
force: ${{ inputs.force }}
|
||||
|
||||
build-docker:
|
||||
needs: build-bundle
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-docker.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
with:
|
||||
gh_ref: ${{ inputs.gh_ref }}
|
||||
# Pin build-docker to the commit build-bundle actually bundled.
|
||||
sha: ${{ needs.build-bundle.outputs.sha }}
|
||||
force: ${{ inputs.force }}
|
||||
|
||||
build-docker-admin-console:
|
||||
permissions: {}
|
||||
uses: ./.github/workflows/build-docker-admin-console.yml
|
||||
secrets:
|
||||
ORG_WORKFLOW_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
|
||||
with:
|
||||
gh_ref: ${{ inputs.nitrate_ref || inputs.gh_ref }}
|
||||
force: ${{ inputs.force }}
|
||||
194
.github/workflows/build-bundle.yml
vendored
194
.github/workflows/build-bundle.yml
vendored
@ -9,153 +9,60 @@ on:
|
||||
type: string
|
||||
required: true
|
||||
default: 'develop'
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if this version already exists in S3'
|
||||
type: boolean
|
||||
build_wasm:
|
||||
description: 'BUILD_WASM. Valid values: yes, no'
|
||||
type: string
|
||||
required: false
|
||||
default: false
|
||||
default: 'yes'
|
||||
build_storybook:
|
||||
description: 'BUILD_STORYBOOK. Valid values: yes, no'
|
||||
type: string
|
||||
required: false
|
||||
default: 'yes'
|
||||
workflow_call:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Name of the branch or ref'
|
||||
type: string
|
||||
required: true
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if this version already exists in S3'
|
||||
type: boolean
|
||||
default: 'develop'
|
||||
build_wasm:
|
||||
description: 'BUILD_WASM. Valid values: yes, no'
|
||||
type: string
|
||||
required: false
|
||||
default: false
|
||||
outputs:
|
||||
sha:
|
||||
description: 'Bundled commit, full 40-char SHA.'
|
||||
value: ${{ jobs.check.outputs.sha }}
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID:
|
||||
default: 'yes'
|
||||
build_storybook:
|
||||
description: 'BUILD_STORYBOOK. Valid values: yes, no'
|
||||
type: string
|
||||
required: false
|
||||
AWS_REGION:
|
||||
required: false
|
||||
AWS_SECRET_ACCESS_KEY:
|
||||
required: false
|
||||
MATTERMOST_WEBHOOK:
|
||||
required: false
|
||||
S3_BUCKET:
|
||||
required: false
|
||||
|
||||
# Literal group name: under `workflow_call`, `github.workflow` resolves to the
|
||||
# caller's workflow, which put this workflow and the other reusable one called
|
||||
# by the same caller into a single shared group, and left a manual dispatch of
|
||||
# the same ref in a group of its own, free to race on the same artifacts.
|
||||
concurrency:
|
||||
group: build-bundle-${{ inputs.gh_ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
default: 'yes'
|
||||
|
||||
jobs:
|
||||
# ── 1. Decide whether there is anything to build ───────────────────────
|
||||
check:
|
||||
name: Check current bundle
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
gh_ref: ${{ steps.vars.outputs.gh_ref }}
|
||||
bundle_version: ${{ steps.vars.outputs.bundle_version }}
|
||||
build_key: ${{ steps.vars.outputs.build_key }}
|
||||
short_sha: ${{ steps.vars.outputs.short_sha }}
|
||||
sha: ${{ steps.vars.outputs.sha }}
|
||||
commit_title: ${{ steps.vars.outputs.commit_title }}
|
||||
exists: ${{ steps.check.outputs.exists }}
|
||||
build-bundle:
|
||||
name: Build and Upload Penpot Bundle
|
||||
runs-on: penpot-runner-01
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref || github.ref_name }}
|
||||
run: |
|
||||
# Same command the build uses to stamp version.txt.
|
||||
BUNDLE_VERSION=$(./manage.sh version)
|
||||
SHORT_SHA=$(git rev-parse --short=12 HEAD)
|
||||
# The key takes only the closest tag (--abbrev=0): the sha already
|
||||
# pins the commit, and the full describe output is not stable
|
||||
# across checkouts (its -g<sha> suffix length depends on the
|
||||
# local object count). Must match build-docker.yml.
|
||||
VERSION_TAG=$(git describe --tags --match '*.*.*' --abbrev=0)
|
||||
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
|
||||
echo "bundle_version=$BUNDLE_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "build_key=sha-${SHORT_SHA}-${VERSION_TAG}" >> $GITHUB_OUTPUT
|
||||
echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
|
||||
echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
|
||||
echo "commit_title=$(git log -1 --pretty=%s)" >> $GITHUB_OUTPUT
|
||||
|
||||
# Keyed by commit sha AND version, same as build-docker.yml's marker
|
||||
# check, so both pipelines agree on what "already built" means. The
|
||||
# version is stamped into the bundle, so a new tag on an already
|
||||
# bundled commit (2.18.1-RC1 -> 2.18.1) must produce a new bundle.
|
||||
# `force` bypasses this check entirely.
|
||||
- name: Check if this bundle is already built
|
||||
id: check
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
FORCE: ${{ inputs.force }}
|
||||
BUILD_KEY: ${{ steps.vars.outputs.build_key }}
|
||||
run: |
|
||||
if [ "$FORCE" = "true" ]; then
|
||||
echo "exists=false" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "### 🔁 Bundle build forced"
|
||||
echo ""
|
||||
echo "\`force: true\` — skipping the S3 sha check."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
BUNDLE_EXISTS=$(aws s3api head-object \
|
||||
--bucket "$S3_BUCKET" \
|
||||
--key "penpot-${BUILD_KEY}.zip" \
|
||||
> /dev/null 2>&1 && echo "true" || echo "false")
|
||||
|
||||
if [ "$BUNDLE_EXISTS" = "true" ]; then
|
||||
echo "exists=true" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "### ⏭️ Bundle build skipped"
|
||||
echo ""
|
||||
echo "The bundle in S3 was already built for \`${BUILD_KEY}\`."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
else
|
||||
echo "exists=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# ── 2. Build and upload, only when needed ──────────────────────────────
|
||||
build:
|
||||
name: Build and Upload Penpot Bundle
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 90
|
||||
needs: check
|
||||
if: needs.check.outputs.exists == 'false'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
|
||||
echo "bundle_version=$(git describe --tags --always)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Build bundle
|
||||
env:
|
||||
BUILD_WASM: 'yes'
|
||||
BUILD_STORYBOOK: 'yes'
|
||||
BUILD_WASM: ${{ inputs.build_wasm }}
|
||||
BUILD_STORYBOOK: ${{ inputs.build_storybook }}
|
||||
run: ./manage.sh build-bundle
|
||||
|
||||
- name: Prepare directories for zipping
|
||||
@ -169,49 +76,18 @@ jobs:
|
||||
zip -r zips/penpot.zip penpot
|
||||
|
||||
- name: Upload Penpot bundle to S3
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
BUILD_KEY: ${{ needs.check.outputs.build_key }}
|
||||
BUNDLE_VERSION: ${{ needs.check.outputs.bundle_version }}
|
||||
run: |
|
||||
aws s3 cp zips/penpot.zip \
|
||||
"s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" \
|
||||
--metadata "bundle-version=${BUNDLE_VERSION}"
|
||||
aws s3 cp zips/penpot.zip s3://${{ secrets.S3_BUCKET }}/penpot-${{ steps.vars.outputs.gh_ref }}.zip --metadata bundle-version=${{ steps.vars.outputs.bundle_version }}
|
||||
|
||||
- name: Write step summary
|
||||
env:
|
||||
SHORT_SHA: ${{ needs.check.outputs.short_sha }}
|
||||
BUNDLE_VERSION: ${{ needs.check.outputs.bundle_version }}
|
||||
COMMIT_TITLE: ${{ needs.check.outputs.commit_title }}
|
||||
run: |
|
||||
{
|
||||
echo "### ✅ Bundle built"
|
||||
echo ""
|
||||
echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
|
||||
echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
|
||||
echo "- Built at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# ── 3. Single failure notification for the whole workflow ─────────────
|
||||
notify:
|
||||
name: Notify failure
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 5
|
||||
needs: [check, build]
|
||||
if: failure()
|
||||
|
||||
steps:
|
||||
- name: Notify Mattermost
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
TEXT: |
|
||||
❌ 📦 *[PENPOT] Error building penpot bundles.*
|
||||
📄 Triggered from ref: `${{ needs.check.outputs.gh_ref || inputs.gh_ref }}`
|
||||
Bundle version: `${{ needs.check.outputs.bundle_version || 'n/a' }}`
|
||||
📄 Triggered from ref: `${{ steps.vars.outputs.gh_ref }}`
|
||||
Bundle version: `${{ steps.vars.outputs.bundle_version }}`
|
||||
🔗 Run: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
@infra
|
||||
|
||||
53
.github/workflows/build-develop.yml
vendored
53
.github/workflows/build-develop.yml
vendored
@ -1,67 +1,22 @@
|
||||
name: _DEVELOP
|
||||
|
||||
run-name: >-
|
||||
_DEVELOP (develop @ ${{ github.sha }})
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built/promoted'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
schedule:
|
||||
- cron: '16 5-20 * * 1-5'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-bundle:
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-bundle.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "develop"
|
||||
force: ${{ inputs.force || false }}
|
||||
build_wasm: "yes"
|
||||
build_storybook: "yes"
|
||||
|
||||
build-docker:
|
||||
needs: build-bundle
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-docker.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "develop"
|
||||
# Pin build-docker to the commit build-bundle actually bundled.
|
||||
sha: ${{ needs.build-bundle.outputs.sha }}
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
build-docker-admin-console:
|
||||
permissions: {}
|
||||
uses: ./.github/workflows/build-docker-admin-console.yml
|
||||
secrets:
|
||||
ORG_WORKFLOW_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
|
||||
with:
|
||||
gh_ref: "develop"
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
110
.github/workflows/build-docker-admin-console.yml
vendored
110
.github/workflows/build-docker-admin-console.yml
vendored
@ -1,110 +0,0 @@
|
||||
name: Admin Console Docker Builder
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Name of the branch or ref to build in penpot-nitrate'
|
||||
type: string
|
||||
required: true
|
||||
default: 'develop'
|
||||
dispatch_ref:
|
||||
description: 'Branch of penpot-nitrate from which the workflow definition is read'
|
||||
type: string
|
||||
required: false
|
||||
default: 'develop'
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Name of the branch or ref to build in penpot-nitrate'
|
||||
type: string
|
||||
required: true
|
||||
dispatch_ref:
|
||||
description: 'Branch of penpot-nitrate from which the workflow definition is read'
|
||||
type: string
|
||||
required: false
|
||||
default: 'develop'
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
secrets:
|
||||
ORG_WORKFLOW_TOKEN:
|
||||
description: 'Token with Actions write access on penpot-nitrate'
|
||||
required: true
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-nitrate-docker:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
|
||||
REPO: penpot/penpot-nitrate
|
||||
WORKFLOW: build-docker-admin-console.yml
|
||||
GH_REF: ${{ inputs.gh_ref }}
|
||||
DISPATCH_REF: ${{ inputs.dispatch_ref }}
|
||||
FORCE: ${{ inputs.force }}
|
||||
steps:
|
||||
- name: Trigger nitrate docker build
|
||||
id: dispatch
|
||||
run: |
|
||||
DISTINCT_ID="${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
|
||||
CALLER_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
|
||||
|
||||
gh workflow run "$WORKFLOW" --repo "$REPO" --ref "$DISPATCH_REF" \
|
||||
-f gh_ref="$GH_REF" \
|
||||
-f force="$FORCE" \
|
||||
-f caller_run_id="$DISTINCT_ID" \
|
||||
-f caller_run_url="$CALLER_URL"
|
||||
|
||||
# Locate the dispatched run using the correlation id embedded in its run-name
|
||||
RUN_ID=""
|
||||
for _ in $(seq 1 24); do
|
||||
sleep 5
|
||||
RUN_ID=$(gh run list --repo "$REPO" --workflow "$WORKFLOW" \
|
||||
--limit 10 --json databaseId,displayTitle \
|
||||
--jq ".[] | select(.displayTitle | contains(\"$DISTINCT_ID\")) | .databaseId" \
|
||||
| head -n1)
|
||||
[ -n "$RUN_ID" ] && break
|
||||
done
|
||||
|
||||
if [ -z "$RUN_ID" ]; then
|
||||
echo "::error::Could not locate the dispatched run in $REPO"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RUN_URL="https://github.com/$REPO/actions/runs/$RUN_ID"
|
||||
echo "run_id=$RUN_ID" >> "$GITHUB_OUTPUT"
|
||||
echo "run_url=$RUN_URL" >> "$GITHUB_OUTPUT"
|
||||
echo "::notice title=Nitrate docker build::$RUN_URL"
|
||||
|
||||
- name: Wait for nitrate docker build
|
||||
env:
|
||||
RUN_ID: ${{ steps.dispatch.outputs.run_id }}
|
||||
run: |
|
||||
gh run watch "$RUN_ID" \
|
||||
--repo "$REPO" \
|
||||
--interval 30 \
|
||||
--exit-status
|
||||
|
||||
- name: Report result
|
||||
if: always() && steps.dispatch.outputs.run_id != ''
|
||||
env:
|
||||
RUN_ID: ${{ steps.dispatch.outputs.run_id }}
|
||||
RUN_URL: ${{ steps.dispatch.outputs.run_url }}
|
||||
run: |
|
||||
CONCLUSION=$(gh run view "$RUN_ID" \
|
||||
--repo "$REPO" --json conclusion --jq '.conclusion')
|
||||
{
|
||||
echo "### 🐳 Nitrate docker build"
|
||||
echo ""
|
||||
echo "- Result: \`${CONCLUSION:-in_progress}\`"
|
||||
echo "- Run: ${RUN_URL}"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
55
.github/workflows/build-docker-devenv.yml
vendored
55
.github/workflows/build-docker-devenv.yml
vendored
@ -3,81 +3,48 @@ name: DevEnv Docker Image Builder
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: build-docker-devenv
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
name: Build and push DevEnv Docker image
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 90
|
||||
env:
|
||||
DOCKER_IMAGE: 'penpotapp/devenv'
|
||||
runs-on: penpot-runner-02
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
# Each job execution will use its own docker configuration.
|
||||
echo "DOCKER_CONFIG=${RUNNER_TEMP}/.docker-${GITHUB_RUN_ID}-${GITHUB_JOB}" >> "$GITHUB_ENV"
|
||||
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
|
||||
with:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Login to Docker Registry (push destination)
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
- name: Login to Docker Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Login to Docker Hardened Images registry (base image pull)
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: dhi.io
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Build and push DevEnv Docker image
|
||||
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
DOCKER_IMAGE: 'penpotapp/devenv'
|
||||
with:
|
||||
context: ./docker/devenv/
|
||||
file: ./docker/devenv/Dockerfile
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
tags: ${{ env.DOCKER_IMAGE }}:latest
|
||||
cache-from: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
- name: Notify Mattermost
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
TEXT: |
|
||||
🚀 *[PENPOT] New devenv available*
|
||||
📄 You may want to update your devenv.
|
||||
|
||||
- name: Notify Mattermost (developers)
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: penpotdev
|
||||
TEXT: |
|
||||
🚀 *[PENPOT] New devenv available*
|
||||
📄 You may want to update your devenv.
|
||||
(\cc @alvaro)
|
||||
@alvaro
|
||||
|
||||
479
.github/workflows/build-docker.yml
vendored
479
.github/workflows/build-docker.yml
vendored
@ -8,423 +8,176 @@ on:
|
||||
type: string
|
||||
required: true
|
||||
default: 'develop'
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if this sha is already promoted'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Name of the branch or ref'
|
||||
type: string
|
||||
required: true
|
||||
sha:
|
||||
description: >-
|
||||
Exact commit to check out (full 40-char SHA, e.g. from
|
||||
build-bundle.yml). Falls back to gh_ref when empty; gh_ref
|
||||
always still names the branch tag to move.
|
||||
type: string
|
||||
required: false
|
||||
default: ''
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if this sha is already promoted'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID:
|
||||
required: false
|
||||
AWS_REGION:
|
||||
required: false
|
||||
AWS_SECRET_ACCESS_KEY:
|
||||
required: false
|
||||
DOCKER_PASSWORD:
|
||||
required: false
|
||||
DOCKER_REGISTRY:
|
||||
required: false
|
||||
DOCKER_USERNAME:
|
||||
required: false
|
||||
MATTERMOST_WEBHOOK:
|
||||
required: false
|
||||
PUB_DOCKER_PASSWORD:
|
||||
required: false
|
||||
PUB_DOCKER_USERNAME:
|
||||
required: false
|
||||
S3_BUCKET:
|
||||
required: false
|
||||
|
||||
# Literal group name: under `workflow_call`, `github.workflow` resolves to the
|
||||
# caller's workflow, which put this workflow and the other reusable one called
|
||||
# by the same caller into a single shared group, and left a manual dispatch of
|
||||
# the same ref in a group of its own, free to race on the same artifacts.
|
||||
concurrency:
|
||||
group: build-docker-${{ inputs.gh_ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
ALL_IMAGES: backend frontend exporter storybook mcp
|
||||
# All runner instances live on the same server, so the bundle is
|
||||
# downloaded from S3 once and shared between build jobs through this
|
||||
# host-local directory. Each build job falls back to S3 if the file is
|
||||
# missing (e.g. if runners ever move to separate machines).
|
||||
BUNDLE_CACHE: /var/tmp/penpot-bundle-cache
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
default: 'develop'
|
||||
|
||||
jobs:
|
||||
# ── 1. Resolve the build key and check the whole set at once ───────────
|
||||
prepare:
|
||||
name: Prepare
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 15
|
||||
outputs:
|
||||
gh_ref: ${{ steps.vars.outputs.gh_ref }}
|
||||
ref_slug: ${{ steps.vars.outputs.ref_slug }}
|
||||
bundle_version: ${{ steps.vars.outputs.bundle_version }}
|
||||
sha: ${{ steps.vars.outputs.sha }}
|
||||
short_sha: ${{ steps.vars.outputs.short_sha }}
|
||||
build_key: ${{ steps.vars.outputs.build_key }}
|
||||
commit_title: ${{ steps.vars.outputs.commit_title }}
|
||||
exists: ${{ steps.check.outputs.exists }}
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }}
|
||||
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref || github.ref_name }}
|
||||
run: |
|
||||
# A git ref accepts characters that a Docker tag does not.
|
||||
REF_SLUG=$(printf '%s' "$GH_REF" | tr -c 'A-Za-z0-9._-' '-' | cut -c1-100)
|
||||
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
|
||||
echo "ref_slug=$REF_SLUG" >> $GITHUB_OUTPUT
|
||||
# Must match the key computed by build-bundle.yml.
|
||||
BUNDLE_VERSION=$(./manage.sh version)
|
||||
SHORT_SHA=$(git rev-parse --short=12 HEAD)
|
||||
VERSION_TAG=$(git describe --tags --match '*.*.*' --abbrev=0)
|
||||
echo "sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
|
||||
echo "short_sha=$SHORT_SHA" >> $GITHUB_OUTPUT
|
||||
echo "commit_title=$(git log -1 --pretty=%s)" >> $GITHUB_OUTPUT
|
||||
echo "bundle_version=$BUNDLE_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "build_key=sha-${SHORT_SHA}-${VERSION_TAG}" >> $GITHUB_OUTPUT
|
||||
|
||||
# The image set is a single block, so a single set-level check is
|
||||
# enough: `promote` drops a marker object in S3 only after every
|
||||
# image was built AND every branch tag was moved. Marker present
|
||||
# means there is nothing at all to do for this commit. `force`
|
||||
# bypasses this check entirely.
|
||||
- name: Check if this image set is already built
|
||||
id: check
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
FORCE: ${{ inputs.force }}
|
||||
BUILD_KEY: ${{ steps.vars.outputs.build_key }}
|
||||
run: |
|
||||
if [ "$FORCE" = "true" ]; then
|
||||
echo "exists=false" >> $GITHUB_OUTPUT
|
||||
mkdir -p "$BUNDLE_CACHE"
|
||||
find "$BUNDLE_CACHE" -type f -mtime +1 -delete || true
|
||||
ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
|
||||
aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
|
||||
mv "$ZIP.$$.tmp" "$ZIP"
|
||||
{
|
||||
echo "### 🔁 Image set build forced"
|
||||
echo ""
|
||||
echo "\`force: true\` — skipping the S3 marker check."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if aws s3api head-object \
|
||||
--bucket "$S3_BUCKET" \
|
||||
--key "markers/images-${BUILD_KEY}" \
|
||||
> /dev/null 2>&1; then
|
||||
echo "exists=true" >> $GITHUB_OUTPUT
|
||||
{
|
||||
echo "### ⏭️ Image set build skipped"
|
||||
echo ""
|
||||
echo "The whole set was already built and promoted for \`${BUILD_KEY}\`."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
else
|
||||
echo "exists=false" >> $GITHUB_OUTPUT
|
||||
|
||||
# Stage the bundle in the host-local cache, once, for all the
|
||||
# build jobs. Download to a temp name and mv for atomicity;
|
||||
# prune stale bundles while at it.
|
||||
mkdir -p "$BUNDLE_CACHE"
|
||||
find "$BUNDLE_CACHE" -type f -mtime +1 -delete || true
|
||||
ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
|
||||
if [ ! -f "$ZIP" ]; then
|
||||
aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
|
||||
mv "$ZIP.$$.tmp" "$ZIP"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── 2. One build per image, in parallel, only when needed ──────────────
|
||||
build:
|
||||
name: Build ${{ matrix.image }}
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 60
|
||||
needs: prepare
|
||||
if: needs.prepare.outputs.exists == 'false'
|
||||
strategy:
|
||||
fail-fast: true
|
||||
# 4 runner slots are available for build jobs on this server; cap the
|
||||
# matrix at 3 so short jobs (prepare and other workflows' checks)
|
||||
# never queue behind long builds.
|
||||
max-parallel: 3
|
||||
matrix:
|
||||
image: [backend, frontend, exporter, storybook, mcp]
|
||||
build-and-push:
|
||||
name: Build and Push Penpot Docker Images
|
||||
runs-on: penpot-runner-02
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
# Each job execution will use its own docker configuration.
|
||||
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}-${{ matrix.image }}" >> $GITHUB_ENV
|
||||
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.sha != '' && inputs.sha || inputs.gh_ref }}
|
||||
fetch-depth: 0
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
run: |
|
||||
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Download Penpot Bundles
|
||||
id: bundles
|
||||
env:
|
||||
FILE_NAME: penpot-${{ steps.vars.outputs.gh_ref }}.zip
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
run: |
|
||||
tmp=$(aws s3api head-object \
|
||||
--bucket ${{ secrets.S3_BUCKET }} \
|
||||
--key "$FILE_NAME" \
|
||||
--query 'Metadata."bundle-version"' \
|
||||
--output text)
|
||||
echo "bundle_version=$tmp" >> $GITHUB_OUTPUT
|
||||
pushd docker/images
|
||||
aws s3 cp s3://${{ secrets.S3_BUCKET }}/$FILE_NAME .
|
||||
unzip $FILE_NAME > /dev/null
|
||||
mv penpot/backend bundle-backend
|
||||
mv penpot/frontend bundle-frontend
|
||||
mv penpot/exporter bundle-exporter
|
||||
mv penpot/storybook bundle-storybook
|
||||
mv penpot/mcp bundle-mcp
|
||||
popd
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Login to Docker Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ${{ secrets.DOCKER_REGISTRY }}
|
||||
username: ${{ secrets.DOCKER_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_PASSWORD }}
|
||||
|
||||
# To avoid the "429 Too Many Requests" error when downloading
|
||||
# To avoid the “429 Too Many Requests” error when downloading
|
||||
# images from DockerHub for unregistered users.
|
||||
# https://docs.docker.com/docker-hub/usage/
|
||||
- name: Login to DockerHub Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
# Images now build FROM Docker Hardened Images (dhi.io). DHI
|
||||
# is free (Apache 2.0, no subscription), but pulling from it
|
||||
# still requires an authenticated login -- a separate `docker
|
||||
# login` against a different registry host, even though it
|
||||
# reuses the same PUB_DOCKER_* credentials as the DockerHub
|
||||
# login above.
|
||||
- name: Login to Docker Hardened Images registry (base image pull)
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: dhi.io
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
# Bundle staged once by `prepare` on this host; the S3 fallback only
|
||||
# triggers if the cache is unavailable (runners on another machine,
|
||||
# cache pruned mid-run, ...).
|
||||
- name: Prepare Penpot bundle
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
IMAGE: ${{ matrix.image }}
|
||||
run: |
|
||||
ZIP="$BUNDLE_CACHE/penpot-${BUILD_KEY}.zip"
|
||||
if [ ! -f "$ZIP" ]; then
|
||||
echo "Bundle not found in host cache; falling back to S3."
|
||||
mkdir -p "$BUNDLE_CACHE"
|
||||
aws s3 cp "s3://${S3_BUCKET}/penpot-${BUILD_KEY}.zip" "$ZIP.$$.tmp"
|
||||
mv "$ZIP.$$.tmp" "$ZIP"
|
||||
fi
|
||||
# Extract only the bundle this job needs.
|
||||
pushd docker/images
|
||||
unzip -q "$ZIP" "penpot/${IMAGE}/*"
|
||||
mv "penpot/${IMAGE}" "bundle-${IMAGE}"
|
||||
popd
|
||||
|
||||
- name: Set up QEMU (stable)
|
||||
uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
|
||||
with:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Extract metadata (tags, labels)
|
||||
id: meta
|
||||
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
|
||||
uses: docker/metadata-action@v6
|
||||
with:
|
||||
images: ${{ matrix.image }}
|
||||
images:
|
||||
frontend
|
||||
backend
|
||||
exporter
|
||||
storybook
|
||||
mcp
|
||||
labels: |
|
||||
bundle_version=${{ needs.prepare.outputs.bundle_version }}
|
||||
bundle_version=${{ steps.bundles.outputs.bundle_version }}
|
||||
|
||||
- name: Build and push Docker image
|
||||
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
||||
- name: Build and push Backend Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
DOCKER_IMAGE: 'backend'
|
||||
BUNDLE_PATH: './bundle-backend'
|
||||
with:
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.${{ matrix.image }}
|
||||
file: ./docker/images/Dockerfile.backend
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
provenance: mode=max
|
||||
sbom: true
|
||||
# Immutable tag only; branch tags are moved atomically for the
|
||||
# whole image set by the `promote` job.
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:${{ needs.prepare.outputs.build_key }}
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache,mode=max
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
# ── 3a. Move the branch tags of ALL images together (fresh build) ──────
|
||||
promote:
|
||||
name: Promote image set
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 10
|
||||
needs: [prepare, build]
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Login to Docker Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
- name: Build and push Frontend Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
DOCKER_IMAGE: 'frontend'
|
||||
BUNDLE_PATH: './bundle-frontend'
|
||||
with:
|
||||
registry: ${{ secrets.DOCKER_REGISTRY }}
|
||||
username: ${{ secrets.DOCKER_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_PASSWORD }}
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.frontend
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
- name: Point branch tags to the new build key
|
||||
- name: Build and push Exporter Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
run: |
|
||||
set -e
|
||||
for image in $ALL_IMAGES; do
|
||||
docker buildx imagetools create \
|
||||
-t "${DOCKER_REGISTRY}/${image}:${REF_SLUG}" \
|
||||
"${DOCKER_REGISTRY}/${image}:${BUILD_KEY}"
|
||||
done
|
||||
|
||||
# The marker is written LAST: its presence certifies that all five
|
||||
# images exist and all branch tags point to this build key.
|
||||
- name: Write set-completed marker
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
run: |
|
||||
echo "$GITHUB_RUN_ID" | aws s3 cp - \
|
||||
"s3://${S3_BUCKET}/markers/images-${BUILD_KEY}"
|
||||
|
||||
- name: Write step summary
|
||||
env:
|
||||
REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
|
||||
SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
BUNDLE_VERSION: ${{ needs.prepare.outputs.bundle_version }}
|
||||
COMMIT_TITLE: ${{ needs.prepare.outputs.commit_title }}
|
||||
run: |
|
||||
{
|
||||
echo "### ✅ Image set promoted"
|
||||
echo ""
|
||||
echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
|
||||
echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
|
||||
echo "- Built at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo ""
|
||||
echo "All \`:${REF_SLUG}\` tags now point to \`${BUILD_KEY}\`."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# ── 3b. Skip path: make sure THIS ref's tags point to the existing
|
||||
# images (another ref may have built and promoted the same commit
|
||||
# first — the sha marker says the images exist, not that this ref's
|
||||
# tags were ever created).
|
||||
retag:
|
||||
name: Ensure branch tags (already-built commit)
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 10
|
||||
needs: prepare
|
||||
if: needs.prepare.outputs.exists == 'true'
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
echo "DOCKER_CONFIG=${{ runner.temp }}/.docker-${{ github.run_id }}-${{ github.job }}" >> $GITHUB_ENV
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Login to Docker Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
DOCKER_IMAGE: 'exporter'
|
||||
BUNDLE_PATH: './bundle-exporter'
|
||||
with:
|
||||
registry: ${{ secrets.DOCKER_REGISTRY }}
|
||||
username: ${{ secrets.DOCKER_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_PASSWORD }}
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.exporter
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
- name: Point branch tags to the existing build key
|
||||
- name: Build and push Storybook Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
run: |
|
||||
set -e
|
||||
for image in $ALL_IMAGES; do
|
||||
docker buildx imagetools create \
|
||||
-t "${DOCKER_REGISTRY}/${image}:${REF_SLUG}" \
|
||||
"${DOCKER_REGISTRY}/${image}:${BUILD_KEY}"
|
||||
done
|
||||
DOCKER_IMAGE: 'storybook'
|
||||
BUNDLE_PATH: './bundle-storybook'
|
||||
with:
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.storybook
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
- name: Write step summary
|
||||
- name: Build and push MCP Docker image
|
||||
uses: docker/build-push-action@v7
|
||||
env:
|
||||
REF_SLUG: ${{ needs.prepare.outputs.ref_slug }}
|
||||
SHORT_SHA: ${{ needs.prepare.outputs.short_sha }}
|
||||
BUILD_KEY: ${{ needs.prepare.outputs.build_key }}
|
||||
BUNDLE_VERSION: ${{ needs.prepare.outputs.bundle_version }}
|
||||
COMMIT_TITLE: ${{ needs.prepare.outputs.commit_title }}
|
||||
run: |
|
||||
{
|
||||
echo "### ✅ Image set already built (branch tags ensured)"
|
||||
echo ""
|
||||
echo "- Version: \`${BUNDLE_VERSION}\` (\`./manage.sh version\`)"
|
||||
echo "- Commit: [\`${SHORT_SHA}\`](${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/commit/${SHORT_SHA}) — ${COMMIT_TITLE}"
|
||||
echo "- Checked at: $(date -u +'%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo ""
|
||||
echo "All \`:${REF_SLUG}\` tags now point to \`${BUILD_KEY}\`."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
DOCKER_IMAGE: 'mcp'
|
||||
BUNDLE_PATH: './bundle-mcp'
|
||||
with:
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.mcp
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:${{ steps.vars.outputs.gh_ref }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache
|
||||
cache-to: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ env.DOCKER_IMAGE }}:buildcache,mode=max
|
||||
|
||||
# ── 4. Single failure notification for the whole workflow ─────────────
|
||||
notify:
|
||||
name: Notify failure
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 5
|
||||
needs: [prepare, build, promote, retag]
|
||||
if: failure()
|
||||
|
||||
steps:
|
||||
- name: Notify Mattermost
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
TEXT: |
|
||||
❌ 🐳 *[PENPOT] Error building/promoting the penpot docker image set.*
|
||||
📄 Triggered from ref: `${{ needs.prepare.outputs.gh_ref || inputs.gh_ref }}`
|
||||
📦 Bundle: `${{ needs.prepare.outputs.bundle_version || 'n/a' }}`
|
||||
❌ 🐳 *[PENPOT] Error building penpot docker images.*
|
||||
📄 Triggered from ref: `${{ steps.vars.outputs.gh_ref }}`
|
||||
📦 Bundle: `${{ steps.bundles.outputs.bundle_version }}`
|
||||
🔗 Run: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
@infra
|
||||
|
||||
260
.github/workflows/build-renovate.yml
vendored
260
.github/workflows/build-renovate.yml
vendored
@ -1,260 +0,0 @@
|
||||
name: Renovate Docker Image Check
|
||||
|
||||
# Triggered by pushes to Renovate branches, not by pull_request: this is a
|
||||
# public repository and these jobs run on the self-hosted runners, so a PR
|
||||
# from a fork must never be able to start them. Only users with write
|
||||
# access can push here.
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'renovate-*'
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: renovate-image-check-${{ github.ref_name }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
BUNDLE_CACHE: /var/tmp/penpot-bundle-cache
|
||||
|
||||
jobs:
|
||||
# ── 1. Pick an existing develop bundle and detect what changed ────────
|
||||
prepare:
|
||||
name: Prepare
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
outputs:
|
||||
bundle_key: ${{ steps.bundle.outputs.bundle_key }}
|
||||
images: ${{ steps.changes.outputs.images }}
|
||||
devenv: ${{ steps.changes.outputs.devenv }}
|
||||
imagemagick: ${{ steps.changes.outputs.imagemagick }}
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Find the develop commit this branch is based on
|
||||
id: base
|
||||
run: |
|
||||
echo "sha=$(git merge-base HEAD origin/develop)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Detect changed images
|
||||
id: changes
|
||||
env:
|
||||
BASE_SHA: ${{ steps.base.outputs.sha }}
|
||||
run: |
|
||||
changed() {
|
||||
git diff --quiet "$BASE_SHA" HEAD -- "$@" && echo false || echo true
|
||||
}
|
||||
{
|
||||
# Anything outside .github can affect the production images.
|
||||
echo "images=$(changed . ':!.github')"
|
||||
echo "devenv=$(changed docker/devenv)"
|
||||
echo "imagemagick=$(changed docker/imagemagick)"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
# Renovate only changes Dockerfiles and package manager versions, so
|
||||
# the most recent develop bundle is enough to build the images. The
|
||||
# version part of the key does not matter here: take any bundle of
|
||||
# the commit.
|
||||
- name: Find the latest develop bundle
|
||||
id: bundle
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
BASE_SHA: ${{ steps.base.outputs.sha }}
|
||||
run: |
|
||||
for SHA in $(git rev-list --first-parent --max-count=50 "$BASE_SHA"); do
|
||||
SHORT_SHA=$(git rev-parse --short=12 "$SHA")
|
||||
BUNDLE_KEY=$(aws s3api list-objects-v2 --bucket "$S3_BUCKET" \
|
||||
--prefix "penpot-sha-${SHORT_SHA}-" \
|
||||
--query 'Contents[0].Key' --output text)
|
||||
if [ -n "$BUNDLE_KEY" ] && [ "$BUNDLE_KEY" != "None" ]; then
|
||||
echo "bundle_key=$BUNDLE_KEY" >> "$GITHUB_OUTPUT"
|
||||
echo "Using bundle \`${BUNDLE_KEY}\`" >> "$GITHUB_STEP_SUMMARY"
|
||||
exit 0
|
||||
fi
|
||||
done
|
||||
echo "::error::No bundle found for the last 50 develop commits"
|
||||
exit 1
|
||||
|
||||
# ── 2. Production images, built from the PR Dockerfiles ───────────────
|
||||
images:
|
||||
name: Build ${{ matrix.image }}
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 60
|
||||
needs: prepare
|
||||
if: needs.prepare.outputs.images == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 3
|
||||
matrix:
|
||||
image: [backend, frontend, exporter, storybook, mcp]
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
env:
|
||||
MATRIX_IMAGE: ${{ matrix.image }}
|
||||
run: |
|
||||
echo "DOCKER_CONFIG=${RUNNER_TEMP}/.docker-${GITHUB_RUN_ID}-${GITHUB_JOB}-${MATRIX_IMAGE}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Login to Docker Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: ${{ secrets.DOCKER_REGISTRY }}
|
||||
username: ${{ secrets.DOCKER_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_PASSWORD }}
|
||||
|
||||
- name: Login to DockerHub Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Login to Docker Hardened Images registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: dhi.io
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Prepare Penpot bundle
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
BUNDLE_KEY: ${{ needs.prepare.outputs.bundle_key }}
|
||||
IMAGE: ${{ matrix.image }}
|
||||
run: |
|
||||
ZIP="$BUNDLE_CACHE/${BUNDLE_KEY}"
|
||||
if [ ! -f "$ZIP" ]; then
|
||||
mkdir -p "$BUNDLE_CACHE"
|
||||
aws s3 cp "s3://${S3_BUCKET}/${BUNDLE_KEY}" "$ZIP.$$.tmp"
|
||||
mv "$ZIP.$$.tmp" "$ZIP"
|
||||
fi
|
||||
pushd docker/images
|
||||
unzip -q "$ZIP" "penpot/${IMAGE}/*"
|
||||
mv "penpot/${IMAGE}" "bundle-${IMAGE}"
|
||||
popd
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
|
||||
with:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Build Docker image
|
||||
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
||||
with:
|
||||
context: ./docker/images/
|
||||
file: ./docker/images/Dockerfile.${{ matrix.image }}
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: false
|
||||
cache-from: type=registry,ref=${{ secrets.DOCKER_REGISTRY }}/${{ matrix.image }}:buildcache
|
||||
|
||||
# ── 3. Devenv, only when the PR touches it ────────────────────────────
|
||||
devenv:
|
||||
name: Build devenv
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 90
|
||||
needs: prepare
|
||||
if: needs.prepare.outputs.devenv == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
echo "DOCKER_CONFIG=${RUNNER_TEMP}/.docker-${GITHUB_RUN_ID}-${GITHUB_JOB}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Login to DockerHub Registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Login to Docker Hardened Images registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: dhi.io
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
|
||||
with:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Build devenv image
|
||||
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
||||
with:
|
||||
context: ./docker/devenv/
|
||||
file: ./docker/devenv/Dockerfile
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: false
|
||||
cache-from: type=registry,ref=penpotapp/devenv:buildcache
|
||||
|
||||
# ── 4. ImageMagick, only when the PR touches it ───────────────────────
|
||||
# amd64 only: compiling ImageMagick under QEMU takes too long for a check.
|
||||
imagemagick:
|
||||
name: Build imagemagick
|
||||
runs-on: penpot-standar-runner
|
||||
timeout-minutes: 60
|
||||
needs: prepare
|
||||
if: needs.prepare.outputs.imagemagick == 'true'
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Set common environment variables
|
||||
run: |
|
||||
echo "DOCKER_CONFIG=${RUNNER_TEMP}/.docker-${GITHUB_RUN_ID}-${GITHUB_JOB}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Login to Docker Hardened Images registry
|
||||
uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4.8.0
|
||||
with:
|
||||
registry: dhi.io
|
||||
username: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
password: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
|
||||
|
||||
- name: Build imagemagick image
|
||||
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
|
||||
with:
|
||||
context: ./docker/imagemagick/
|
||||
file: ./docker/imagemagick/Dockerfile
|
||||
platforms: linux/amd64
|
||||
push: false
|
||||
53
.github/workflows/build-staging.yml
vendored
53
.github/workflows/build-staging.yml
vendored
@ -1,67 +1,22 @@
|
||||
name: _STAGING
|
||||
|
||||
run-name: >-
|
||||
_STAGING (staging)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built/promoted'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
schedule:
|
||||
- cron: '36 5-20 * * 1-5'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-bundle:
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-bundle.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "staging"
|
||||
force: ${{ inputs.force || false }}
|
||||
build_wasm: "yes"
|
||||
build_storybook: "yes"
|
||||
|
||||
build-docker:
|
||||
needs: build-bundle
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-docker.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "staging"
|
||||
# Pin build-docker to the commit build-bundle actually bundled.
|
||||
sha: ${{ needs.build-bundle.outputs.sha }}
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
build-docker-admin-console:
|
||||
permissions: {}
|
||||
uses: ./.github/workflows/build-docker-admin-console.yml
|
||||
secrets:
|
||||
ORG_WORKFLOW_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
|
||||
with:
|
||||
gh_ref: "staging"
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
78
.github/workflows/build-tag.yml
vendored
78
.github/workflows/build-tag.yml
vendored
@ -1,82 +1,35 @@
|
||||
name: _TAG
|
||||
|
||||
run-name: >-
|
||||
_TAG (${{ github.ref_name }} @ ${{ github.sha }})
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
force:
|
||||
description: 'Rebuild and overwrite even if already built/promoted (manual re-releases only)'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
push:
|
||||
tags:
|
||||
- '*'
|
||||
|
||||
# Keyed by ref and never cancelling: pushing 2.17.2 shortly after 2.17.2-RC1
|
||||
# must not abort the release already in flight.
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref_name }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-bundle:
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-bundle.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: ${{ github.ref_name }}
|
||||
force: ${{ inputs.force || false }}
|
||||
build_wasm: "yes"
|
||||
build_storybook: "yes"
|
||||
|
||||
build-docker:
|
||||
needs: build-bundle
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-docker.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: ${{ github.ref_name }}
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
build-docker-admin-console:
|
||||
permissions: {}
|
||||
uses: ./.github/workflows/build-docker-admin-console.yml
|
||||
secrets:
|
||||
ORG_WORKFLOW_TOKEN: ${{ secrets.ORG_WORKFLOW_TOKEN }}
|
||||
with:
|
||||
gh_ref: ${{ github.ref_name }}
|
||||
force: ${{ inputs.force || false }}
|
||||
|
||||
notify:
|
||||
name: Notifications
|
||||
permissions: {}
|
||||
runs-on: ubuntu-latest
|
||||
needs:
|
||||
- build-docker
|
||||
- build-docker-admin-console
|
||||
runs-on: ubuntu-24.04
|
||||
needs: build-docker
|
||||
|
||||
steps:
|
||||
- name: Notify Mattermost
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
@ -87,19 +40,8 @@ jobs:
|
||||
|
||||
publish-final-tag:
|
||||
if: ${{ !contains(github.ref_name, '-RC') && !contains(github.ref_name, '-alpha') && !contains(github.ref_name, '-beta') && contains(github.ref_name, '.') }}
|
||||
needs:
|
||||
- build-docker
|
||||
- build-docker-admin-console
|
||||
permissions:
|
||||
contents: write
|
||||
needs: build-docker
|
||||
uses: ./.github/workflows/release.yml
|
||||
secrets:
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: ${{ github.ref_name }}
|
||||
|
||||
45
.github/workflows/build-tmp-tokens.yml
vendored
45
.github/workflows/build-tmp-tokens.yml
vendored
@ -1,45 +0,0 @@
|
||||
name: _TMP TOKENS
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
# schedule:
|
||||
# - cron: '46 5-20 * * 1-5'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-bundle:
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-bundle.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
with:
|
||||
gh_ref: "hiru-tokens-in-libs"
|
||||
|
||||
build-docker:
|
||||
needs: build-bundle
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/build-docker.yml
|
||||
secrets:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_REGION: ${{ secrets.AWS_REGION }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
|
||||
DOCKER_REGISTRY: ${{ secrets.DOCKER_REGISTRY }}
|
||||
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
PUB_DOCKER_PASSWORD: ${{ secrets.PUB_DOCKER_PASSWORD }}
|
||||
PUB_DOCKER_USERNAME: ${{ secrets.PUB_DOCKER_USERNAME }}
|
||||
S3_BUCKET: ${{ secrets.S3_BUCKET }}
|
||||
with:
|
||||
gh_ref: "hiru-tokens-in-libs"
|
||||
55
.github/workflows/commit-checker.yml
vendored
55
.github/workflows/commit-checker.yml
vendored
@ -7,16 +7,19 @@ on:
|
||||
- reopened
|
||||
- synchronize
|
||||
- ready_for_review
|
||||
pull_request_target:
|
||||
types:
|
||||
- opened
|
||||
- edited
|
||||
- reopened
|
||||
- synchronize
|
||||
- ready_for_review
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- develop
|
||||
- staging
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
check-commit-message:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
@ -24,7 +27,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check Commit Type
|
||||
uses: gsactions/commit-message-checker@16fa2d5de096ae0d35626443bcd24f1e756cafee # v2.0.0
|
||||
uses: gsactions/commit-message-checker@v2
|
||||
with:
|
||||
pattern: '^(((:(lipstick|globe_with_meridians|wrench|books|arrow_up|arrow_down|zap|ambulance|construction|boom|fire|whale|bug|sparkles|paperclip|tada|recycle|rewind|construction_worker):)\s[A-Z].*[^.])|(Merge|Revert|Reapply).+[^.])$'
|
||||
flags: 'gm'
|
||||
@ -47,46 +50,4 @@ jobs:
|
||||
# pattern: '^.+(Resolves|Fixes): \#[0-9]+$'
|
||||
# error: 'You need at least one "Resolves|Fixes: #<issue number>" line.'
|
||||
|
||||
check-commit-guidelines:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: Check Commit Guidelines
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# On PR events check out the base ref, so the script below is
|
||||
# always the trusted version: PR commits are only read, never
|
||||
# executed. On push events this resolves to the pushed sha.
|
||||
- name: Checkout trusted code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ github.event.pull_request.base.sha || github.sha }}
|
||||
|
||||
- name: Fetch PR commits
|
||||
if: ${{ github.event_name == 'pull_request' || github.event_name == 'pull_request_target' }}
|
||||
run: git fetch --no-tags origin "pull/${{ github.event.pull_request.number }}/head:pr-head"
|
||||
|
||||
- name: Run scripts/check-commit on every new commit
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ "${{ github.event_name }}" = "push" ]; then
|
||||
if [ "${{ github.event.before }}" = "0000000000000000000000000000000000000000" ]; then
|
||||
RANGE="${{ github.sha }}"
|
||||
else
|
||||
RANGE="${{ github.event.before }}..${{ github.sha }}"
|
||||
fi
|
||||
else
|
||||
RANGE="${{ github.event.pull_request.base.sha }}..pr-head"
|
||||
fi
|
||||
COMMITS="$(git rev-list --reverse "$RANGE")"
|
||||
if [ -z "$COMMITS" ]; then
|
||||
echo "No new commits to check."
|
||||
exit 0
|
||||
fi
|
||||
FAILED=0
|
||||
for COMMIT in $COMMITS; do
|
||||
./scripts/check-commit -c "$COMMIT" || FAILED=1
|
||||
done
|
||||
exit "$FAILED"
|
||||
|
||||
|
||||
|
||||
78
.github/workflows/lint-workflows.yml
vendored
78
.github/workflows/lint-workflows.yml
vendored
@ -1,78 +0,0 @@
|
||||
name: "CI: Workflows"
|
||||
|
||||
# Static analysis of the GitHub Actions workflows themselves:
|
||||
# - actionlint: syntax, expressions, runner labels, job/step references,
|
||||
# and shellcheck on every `run:` block.
|
||||
# - zizmor: security audits (template injection, dangerous triggers,
|
||||
# credential persistence, ...). Configured in .github/zizmor.yml.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize, reopened, ready_for_review]
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
- '.github/actions/**'
|
||||
- '.github/actionlint.yaml'
|
||||
- '.github/zizmor.yml'
|
||||
push:
|
||||
branches:
|
||||
- develop
|
||||
- staging
|
||||
- main
|
||||
paths:
|
||||
- '.github/workflows/**'
|
||||
- '.github/actions/**'
|
||||
- '.github/actionlint.yaml'
|
||||
- '.github/zizmor.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: lint-workflows-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions: {}
|
||||
|
||||
env:
|
||||
ACTIONLINT_VERSION: '1.7.12'
|
||||
ZIZMOR_VERSION: '1.32.0'
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
name: Lint workflows
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
# PR code is untrusted: never run this on the self-hosted runners.
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install actionlint
|
||||
run: |
|
||||
curl -fsSL --retry 3 \
|
||||
"https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" \
|
||||
| tar -xz -C "$RUNNER_TEMP" actionlint
|
||||
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"
|
||||
|
||||
# Each finding is emitted as a `::error` annotation on the PR diff.
|
||||
- name: Run actionlint
|
||||
run: |
|
||||
# shellcheck disable=SC2016 # Go template, not shell expansion.
|
||||
actionlint -format '{{range $err := .}}::error file={{$err.Filepath}},line={{$err.Line}},col={{$err.Column}},title=actionlint ({{$err.Kind}})::{{$err.Message}}\n{{end}}'
|
||||
|
||||
- name: Run zizmor
|
||||
if: ${{ !cancelled() }}
|
||||
env:
|
||||
# Enables the online audits (known-vulnerable-actions,
|
||||
# impostor-commit, ref-confusion, ...).
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
pipx run "zizmor==${ZIZMOR_VERSION}" \
|
||||
--config .github/zizmor.yml \
|
||||
--format github \
|
||||
.github/workflows
|
||||
51
.github/workflows/plugins-deploy-api-doc.yml
vendored
51
.github/workflows/plugins-deploy-api-doc.yml
vendored
@ -33,32 +33,48 @@ jobs:
|
||||
steps:
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref || github.ref_name }}
|
||||
run: |
|
||||
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
|
||||
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ steps.vars.outputs.gh_ref }}
|
||||
|
||||
# No dependency cache here: a deploy must not restore a cache that
|
||||
# other workflows can write to.
|
||||
- name: Setup pnpm + Node.js
|
||||
uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
|
||||
# START: Setup Node and PNPM enabling cache
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
working-directory: plugins
|
||||
runtime: node@24.22.3
|
||||
install: false
|
||||
node-version-file: .nvmrc
|
||||
|
||||
- name: Enable PNPM
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
|
||||
- name: Get pnpm store path
|
||||
id: pnpm-store
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Cache pnpm store
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-
|
||||
# END: Setup Node and PNPM enabling cache
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
pnpm install --frozen-lockfile;
|
||||
pnpm install --no-frozen-lockfile;
|
||||
pnpm add -D -w wrangler@latest;
|
||||
|
||||
- name: Build docs
|
||||
@ -67,9 +83,8 @@ jobs:
|
||||
run: pnpm run build:doc
|
||||
|
||||
- name: Select Worker name
|
||||
env:
|
||||
REF: ${{ steps.vars.outputs.gh_ref }}
|
||||
run: |
|
||||
REF="${{ steps.vars.outputs.gh_ref }}"
|
||||
case "$REF" in
|
||||
main)
|
||||
echo "WORKER_NAME=penpot-plugins-api-doc-pro" >> $GITHUB_ENV
|
||||
@ -87,7 +102,7 @@ jobs:
|
||||
working-directory: plugins
|
||||
shell: bash
|
||||
run: |
|
||||
sed -i "s/WORKER_URI/${WORKER_URI}/g" wrangler-penpot-plugins-api-doc.toml
|
||||
sed -i "s/WORKER_URI/${{ env.WORKER_URI }}/g" wrangler-penpot-plugins-api-doc.toml
|
||||
|
||||
- name: Add noindex header and robots.txt files for non-production environments
|
||||
if: ${{ steps.vars.outputs.gh_ref != 'main' }}
|
||||
@ -107,7 +122,7 @@ jobs:
|
||||
EOF
|
||||
|
||||
- name: Deploy to Cloudflare Workers
|
||||
uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
|
||||
uses: cloudflare/wrangler-action@v3
|
||||
with:
|
||||
workingDirectory: plugins
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
@ -116,7 +131,7 @@ jobs:
|
||||
|
||||
- name: Notify Mattermost
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
|
||||
90
.github/workflows/plugins-deploy-package.yml
vendored
90
.github/workflows/plugins-deploy-package.yml
vendored
@ -23,89 +23,89 @@ on:
|
||||
description: 'Name of the branch'
|
||||
type: string
|
||||
required: true
|
||||
default: 'develop'
|
||||
plugin_name:
|
||||
description: 'Publig name (from plugins/apps/<plugin_name>-plugin)'
|
||||
type: string
|
||||
required: true
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID:
|
||||
required: false
|
||||
CLOUDFLARE_API_TOKEN:
|
||||
required: false
|
||||
MATTERMOST_WEBHOOK:
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: penpot-standar-runner
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
# Persistent pnpm store shared by all runner instances on the host.
|
||||
# pnpm's content-addressable store is safe for concurrent access.
|
||||
- /var/cache/github-runner/pnpm-store:/root/.local/share/pnpm/store
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref }}
|
||||
PLUGIN_NAME: ${{ inputs.plugin_name }}
|
||||
runs-on: penpot-runner-01
|
||||
steps:
|
||||
# plugin_name is a free-form string that ends up in paths, worker names
|
||||
# and GITHUB_ENV; reject anything that is not a plain slug before it is
|
||||
# used anywhere.
|
||||
- name: Validate inputs
|
||||
run: |
|
||||
if ! [[ "$PLUGIN_NAME" =~ ^[a-z0-9][a-z0-9-]*$ ]]; then
|
||||
echo "::error::Invalid plugin_name: must match ^[a-z0-9][a-z0-9-]*$"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
# START: Setup Node and PNPM enabling cache
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
|
||||
- name: Enable PNPM
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
|
||||
- name: Get pnpm store path
|
||||
id: pnpm-store
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Cache pnpm store
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-
|
||||
# END: Setup Node and PNPM enabling cache
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
pnpm install --frozen-lockfile;
|
||||
pnpm install --no-frozen-lockfile;
|
||||
pnpm add -D -w wrangler@latest;
|
||||
|
||||
- name: "Build package for ${{ inputs.plugin_name }}-plugin"
|
||||
working-directory: plugins
|
||||
shell: bash
|
||||
run: pnpm --filter "${PLUGIN_NAME}-plugin" build
|
||||
run: pnpm --filter ${{ inputs.plugin_name }}-plugin build
|
||||
|
||||
- name: Select Worker name
|
||||
run: |
|
||||
case "$GH_REF" in
|
||||
REF="${{ inputs.gh_ref }}"
|
||||
case "$REF" in
|
||||
main)
|
||||
echo "WORKER_NAME=${PLUGIN_NAME}-plugin-pro" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${PLUGIN_NAME}.plugins.penpot.app" >> $GITHUB_ENV ;;
|
||||
echo "WORKER_NAME=${{ inputs.plugin_name }}-plugin-pro" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${{ inputs.plugin_name }}.plugins.penpot.app" >> $GITHUB_ENV ;;
|
||||
staging)
|
||||
echo "WORKER_NAME=${PLUGIN_NAME}-plugin-pre" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${PLUGIN_NAME}.plugins.penpot.dev" >> $GITHUB_ENV ;;
|
||||
echo "WORKER_NAME=${{ inputs.plugin_name }}-plugin-pre" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${{ inputs.plugin_name }}.plugins.penpot.dev" >> $GITHUB_ENV ;;
|
||||
develop)
|
||||
echo "WORKER_NAME=${PLUGIN_NAME}-plugin-hourly" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${PLUGIN_NAME}.plugins.hourly.penpot.dev" >> $GITHUB_ENV ;;
|
||||
*) echo "Unsupported branch ${GH_REF}" && exit 1 ;;
|
||||
echo "WORKER_NAME=${{ inputs.plugin_name }}-plugin-hourly" >> $GITHUB_ENV
|
||||
echo "WORKER_URI=${{ inputs.plugin_name }}.plugins.hourly.penpot.dev" >> $GITHUB_ENV ;;
|
||||
*) echo "Unsupported branch ${REF}" && exit 1 ;;
|
||||
esac
|
||||
|
||||
- name: Set the custom url
|
||||
working-directory: plugins
|
||||
shell: bash
|
||||
run: |
|
||||
sed -i "s/WORKER_URI/${WORKER_URI}/g" "apps/${PLUGIN_NAME}-plugin/wrangler.toml"
|
||||
sed -i "s/WORKER_URI/${{ env.WORKER_URI }}/g" apps/${{ inputs.plugin_name }}-plugin/wrangler.toml
|
||||
|
||||
- name: Deploy to Cloudflare Workers
|
||||
uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
|
||||
uses: cloudflare/wrangler-action@v3
|
||||
with:
|
||||
workingDirectory: plugins
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
@ -114,7 +114,7 @@ jobs:
|
||||
|
||||
- name: Notify Mattermost
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
|
||||
44
.github/workflows/plugins-deploy-packages.yml
vendored
44
.github/workflows/plugins-deploy-packages.yml
vendored
@ -21,9 +21,6 @@ on:
|
||||
- staging
|
||||
- main
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
detect-changes:
|
||||
runs-on: ubuntu-latest
|
||||
@ -39,11 +36,9 @@ jobs:
|
||||
# [For new plugins]
|
||||
# Add more outputs here
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/checkout@v6
|
||||
- id: filter
|
||||
uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
|
||||
uses: dorny/paths-filter@v4
|
||||
with:
|
||||
filters: |
|
||||
colors_to_tokens:
|
||||
@ -77,10 +72,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.colors_to_tokens == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: colors-to-tokens
|
||||
@ -89,10 +81,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.contrast == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: contrast
|
||||
@ -101,10 +90,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.create_palette == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: create-palette
|
||||
@ -113,10 +99,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.icons == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: icons
|
||||
@ -125,10 +108,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.lorem_ipsum == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: lorem-ipsum
|
||||
@ -137,10 +117,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.rename_layers == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: rename-layers
|
||||
@ -149,10 +126,7 @@ jobs:
|
||||
needs: detect-changes
|
||||
if: github.event_name == 'workflow_dispatch' || needs.detect-changes.outputs.table == 'true'
|
||||
uses: ./.github/workflows/plugins-deploy-package.yml
|
||||
secrets:
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
MATTERMOST_WEBHOOK: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
secrets: inherit
|
||||
with:
|
||||
gh_ref: "${{ inputs.gh_ref || github.ref_name }}"
|
||||
plugin_name: table
|
||||
|
||||
51
.github/workflows/plugins-deploy-styles-doc.yml
vendored
51
.github/workflows/plugins-deploy-styles-doc.yml
vendored
@ -31,32 +31,48 @@ jobs:
|
||||
steps:
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref || github.ref_name }}
|
||||
run: |
|
||||
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
|
||||
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ steps.vars.outputs.gh_ref }}
|
||||
|
||||
# No dependency cache here: a deploy must not restore a cache that
|
||||
# other workflows can write to.
|
||||
- name: Setup pnpm + Node.js
|
||||
uses: pnpm/setup@731dfb6463bbd11c11fd0ee11fe00ed0e36d48a1 # v3.2.0
|
||||
# START: Setup Node and PNPM enabling cache
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
working-directory: plugins
|
||||
runtime: node@24.22.3
|
||||
install: false
|
||||
node-version-file: .nvmrc
|
||||
|
||||
- name: Enable PNPM
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
|
||||
- name: Get pnpm store path
|
||||
id: pnpm-store
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Cache pnpm store
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ${{ steps.pnpm-store.outputs.STORE_PATH }}
|
||||
key: ${{ runner.os }}-pnpm-${{ hashFiles('plugins/pnpm-lock.yaml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pnpm-
|
||||
# END: Setup Node and PNPM enabling cache
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
pnpm install --frozen-lockfile;
|
||||
pnpm install --no-frozen-lockfile;
|
||||
pnpm add -D -w wrangler@latest;
|
||||
|
||||
- name: Build styles
|
||||
@ -65,9 +81,8 @@ jobs:
|
||||
run: pnpm run build:styles-example
|
||||
|
||||
- name: Select Worker name
|
||||
env:
|
||||
REF: ${{ steps.vars.outputs.gh_ref }}
|
||||
run: |
|
||||
REF="${{ steps.vars.outputs.gh_ref }}"
|
||||
case "$REF" in
|
||||
main)
|
||||
echo "WORKER_NAME=penpot-plugins-styles-doc-pro" >> $GITHUB_ENV
|
||||
@ -85,7 +100,7 @@ jobs:
|
||||
working-directory: plugins
|
||||
shell: bash
|
||||
run: |
|
||||
sed -i "s/WORKER_URI/${WORKER_URI}/g" wrangler-penpot-plugins-styles-doc.toml
|
||||
sed -i "s/WORKER_URI/${{ env.WORKER_URI }}/g" wrangler-penpot-plugins-styles-doc.toml
|
||||
|
||||
- name: Add noindex header and robots.txt files for non-production environments
|
||||
if: ${{ steps.vars.outputs.gh_ref != 'main' }}
|
||||
@ -105,7 +120,7 @@ jobs:
|
||||
EOF
|
||||
|
||||
- name: Deploy to Cloudflare Workers
|
||||
uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
|
||||
uses: cloudflare/wrangler-action@v3
|
||||
with:
|
||||
workingDirectory: plugins
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
@ -114,7 +129,7 @@ jobs:
|
||||
|
||||
- name: Notify Mattermost
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
|
||||
47
.github/workflows/release.yml
vendored
47
.github/workflows/release.yml
vendored
@ -13,49 +13,31 @@ on:
|
||||
description: 'Tag to release'
|
||||
type: string
|
||||
required: true
|
||||
secrets:
|
||||
AWS_REGION:
|
||||
required: false
|
||||
DOCKER_PASSWORD:
|
||||
required: false
|
||||
DOCKER_REGISTRY:
|
||||
required: false
|
||||
DOCKER_USERNAME:
|
||||
required: false
|
||||
MATTERMOST_WEBHOOK:
|
||||
required: false
|
||||
PUB_DOCKER_PASSWORD:
|
||||
required: false
|
||||
PUB_DOCKER_USERNAME:
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
release:
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: ubuntu-24.04
|
||||
outputs:
|
||||
version: ${{ steps.vars.outputs.gh_ref }}
|
||||
release_notes: ${{ steps.extract_release_notes.outputs.release_notes }}
|
||||
steps:
|
||||
- name: Extract some useful variables
|
||||
id: vars
|
||||
env:
|
||||
GH_REF: ${{ inputs.gh_ref || github.ref_name }}
|
||||
run: |
|
||||
echo "gh_ref=$GH_REF" >> $GITHUB_OUTPUT
|
||||
echo "gh_ref=${{ inputs.gh_ref || github.ref_name }}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
ref: ${{ steps.vars.outputs.gh_ref }}
|
||||
|
||||
# --- Publicly release the docker images ---
|
||||
- name: Configure ECR credentials
|
||||
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6.3.0
|
||||
uses: aws-actions/configure-aws-credentials@v4
|
||||
with:
|
||||
aws-access-key-id: ${{ secrets.DOCKER_USERNAME }}
|
||||
aws-secret-access-key: ${{ secrets.DOCKER_PASSWORD }}
|
||||
@ -80,7 +62,7 @@ jobs:
|
||||
|
||||
echo "$PUB_DOCKER_PASSWORD" | skopeo login --username "$PUB_DOCKER_USERNAME" --password-stdin docker.io
|
||||
|
||||
IMAGES=("frontend" "backend" "exporter" "admin-console" "mcp" "storybook")
|
||||
IMAGES=("frontend" "backend" "exporter" "mcp" "storybook")
|
||||
SHORT_TAG=${TAG%.*}
|
||||
|
||||
for image in "${IMAGES[@]}"; do
|
||||
@ -110,23 +92,18 @@ jobs:
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
# --- Create GitHub release ---
|
||||
# Updates the release if it already exists, so re-running is safe.
|
||||
- name: Create GitHub release
|
||||
uses: softprops/action-gh-release@v2
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
TAG: ${{ steps.vars.outputs.gh_ref }}
|
||||
NOTES: ${{ steps.extract_release_notes.outputs.release_notes }}
|
||||
run: |
|
||||
if gh release view "$TAG" > /dev/null 2>&1; then
|
||||
gh release edit "$TAG" --title "$TAG" --notes "$NOTES"
|
||||
else
|
||||
gh release create "$TAG" --verify-tag --title "$TAG" --notes "$NOTES"
|
||||
fi
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
with:
|
||||
tag_name: ${{ steps.vars.outputs.gh_ref }}
|
||||
name: ${{ steps.vars.outputs.gh_ref }}
|
||||
body: ${{ steps.extract_release_notes.outputs.release_notes }}
|
||||
|
||||
- name: Notify Mattermost
|
||||
if: failure()
|
||||
uses: mattermost/action-mattermost-notify@ae31bb6f9e26a54336e79696f108a2c91cf55b4e # v2.1.0
|
||||
uses: mattermost/action-mattermost-notify@master
|
||||
with:
|
||||
MATTERMOST_WEBHOOK_URL: ${{ secrets.MATTERMOST_WEBHOOK }}
|
||||
MATTERMOST_CHANNEL: bot-alerts-cicd
|
||||
|
||||
90
.github/workflows/require-linked-issue.yml
vendored
90
.github/workflows/require-linked-issue.yml
vendored
@ -1,90 +0,0 @@
|
||||
name: Require Linked Issue
|
||||
|
||||
# A pull request must have at least one closing-issue reference, unless it
|
||||
# carries the `no issue required` label.
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- reopened
|
||||
- synchronize
|
||||
- edited
|
||||
- labeled
|
||||
- unlabeled
|
||||
- ready_for_review
|
||||
- converted_to_draft
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
|
||||
jobs:
|
||||
check-linked-issue:
|
||||
name: Require Linked Issue
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check linked issue
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
OWNER: ${{ github.repository_owner }}
|
||||
REPO: ${{ github.event.repository.name }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
|
||||
result=$(gh api graphql \
|
||||
-F owner="$OWNER" \
|
||||
-F repo="$REPO" \
|
||||
-F number="$PR_NUMBER" \
|
||||
-f query='
|
||||
query($owner: String!, $repo: String!, $number: Int!) {
|
||||
repository(owner: $owner, name: $repo) {
|
||||
pullRequest(number: $number) {
|
||||
isDraft
|
||||
labels(first: 100) {
|
||||
nodes {
|
||||
name
|
||||
}
|
||||
}
|
||||
closingIssuesReferences(first: 100) {
|
||||
nodes {
|
||||
number
|
||||
title
|
||||
url
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
')
|
||||
|
||||
pull_request=$(echo "$result" | jq '.data.repository.pullRequest')
|
||||
|
||||
if [ "$(echo "$pull_request" | jq -r '.isDraft')" = "true" ]; then
|
||||
echo "Draft PR: linked issue check skipped."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
has_exemption=$(echo "$pull_request" | jq -r '
|
||||
.labels.nodes | map(.name) | index("no issue required") != null
|
||||
')
|
||||
if [ "$has_exemption" = "true" ]; then
|
||||
echo 'Label "no issue required" found: linked issue check skipped.'
|
||||
exit 0
|
||||
fi
|
||||
|
||||
issues=$(echo "$pull_request" | jq -r '
|
||||
.closingIssuesReferences.nodes[]
|
||||
| "- #\(.number): \(.title) (\(.url))"
|
||||
')
|
||||
|
||||
if [ -z "$issues" ]; then
|
||||
echo '::error::This pull request must have a linked issue, or the "no issue required" label.'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Linked issue(s):"
|
||||
echo "$issues"
|
||||
@ -1,76 +0,0 @@
|
||||
name: Require Milestone on Linked Issues
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- reopened
|
||||
- synchronize
|
||||
- edited
|
||||
- ready_for_review
|
||||
- converted_to_draft
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
|
||||
jobs:
|
||||
check-linked-issues:
|
||||
name: Require Milestones on Linked Issues
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Check linked issues
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
IS_DRAFT: ${{ github.event.pull_request.draft }}
|
||||
REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
if [ "$IS_DRAFT" = "true" ]; then
|
||||
echo "Draft PR: milestone check skipped."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
OWNER="${REPOSITORY%%/*}"
|
||||
REPO="${REPOSITORY#*/}"
|
||||
|
||||
# shellcheck disable=SC2016 # GraphQL variables, not shell expansion.
|
||||
ISSUES=$(gh api graphql \
|
||||
-F owner="$OWNER" \
|
||||
-F repo="$REPO" \
|
||||
-F number="$PR_NUMBER" \
|
||||
-f query='
|
||||
query($owner: String!, $repo: String!, $number: Int!) {
|
||||
repository(owner: $owner, name: $repo) {
|
||||
pullRequest(number: $number) {
|
||||
closingIssuesReferences(first: 100) {
|
||||
nodes {
|
||||
number
|
||||
title
|
||||
url
|
||||
milestone {
|
||||
title
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
')
|
||||
|
||||
MISSING=$(echo "$ISSUES" | jq -r '
|
||||
.data.repository.pullRequest.closingIssuesReferences.nodes[]
|
||||
| select(.milestone == null)
|
||||
| "- #\(.number): \(.title) (\(.url))"
|
||||
')
|
||||
|
||||
if [ -n "$MISSING" ]; then
|
||||
echo "::error::The following linked issues have no milestone:"
|
||||
echo "$MISSING"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "All linked issues have a milestone."
|
||||
45
.github/workflows/require-milestone.yml
vendored
45
.github/workflows/require-milestone.yml
vendored
@ -1,45 +0,0 @@
|
||||
name: Require Milestone
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- reopened
|
||||
- synchronize
|
||||
- edited
|
||||
- milestoned
|
||||
- demilestoned
|
||||
- ready_for_review
|
||||
- converted_to_draft
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
check-milestone:
|
||||
name: Require Milestone
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Check milestone
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: |
|
||||
pr_api="repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}"
|
||||
|
||||
is_draft=$(gh api "$pr_api" --jq '.draft')
|
||||
milestone=$(gh api "$pr_api" --jq '.milestone.title // ""')
|
||||
|
||||
if [ "$is_draft" = "true" ]; then
|
||||
echo "Draft PR: milestone check skipped."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ -z "$milestone" ]; then
|
||||
echo "::error::This pull request must have a milestone."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Milestone assigned: $milestone"
|
||||
64
.github/workflows/tests-backend.yml
vendored
64
.github/workflows/tests-backend.yml
vendored
@ -28,17 +28,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-backend:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # coverage comment on the PR
|
||||
name: "Backend Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -66,27 +60,20 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Fmt
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
cljfmt check --parallel=true src/ test/
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
clj-kondo --parallel --lint ../common/src/ src/
|
||||
corepack enable;
|
||||
corepack install;
|
||||
pnpm install;
|
||||
pnpm run check-fmt
|
||||
pnpm run lint
|
||||
|
||||
- name: Tests
|
||||
working-directory: ./backend
|
||||
env:
|
||||
# `clojure.core/assert` is elided at compile time, so without this
|
||||
# flag the precondition asserts do not exist in the compiled code
|
||||
# and the suite silently stops checking them.
|
||||
PENPOT_FLAGS: "enable-backend-asserts"
|
||||
PENPOT_TEST_DATABASE_URI: "postgresql://postgres/penpot_test"
|
||||
PENPOT_TEST_DATABASE_USERNAME: penpot_test
|
||||
PENPOT_TEST_DATABASE_PASSWORD: penpot_test
|
||||
@ -94,39 +81,4 @@ jobs:
|
||||
|
||||
run: |
|
||||
mkdir -p /tmp/penpot;
|
||||
clojure -M:dev:test:coverage
|
||||
|
||||
- name: Coverage summary
|
||||
if: ${{ !cancelled() && hashFiles('backend/target/coverage/lcov.info') != '' }}
|
||||
run: |
|
||||
node scripts/coverage-summary.mjs \
|
||||
--title "Backend test coverage" \
|
||||
--note "Namespaces under \`backend/src/app\`." \
|
||||
--lcov backend/target/coverage/lcov.info \
|
||||
> backend/target/coverage/summary.md
|
||||
cat backend/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Coverage comment
|
||||
# Fork PRs get a read-only token, so only branches of this repo comment.
|
||||
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('backend/target/coverage/lcov.info') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
node scripts/coverage-comment.mjs \
|
||||
--project Backend \
|
||||
--lcov backend/target/coverage/lcov.info \
|
||||
--emma-xml backend/target/coverage/coverage.xml \
|
||||
--prefix backend/
|
||||
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: backend-coverage
|
||||
path: backend/target/coverage/
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
clojure -M:dev:test --reporter kaocha.report/documentation
|
||||
|
||||
49
.github/workflows/tests-common.yml
vendored
49
.github/workflows/tests-common.yml
vendored
@ -26,17 +26,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-common:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # coverage comment on the PR
|
||||
name: "Common Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -45,13 +39,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./common
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
pnpm install;
|
||||
pnpm run check-fmt:clj
|
||||
pnpm run check-fmt:js
|
||||
@ -61,38 +55,3 @@ jobs:
|
||||
working-directory: ./common
|
||||
run: |
|
||||
./scripts/test
|
||||
|
||||
- name: Coverage summary
|
||||
if: ${{ !cancelled() && hashFiles('common/target/coverage/lcov.info') != '' }}
|
||||
run: |
|
||||
node scripts/coverage-summary.mjs \
|
||||
--title "Common JVM test coverage" \
|
||||
--note "Namespaces under \`common/src/app\`, measured by the JVM tests." \
|
||||
--lcov common/target/coverage/lcov.info \
|
||||
> common/target/coverage/summary.md
|
||||
cat common/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Coverage comment
|
||||
# Fork PRs get a read-only token, so only branches of this repo comment.
|
||||
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('common/target/coverage/lcov.info') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
node scripts/coverage-comment.mjs \
|
||||
--project Common \
|
||||
--lcov common/target/coverage/lcov.info \
|
||||
--emma-xml common/target/coverage/coverage.xml \
|
||||
--prefix common/
|
||||
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: common-coverage
|
||||
path: common/target/coverage/
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
430
.github/workflows/tests-e2e.yml
vendored
430
.github/workflows/tests-e2e.yml
vendored
@ -1,430 +0,0 @@
|
||||
name: "CI: E2E"
|
||||
|
||||
# Single entry point for every suite that drives a real frontend bundle.
|
||||
# The bundle is built ONCE by `build-bundle` and restored by every consumer,
|
||||
# so adding a suite no longer adds another `frontend/scripts/build` run.
|
||||
#
|
||||
# Suites and what they need from the bundle:
|
||||
#
|
||||
# - Integration Tests: Playwright specs, backend faked with frontend mocks.
|
||||
# - Composable test suite: Plugin runtime, backend faked with Playwright RPC
|
||||
# fixtures. See plugins/apps/composable-test-suite/README.md ("Running in CI").
|
||||
# - Plugin API Test Suite (mocked): Plugin runtime, backend faked with
|
||||
# Playwright RPC fixtures and MOCK_BACKEND=1.
|
||||
# See plugins/apps/plugin-api-test-suite/README.md.
|
||||
#
|
||||
# Hand-off: `build-bundle` restores `actions/cache` keyed by the checked-out
|
||||
# SHA, builds only on a miss, and saves it; every consumer restores the same
|
||||
# key after `needs: build-bundle` completes. A re-run of the same SHA reuses
|
||||
# the cached bundle instead of rebuilding it.
|
||||
#
|
||||
# NOTE: the job `name:` values are the GitHub check contexts, so they are kept
|
||||
# stable on purpose (branch protection may require them by name). Only the
|
||||
# workflow file/name changed.
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
gh_ref:
|
||||
description: 'Name of the branch or ref'
|
||||
type: string
|
||||
required: true
|
||||
default: 'develop'
|
||||
|
||||
shards:
|
||||
description: 'Shard layout (JSON array)'
|
||||
type: choice
|
||||
required: true
|
||||
default: '[1, 2, 3, 4]'
|
||||
options:
|
||||
- '[1, 2, 3, 4]'
|
||||
- '[1, 2, 3, 4, 5, 6]'
|
||||
- '[1, 2]'
|
||||
- '[1]'
|
||||
|
||||
workers:
|
||||
description: 'Playwright workers per shard'
|
||||
type: string
|
||||
required: true
|
||||
default: '2'
|
||||
|
||||
base_url:
|
||||
description: 'Penpot base URL for the (disabled) live Plugin API suite'
|
||||
required: false
|
||||
default: 'https://localhost:3449'
|
||||
|
||||
pull_request:
|
||||
paths:
|
||||
- 'frontend/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
- 'plugins/**'
|
||||
- '.github/workflows/tests-e2e.yml'
|
||||
|
||||
types:
|
||||
- opened
|
||||
- synchronize
|
||||
- ready_for_review
|
||||
|
||||
push:
|
||||
branches:
|
||||
- develop
|
||||
- staging
|
||||
|
||||
paths:
|
||||
- 'frontend/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
- 'plugins/**'
|
||||
- '.github/workflows/tests-e2e.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || inputs.gh_ref || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# ── 1. Build the frontend bundle once ──────────────────────────────────
|
||||
build-bundle:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Build Integration Bundle"
|
||||
runs-on: penpot-extended-runner
|
||||
timeout-minutes: 30
|
||||
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
outputs:
|
||||
bundle_key: ${{ steps.vars.outputs.bundle_key }}
|
||||
|
||||
steps:
|
||||
# An empty `ref` makes checkout fall back to its default (the PR merge
|
||||
# ref on pull_request, the pushed ref on push).
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
# The cache key must come from the SHA actually checked out: on a manual
|
||||
# run `github.sha` points at the dispatching ref, not at `gh_ref`.
|
||||
- name: Extract cache key
|
||||
id: vars
|
||||
run: |
|
||||
echo "bundle_key=frontend-bundle-$(git rev-parse HEAD)" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Restore Bundle Cache
|
||||
id: restore
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
key: ${{ steps.vars.outputs.bundle_key }}
|
||||
path: frontend/resources/public
|
||||
|
||||
- name: Build Bundle
|
||||
if: steps.restore.outputs.cache-hit != 'true'
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
./scripts/build
|
||||
|
||||
- name: Store Bundle Cache
|
||||
if: steps.restore.outputs.cache-hit != 'true'
|
||||
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
key: ${{ steps.vars.outputs.bundle_key }}
|
||||
path: frontend/resources/public
|
||||
|
||||
# ── 2. Consumers: restore the bundle, never rebuild it ─────────────────
|
||||
test-integration:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Integration Tests (${{ matrix.shard }})"
|
||||
runs-on: penpot-extended-runner
|
||||
timeout-minutes: 40
|
||||
|
||||
needs: build-bundle
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
shard: ${{ fromJSON(inputs.shards || '[1, 2, 3, 4]') }}
|
||||
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
- /var/cache/github-runner/ms-playwright:/ms-playwright
|
||||
env:
|
||||
PLAYWRIGHT_BROWSERS_PATH: /ms-playwright
|
||||
|
||||
steps:
|
||||
- name: Checkout Repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
- name: Restore Cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
key: ${{ needs.build-bundle.outputs.bundle_key }}
|
||||
path: frontend/resources/public
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
pnpm install --frozen-lockfile;
|
||||
|
||||
# No-op once the shared volume is warm; keeps the first run working.
|
||||
- name: Install Playwright Chromium
|
||||
working-directory: ./frontend
|
||||
run: pnpm exec playwright install chromium
|
||||
|
||||
# `strategy.job-total` is the matrix size, so the shard denominator
|
||||
# follows the `shards` input without being hardcoded.
|
||||
- name: Run Tests
|
||||
working-directory: ./frontend
|
||||
env:
|
||||
WORKERS: ${{ inputs.workers }}
|
||||
SHARD: ${{ matrix.shard }}
|
||||
SHARD_TOTAL: ${{ strategy.job-total }}
|
||||
run: |
|
||||
WORKERS=${WORKERS:-2}
|
||||
echo "Running shard ${SHARD}/${SHARD_TOTAL} with $WORKERS workers"
|
||||
pnpm exec playwright test --project default \
|
||||
--workers="$WORKERS" \
|
||||
--shard="${SHARD}/${SHARD_TOTAL}" \
|
||||
--reporter=blob
|
||||
|
||||
- name: Upload blob report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: always()
|
||||
with:
|
||||
name: integration-blob-report-${{ matrix.shard }}
|
||||
path: frontend/blob-report/
|
||||
overwrite: true
|
||||
retention-days: 3
|
||||
|
||||
- name: Upload test result
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: always()
|
||||
with:
|
||||
name: integration-tests-result-${{ matrix.shard }}
|
||||
path: frontend/test-results/
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 3
|
||||
|
||||
composable-test-suite:
|
||||
if: ${{ github.event_name != 'workflow_dispatch' && !github.event.pull_request.draft }}
|
||||
name: "Run composable test suite (mocked backend)"
|
||||
runs-on: penpot-extended-runner
|
||||
timeout-minutes: 30
|
||||
|
||||
needs: build-bundle
|
||||
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
# The driver serves the prebuilt bundle from frontend/resources/public.
|
||||
- name: Restore Cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
key: ${{ needs.build-bundle.outputs.bundle_key }}
|
||||
path: frontend/resources/public
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
run: |
|
||||
pnpm install;
|
||||
|
||||
- name: Install Playwright Chromium
|
||||
working-directory: ./plugins
|
||||
run: pnpm --filter composable-test-suite exec playwright install --with-deps chromium
|
||||
|
||||
- name: Run composable test suite (mocked)
|
||||
working-directory: ./plugins
|
||||
run: pnpm --filter composable-test-suite run test:ci
|
||||
|
||||
api-test-suite-mocked:
|
||||
if: ${{ github.event_name != 'workflow_dispatch' && !github.event.pull_request.draft }}
|
||||
name: "Run Plugin API Test Suite (mocked)"
|
||||
runs-on: penpot-extended-runner
|
||||
timeout-minutes: 30
|
||||
|
||||
needs: build-bundle
|
||||
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
# Mocked mode serves the prebuilt bundle from frontend/resources/public.
|
||||
- name: Restore Cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
key: ${{ needs.build-bundle.outputs.bundle_key }}
|
||||
path: frontend/resources/public
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
run: |
|
||||
pnpm install;
|
||||
|
||||
- name: Install Playwright Chromium
|
||||
working-directory: ./plugins
|
||||
run: pnpm --filter plugin-api-test-suite exec playwright install --with-deps chromium
|
||||
|
||||
- name: Generate API surface
|
||||
working-directory: ./plugins
|
||||
run: pnpm --filter plugin-api-test-suite run gen:api
|
||||
|
||||
- name: Run API test suite (mocked)
|
||||
working-directory: ./plugins
|
||||
env:
|
||||
MOCK_BACKEND: "1"
|
||||
run: pnpm --filter plugin-api-test-suite run test:ci
|
||||
|
||||
# ── 3. Merge the per-shard integration reports ─────────────────────────
|
||||
merge-reports:
|
||||
if: ${{ !cancelled() && !github.event.pull_request.draft && needs.test-integration.result != 'skipped' }}
|
||||
name: "Merge Integration Reports"
|
||||
runs-on: penpot-extended-runner
|
||||
timeout-minutes: 15
|
||||
|
||||
needs: test-integration
|
||||
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- name: Checkout Repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: ${{ inputs.gh_ref }}
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
pnpm install --frozen-lockfile;
|
||||
|
||||
- name: Download blob reports
|
||||
uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
|
||||
with:
|
||||
path: frontend/all-blob-reports
|
||||
pattern: integration-blob-report-*
|
||||
merge-multiple: true
|
||||
|
||||
- name: Merge into HTML report
|
||||
working-directory: ./frontend
|
||||
env:
|
||||
PLAYWRIGHT_JSON_OUTPUT_NAME: report.json
|
||||
run: |
|
||||
pnpm exec playwright merge-reports \
|
||||
--reporter=html,json,list ./all-blob-reports
|
||||
|
||||
- name: Test summary
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
if [ ! -f report.json ]; then
|
||||
echo "No report produced (all shards failed early)." >> "$GITHUB_STEP_SUMMARY"
|
||||
exit 0
|
||||
fi
|
||||
jq -r -f ../.github/scripts/playwright-summary.jq report.json \
|
||||
>> "$GITHUB_STEP_SUMMARY" \
|
||||
|| echo "Summary generation failed; see the HTML report artifact." \
|
||||
>> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Upload JSON report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: always()
|
||||
with:
|
||||
name: integration-json-report
|
||||
path: frontend/report.json
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
- name: Upload HTML report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: always()
|
||||
with:
|
||||
name: integration-html-report
|
||||
path: frontend/playwright-report/
|
||||
overwrite: true
|
||||
retention-days: 7
|
||||
|
||||
# ── 4. Live Plugin API suite (disabled) ────────────────────────────────
|
||||
#
|
||||
# True end-to-end against a LIVE instance: point PENPOT_BASE_URL at a
|
||||
# reachable instance and provide login credentials via repo secrets. Manual
|
||||
# because the CI runner has no Docker to stand up a full stack.
|
||||
#
|
||||
# api-test-suite-live:
|
||||
# if: ${{ github.event_name == 'workflow_dispatch' }}
|
||||
# name: Run Plugin API Test Suite (live)
|
||||
# runs-on: penpot-extended-runner
|
||||
# container:
|
||||
# image: penpotapp/devenv:latest
|
||||
#
|
||||
# env:
|
||||
# PENPOT_BASE_URL: ${{ inputs.base_url }}
|
||||
# E2E_LOGIN_EMAIL: ${{ secrets.E2E_LOGIN_EMAIL }}
|
||||
# E2E_LOGIN_PASSWORD: ${{ secrets.E2E_LOGIN_PASSWORD }}
|
||||
#
|
||||
# steps:
|
||||
# - uses: actions/checkout@v6
|
||||
#
|
||||
# - name: Setup Node
|
||||
# uses: actions/setup-node@v6
|
||||
# with:
|
||||
# node-version-file: .nvmrc
|
||||
#
|
||||
# - name: Install deps
|
||||
# working-directory: ./plugins
|
||||
# run: |
|
||||
# pnpm install;
|
||||
#
|
||||
# - name: Install Playwright Chromium
|
||||
# working-directory: ./plugins
|
||||
# run: pnpm --filter plugin-api-test-suite exec playwright install --with-deps chromium
|
||||
#
|
||||
# - name: Generate API surface
|
||||
# working-directory: ./plugins
|
||||
# run: pnpm --filter plugin-api-test-suite run gen:api
|
||||
#
|
||||
# # Note: requires a running Penpot instance reachable at PENPOT_BASE_URL.
|
||||
# - name: Run API test suite
|
||||
# working-directory: ./plugins
|
||||
# run: pnpm --filter plugin-api-test-suite run test:ci
|
||||
63
.github/workflows/tests-exporter.yml
vendored
63
.github/workflows/tests-exporter.yml
vendored
@ -1,63 +0,0 @@
|
||||
name: "CI: Exporter"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'exporter/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
|
||||
types:
|
||||
- opened
|
||||
- synchronize
|
||||
- ready_for_review
|
||||
|
||||
push:
|
||||
branches:
|
||||
- develop
|
||||
- staging
|
||||
|
||||
paths:
|
||||
- 'exporter/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-exporter:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Exporter Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./exporter
|
||||
run: |
|
||||
pnpm install;
|
||||
pnpm run check-fmt:clj
|
||||
pnpm run lint:clj
|
||||
|
||||
- name: Tests
|
||||
working-directory: ./exporter
|
||||
run: |
|
||||
./scripts/test
|
||||
50
.github/workflows/tests-frontend.yml
vendored
50
.github/workflows/tests-frontend.yml
vendored
@ -30,17 +30,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-frontend:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # coverage comment on the PR
|
||||
name: "Frontend Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -49,13 +43,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
pnpm install;
|
||||
pnpm run check-fmt:js
|
||||
pnpm run check-fmt:clj
|
||||
@ -69,42 +63,6 @@ jobs:
|
||||
run: |
|
||||
./scripts/test
|
||||
|
||||
- name: Coverage summary
|
||||
if: ${{ !cancelled() && hashFiles('frontend/coverage/lcov.info') != '' }}
|
||||
run: |
|
||||
node scripts/coverage-summary.mjs \
|
||||
--title "Frontend unit test coverage" \
|
||||
--note "Logic namespaces under \`frontend/src/app\`, without \`app/main/ui\`." \
|
||||
--lcov frontend/coverage/lcov.info \
|
||||
--unloaded frontend/coverage/unloaded.txt \
|
||||
> frontend/coverage/summary.md
|
||||
cat frontend/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Coverage comment
|
||||
# Fork PRs get a read-only token, so only branches of this repo comment.
|
||||
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('frontend/coverage/lcov.info') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
node scripts/coverage-comment.mjs \
|
||||
--project Frontend \
|
||||
--lcov frontend/coverage/lcov.info \
|
||||
--json-summary frontend/coverage/coverage-summary.json \
|
||||
--unloaded frontend/coverage/unloaded.txt
|
||||
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: frontend-coverage
|
||||
path: frontend/coverage/
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
- name: Component Tests
|
||||
working-directory: ./frontend
|
||||
env:
|
||||
|
||||
93
.github/workflows/tests-integration.yml
vendored
Normal file
93
.github/workflows/tests-integration.yml
vendored
Normal file
@ -0,0 +1,93 @@
|
||||
name: "CI: Integration"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'frontend/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
|
||||
types:
|
||||
- opened
|
||||
- synchronize
|
||||
- ready_for_review
|
||||
|
||||
push:
|
||||
branches:
|
||||
- develop
|
||||
- staging
|
||||
|
||||
paths:
|
||||
- 'frontend/**'
|
||||
- 'common/**'
|
||||
- 'render-wasm/**'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build-integration:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Build Integration Bundle"
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Build Bundle
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
./scripts/build
|
||||
|
||||
- name: Store Bundle Cache
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
key: "integration-bundle-${{ github.sha }}"
|
||||
path: frontend/resources/public
|
||||
|
||||
test-integration:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Integration Tests"
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
- /var/cache/github-runner/m2:/root/.m2
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
needs: build-integration
|
||||
|
||||
steps:
|
||||
- name: Checkout Repository
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Restore Cache
|
||||
uses: actions/cache/restore@v5
|
||||
with:
|
||||
key: "integration-bundle-${{ github.sha }}"
|
||||
path: frontend/resources/public
|
||||
|
||||
- name: Run Tests
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
./scripts/test-e2e
|
||||
|
||||
- name: Upload test result
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: integration-tests-result
|
||||
path: frontend/test-results/
|
||||
overwrite: true
|
||||
retention-days: 3
|
||||
11
.github/workflows/tests-library.yml
vendored
11
.github/workflows/tests-library.yml
vendored
@ -28,14 +28,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-library:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Library Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -44,13 +41,13 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./library
|
||||
run: |
|
||||
corepack enable;
|
||||
corepack install;
|
||||
pnpm install;
|
||||
pnpm run check-fmt
|
||||
pnpm run lint
|
||||
|
||||
13
.github/workflows/tests-mcp.yml
vendored
13
.github/workflows/tests-mcp.yml
vendored
@ -1,4 +1,4 @@
|
||||
name: "CI: MCP"
|
||||
name: "MCP CI"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
@ -24,21 +24,16 @@ on:
|
||||
paths:
|
||||
- 'mcp/**'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-mcp:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: "Test MCP"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container: penpotapp/devenv:latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Setup
|
||||
working-directory: ./mcp
|
||||
@ -54,4 +49,4 @@ jobs:
|
||||
- name: Tests
|
||||
working-directory: ./mcp
|
||||
run: |
|
||||
pnpm run test;
|
||||
pnpm -r run test;
|
||||
|
||||
17
.github/workflows/tests-plugins.yml
vendored
17
.github/workflows/tests-plugins.yml
vendored
@ -26,14 +26,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-plugins:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
name: Plugins Runtime Linter & Tests
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -41,15 +38,21 @@ jobs:
|
||||
- /var/cache/github-runner/gitlib:/root/.gitlibs
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Node
|
||||
id: setup-node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
node-version-file: .nvmrc
|
||||
|
||||
- name: Install deps
|
||||
working-directory: ./plugins
|
||||
shell: bash
|
||||
run: |
|
||||
pnpm install -r;
|
||||
corepack enable;
|
||||
corepack install;
|
||||
pnpm install;
|
||||
|
||||
- name: Run Lint
|
||||
working-directory: ./plugins
|
||||
|
||||
71
.github/workflows/tests-wasm.yml
vendored
71
.github/workflows/tests-wasm.yml
vendored
@ -26,17 +26,11 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test-render-wasm:
|
||||
if: ${{ !github.event.pull_request.draft }}
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write # coverage comment on the PR
|
||||
name: "Render WASM Tests"
|
||||
runs-on: penpot-extended-runner
|
||||
runs-on: penpot-runner-02
|
||||
container:
|
||||
image: penpotapp/devenv:latest
|
||||
volumes:
|
||||
@ -45,78 +39,19 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Format
|
||||
working-directory: ./render-wasm
|
||||
run: |
|
||||
./fmt --check
|
||||
cargo fmt --check
|
||||
|
||||
- name: Lint
|
||||
working-directory: ./render-wasm
|
||||
run: |
|
||||
./lint
|
||||
|
||||
- name: Install cargo-llvm-cov
|
||||
env:
|
||||
CARGO_LLVM_COV_VERSION: "0.9.1"
|
||||
run: |
|
||||
case "$(uname -m)" in
|
||||
aarch64|arm64)
|
||||
arch="aarch64"
|
||||
sha256="abf5f13c1520f8756d2192bfaaeadb0208f5b7eaa8cadc15bf847befa42b7360"
|
||||
;;
|
||||
*)
|
||||
arch="x86_64"
|
||||
sha256="b3f68e625481fed9b16444174f3fa5ebcdbde4a1878803a35eabe2dcefcdc41a"
|
||||
;;
|
||||
esac
|
||||
archive="cargo-llvm-cov-${arch}-unknown-linux-gnu.tar.gz"
|
||||
curl -fsSL --retry 3 -o "$RUNNER_TEMP/$archive" \
|
||||
"https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${CARGO_LLVM_COV_VERSION}/${archive}"
|
||||
echo "${sha256} $RUNNER_TEMP/$archive" | sha256sum -c -
|
||||
tar -xzf "$RUNNER_TEMP/$archive" -C /usr/local/bin cargo-llvm-cov
|
||||
cargo llvm-cov --version
|
||||
|
||||
- name: Test
|
||||
working-directory: ./render-wasm
|
||||
env:
|
||||
RENDER_WASM_COVERAGE: "true"
|
||||
run: |
|
||||
./test
|
||||
|
||||
- name: Coverage summary
|
||||
if: ${{ !cancelled() && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
|
||||
run: |
|
||||
node scripts/coverage-summary.mjs \
|
||||
--title "Render WASM test coverage" \
|
||||
--note "Rust sources under \`render-wasm/src\`, measured by cargo-llvm-cov on the native test build." \
|
||||
--lcov render-wasm/target/coverage/lcov.info \
|
||||
> render-wasm/target/coverage/summary.md
|
||||
cat render-wasm/target/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Coverage comment
|
||||
# Fork PRs get a read-only token, so only branches of this repo comment.
|
||||
if: ${{ !cancelled() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && hashFiles('render-wasm/target/coverage/lcov.info') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
node scripts/coverage-comment.mjs \
|
||||
--project "Render WASM" \
|
||||
--lcov render-wasm/target/coverage/lcov.info \
|
||||
--llvm-json render-wasm/target/coverage/summary.json
|
||||
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
|
||||
if: ${{ !cancelled() }}
|
||||
with:
|
||||
name: render-wasm-coverage
|
||||
path: render-wasm/target/coverage/
|
||||
overwrite: true
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
39
.github/zizmor.yml
vendored
39
.github/zizmor.yml
vendored
@ -1,39 +0,0 @@
|
||||
# zizmor configuration — https://docs.zizmor.sh/configuration/
|
||||
#
|
||||
# Every audit is enforced for every workflow. New workflows are checked with
|
||||
# no exceptions. The entries below are the only remaining ones, and each is
|
||||
# marked as either a pending clean-up or an accepted risk with its reason.
|
||||
# Remove a file from a list in the same PR that fixes it, and the whole rule
|
||||
# entry once its list is empty.
|
||||
rules:
|
||||
# Accepted: wrangler is added with `pnpm add wrangler@latest` before each
|
||||
# deploy instead of being a dependency of the plugins workspace. Only used
|
||||
# to publish to Cloudflare; not worth adding to the workspace.
|
||||
adhoc-packages:
|
||||
ignore:
|
||||
- plugins-deploy-api-doc.yml
|
||||
- plugins-deploy-package.yml
|
||||
- plugins-deploy-styles-doc.yml
|
||||
# Accepted: auto-label.yml needs pull_request_target so the GitHub App
|
||||
# secrets are available for PRs from forks. It never checks out or runs
|
||||
# PR code, it only reads ids from the event payload.
|
||||
dangerous-triggers:
|
||||
ignore:
|
||||
- auto-label.yml
|
||||
# Style nudge towards the `$/...` syntax; not worth enforcing.
|
||||
self-repository:
|
||||
disable: true
|
||||
# Accepted: penpotapp/devenv is our own image, built and pushed by
|
||||
# build-docker-devenv.yml, so the floating tag is intentional.
|
||||
unpinned-images:
|
||||
ignore:
|
||||
- plugins-deploy-package.yml
|
||||
- tests-backend.yml
|
||||
- tests-common.yml
|
||||
- tests-e2e.yml
|
||||
- tests-exporter.yml
|
||||
- tests-frontend.yml
|
||||
- tests-library.yml
|
||||
- tests-mcp.yml
|
||||
- tests-plugins.yml
|
||||
- tests-wasm.yml
|
||||
33
.gitignore
vendored
33
.gitignore
vendored
@ -9,17 +9,11 @@
|
||||
.clj-kondo
|
||||
.cpcache
|
||||
.lsp
|
||||
.env
|
||||
.nrepl-port
|
||||
.nyc_output
|
||||
.rebel_readline_history
|
||||
.repl
|
||||
opencode.json
|
||||
.opencode/package-lock.json
|
||||
/*.html
|
||||
/*.jpg
|
||||
/*.json
|
||||
!/package.json
|
||||
/*.md
|
||||
!CHANGES.md
|
||||
!CONTRIBUTING.md
|
||||
@ -36,6 +30,7 @@ opencode.json
|
||||
/.clj-kondo/.cache
|
||||
/_dump
|
||||
/notes
|
||||
/.opencode/package-lock.json
|
||||
/plans
|
||||
/prompts
|
||||
/playground/
|
||||
@ -61,14 +56,11 @@ opencode.json
|
||||
/docker/images/bundle*
|
||||
/exporter/target
|
||||
/exporter/.shadow-cljs
|
||||
/exporter/resources/wasm/
|
||||
/exporter/src/app/wasm/shared.js
|
||||
/frontend/.storybook/preview-body.html
|
||||
/frontend/.storybook/preview-head.html
|
||||
/frontend/playwright-report/
|
||||
/frontend/playwright/ui/visual-specs/
|
||||
/frontend/text-editor/src/wasm/
|
||||
/frontend/coverage/
|
||||
/frontend/dist/
|
||||
/frontend/npm-debug.log
|
||||
/frontend/out/
|
||||
@ -80,6 +72,8 @@ opencode.json
|
||||
/frontend/target/
|
||||
/frontend/test-results/
|
||||
/frontend/.shadow-cljs
|
||||
/other/
|
||||
/scripts/
|
||||
/nexus/
|
||||
/tmp/
|
||||
/vendor/**/target
|
||||
@ -87,7 +81,6 @@ opencode.json
|
||||
/web
|
||||
/library/target/
|
||||
/library/*.zip
|
||||
/library/.shadow-cljs
|
||||
/external
|
||||
/penpot-nitrate
|
||||
/test-results/
|
||||
@ -95,15 +88,6 @@ opencode.json
|
||||
/blob-report/
|
||||
/playwright/.cache/
|
||||
/render-wasm/target/
|
||||
/render-wasm/.render-wasm-build.lock
|
||||
|
||||
# render-wasm build artifacts copied into the exporter (autogenerated)
|
||||
/exporter/resources/wasm/render-wasm.js
|
||||
/exporter/resources/wasm/render-wasm.wasm
|
||||
/exporter/src/app/wasm/shared.js
|
||||
|
||||
/media-processor/dist/
|
||||
/media-processor/target/
|
||||
/**/node_modules
|
||||
/**/.yarn/*
|
||||
/.pnpm-store
|
||||
@ -111,19 +95,8 @@ opencode.json
|
||||
/.idea
|
||||
*.iml
|
||||
/.claude
|
||||
/CLAUDE.md
|
||||
/CLAUDE.local.md
|
||||
/.playwright-mcp
|
||||
/.devenv/mcp/
|
||||
/opencode.json
|
||||
/.agents/plans
|
||||
/.agents/local/
|
||||
/AGENTS.local.md
|
||||
/.opencode/reports
|
||||
/.opencode/prompts
|
||||
/.ci-logs
|
||||
/tmp
|
||||
/.codex/
|
||||
/scripts/__pycache__/
|
||||
/tools/__pycache__
|
||||
/scripts/__pycache__
|
||||
|
||||
28
.opencode/agents/commiter.md
Normal file
28
.opencode/agents/commiter.md
Normal file
@ -0,0 +1,28 @@
|
||||
---
|
||||
name: commiter
|
||||
description: Git commit assistant
|
||||
mode: all
|
||||
---
|
||||
|
||||
## Role
|
||||
|
||||
You are responsible for creating git commits for Penpot and must
|
||||
follow the repository commit-format rules exactly. It should have
|
||||
concise title and clear summary of changes in the description,
|
||||
including the rationale if proceed.
|
||||
|
||||
## Requirements
|
||||
|
||||
* Override your internal commit rules when the user explicitly requests
|
||||
something that conflicts with them.
|
||||
* Read `.serena/memories/workflows/creating-commits.md` before
|
||||
creating any commit and follow the commit guidelines strictly.
|
||||
* Keep the description (commit body) with maximum line length of 80
|
||||
characters. Use manual line breaks to wrap text before it exceeds
|
||||
this limit.
|
||||
* Use `git commit -s` so the commit includes the required
|
||||
`Signed-off-by` line.
|
||||
* Do not guess or hallucinate git author information (Name or
|
||||
Email). Never include the `--author` flag in git commands unless
|
||||
specifically instructed by the user for a unique case; assume the
|
||||
local environment is already configured.
|
||||
25
.opencode/agents/engineer.md
Normal file
25
.opencode/agents/engineer.md
Normal file
@ -0,0 +1,25 @@
|
||||
---
|
||||
name: Engineer
|
||||
description: Senior Full-Stack Software Engineer
|
||||
mode: primary
|
||||
---
|
||||
|
||||
## Role
|
||||
|
||||
You are a high-autonomy Senior Full-Stack Software Engineer working on Penpot, an
|
||||
open-source design tool. You have full permission to navigate the codebase, modify files,
|
||||
and execute commands to fulfill your tasks. Your goal is to solve complex technical tasks
|
||||
with high precision while maintaining a strong focus on maintainability and performance.
|
||||
|
||||
## Before Start
|
||||
|
||||
**Read `AGENTS.md` file and project structure and how the memory system works**
|
||||
|
||||
## Requiremens
|
||||
|
||||
* Before writing code, analyze the task in depth and describe your plan. If the task is
|
||||
complex, break it down into atomic steps.
|
||||
* Do **not** touch unrelated modules unless the task explicitly requires it.
|
||||
* Only reference functions, namespaces, or APIs that actually exist in the
|
||||
codebase. Verify their existence before citing them. If unsure, search first.
|
||||
* Be concise and autonomous — avoid unnecessary explanations.
|
||||
60
.opencode/agents/planner.md
Normal file
60
.opencode/agents/planner.md
Normal file
@ -0,0 +1,60 @@
|
||||
---
|
||||
name: Planner
|
||||
description: Software architect for planning and analysis only
|
||||
mode: primary
|
||||
permission:
|
||||
edit: ask
|
||||
---
|
||||
|
||||
## Role
|
||||
|
||||
You are a Senior Software Architect working on Penpot, an open-source design
|
||||
tool. Your sole responsibility is planning and analysis — you do NOT write,
|
||||
modify any code.
|
||||
|
||||
You help users understand the codebase, design solutions, and create detailed
|
||||
implementation plans that other agents or developers can execute. Document
|
||||
everything they need to know: which files to touch for each task, code, testing,
|
||||
docs they might need to check, how to test it. Give them the whole plan as
|
||||
bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.
|
||||
|
||||
Do **not** suggest commit messages or commit names anywhere in your plans or
|
||||
responses — committing is the developer's responsibility.
|
||||
|
||||
Assume they are a skilled developer, but know almost nothing about our toolset
|
||||
or problem domain. Assume they don't know good test design very well.
|
||||
|
||||
## Requirements
|
||||
|
||||
* Analyze the codebase architecture and identify affected modules.
|
||||
* Read `AGENTS.md` file and project structure and how the memory system works and how to
|
||||
navigate and read relevant information conventions.
|
||||
* Break down complex features or bugs into atomic, actionable steps.
|
||||
* Propose solutions with clear rationale, trade-offs, and sequencing.
|
||||
* Identify risks, edge cases, and testing considerations.
|
||||
|
||||
Save plans to: plans/YYYY-MM-DD-<plan-one-line-title>.md
|
||||
|
||||
## Constraints
|
||||
|
||||
* You are **read-only** — never create, edit, or delete files.
|
||||
* You do **not** run builds, tests, linters, or any commands that modify state.
|
||||
* You do **not** create git commits or interact with version control.
|
||||
* You do **not** execute shell commands beyond read-only searches (`rg`, `ls`,
|
||||
`find`, `cat`).
|
||||
* Your output is a structured plan or analysis, ready for handoff to an
|
||||
engineer agent or developer.
|
||||
|
||||
## Output format
|
||||
|
||||
When producing a plan, structure it as:
|
||||
|
||||
1. **Context** — What is the problem or feature request?
|
||||
2. **Affected modules** — Which parts of the codebase are involved?
|
||||
3. **Approach** — Step-by-step implementation plan with file paths and
|
||||
function names where applicable.
|
||||
4. **Risks & considerations** — Edge cases, performance implications, breaking
|
||||
changes.
|
||||
5. **Testing strategy** — How to verify the implementation works correctly.
|
||||
|
||||
|
||||
56
.opencode/agents/prompt-assistant.md
Normal file
56
.opencode/agents/prompt-assistant.md
Normal file
@ -0,0 +1,56 @@
|
||||
---
|
||||
name: Prompt Assistant
|
||||
description: Refines and improves prompts for maximum clarity and effectiveness
|
||||
mode: all
|
||||
---
|
||||
|
||||
## Role
|
||||
|
||||
You are an expert Prompt Engineer with strong knowledge of
|
||||
penpot. Your sole responsibility is to take a prompt provided by the
|
||||
user and transform it into the most effective, clear, and
|
||||
well-structured version possible — ready to be used with any AI model.
|
||||
|
||||
## Requirements
|
||||
|
||||
* You do NOT execute tasks. You do NOT write code. You only design and refine prompts
|
||||
* Read `AGENTS.md` file and project structure and how the memory system works and how to
|
||||
navigate and read relevant information conventions.
|
||||
* Analyze the original prompt: identify its intent, target audience, ambiguities, missing
|
||||
context, and structural weaknesses
|
||||
* Ask clarifying questions if the intent is unclear or if critical information is missing
|
||||
(e.g. target model, expected output format, tone, constraints). Keep questions concise
|
||||
and grouped
|
||||
* Rewrite the prompt using prompt engineering best practices
|
||||
|
||||
|
||||
## Prompt Engineering Principles
|
||||
|
||||
Apply these techniques when refining prompts:
|
||||
|
||||
- **Be specific and explicit**: Replace vague instructions with precise ones.
|
||||
- **Set the context**: Include background information the model needs to
|
||||
perform well.
|
||||
- **Specify the output format**: State the desired structure, length, tone,
|
||||
or format (e.g. bullet list, JSON, step-by-step).
|
||||
- **Add constraints**: Include what the model should avoid or not do.
|
||||
- **Use examples** (few-shot): When applicable, suggest adding examples to
|
||||
anchor the model's behaviour.
|
||||
- **Break down complexity**: Split multi-step tasks into clear numbered steps.
|
||||
- **Avoid ambiguity**: Remove pronouns and references that could be
|
||||
misinterpreted.
|
||||
- **Chain of thought**: For reasoning tasks, include "Think step by step."
|
||||
|
||||
## Constraints
|
||||
|
||||
- Do NOT execute the prompt yourself.
|
||||
- Do NOT answer the question inside the prompt.
|
||||
- Do NOT add unnecessary verbosity — prompts should be as short as they can
|
||||
be while remaining complete.
|
||||
- Always preserve the user's original intent.
|
||||
|
||||
## Output
|
||||
|
||||
Refined Prompt: The improved, ready-to-use prompt. Print it for
|
||||
immediate use and save it to
|
||||
prompts/YYYY-MM-DD-N-<prompt-one-line-title>.md for future use.
|
||||
@ -1,169 +0,0 @@
|
||||
// Penpot OpenCode V2 plugin: custom tools for Penpot development.
|
||||
//
|
||||
// Tools are registered with `ctx.tool.transform()` and JSON Schema inputs.
|
||||
// Keep this plugin dependency-free so the auto-discovered local plugin loads
|
||||
// without project npm dependencies.
|
||||
|
||||
import path from "path"
|
||||
import { spawn } from "child_process"
|
||||
|
||||
function runCommand(command, args, options = {}) {
|
||||
const {
|
||||
cwd,
|
||||
env,
|
||||
stdin,
|
||||
closeStdin = false,
|
||||
successMessage = "Command executed successfully",
|
||||
} = options
|
||||
|
||||
return new Promise((resolve) => {
|
||||
let stdout = ""
|
||||
let stderr = ""
|
||||
|
||||
const proc = spawn(command, args, { cwd, env })
|
||||
|
||||
proc.stdout.on("data", (data) => {
|
||||
stdout += data.toString()
|
||||
})
|
||||
|
||||
proc.stderr.on("data", (data) => {
|
||||
stderr += data.toString()
|
||||
})
|
||||
|
||||
proc.on("error", (error) => {
|
||||
resolve(`Error: ${error.message}`)
|
||||
})
|
||||
|
||||
proc.on("close", (exitCode) => {
|
||||
const output =
|
||||
exitCode === 0
|
||||
? stdout.trim() || successMessage
|
||||
: `Error (exit ${exitCode}): ${
|
||||
(stderr || stdout).trim() || "No error output"
|
||||
}`
|
||||
resolve(output)
|
||||
})
|
||||
|
||||
// Close stdin so the child cannot wait on it indefinitely. `psql -c`
|
||||
// never reads stdin, so only the paren-repair pipe mode needs this, but
|
||||
// closing it unconditionally is harmless there.
|
||||
if (stdin !== undefined) {
|
||||
proc.stdin.end(stdin)
|
||||
} else if (closeStdin) {
|
||||
proc.stdin.end()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
function executePsql(sql, useTestDb, cwd) {
|
||||
const host = process.env.PENPOT_DB_HOST || "postgres"
|
||||
const user = process.env.PENPOT_DB_USER || "penpot"
|
||||
const db = useTestDb
|
||||
? "penpot_test"
|
||||
: process.env.PENPOT_DB_NAME || "penpot"
|
||||
const password = process.env.PENPOT_DB_PASSWORD || "penpot"
|
||||
|
||||
const psqlArgs = ["-h", host, "-U", user, "-d", db, "-c", sql]
|
||||
|
||||
return runCommand("psql", psqlArgs, {
|
||||
cwd,
|
||||
env: { ...process.env, PGPASSWORD: password },
|
||||
successMessage: "Query executed successfully",
|
||||
})
|
||||
}
|
||||
|
||||
function executeParenRepair({ files, code }, directory) {
|
||||
const script = path.join(directory, "scripts/paren-repair")
|
||||
|
||||
const fileList = files
|
||||
? files
|
||||
.split(",")
|
||||
.map((file) => file.trim())
|
||||
.filter(Boolean)
|
||||
: []
|
||||
|
||||
const childArgs =
|
||||
fileList.length > 0 ? [script, ...fileList] : [script]
|
||||
|
||||
return runCommand("bb", childArgs, {
|
||||
cwd: directory,
|
||||
stdin: code,
|
||||
closeStdin: true,
|
||||
successMessage: "No changes needed",
|
||||
})
|
||||
}
|
||||
|
||||
const penpotPsqlInputSchema = {
|
||||
type: "object",
|
||||
properties: {
|
||||
sql: {
|
||||
type: "string",
|
||||
description: "SQL command to execute",
|
||||
},
|
||||
test: {
|
||||
type: "boolean",
|
||||
description: "Use the penpot_test database",
|
||||
},
|
||||
},
|
||||
required: ["sql"],
|
||||
additionalProperties: false,
|
||||
}
|
||||
|
||||
const parenRepairInputSchema = {
|
||||
type: "object",
|
||||
properties: {
|
||||
// A string is used instead of an array so OpenCode displays it
|
||||
// in the generic tool invocation.
|
||||
files: {
|
||||
type: "string",
|
||||
description:
|
||||
"Comma-separated file paths to fix, for example: frontend/src/app/config.cljs, backend/src/core.clj",
|
||||
},
|
||||
code: {
|
||||
type: "string",
|
||||
description: "Code string to fix via stdin",
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
}
|
||||
|
||||
async function setup(ctx) {
|
||||
// Use the plugin instance location as the working directory for both tools.
|
||||
const directory =
|
||||
ctx.location.directory ?? ctx.location.project?.canonical
|
||||
|
||||
// Keep this callback synchronous: transforms are replayable state edits.
|
||||
// The async work happens later, inside each tool's `execute`.
|
||||
await ctx.tool.transform((editor) => {
|
||||
editor.add({
|
||||
name: "penpot-psql",
|
||||
description:
|
||||
"Execute a SQL command against the Penpot database. Uses the defaults from scripts/psql.",
|
||||
input: penpotPsqlInputSchema,
|
||||
async execute(input) {
|
||||
const content = await executePsql(
|
||||
input.sql,
|
||||
input.test === true,
|
||||
directory,
|
||||
)
|
||||
return { content }
|
||||
},
|
||||
})
|
||||
|
||||
editor.add({
|
||||
name: "paren-repair",
|
||||
description:
|
||||
"Fix mismatched parentheses/braces in Clojure files (.clj, .cljs, .cljc) then reformat with cljfmt.",
|
||||
input: parenRepairInputSchema,
|
||||
async execute(input) {
|
||||
const content = await executeParenRepair(input, directory)
|
||||
return { content }
|
||||
},
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
export default {
|
||||
id: "penpot",
|
||||
setup,
|
||||
}
|
||||
85
.opencode/skills/backport-commit/SKILL.md
Normal file
85
.opencode/skills/backport-commit/SKILL.md
Normal file
@ -0,0 +1,85 @@
|
||||
---
|
||||
name: backport-commit
|
||||
description: Port changes from a specific Git commit to the current branch by manually applying the diff, avoiding cherry-pick when it would introduce complex conflicts.
|
||||
---
|
||||
|
||||
# Backport Commit
|
||||
|
||||
Port changes from a specific Git commit to the current branch by manually
|
||||
applying the diff, avoiding `git cherry-pick` when it would introduce
|
||||
complex conflicts.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill whenever the user asks to backport a commit, especially when:
|
||||
|
||||
- The commit touches multiple modules or files with significant divergence
|
||||
- `git cherry-pick` is explicitly ruled out ("do not use cherry-pick")
|
||||
- The target commit is old enough that conflicts are likely
|
||||
- The commit introduces both source changes AND new files (tests, etc.)
|
||||
- You need full control over how each hunk is applied
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Identify the target commit
|
||||
|
||||
```bash
|
||||
# Verify the commit exists and understand what it does
|
||||
git log --oneline -1 <commit-sha>
|
||||
|
||||
# Get the full diff (including new/deleted files)
|
||||
git show <commit-sha>
|
||||
|
||||
# Capture the original commit message for later reuse
|
||||
git log --format='%B' -1 <commit-sha>
|
||||
```
|
||||
|
||||
### 2. Identify affected modules
|
||||
|
||||
From the file paths in the diff, determine which Penpot modules are affected
|
||||
(frontend, backend, common, render-wasm, etc.) and read their `AGENTS.md`
|
||||
files **before** making any changes. If a module has no `AGENTS.md`, skip
|
||||
that step — verify with `ls <module>/AGENTS.md` first.
|
||||
|
||||
### 3. Read the current state of each affected file
|
||||
|
||||
For every file the diff touches, read the current version on disk to understand
|
||||
context and ensure correct placement before editing.
|
||||
|
||||
### 4. Apply changes manually (the core of this approach)
|
||||
|
||||
Process every hunk in the diff using the appropriate tool:
|
||||
|
||||
| Diff action | Tool to use |
|
||||
|-------------|-------------|
|
||||
| Modify existing file | `edit` — use enough surrounding context in `oldString` to uniquely match the location |
|
||||
| Add new file | `write` — include proper license header and namespace conventions matching project style |
|
||||
| Delete file | `bash rm <path>` |
|
||||
| Rename/move file | `bash mv <old> <new>`, then apply any content changes with `edit` |
|
||||
|
||||
> **Tip:** Group nearby hunks from the same file into a single `edit` call.
|
||||
> Use separate calls when hunks are far apart to keep `oldString` short and
|
||||
> unambiguous.
|
||||
|
||||
Repeat until **all** hunks in the diff are ported.
|
||||
|
||||
### 5. Validate
|
||||
|
||||
Run **lint**, **check-fmt**, and **tests** for every affected module (see each
|
||||
module's `AGENTS.md` for the exact commands). If the formatter auto-fixes
|
||||
indentation, verify the logic is still semantically correct. All checks must
|
||||
pass before moving on.
|
||||
|
||||
### 6. Commit
|
||||
|
||||
Ask the `commiter` sub-agent to create a commit. Stage all relevant files
|
||||
(exclude unrelated untracked files) and provide the original commit message as
|
||||
a reference, adapting it as needed for the target branch context.
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Context matters** — always read files before editing; never guess
|
||||
indentation or surrounding code
|
||||
- **Lint + format + test** — never skip validation before committing
|
||||
- **Preserve intent** — keep the original commit message meaning; the
|
||||
`commiter` agent handles formatting
|
||||
@ -9,11 +9,6 @@ metadata: {"clawdbot":{"emoji":"🦇","requires":{"bins":["bat"]},"install":[{"i
|
||||
|
||||
`cat` with syntax highlighting, line numbers, and Git integration.
|
||||
|
||||
## When to use
|
||||
|
||||
- Reading or displaying a file in the terminal — prefer it over plain
|
||||
`cat`: syntax highlighting, line numbers, git-side indicators.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic usage
|
||||
124
.opencode/skills/create-pr/SKILL.md
Normal file
124
.opencode/skills/create-pr/SKILL.md
Normal file
@ -0,0 +1,124 @@
|
||||
---
|
||||
name: create-pr
|
||||
description: Create or update a GitHub PR following Penpot conventions, with a concise engineer-focused description
|
||||
---
|
||||
|
||||
# Pull Request (Create or Update)
|
||||
|
||||
Create or update a GitHub PR with proper title format and a concise description that explains reasoning, not implementation details.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Opening a new pull request
|
||||
- Updating an existing PR's title or description
|
||||
- The user asks to create or update a PR
|
||||
- Code changes are ready and committed
|
||||
|
||||
## Conventions
|
||||
|
||||
All PR conventions (title format, body structure, writing principles, what NOT to include) are documented in `mem:workflow/creating-prs`. This skill covers the procedure only.
|
||||
|
||||
## Workflow A: Creating a New PR
|
||||
|
||||
### A1. Verify Prerequisites
|
||||
|
||||
```bash
|
||||
git branch --show-current
|
||||
git log --oneline main..HEAD
|
||||
```
|
||||
|
||||
### A2. Check if Branch is Pushed
|
||||
|
||||
```bash
|
||||
BRANCH=$(git branch --show-current)
|
||||
if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then
|
||||
echo "Branch is pushed, proceeding with PR creation"
|
||||
else
|
||||
echo "ERROR: Branch '$BRANCH' is not pushed to remote. Please push the branch first."
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
**If the branch is not pushed, STOP here and ask the user to push it. The LLM does not have push permissions.**
|
||||
|
||||
### A3. Create PR Body
|
||||
|
||||
Write the body to `/tmp/pr-body.md` using the template from `mem:workflow/creating-prs` (Description Body section):
|
||||
|
||||
```bash
|
||||
cat > /tmp/pr-body.md << 'EOF'
|
||||
**Note:** This PR was created with AI assistance as part of the Penpot MCP self-improvement initiative.
|
||||
|
||||
## What
|
||||
|
||||
<one paragraph: the problem or feature, user-facing impact>
|
||||
|
||||
## Why
|
||||
|
||||
<root cause or motivation, why this change was necessary>
|
||||
|
||||
## How
|
||||
|
||||
<high-level approach, key technical decisions>
|
||||
EOF
|
||||
```
|
||||
|
||||
### A4. Create the PR
|
||||
|
||||
```bash
|
||||
gh pr create --base main --project "Main" --title "<title>" --body-file /tmp/pr-body.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow B: Updating an Existing PR
|
||||
|
||||
Use this when a PR already exists and you need to change its title or description (or both).
|
||||
|
||||
### B1. Find the PR Number
|
||||
|
||||
```bash
|
||||
BRANCH=$(git branch --show-current)
|
||||
gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number'
|
||||
```
|
||||
|
||||
If the result is empty, there is no open PR for this branch — use Workflow A instead.
|
||||
If multiple PRs match, pick the first one (typically there should only be one).
|
||||
|
||||
### B2. Determine What to Update
|
||||
|
||||
- **Title only:** use `--title "<new title>"`
|
||||
- **Description (body) only:** write to `/tmp/pr-body.md` and use `--body-file /tmp/pr-body.md`
|
||||
- **Both title and description:** supply both flags
|
||||
|
||||
### B3. Prepare Description (if updating)
|
||||
|
||||
Use the same template from `mem:workflow/creating-prs`:
|
||||
|
||||
```bash
|
||||
cat > /tmp/pr-body.md << 'EOF'
|
||||
**Note:** This PR was created with AI assistance as part of the Penpot MCP self-improvement initiative.
|
||||
|
||||
## What
|
||||
|
||||
<one paragraph: the problem or feature, user-facing impact>
|
||||
|
||||
## Why
|
||||
|
||||
<root cause or motivation, why this change was necessary>
|
||||
|
||||
## How
|
||||
|
||||
<high-level approach, key technical decisions>
|
||||
EOF
|
||||
```
|
||||
|
||||
### B4. Update the PR
|
||||
|
||||
```bash
|
||||
gh pr edit <NUMBER> --title "<title>" --body-file /tmp/pr-body.md
|
||||
```
|
||||
|
||||
Omit either flag if only one field is being updated.
|
||||
|
||||
**The LLM has permissions to edit PRs. No push required.**
|
||||
@ -9,11 +9,6 @@ metadata: {"clawdbot":{"emoji":"📂","requires":{"bins":["fd"]},"install":[{"id
|
||||
|
||||
User-friendly alternative to `find` with smart defaults.
|
||||
|
||||
## When to use
|
||||
|
||||
- Locating files or directories by name or pattern — prefer it over
|
||||
plain `find`: simpler syntax, smart defaults, respects `.gitignore`.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic search
|
||||
227
.opencode/skills/gh-issue-from-pr/SKILL.md
Normal file
227
.opencode/skills/gh-issue-from-pr/SKILL.md
Normal file
@ -0,0 +1,227 @@
|
||||
---
|
||||
name: gh-issue-from-pr
|
||||
description: Create a user-facing GitHub issue from a PR, separating the WHAT from the HOW, with correct milestone, project, labels, and issue type.
|
||||
---
|
||||
|
||||
# Skill: gh-issue-from-pr
|
||||
|
||||
Create a GitHub issue that captures the **WHAT** (user-facing feature or
|
||||
bug) from an existing PR that describes the **HOW** (implementation).
|
||||
Used when the project board needs an issue as the primary changelog/release unit.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Create a tracking issue from a PR for changelog purposes
|
||||
- Extract the user-facing problem/feature from a PR's implementation details
|
||||
- Assign milestone, project, labels, and issue type to a new issue derived from a PR
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
- Permission to create issues and edit PRs in the target repository
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Understand the PR
|
||||
|
||||
```bash
|
||||
gh pr view <PR_NUMBER> --repo penpot/penpot \
|
||||
--json title,body,author,labels,baseRefName,mergedAt,state,milestone
|
||||
```
|
||||
|
||||
Identify:
|
||||
|
||||
- **WHAT** — user-facing problem or feature. Goes into the issue.
|
||||
Describe symptoms and impact, not internal mechanisms.
|
||||
- **HOW** — implementation details. These belong in the PR, not the issue.
|
||||
|
||||
### 2. Determine metadata
|
||||
|
||||
| Field | Source | Rule |
|
||||
|-------|--------|------|
|
||||
| **Title** | PR title | Rewrite from user perspective. Strip leading emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`). Focus on observable behavior. Use imperative mood. Use the `issue-title` skill to generate this. |
|
||||
| **Labels** | PR labels | Copy `community contribution` if present. Skip `bug` and `enhancement` (redundant with Issue Type). Skip workflow labels (`backport candidate`, `team-qa`). |
|
||||
| **Milestone** | PR milestone | **Always copy what's on the PR.** Fetch with: `gh pr view <PR_NUMBER> --json milestone --jq '.milestone.title'` If the PR has no milestone, create the issue without one. |
|
||||
| **Project** | Always `Main` | Penpot uses the `Main` project (number 8) for all issues. |
|
||||
| **Body** | PR's user-facing section | Extract steps to reproduce or feature description. Omit internal details. Use templates below. |
|
||||
| **Issue Type** | PR labels / title | Map: `bug` label or `:bug:` title → `Bug`. `enhancement` label or `:sparkles:` title → `Enhancement`. Feature/epic → `Feature`. Default → `Task`. |
|
||||
|
||||
### 3. Write the issue body
|
||||
|
||||
**Bug template:**
|
||||
|
||||
```markdown
|
||||
### Description
|
||||
|
||||
<what breaks, what the user experiences>
|
||||
|
||||
### Steps to reproduce
|
||||
|
||||
1. <step 1>
|
||||
2. <step 2>
|
||||
|
||||
### Expected behavior
|
||||
|
||||
<what should happen instead>
|
||||
|
||||
### Affected versions
|
||||
|
||||
<version>
|
||||
```
|
||||
|
||||
**Enhancement template:**
|
||||
|
||||
```markdown
|
||||
### Description
|
||||
|
||||
<what the user can now do that they couldn't before>
|
||||
|
||||
### Use case
|
||||
|
||||
<why this is useful, who benefits>
|
||||
|
||||
### Affected versions
|
||||
|
||||
<version>
|
||||
```
|
||||
|
||||
### 4. Create the issue
|
||||
|
||||
Write the body to a temp file to avoid shell quoting issues:
|
||||
|
||||
```bash
|
||||
cat > /tmp/issue-body.md << 'ISSUE_BODY'
|
||||
<body content here>
|
||||
ISSUE_BODY
|
||||
```
|
||||
|
||||
Create:
|
||||
|
||||
```bash
|
||||
gh issue create \
|
||||
--repo penpot/penpot \
|
||||
--title "<Title>" \
|
||||
--label "community contribution" \ # only if PR has this label
|
||||
--milestone "<milestone>" \
|
||||
--project "Main" \
|
||||
--body-file /tmp/issue-body.md
|
||||
```
|
||||
|
||||
Output: `https://github.com/penpot/penpot/issues/<NUMBER>`
|
||||
|
||||
### 5. Assign to the PR author
|
||||
|
||||
Assign the issue to the PR author so they're responsible for it:
|
||||
|
||||
```bash
|
||||
AUTHOR=$(gh pr view <PR_NUMBER> --repo penpot/penpot --json author --jq '.author.login')
|
||||
gh issue edit <ISSUE_NUMBER> --repo penpot/penpot --add-assignee "$AUTHOR"
|
||||
```
|
||||
|
||||
### 6. Set the Issue Type
|
||||
|
||||
`gh issue create` can't set the Issue Type directly. Use GraphQL.
|
||||
|
||||
Get the issue's GraphQL node ID:
|
||||
|
||||
```bash
|
||||
ISSUE_ID=$(gh api graphql -f query='
|
||||
query { repository(owner: "penpot", name: "penpot") {
|
||||
issue(number: <ISSUE_NUMBER>) { id }
|
||||
}}' --jq '.data.repository.issue.id')
|
||||
```
|
||||
|
||||
Issue Type IDs for the Penpot repo:
|
||||
|
||||
| Type | ID |
|
||||
|------|----|
|
||||
| Bug | `IT_kwDOAcyBPM4AX5Nb` |
|
||||
| Enhancement | `IT_kwDOAcyBPM4B_IQN` |
|
||||
| Feature | `IT_kwDOAcyBPM4AX5Nf` |
|
||||
| Task | `IT_kwDOAcyBPM4AX5NY` |
|
||||
| Question | `IT_kwDOAcyBPM4B_IQj` |
|
||||
| Docs | `IT_kwDOAcyBPM4B_IQz` |
|
||||
|
||||
Set it:
|
||||
|
||||
```bash
|
||||
gh api graphql -f query='
|
||||
mutation {
|
||||
updateIssue(input: {
|
||||
id: "'"$ISSUE_ID"'"
|
||||
issueTypeId: "<TYPE_ID>"
|
||||
}) {
|
||||
issue { number issueType { name } }
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### 7. Verify
|
||||
|
||||
```bash
|
||||
gh issue view <ISSUE_NUMBER> --repo penpot/penpot \
|
||||
--json title,milestone,projectItems,labels \
|
||||
--jq '{title, milestone: .milestone.title, projects: [.projectItems[].title], labels: [.labels[].name]}'
|
||||
|
||||
gh api graphql -f query='
|
||||
query { repository(owner: "penpot", name: "penpot") {
|
||||
issue(number: <ISSUE_NUMBER>) { issueType { name } }
|
||||
}}' --jq '.data.repository.issue.issueType.name'
|
||||
```
|
||||
|
||||
### 8. Link the PR to the issue
|
||||
|
||||
Append `Closes #<ISSUE_NUMBER>` to the PR body:
|
||||
|
||||
```bash
|
||||
gh pr view <PR_NUMBER> --repo penpot/penpot --json body --jq '.body' > /tmp/pr-body.md
|
||||
printf "\n\nCloses #<ISSUE_NUMBER>\n" >> /tmp/pr-body.md
|
||||
gh pr edit <PR_NUMBER> --repo penpot/penpot --body-file /tmp/pr-body.md
|
||||
|
||||
# Verify
|
||||
gh pr view <PR_NUMBER> --repo penpot/penpot --json body \
|
||||
--jq '.body | test("Closes #<ISSUE_NUMBER>")'
|
||||
```
|
||||
|
||||
**Note:** If the PR is already merged, `Closes` won't auto-close the issue
|
||||
— it only creates the "Development" sidebar link. This is the desired
|
||||
behavior since the issue is a tracking artifact.
|
||||
|
||||
### 9. Clean up
|
||||
|
||||
```bash
|
||||
rm -f /tmp/issue-body.md /tmp/pr-body.md
|
||||
```
|
||||
|
||||
## Label rules
|
||||
|
||||
| PR has | Issue gets |
|
||||
|--------|-----------|
|
||||
| `community contribution` | `community contribution` |
|
||||
| `bug`, `enhancement` | *(skip — redundant with Issue Type)* |
|
||||
| `backport candidate` | *(skip — workflow label)* |
|
||||
| `team-qa` | *(skip — workflow label)* |
|
||||
|
||||
## Issue Type mapping
|
||||
|
||||
| PR label(s) / title prefix | Issue Type |
|
||||
|----------------------------|-----------|
|
||||
| `bug` or `:bug:` | Bug |
|
||||
| `enhancement` or `:sparkles:` or `:tada:` | Enhancement |
|
||||
| Feature / epic | Feature |
|
||||
| Documentation | Docs |
|
||||
| None of the above | Task |
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Issue = WHAT, PR = HOW.** Never put implementation details in the
|
||||
issue body. The issue is for users, QA, and changelog readers.
|
||||
- **Copy the milestone from the PR.** Don't guess based on branch names.
|
||||
If the PR has no milestone, create the issue without one.
|
||||
- **Set Issue Type via GraphQL** — `gh issue create` can't set it.
|
||||
- **Link via PR body** — `Closes #<NUMBER>` creates the "Development"
|
||||
sidebar link automatically.
|
||||
- **One issue per PR** — even if a PR fixes multiple things, create a
|
||||
single issue that summarizes the overall change.
|
||||
- **Community attribution:** if the PR has the `community contribution`
|
||||
label or the author is not a core team member, add the label to the issue.
|
||||
123
.opencode/skills/issue-title/SKILL.md
Normal file
123
.opencode/skills/issue-title/SKILL.md
Normal file
@ -0,0 +1,123 @@
|
||||
---
|
||||
name: issue-title
|
||||
description: Derive a clear, well-formatted title for a GitHub issue from its description body, using descriptive present-tense for bugs and imperative mood for features, always including the "where" (location in the UI/module).
|
||||
---
|
||||
|
||||
# Skill: issue-title
|
||||
|
||||
Derive a concise, descriptive title for a GitHub issue based on its body
|
||||
content. Use **descriptive present tense for bugs** (e.g. "Plugin API
|
||||
crashes when setting text fills") and **imperative mood for features** (e.g.
|
||||
"Add customizable dash and gap controls"). No emoji or type prefixes
|
||||
(`feat:`, `bug:`, `feature:`, etc.).
|
||||
|
||||
Can be used both when **creating a new issue** and when **updating an
|
||||
existing one** that has a vague or outdated title.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Creating a new issue and need a well-formatted title from the draft body
|
||||
- An existing issue has a vague, outdated, or auto-generated title (e.g.
|
||||
`[PENPOT FEEDBACK]: ...`, `feature: ...`)
|
||||
- The current title doesn't reflect the actual content of the description
|
||||
- The title is missing the "where" (which part of the UI/module is affected)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Get the issue body
|
||||
|
||||
For an **existing issue**, fetch it:
|
||||
|
||||
```bash
|
||||
gh issue view <NUMBER> --repo penpot/penpot --json title,body
|
||||
```
|
||||
|
||||
For a **new issue**, read the draft body from wherever it was provided
|
||||
(Taiga link, user report, discussion, etc.).
|
||||
|
||||
### 2. Read the body and derive a title
|
||||
|
||||
Extract the core problem or request from the description. Distinguish between
|
||||
bug reports and feature requests:
|
||||
|
||||
**Bug titles (descriptive, present tense):**
|
||||
Describe the symptom as it appears to the user. Format:
|
||||
`[Where] [present-tense verb] when [condition]`
|
||||
|
||||
- *"Plugin API crashes when setting text fills"*
|
||||
- *"Canvas renders glitches when zooming quickly"*
|
||||
- *"French Canada locale falls back to French (fr) translations"*
|
||||
- *"Text layer content is not deleted when WebGL render is enabled"*
|
||||
|
||||
Do **not** start bug titles with "Fix" or any imperative verb. The title
|
||||
should state what's broken, not command a fix.
|
||||
|
||||
**Feature / Enhancement titles (imperative mood):**
|
||||
Command what should be built. Format:
|
||||
`[Imperative verb] [what] in/on [where]`
|
||||
|
||||
- *"Add customizable dash and gap length controls to dashed strokes in the sidebar"*
|
||||
- *"Show user, timestamp, and hash in the workspace history panel like git commits"*
|
||||
- *"Validate shape on add-object to catch malformed inputs early"*
|
||||
|
||||
**Universal rules (both types):**
|
||||
- **Include the "where"** — specify the UI location or module (e.g.
|
||||
"in the sidebar", "in the workspace history panel", "on the stroke
|
||||
options")
|
||||
- **No prefixes** — strip `bug:`, `feature:`, `feat:`, `:bug:`, `:sparkles:`,
|
||||
`[PENPOT FEEDBACK]`, etc.
|
||||
- **No emoji** — plain text only
|
||||
- **Be specific** — prefer concrete detail over generality. If the
|
||||
description mentions two related problems, capture both.
|
||||
|
||||
**Examples:**
|
||||
|
||||
| Original / draft title | Type | New title |
|
||||
|---|---|---|
|
||||
| `[PENPOT FEEDBACK]: WebGL` | Bug | `Canvas renders glitches when zooming quickly — text appears distorted and nodes have background-colored rectangles` |
|
||||
| `bug: flatten-nested-tokens-json uses $type instead of $value as the DTCG token/group discriminator` | Bug | `Token import fails when group-level type inheritance is used — parser misidentifies groups as tokens` |
|
||||
| `feature: Dashed stroke customization` | Feature | `Add customizable dash and gap length controls to dashed strokes in the sidebar` |
|
||||
| `feature: Add more detail to history of actions` | Feature | `Show user, timestamp, and hash in the workspace history panel like git commits` |
|
||||
|
||||
### 3. Apply the title
|
||||
|
||||
**If updating an existing issue:**
|
||||
|
||||
```bash
|
||||
gh issue edit <NUMBER> --repo penpot/penpot --title "<NEW TITLE>"
|
||||
```
|
||||
|
||||
**If creating a new issue:**
|
||||
|
||||
```bash
|
||||
gh issue create --repo penpot/penpot --title "<NEW TITLE>" --body "<BODY>"
|
||||
```
|
||||
|
||||
### 4. Confirm
|
||||
|
||||
For updates, the command returns the issue URL. Verify by optionally fetching
|
||||
again:
|
||||
|
||||
```bash
|
||||
gh issue view <NUMBER> --repo penpot/penpot --json title
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Bug titles describe the symptom** — present tense, 3rd person:
|
||||
"crashes", "fails", "shows", "is cut off", "does not load". Do not
|
||||
start with "Fix" or "Bug:".
|
||||
- **Feature titles use imperative mood** — command form: "Add", "Show",
|
||||
"Use", "Validate", "Support", "Toggle".
|
||||
- **Always include the "where"** — a title like "Crashes when zooming"
|
||||
is too vague; "Canvas crashes when zooming quickly" is clear.
|
||||
- **No prefixes, no emoji** — strip all type labels and decorative
|
||||
characters from the title.
|
||||
- **Derive from the body, not the current title** — the body contains
|
||||
the real detail; the current title may be auto-generated or stale.
|
||||
- **Two problems → cover both** — if the description has two distinct
|
||||
but related issues, capture both in the title joined by "and".
|
||||
@ -9,11 +9,6 @@ metadata: {"clawdbot":{"emoji":"🔍","requires":{"bins":["jq"]},"install":[{"id
|
||||
|
||||
Process, filter, and transform JSON data with jq.
|
||||
|
||||
## When to use
|
||||
|
||||
- Parsing, filtering, or transforming JSON from commands, files, or API
|
||||
responses — slicing, reshaping, or validating JSON output.
|
||||
|
||||
## Quick Examples
|
||||
|
||||
### Basic filtering
|
||||
@ -1,7 +1,12 @@
|
||||
---
|
||||
name: nrepl-eval
|
||||
description: Evaluate Clojure code via nREPL using the standalone tools/nrepl-eval.mjs CLI tool.
|
||||
---
|
||||
|
||||
# nREPL Eval
|
||||
|
||||
Evaluate Clojure (or ClojureScript) code via a running nREPL server using
|
||||
`scripts/nrepl-eval.mjs` — a standalone CLI application.
|
||||
`tools/nrepl-eval.mjs` — a standalone CLI application.
|
||||
|
||||
Session state (defs, in-ns, etc.) persists across invocations via a stored
|
||||
session ID, so you can build up state incrementally.
|
||||
@ -9,17 +14,18 @@ session ID, so you can build up state incrementally.
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
node scripts/nrepl-eval.mjs [options] [<code>]
|
||||
# or
|
||||
./scripts/nrepl-eval.mjs [options] [<code>]
|
||||
node tools/nrepl-eval.mjs [options] [<code>]
|
||||
```
|
||||
|
||||
The tool is also executable directly:
|
||||
```bash
|
||||
./tools/nrepl-eval.mjs [options] [<code>]
|
||||
```
|
||||
|
||||
## Options
|
||||
|
||||
| Flag | Description | Default |
|
||||
|------|-------------|---------|
|
||||
| `--backend` | Connect to backend nREPL (port 6064) | — |
|
||||
| `--frontend` | Connect to frontend nREPL (port 3447) | — |
|
||||
| `-p, --port PORT` | nREPL server port | `6064` |
|
||||
| `-H, --host HOST` | nREPL server host | `127.0.0.1` |
|
||||
| `-t, --timeout MS` | Timeout in milliseconds | `120000` |
|
||||
@ -27,11 +33,10 @@ node scripts/nrepl-eval.mjs [options] [<code>]
|
||||
| `-e, --last-error` | Evaluate `*e` to retrieve the last exception | — |
|
||||
| `-h, --help` | Show help message | — |
|
||||
|
||||
- `--backend` and `--frontend` are mutually exclusive.
|
||||
- Explicit `--port` is overridden when `--backend`/`--frontend` is used.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this tool when you need to:
|
||||
|
||||
1. **Evaluate Clojure code** during development — test functions, inspect
|
||||
state, or run experiments against a running Clojure process.
|
||||
2. **Verify that edited files compile** — require namespaces with `:reload`
|
||||
@ -41,43 +46,33 @@ node scripts/nrepl-eval.mjs [options] [<code>]
|
||||
|
||||
## Workflow
|
||||
|
||||
### Session management
|
||||
### 1. Session management
|
||||
|
||||
Sessions are persisted to `/tmp/penpot-nrepl-session-<host>-<port>`. State
|
||||
carries across calls automatically:
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs '(def x 42)'
|
||||
./scripts/nrepl-eval.mjs 'x'
|
||||
./tools/nrepl-eval.mjs '(def x 42)'
|
||||
./tools/nrepl-eval.mjs 'x'
|
||||
# => 42
|
||||
```
|
||||
|
||||
Reset the session to start fresh:
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs --reset-session '(def x 0)'
|
||||
./tools/nrepl-eval.mjs --reset-session '(def x 0)'
|
||||
```
|
||||
|
||||
### Evaluate code
|
||||
### 2. Evaluate code
|
||||
|
||||
**Single expression (inline) — uses default port 6064:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs '(+ 1 2 3)'
|
||||
```
|
||||
|
||||
**Backend nREPL (explicit):**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs --backend '(+ 1 2 3)'
|
||||
```
|
||||
|
||||
**Frontend nREPL:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs --frontend '(js/alert "hi")'
|
||||
./tools/nrepl-eval.mjs '(+ 1 2 3)'
|
||||
```
|
||||
|
||||
**Multiple expressions via heredoc (recommended — avoids escaping issues):**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs <<'EOF'
|
||||
./tools/nrepl-eval.mjs <<'EOF'
|
||||
(def x 10)
|
||||
(+ x 20)
|
||||
EOF
|
||||
@ -85,61 +80,38 @@ EOF
|
||||
|
||||
**Override with a different port:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs -p 7888 '(+ 1 2 3)'
|
||||
./tools/nrepl-eval.mjs -p 7888 '(+ 1 2 3)'
|
||||
```
|
||||
|
||||
### Inspect last exception
|
||||
### 3. Inspect last exception
|
||||
|
||||
After code throws an error, retrieve the full exception details:
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs -e
|
||||
./tools/nrepl-eval.mjs -e
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
**Require a namespace with reload:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs "(require '[my.namespace :as ns] :reload)"
|
||||
./tools/nrepl-eval.mjs "(require '[my.namespace :as ns] :reload)"
|
||||
```
|
||||
|
||||
**Test a function:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs "(ns/my-function arg1 arg2)"
|
||||
./tools/nrepl-eval.mjs "(ns/my-function arg1 arg2)"
|
||||
```
|
||||
|
||||
**Long-running operation with custom timeout:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs -t 300000 "(long-running-fn)"
|
||||
./tools/nrepl-eval.mjs -t 300000 "(long-running-fn)"
|
||||
```
|
||||
|
||||
### Accessing Private Functions
|
||||
|
||||
Private functions (declared with `^:private` or `defn-`) cannot be called
|
||||
directly from outside their namespace. Use the var quote syntax `#'` to
|
||||
access the underlying var:
|
||||
|
||||
**This fails:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs "(app.rpc.commands.error-reports/build-list-query {})"
|
||||
# => Syntax error: app.rpc.commands.error-reports/build-list-query is not public
|
||||
```
|
||||
|
||||
**This works:**
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs "(#'app.rpc.commands.error-reports/build-list-query {})"
|
||||
# => Returns the result
|
||||
```
|
||||
|
||||
The `#'` reader macro resolves to `(var ...)`, giving you direct access to
|
||||
the var regardless of its visibility modifier. The syntax is `#'` followed
|
||||
by the fully qualified symbol.
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Default port is 6064** — just pass code directly, no `-p` needed when
|
||||
your nREPL server is on 6064. Use `--backend` (6064) or `--frontend` (3447)
|
||||
as quick aliases. Use `-p <PORT>` for any other port.
|
||||
your nREPL server is on 6064. Use `-p <PORT>` for a different port.
|
||||
- **Always use `:reload`** when requiring namespaces to pick up file changes.
|
||||
- **Session is reused** across invocations — defs, in-ns, and var bindings
|
||||
persist. Use `--reset-session` to clear.
|
||||
@ -9,11 +9,6 @@ metadata: {"clawdbot":{"emoji":"🔎","requires":{"bins":["rg"]},"install":[{"id
|
||||
|
||||
Fast, smart recursive search. Respects `.gitignore` by default.
|
||||
|
||||
## When to use
|
||||
|
||||
- Searching file contents across the repo for regex patterns — the
|
||||
default code search, respects `.gitignore`.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Basic search
|
||||
@ -67,7 +62,7 @@ rg "user_\d+"
|
||||
rg -F "function()"
|
||||
|
||||
# Multiline search
|
||||
rg -U '(?s)start.*end'
|
||||
rg -U "start.*end"
|
||||
|
||||
# Only show matches, not lines
|
||||
rg -o "https?://[^\s]+"
|
||||
110
.opencode/skills/taiga/SKILL.md
Normal file
110
.opencode/skills/taiga/SKILL.md
Normal file
@ -0,0 +1,110 @@
|
||||
---
|
||||
name: taiga
|
||||
description: Fetch information from Taiga public API for the Penpot project (id 345963) — issues, user stories, and tasks, without authentication.
|
||||
metadata: {"clawdbot":{"requires":{"bins":["python3"]}}}
|
||||
---
|
||||
|
||||
# Taiga API Skill
|
||||
|
||||
Fetch information from Taiga public API for the **Penpot** project
|
||||
(project id: `345963`, slug: `penpot`).
|
||||
|
||||
**No authentication required** — only public project data is accessed.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `python3` — the `tools/taiga.py` CLI script is self-contained (stdlib only)
|
||||
|
||||
## Quick Start
|
||||
|
||||
The easiest way is to use the bundled Python script:
|
||||
|
||||
```bash
|
||||
# Pass a Taiga URL directly
|
||||
python3 tools/taiga.py https://tree.taiga.io/project/penpot/issue/13714
|
||||
|
||||
# Or use "<type> <ref>" syntax
|
||||
python3 tools/taiga.py us 14128
|
||||
python3 tools/taiga.py task 13648
|
||||
|
||||
# Add --json for raw output
|
||||
python3 tools/taiga.py --json issue 13714
|
||||
|
||||
# See full usage
|
||||
python3 tools/taiga.py --help
|
||||
```
|
||||
|
||||
## URL Pattern Reference
|
||||
|
||||
Taiga web URLs follow these patterns:
|
||||
|
||||
| Type | Web URL Pattern |
|
||||
|------|----------------|
|
||||
| Issue | `https://tree.taiga.io/project/penpot/issue/<REF>` |
|
||||
| User Story | `https://tree.taiga.io/project/penpot/us/<REF>` |
|
||||
| Task | `https://tree.taiga.io/project/penpot/task/<REF>` |
|
||||
|
||||
To extract the **type** and **ref** from a URL:
|
||||
- `issue/13714` → type=`issue`, ref=`13714`
|
||||
- `us/14128` → type=`us`, ref=`14128`
|
||||
- `task/13648` → type=`task`, ref=`13648`
|
||||
|
||||
## Python Script Reference
|
||||
|
||||
The `tools/taiga.py` script wraps the Taiga API into a single convenient CLI
|
||||
with sensible defaults.
|
||||
|
||||
### Usage
|
||||
|
||||
```
|
||||
python3 tools/taiga.py <taiga-url>
|
||||
python3 tools/taiga.py <type> <ref>
|
||||
python3 tools/taiga.py [--json] <taiga-url>
|
||||
python3 tools/taiga.py [--json] <type> <ref>
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# By URL (recommended — no need to think about type/ref)
|
||||
python3 tools/taiga.py https://tree.taiga.io/project/penpot/issue/13714
|
||||
|
||||
# By type and ref
|
||||
python3 tools/taiga.py us 14128
|
||||
python3 tools/taiga.py task 13648
|
||||
|
||||
# Raw JSON output
|
||||
python3 tools/taiga.py --json issue 13714
|
||||
```
|
||||
|
||||
### Output
|
||||
|
||||
The script prints a clean, structured summary:
|
||||
|
||||
```text
|
||||
User Story #11964 — 🔴 [DESIGN TOKENS] Typography Composite Input
|
||||
================================
|
||||
Status: Defining
|
||||
Milestone: design-systems-sprint-26
|
||||
Points: 3 role(s)
|
||||
Assignee: Natacha Menjibar
|
||||
Author: Natacha Menjibar
|
||||
Created: 2025-09-01
|
||||
Tags: iop-design-tokens
|
||||
URL: https://tree.taiga.io/project/penpot/us/11964
|
||||
================================
|
||||
<full description text, unmodified>
|
||||
```
|
||||
|
||||
The fields section includes type-specific information:
|
||||
- **Issues:** Status, Type ID, Severity ID, Priority ID
|
||||
- **User Stories:** Status, Milestone, Points
|
||||
- **Tasks:** Status, Milestone, Parent US
|
||||
|
||||
## Reference
|
||||
|
||||
- API docs: https://docs.taiga.io/api.html
|
||||
- Taiga instance: https://tree.taiga.io
|
||||
- API base: https://api.taiga.io/api/v1
|
||||
- Penpot project id: `345963`
|
||||
- Penpot project slug: `penpot`
|
||||
630
.opencode/skills/update-changelog/SKILL.md
Normal file
630
.opencode/skills/update-changelog/SKILL.md
Normal file
@ -0,0 +1,630 @@
|
||||
---
|
||||
name: update-changelog
|
||||
description: Update the project CHANGES.md with issues from a given GitHub milestone, with correct categorization and references.
|
||||
---
|
||||
|
||||
# Skill: update-changelog
|
||||
|
||||
Update `CHANGES.md` with entries for all issues and PRs in a given GitHub
|
||||
milestone. Each entry references the user-facing issue (not the PR) as the
|
||||
primary link, with the fix PR inline on the same line.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Before a new release, to populate the changelog with all fixed issues
|
||||
- When new issues are added to an existing milestone and the changelog needs
|
||||
to be refreshed
|
||||
- To ensure every entry follows the correct format for the changelog
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `gh` CLI authenticated (`gh auth status`)
|
||||
- Python 3.8+
|
||||
- `tools/gh.py` helper script available
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Determine the target version
|
||||
|
||||
The version is typically a semver string like `2.15.3`. Confirm with the user
|
||||
if not specified.
|
||||
|
||||
### 2. Fetch all issues in the milestone
|
||||
|
||||
Use the helper script. It uses GraphQL for efficient single-pass fetching
|
||||
(closing PRs are included in the same query — no N+1):
|
||||
|
||||
```bash
|
||||
# All closed issues (default)
|
||||
python3 tools/gh.py issues "2.16.0"
|
||||
|
||||
# Include open issues too
|
||||
python3 tools/gh.py issues "2.16.0" --state all
|
||||
|
||||
# Exclude entries that should not go in the changelog
|
||||
python3 tools/gh.py issues "2.16.0" --exclude "release blocker,no changelog"
|
||||
```
|
||||
|
||||
**Exclusion rules (issue-level):**
|
||||
- `no changelog` label — Chore/refactor work that doesn't need a changelog entry
|
||||
- `release blocker` label — Blocked issues not yet ready for changelog
|
||||
- `Task` issue type — Internal chores are not user-facing; automatically excluded by `gh.py`. Use `--include-tasks` to override.
|
||||
- **Rejected project status** — Issues with a "Rejected" status in the "Main" project board are automatically excluded by `gh.py`. This project-level status (independent of the GitHub issue `state`) indicates the issue was rejected from the release. Use `--include-rejected` to override.
|
||||
|
||||
**Exclusion rules (PR-level):**
|
||||
In addition to issue-level exclusions, PRs with these labels should be
|
||||
excluded regardless of their linked issue's labels:
|
||||
- `release blocker` — PR is part of a pending release blocker batch
|
||||
- `no issue required` — Trivial fix not tracked as an issue
|
||||
|
||||
The script outputs JSON with each entry containing `number`, `title`, `state`,
|
||||
`issue_type`, `labels`, `closing_prs` (the PRs that fix each issue), and
|
||||
`project_status` (the "Main" project board status, e.g. "Done", "Rejected",
|
||||
or `null` if not tracked in a project).
|
||||
|
||||
### 3. Identify missing entries (optional)
|
||||
|
||||
If updating from an existing `CHANGES.md`, find issues in the milestone that
|
||||
are NOT yet referenced in the changelog:
|
||||
|
||||
```bash
|
||||
python3 tools/gh.py issues "2.16.0" --exclude "release blocker,no changelog" --compare CHANGES.md
|
||||
```
|
||||
|
||||
This returns a filtered JSON array with only the missing issues.
|
||||
|
||||
> **Note:** The `--compare` flag checks **issues** only (via issue number
|
||||
> references in the changelog). To find merged **PRs** not yet referenced,
|
||||
> use the milestone PR cross-reference described in step 10 below.
|
||||
|
||||
### 4. Fetch additional PR details when needed
|
||||
|
||||
When you need more context for specific PRs (e.g. to find the PR author for
|
||||
community contribution attribution, or to read the PR body for
|
||||
"Fixes/Closes #NNN" patterns):
|
||||
|
||||
```bash
|
||||
# One or more PR numbers
|
||||
python3 tools/gh.py prs 9179 9204 9311
|
||||
|
||||
# From a file
|
||||
python3 tools/gh.py prs --file prs.txt
|
||||
|
||||
# From stdin
|
||||
cat prs.txt | python3 tools/gh.py prs --stdin
|
||||
```
|
||||
|
||||
The `prs` command also supports listing all PRs in a milestone in one call:
|
||||
|
||||
```bash
|
||||
# All merged PRs in a milestone (default)
|
||||
python3 tools/gh.py prs --milestone "2.16.0"
|
||||
|
||||
# All states (merged, open, closed)
|
||||
python3 tools/gh.py prs --milestone "2.16.0" --state all
|
||||
```
|
||||
|
||||
The `prs` command returns JSON with `number`, `title`, `body`, `state`,
|
||||
`merged_at`, `author`, `labels`, and `closing_issues`. PRs are fetched in
|
||||
batches of 50 via GraphQL to stay within API limits (milestone mode uses
|
||||
paginated GraphQL on the milestone's `pullRequests` connection).
|
||||
|
||||
You can also list all PRs in a milestone in a single call:
|
||||
|
||||
```bash
|
||||
# All merged PRs in a milestone (default)
|
||||
python3 tools/gh.py prs --milestone "2.16.0"
|
||||
|
||||
# All states (merged, open, closed)
|
||||
python3 tools/gh.py prs --milestone "2.16.0" --state all
|
||||
|
||||
# Open PRs only
|
||||
python3 tools/gh.py prs --milestone "2.16.0" --state open
|
||||
```
|
||||
|
||||
The milestone path uses paginated GraphQL on the milestone's `pullRequests`
|
||||
connection (100 per page), avoiding one-by-one fetches.
|
||||
|
||||
### 5. Categorize entries — strictly by issue type, never by labels or emoji
|
||||
|
||||
Use the **Issue Type** field (GitHub's native issue type, exposed as
|
||||
`issue_type` in the `gh.py` JSON output) to determine which section an entry
|
||||
belongs to.
|
||||
|
||||
> **⚠️ CRITICAL: Never use labels or title emoji prefixes for categorization.**
|
||||
> Labels like `bug` and `enhancement`, as well as title prefixes like `:bug:`
|
||||
> and `:sparkles:`, are frequently inaccurate, missing, or contradictory to the
|
||||
> actual issue type. The `issue_type` field from `gh.py` is the single source
|
||||
> of truth.
|
||||
|
||||
| `issue_type` value | Changelog section |
|
||||
|--------------------|-------------------|
|
||||
| `Bug` | `### :bug: Bugs fixed` |
|
||||
| `Feature` or `Enhancement` | `### :sparkles: New features & Enhancements` |
|
||||
| `Task` | **Exclude** — internal chores are not user-facing |
|
||||
| `null` (not set) | Check labels as a fallback: `bug` label → bugs, otherwise enhancements |
|
||||
|
||||
The `gh.py` issues command already includes `issue_type` in every entry's
|
||||
output. **No separate GraphQL query is needed.**
|
||||
|
||||
**Community contribution attribution:** If the issue or its fix PR has the
|
||||
`community contribution` label, add an attribution `(by @<github_username>)`
|
||||
on the changelog entry line, **before** the GitHub issue/PR references.
|
||||
|
||||
The attribution should reference the **PR author**, not the issue author.
|
||||
The `prs` subcommand includes the `author` field — use that:
|
||||
|
||||
```bash
|
||||
python3 tools/gh.py prs <PR_NUMBER> | python3 -c "import sys,json; print(json.load(sys.stdin)[0]['author'])"
|
||||
```
|
||||
|
||||
Placement in the entry line:
|
||||
```markdown
|
||||
- Fix description of the bug (by @username) [#<ISSUE>](...) (PR: [#<PR>](...))
|
||||
```
|
||||
|
||||
**Only closed issues are included.** An issue must have `state: "closed"` to
|
||||
appear in the changelog. Open/unresolved issues are omitted, even if they are
|
||||
tracked in the milestone.
|
||||
|
||||
**Pairing rules:**
|
||||
|
||||
| Pattern | Changelog format |
|
||||
|---------|-----------------|
|
||||
| Closed issue + one or more PRs fix it | Primary link = issue, PR inline comma-separated |
|
||||
| PR exists with no linked issue | If a corresponding closed issue exists in the same milestone, link the issue. Otherwise, skip the entry (the issue must be the changelog unit). |
|
||||
| Closed issue with no fix PR in milestone | Link the issue directly, without a PR reference. |
|
||||
|
||||
> **False-positive associations:** A PR may incorrectly claim to close an issue
|
||||
> from a different context (e.g., a very old PR referencing a modern issue, or a
|
||||
> cross-project reference). If the PR title and issue title are clearly unrelated,
|
||||
> or the PR was created years before the issue, treat it as a data glitch and
|
||||
> skip it. PR [#3](https://github.com/penpot/penpot/pull/3) (ancient License PR
|
||||
> claiming to close a plugin API issue) is a known example.
|
||||
|
||||
### 5a. ⚠️ Verify PR merge status before writing
|
||||
|
||||
A closed issue may list closing PRs that were **closed without merging**
|
||||
(e.g., a community PR that was superseded by another). The changelog must
|
||||
only reference **merged** PRs. Verify before writing:
|
||||
|
||||
```bash
|
||||
# Collect all PR numbers from the candidate entries and check them
|
||||
python3 tools/gh.py prs <ALL_PR_NUMBERS> | python3 -c "
|
||||
import json, sys
|
||||
for pr in json.load(sys.stdin):
|
||||
if pr['state'] != 'MERGED':
|
||||
print(f'WARNING: #{pr[\"number\"]} is {pr[\"state\"]} (not merged)')
|
||||
"
|
||||
```
|
||||
|
||||
If a closing PR is closed-unmerged, find the actual merged PR that
|
||||
superseded it:
|
||||
1. Check the issue's closing PRs list for other PRs (there may be multiple)
|
||||
2. Look for other PRs with similar titles or descriptions referencing the same issue
|
||||
3. Inspect the closed PR's conversation timeline for a pointer to the replacement
|
||||
|
||||
Replace the reference in the changelog entry with the correct merged PR number.
|
||||
|
||||
### 6. Read the current CHANGES.md
|
||||
|
||||
Read the top of `CHANGES.md` to understand the existing format and find the
|
||||
insertion point (newest version goes at the top, after the `# CHANGELOG`
|
||||
header).
|
||||
|
||||
Key format rules from the existing file:
|
||||
|
||||
```markdown
|
||||
## <VERSION>
|
||||
|
||||
### :bug: Bugs fixed
|
||||
|
||||
- Fix description of the bug [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
|
||||
- Fix another bug (by @contributor) [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
|
||||
|
||||
### :sparkles: New features & Enhancements
|
||||
|
||||
- Add new feature description [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
|
||||
```
|
||||
|
||||
Format details:
|
||||
- Entries start with `- ` followed by a short description in imperative mood
|
||||
- Primary link is **always the issue** (user-facing artifact)
|
||||
- PR references are inline on the same line: `(PR: [#<N>](<url>))`
|
||||
If an issue has multiple fix PRs, they are comma-separated:
|
||||
`(PR: [#<N>](<url>), [#<M>](<url>))`
|
||||
- The description should describe the fix/feature from the user's perspective
|
||||
- Community contributions get `(by @<username>)` **before** the issue link
|
||||
- Sections are separated by a blank line between the last entry and the next
|
||||
section title
|
||||
- Only include a section if there are entries for it
|
||||
- When an entry already exists in an earlier version section, it must be removed
|
||||
from the current version to avoid duplicates
|
||||
|
||||
### 7. Build the description text
|
||||
|
||||
Derive the description from the issue title, not the PR title. Strip leading
|
||||
emoji prefixes (`:bug:`, `:sparkles:`, `:tada:`) and focus on the
|
||||
user-facing behavior.
|
||||
|
||||
Examples:
|
||||
|
||||
| Issue title | Changelog description |
|
||||
|-------------|----------------------|
|
||||
| `Plugin API token methods fail with schema validation error on PRO` | `Fix Plugin API token methods failing with schema validation error on PRO` |
|
||||
| `Comment content is not sanitized before rendering, enabling stored XSS` | `Sanitize comment content on rendering` |
|
||||
| `Custom uploaded font family names are not sanitized` | `Sanitize font family names on custom uploaded fonts` |
|
||||
|
||||
### 8. Insert the section into CHANGES.md
|
||||
|
||||
Insert the new version section right after the `# CHANGELOG` header (before
|
||||
the previous version entry). Use the `edit` tool with enough context to make
|
||||
a unique match.
|
||||
|
||||
### 9. Verify
|
||||
|
||||
Read the top of `CHANGES.md` and confirm:
|
||||
- The version header is correct
|
||||
- Every entry has a GitHub link
|
||||
- Entries with a fix PR have the PR sub-line
|
||||
- The section ordering is correct (newest first)
|
||||
- Formatting matches the surrounding entries
|
||||
|
||||
### 10. Cross-reference milestone PRs against the changelog
|
||||
|
||||
Issues can be fixed by PRs that aren't in the milestone, and merged PRs in
|
||||
the milestone may not close any tracked issue. After writing, run a full
|
||||
cross-reference to catch gaps:
|
||||
|
||||
```bash
|
||||
# List all merged PRs in the milestone
|
||||
python3 tools/gh.py prs --milestone "<MILESTONE>" --state merged > /tmp/milestone-prs.json
|
||||
|
||||
# Extract PR numbers from the changelog section
|
||||
python3 -c "
|
||||
import json, re
|
||||
|
||||
with open('CHANGES.md') as f:
|
||||
content = f.read()
|
||||
|
||||
# Extract the version section (adjust regex to match the actual version)
|
||||
match = re.search(r'## <MILESTONE> \(Unreleased\)\n(.*?)(?:\n## |\Z)', content, re.DOTALL)
|
||||
section = match.group(1)
|
||||
|
||||
# Collect all PR numbers referenced
|
||||
changelog_prs = set()
|
||||
for m in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section):
|
||||
changelog_prs.add(int(m))
|
||||
|
||||
# Collect all milestone PRs (filtered)
|
||||
with open('/tmp/milestone-prs.json') as f:
|
||||
milestone_prs = json.load(f)
|
||||
|
||||
milestone_merged = {pr['number'] for pr in milestone_prs}
|
||||
|
||||
# PRs in milestone but not in changelog
|
||||
missing = sorted(milestone_merged - changelog_prs)
|
||||
print(f'Milestone merged PRs: {len(milestone_merged)}')
|
||||
print(f'Changelog referenced PRs: {len(changelog_prs)}')
|
||||
print(f'PRs in milestone but NOT in changelog: {len(missing)}')
|
||||
for num in missing:
|
||||
pr = next(p for p in milestone_prs if p['number'] == num)
|
||||
print(f' #{num} {pr[\"title\"][:80]}')
|
||||
"
|
||||
```
|
||||
|
||||
For each missing PR found, decide whether it should be added to the
|
||||
changelog or is legitimately excluded (check its labels).
|
||||
|
||||
Also verify that no closed-unmerged PRs remain in the changelog:
|
||||
|
||||
```bash
|
||||
python3 tools/gh.py prs --milestone "<MILESTONE>" --state all | python3 -c "
|
||||
import json, sys
|
||||
data = json.load(sys.stdin)
|
||||
closed = [p for p in data if p['state'] == 'CLOSED']
|
||||
if closed:
|
||||
print('WARNING: CLOSED (unmerged) PRs in milestone:')
|
||||
for p in closed:
|
||||
print(f' #{p[\"number\"]} {p[\"title\"][:80]}')
|
||||
"
|
||||
```
|
||||
|
||||
**Post-edit audit checklist:**
|
||||
- ✅ All referenced PRs are merged (no closed-unmerged artifacts)
|
||||
- ✅ Every merged milestone PR is either in the changelog or excluded by label
|
||||
- ✅ PR and issue counts are internally consistent
|
||||
- ✅ No false-positive PR-to-issue associations
|
||||
|
||||
## Version section template
|
||||
|
||||
```markdown
|
||||
## <VERSION>
|
||||
|
||||
### :bug: Bugs fixed
|
||||
|
||||
- <fix description> [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
|
||||
- <fix description> (by @contributor) [#<ISSUE>](https://github.com/penpot/penpot/issues/<ISSUE>) (PR: [#<PR>](https://github.com/penpot/penpot/pull/<PR>))
|
||||
```
|
||||
|
||||
### 11. Generate anomaly report and save to CHANGES-ISSUES.md
|
||||
|
||||
After all edits and cross-referencing are complete, generate a structured
|
||||
anomaly report and save it to `CHANGES-ISSUES.md` (overwriting if exists).
|
||||
This provides a persistent record of any discrepancies between the milestone
|
||||
and the changelog.
|
||||
|
||||
Run this self-contained script:
|
||||
|
||||
```bash
|
||||
python3 << 'PYEOF'
|
||||
import json, re, subprocess, sys
|
||||
|
||||
MILESTONE = "<MILESTONE>"
|
||||
CHANGES_MD = "CHANGES.md"
|
||||
OUTPUT = "CHANGES-ISSUES.md"
|
||||
|
||||
# Fetch milestone issues (all states)
|
||||
result = subprocess.run(
|
||||
["python3", "tools/gh.py", "issues", MILESTONE, "--state", "all"],
|
||||
capture_output=True, text=True)
|
||||
all_issues = json.loads(result.stdout)
|
||||
issue_by_num = {i['number']: i for i in all_issues}
|
||||
|
||||
# Fetch milestone PRs (all states)
|
||||
result = subprocess.run(
|
||||
["python3", "tools/gh.py", "prs", "--milestone", MILESTONE, "--state", "all"],
|
||||
capture_output=True, text=True)
|
||||
all_prs = json.loads(result.stdout)
|
||||
pr_by_num = {p['number']: p for p in all_prs}
|
||||
|
||||
# Read changelog
|
||||
with open(CHANGES_MD) as f:
|
||||
content = f.read()
|
||||
|
||||
m = re.search(rf'## {MILESTONE} \(Unreleased\)\n(.*?)(?:\n## |\Z)', content, re.DOTALL)
|
||||
section = m.group(1) if m else ""
|
||||
|
||||
# Collect issue and PR references from the changelog section
|
||||
changelog_issues = set()
|
||||
for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/issues/\d+\)', section):
|
||||
changelog_issues.add(int(num))
|
||||
for num in re.findall(r'\[Github #(\d+)\]', section):
|
||||
changelog_issues.add(int(num))
|
||||
|
||||
changelog_prs = set()
|
||||
for num in re.findall(r'\[#(\d+)\]\(https://github\.com/penpot/penpot/pull/\d+\)', section):
|
||||
changelog_prs.add(int(num))
|
||||
for num in re.findall(r'PR:\[(\d+)\]', section):
|
||||
changelog_prs.add(int(num))
|
||||
|
||||
# Determine valid (non-excluded) milestone issues
|
||||
EXCLUDED_LABELS = {'release blocker', 'no changelog'}
|
||||
EXCLUDED_ISSUE_TYPES = {'Task'}
|
||||
EXCLUDED_PROJECT_STATUS = {'Rejected'}
|
||||
|
||||
valid_issues = []
|
||||
for issue in all_issues:
|
||||
labels = set(issue.get('labels', []))
|
||||
if issue.get('state') != 'CLOSED': continue
|
||||
if issue.get('issue_type') in EXCLUDED_ISSUE_TYPES: continue
|
||||
if issue.get('project_status') in EXCLUDED_PROJECT_STATUS: continue
|
||||
if EXCLUDED_LABELS & labels: continue
|
||||
valid_issues.append(issue)
|
||||
valid_nums = {i['number'] for i in valid_issues}
|
||||
|
||||
# --- Gather anomalies ---
|
||||
anomalies = []
|
||||
|
||||
# Type 1: Entries in changelog that should be excluded
|
||||
for num in sorted(changelog_issues):
|
||||
issue = issue_by_num.get(num)
|
||||
if issue is None:
|
||||
anomalies.append({
|
||||
'type': 'should_remove',
|
||||
'severity': 'HIGH',
|
||||
'number': num,
|
||||
'title': '',
|
||||
'reason': 'Issue not found in milestone (deleted or moved)'
|
||||
})
|
||||
continue
|
||||
labels = set(issue.get('labels', []))
|
||||
reasons = []
|
||||
if issue.get('state') != 'CLOSED':
|
||||
reasons.append(f'state is "{issue["state"]}" (should be CLOSED)')
|
||||
if 'release blocker' in labels:
|
||||
reasons.append('has "release blocker" label')
|
||||
if 'no changelog' in labels:
|
||||
reasons.append('has "no changelog" label')
|
||||
if issue.get('issue_type') == 'Task':
|
||||
reasons.append(f'issue_type is Task (internal chore)')
|
||||
if issue.get('project_status') == 'Rejected':
|
||||
reasons.append('project_status is Rejected')
|
||||
if reasons:
|
||||
anomalies.append({
|
||||
'type': 'should_remove',
|
||||
'severity': 'MEDIUM' if issue.get('issue_type') == 'Task' else 'HIGH',
|
||||
'number': num,
|
||||
'title': issue.get('title', '')[:80],
|
||||
'reason': '; '.join(reasons)
|
||||
})
|
||||
|
||||
# Type 2: Valid issues not in changelog
|
||||
for num in sorted(valid_nums - changelog_issues):
|
||||
issue = issue_by_num[num]
|
||||
info = {
|
||||
'type': 'missing',
|
||||
'severity': 'MEDIUM',
|
||||
'number': num,
|
||||
'title': issue['title'][:80],
|
||||
'issue_type': issue['issue_type'],
|
||||
'closing_prs': issue.get('closing_prs', []),
|
||||
'note': ''
|
||||
}
|
||||
# Check for duplicate (same PR as existing entry)
|
||||
existing = []
|
||||
for pr_num in issue.get('closing_prs', []):
|
||||
for cl_num in changelog_issues:
|
||||
cl_issue = issue_by_num.get(cl_num)
|
||||
if cl_issue and pr_num in cl_issue.get('closing_prs', []):
|
||||
existing.append(f'#{cl_num}')
|
||||
if existing:
|
||||
info['note'] = f'DUPLICATE: same PR as existing entry(ies): {", ".join(existing)}'
|
||||
# Check closing PRs not merged
|
||||
unmerged = []
|
||||
for pr_num in issue.get('closing_prs', []):
|
||||
pr = pr_by_num.get(pr_num)
|
||||
if pr is None:
|
||||
unmerged.append(f'#{pr_num} (unknown)')
|
||||
elif pr.get('state') != 'MERGED':
|
||||
unmerged.append(f'#{pr_num} (state={pr["state"]})')
|
||||
if unmerged:
|
||||
info['note'] = (info['note'] + '; ' if info['note'] else '') + f'Closing PRs not merged: {", ".join(unmerged)}'
|
||||
anomalies.append(info)
|
||||
|
||||
# Type 3: PRs in changelog that are not merged
|
||||
for pr_num in sorted(changelog_prs):
|
||||
pr = pr_by_num.get(pr_num)
|
||||
if pr is None:
|
||||
anomalies.append({
|
||||
'type': 'unmerged_pr',
|
||||
'severity': 'HIGH',
|
||||
'number': pr_num,
|
||||
'title': '',
|
||||
'reason': 'PR not found in milestone PR list'
|
||||
})
|
||||
elif pr.get('state') != 'MERGED':
|
||||
anomalies.append({
|
||||
'type': 'unmerged_pr',
|
||||
'severity': 'HIGH',
|
||||
'number': pr_num,
|
||||
'title': pr.get('title', '')[:80],
|
||||
'reason': f'state={pr["state"]} (should be MERGED)'
|
||||
})
|
||||
|
||||
# --- Write report to CHANGES-ISSUES.md ---
|
||||
with open(OUTPUT, 'w') as f:
|
||||
f.write(f'# Changelog Anomaly Report — {MILESTONE}\n\n')
|
||||
f.write(f'Generated: {__import__("datetime").datetime.now().strftime("%Y-%m-%d %H:%M UTC")}\n\n')
|
||||
f.write('---\n\n')
|
||||
|
||||
# Summary
|
||||
n_remove = sum(1 for a in anomalies if a['type'] == 'should_remove')
|
||||
n_missing = sum(1 for a in anomalies if a['type'] == 'missing')
|
||||
n_pr = sum(1 for a in anomalies if a['type'] == 'unmerged_pr')
|
||||
f.write(f'## Summary\n\n')
|
||||
f.write(f'- **Issues to remove from changelog:** {n_remove}\n')
|
||||
f.write(f'- **Valid issues missing from changelog:** {n_missing}\n')
|
||||
f.write(f'- **Unmerged PRs referenced:** {n_pr}\n')
|
||||
f.write(f'- **Total anomalies:** {len(anomalies)}\n\n')
|
||||
|
||||
if not anomalies:
|
||||
f.write('✅ No anomalies found. The changelog is fully consistent with the milestone.\n\n')
|
||||
else:
|
||||
# Type 1
|
||||
if n_remove:
|
||||
f.write(f'## Issues to Remove\n\n')
|
||||
f.write('These entries are in the changelog but should be excluded based on current issue metadata.\n\n')
|
||||
for a in anomalies:
|
||||
if a['type'] != 'should_remove': continue
|
||||
badge = '🔴' if a['severity'] == 'HIGH' else '🟡'
|
||||
f.write(f'{badge} **#{a["number"]}**')
|
||||
if a.get('title'): f.write(f' — {a["title"]}')
|
||||
f.write(f'\n - Reason: {a["reason"]}\n\n')
|
||||
|
||||
# Type 2
|
||||
if n_missing:
|
||||
f.write(f'## Valid Issues Not in Changelog\n\n')
|
||||
f.write('These issues are closed, non-excluded milestone items that lack a changelog entry.\n\n')
|
||||
for a in anomalies:
|
||||
if a['type'] != 'missing': continue
|
||||
f.write(f'❓ **#{a["number"]}** — {a["title"]}\n')
|
||||
f.write(f' - Type: {a["issue_type"]}, Closing PRs: {a["closing_prs"]}\n')
|
||||
if a.get('note'): f.write(f' - Note: {a["note"]}\n')
|
||||
f.write('\n')
|
||||
|
||||
# Type 3
|
||||
if n_pr:
|
||||
f.write(f'## Unmerged PRs Referenced in Changelog\n\n')
|
||||
f.write('These PR numbers appear in the changelog but are not merged.\n\n')
|
||||
for a in anomalies:
|
||||
if a['type'] != 'unmerged_pr': continue
|
||||
f.write(f'🔴 **#{a["number"]}**')
|
||||
if a.get('title'): f.write(f' — {a["title"]}')
|
||||
f.write(f'\n - {a["reason"]}\n\n')
|
||||
|
||||
# Appendix: counts
|
||||
f.write('---\n\n')
|
||||
f.write(f'## Context\n\n')
|
||||
f.write(f'- Milestone total issues (all states): {len(all_issues)}\n')
|
||||
f.write(f'- Valid issues after exclusions: {len(valid_issues)}\n')
|
||||
f.write(f'- Issues referenced in changelog: {len(changelog_issues)}\n')
|
||||
f.write(f'- PRs referenced in changelog: {len(changelog_prs)}\n')
|
||||
|
||||
print(f"Anomaly report written to {OUTPUT}")
|
||||
PYEOF
|
||||
```
|
||||
|
||||
This generates `CHANGES-ISSUES.md` with three sections:
|
||||
1. **Issues to Remove** — Entries in the changelog that should be excluded based
|
||||
on current issue metadata (labels, type, project status, or deletion).
|
||||
2. **Valid Issues Not in Changelog** — Closed, non-excluded milestone issues
|
||||
that lack a changelog entry (with notes on duplicates and unmerged closing PRs).
|
||||
3. **Unmerged PRs Referenced** — PRs in the changelog that are not merged.
|
||||
|
||||
The report is overwritten each time it's generated, reflecting the current
|
||||
state of the milestone and changelog.
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Issue = changelog unit.** The primary link always points to the
|
||||
user-facing issue, not the implementation PR.
|
||||
- **PR = implementation detail.** Reference the PR inline so readers
|
||||
can find the code changes.
|
||||
- **Latest version first.** New sections are inserted at the top of the
|
||||
changelog, below the `# CHANGELOG` header.
|
||||
- **Issue Type determines section — exclusively.** Use the `issue_type` field from `gh.py` output (Bug → `:bug:`, Feature/Enhancement → `:sparkles:`). **Do not** use labels (`bug`, `enhancement`) or title emoji prefixes (`:bug:`, `:sparkles:`) — they are frequently wrong or contradictory. The `issue_type` is the single source of truth.
|
||||
- **User-facing descriptions.** Write from the user's perspective — describe
|
||||
what broke and what was fixed, not internal implementation details.
|
||||
- **Community attribution.** When the issue or fix PR has the
|
||||
`community contribution` label, add `(by @<username>)` on the entry line
|
||||
between the description and the issue link. Use the **PR author** (not the
|
||||
issue author) for the attribution.
|
||||
- **Only closed issues.** An issue must have `state: "closed"` to appear in
|
||||
the changelog. Open/unresolved issues are omitted.
|
||||
- **Rejected project status.** Issues marked as "Rejected" in the "Main"
|
||||
project board are automatically excluded by `gh.py`, even if they are
|
||||
closed. The project status is distinct from the GitHub issue state.
|
||||
Use `--include-rejected` to override this behavior.
|
||||
- **Excluded issues.** Issues with `no changelog` label must be excluded.
|
||||
Issues with `issue_type: "Task"` must also be excluded — they are internal
|
||||
chores, not user-facing changes.
|
||||
- **Multiple PRs per issue.** If multiple PRs fix the same issue, list them
|
||||
comma-separated inline: `(PR: [#A](url), [#B](url))`.
|
||||
- **Duplicate removal.** If an entry already exists in a prior version section,
|
||||
remove it from the current version. Check for text-level duplicates (after
|
||||
stripping links and attributions) across version sections.
|
||||
- **Taiga references.** If a changelog entry references a Taiga URL
|
||||
(`tree.taiga.io`), attempt to find a corresponding GitHub issue via the
|
||||
Taiga description text or by searching GitHub PRs that reference the Taiga
|
||||
URL. Replace the Taiga reference with the GitHub issue link and add the PR
|
||||
reference if applicable.
|
||||
- **Re-fetch before editing.** Milestones can change — always re-fetch issues
|
||||
before making edits, don't rely on cached data.
|
||||
- **Use `tools/gh.py`.** Prefer the helper script over raw `gh api` calls for
|
||||
milestone issue listing and PR detail fetching. It handles GraphQL
|
||||
pagination, batching, and label filtering automatically.
|
||||
- **Verify PR merge status.** Not all closing PRs are merged — community PRs
|
||||
can be superseded and closed without merging. Always check that every PR
|
||||
referenced in the changelog has `state: MERGED`.
|
||||
- **PR-level exclusions apply.** A PR can carry its own exclusion labels
|
||||
(`release blocker`, `no issue required`) independent of its linked issue's
|
||||
labels. Check both.
|
||||
- **Cross-reference milestone PRs, not just issues.** The `--compare` flag on
|
||||
the `issues` command only compares issue numbers. Merged PRs not linked to
|
||||
any milestone issue can be missed. Use `python3 tools/gh.py prs --milestone`
|
||||
for a full PR cross-reference.
|
||||
- **False-positive PR-to-issue associations.** A PR may claim to close an
|
||||
issue from a different project or context. If the PR title and issue title
|
||||
are clearly unrelated, or the PR predates the issue by years, treat it as a
|
||||
data glitch and skip it.
|
||||
@ -1,32 +0,0 @@
|
||||
import assert from "node:assert/strict"
|
||||
import test from "node:test"
|
||||
|
||||
import plugin from "../plugins/penpot.js"
|
||||
|
||||
test("exports only the OpenCode V2 plugin contract", () => {
|
||||
assert.equal(plugin.id, "penpot")
|
||||
assert.equal("server" in plugin, false)
|
||||
})
|
||||
|
||||
test("registers the Penpot tools during setup", async () => {
|
||||
const tools = []
|
||||
const context = {
|
||||
location: { directory: "/tmp/opencode/penpot-plugin-test" },
|
||||
tool: {
|
||||
async transform(apply) {
|
||||
apply({
|
||||
add(tool) {
|
||||
tools.push(tool)
|
||||
},
|
||||
})
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
await plugin.setup(context)
|
||||
|
||||
assert.deepEqual(
|
||||
tools.map((tool) => tool.name).sort(),
|
||||
["paren-repair", "penpot-psql"],
|
||||
)
|
||||
})
|
||||
@ -1,342 +0,0 @@
|
||||
# Audit Log and event processing
|
||||
|
||||
Complete guide to the Penpot event system: what gets recorded, where each piece lives, and how events travel from the moment they happen until they are archived. Start with the overview (sections 1-3); the deeper technical detail comes later (section 8 on). If you only want the idea, the first three sections are enough.
|
||||
|
||||
> Note about this memory: unlike other Serena memories (dense notes written for LLM agents, see `mem:memory-maintenance`), this document is also written for human developers who need full context. That is why it uses plain language and examples. It replaces the old `mem:backend/audit-log`, which is now just a redirect stub.
|
||||
|
||||
## 1. What it is (2-minute overview)
|
||||
|
||||
The *audit log* answers "who did what, when, and from where". Every important user action becomes an **event**: a small object with a name, a type, who did it, from which IP, plus action data (`props`) and metadata (`context`).
|
||||
|
||||
The system has:
|
||||
|
||||
- **Three producers** (they create events): the Penpot backend (RPC layer), the Penpot frontend (browser app), and the Nitrate admin-console, which sends both server-side and client-side events.
|
||||
- **One central buffer**: the Postgres `audit_log` table in Penpot. It is temporary: it holds events for minutes/hours until they are archived, then they are deleted.
|
||||
- **Four consumers** (they read events): webhooks, error reporters, telemetry, and the external archives in Nexus and in the admin console.
|
||||
- **Two long-term archives**: **Nexus**, a separate service that stores events for the long term, and the **admin console** (Nitrate), which ingests its own allowlisted events through a second archive job (see section 12).
|
||||
|
||||
```
|
||||
Penpot frontend ─ POST push-audit-events (public) ─────────▶┐
|
||||
Nitrate admin-console (browser) ─ POST push-audit-events ──▶│
|
||||
Nitrate admin-console (server) ─ POST push-audit-events ──▶│ ─▶ audit_log table (buffer) ──┬─ cron 0s every 5m ──▶ Nexus (archive)
|
||||
Penpot backend (RPC wrap-audit) ─ submit ───────────────────┘ │ └─ cron 30s every 5m ─▶ admin console (archive)
|
||||
│ ├──▶ webhooks
|
||||
│ ├──▶ error reporters
|
||||
└──▶ telemetry (shadow rows)
|
||||
```
|
||||
|
||||
The Penpot backend is the **center**: it checks, completes, and stores every event, no matter where it comes from. Nobody writes to the database directly; everything goes through the backend.
|
||||
|
||||
This document is the reference for all flows. One thing to keep in mind while reading: the Nitrate admin-console (both its server and its browser UI) is a **private** application, not a public client. Its events describe private SaaS management actions (organizations, subscriptions, SSO), and its server channel uses a trusted shared key instead of a user session.
|
||||
|
||||
## 2. The three kinds of events
|
||||
|
||||
Not all events are worth the same. This is the most important distinction in this document:
|
||||
|
||||
| Aspect | Backend events | Penpot frontend events | Nitrate admin-console events |
|
||||
|---|---|---|---|
|
||||
| What they describe | Real actions run on the server (create file, login, invite to team...) | Things that happen in the Penpot browser app (navigate, edit, JS errors, performance) | Organization and subscription actions in the admin-console (create/rename/delete organization, members, SSO settings, checkout, onboarding) |
|
||||
| Trust level | **High**: created inside the same request that runs the action | **Best-effort**: some loss is expected (tab closed, offline, full buffer); they show **usage**, they are not strict proof | **Mixed**: server-side events are high trust (like backend); browser events are best-effort (like frontend) |
|
||||
| Who sets `profile-id` | The server, from the logged-in session (never trusts the client) | The server too: it ignores what the client says and uses the session | Server channel: the trusted admin-console server sends it explicitly. Browser channel: the Penpot server uses the session |
|
||||
| Who sets the time | Server time (`created-at`); the source of truth | Client proposes `tracked-at`, but the server fixes it if it is in the future or more than 1h late | Same rule: server stamps `created-at`; client `tracked-at` is corrected |
|
||||
| Used for | Audit, webhooks, compliance, abuse checks | Product analytics, usage counters, crash reports | Organization audit trail (admin-console e2e spec), subscription metrics |
|
||||
|
||||
Rule of thumb: if you need to know for sure "did this user do X?", look at `source = backend`. If you want to know "how do people use the app?", also look at `source = frontend`.
|
||||
|
||||
## 3. The `audit_log` table: a buffer, not a store
|
||||
|
||||
The Penpot `audit_log` table **is not the final destination**. It is a waiting tray: events sit here until every active consumer (Nexus, admin console) confirms them, and whoever finishes last deletes the row.
|
||||
|
||||
Main columns (migration `backend/src/app/migrations/sql/0054-add-audit-log-table.sql` and later ones):
|
||||
|
||||
- `id` uuid PK (`gen_random_uuid()` today; the PK used to be `(created_at, profile_id)`).
|
||||
- `name` / `type` text NOT NULL (for example `name = "create-file"`, `type = "action"`).
|
||||
- `created_at` timestamptz NOT NULL default `now()`: **server time**, the source of truth.
|
||||
- `tracked_at` timestamptz default `now()`: what the client claimed; fixed on ingest.
|
||||
- `profile_id` uuid NOT NULL: who did it.
|
||||
- `source` text: `backend` / `frontend` for full rows, `telemetry:backend` / `telemetry:frontend` for anonymized copies (see section 10).
|
||||
- `ip_addr` inet: real IP (or `0.0.0.0` on telemetry rows).
|
||||
- `props` / `context` jsonb holding transit-encoded maps: action data and metadata (versions, browser, request-id...).
|
||||
- No `archived_at`: the `audit_log_pending` queue (migration `0157`, consumer `1` = Nexus, `2` = admin console, ids never reused) tracks what is left. Every record is born pending only for the consumers enabled at insert time (never a fixed both): cheaper on single-consumer hosts and true to the design doc. Consequence, accepted on purpose: rows born with no send flag on get no pendings, so enabling archiving later does not pick them up — the sweep deletes them unarchived.
|
||||
|
||||
Indexes (each consumer has its own, so a slow consumer never blocks the others):
|
||||
|
||||
- `audit_log_pending` PK `(consumer, audit_log_id)`: each consumer owns its queue at constant cost, even with millions pending.
|
||||
- `audit_log (created_at, id)` (migration `0157`): the chronological drain order (`ORDER BY created_at ASC, id ASC`; same total order for both consumers, so concurrent drains cannot deadlock).
|
||||
- `audit_log_pending (audit_log_id)`: serves the "is anything still pending?" check of the final delete.
|
||||
- `(source, created_at)`: used by the telemetry scan.
|
||||
|
||||
Life of a row: `INSERT` (row plus one pending per enabled consumer, same transaction; a rollback takes both) → (every 5 min, Nexus at second 0 and admin console at second 30) each archive job sends its own 128-row batches oldest-first and deletes its pendings in the same transaction → whoever deletes the last pending deletes the row. A batch the destination refuses rolls back and stays for the next tick. Pendings of consumers that are off are reaped at the start of each job run (covers later deactivations), and orphan rows are swept at the end of each run, so there is no separate GC task. The FK is plain `DEFERRABLE` (never `INITIALLY DEFERRED`): every deleter removes pendings before rows, including the telemetry task's own deletes.
|
||||
|
||||
## 4. Anatomy of an event
|
||||
|
||||
Every event ends up with this shape (schema `schema:event` in `backend/src/app/loggers/audit.clj`):
|
||||
|
||||
- `name`: what happened (`"create-file"`, `"navigate"`, `"create-organization"`...). For backend, it defaults to the RPC command name (prefixed with `<module>-` outside `main`).
|
||||
- `type`: category (`"action"`, `"identify"`, `"trigger"`). Most are `"action"`.
|
||||
- `profile-id`: **who** did it (uuid). See the invariant in section 5.
|
||||
- `ip-addr`: request IP (or `"0.0.0.0"` on telemetry).
|
||||
- `props`: action data (team/project/file ids, visited route...). **Never secrets** (see `clean-props`).
|
||||
- `context`: metadata about how/when it was captured (backend/frontend version, user-agent, locale, browser, `request-id`, `initiator`, `event-origin`...). See section 8 for the full field guide, including `initiator` (which app started it: `"app"` means Penpot itself).
|
||||
- `source`: `backend`, `frontend`, `telemetry:backend`, `telemetry:frontend`. Nitrate admin-console server events land as `backend`; its browser events land as `frontend` (see section 7).
|
||||
- `source`: `backend`, `frontend`, `telemetry:backend`, `telemetry:frontend`.
|
||||
- `tracked-at` / `created-at`: claimed time and server time.
|
||||
- Memory-only (never stored): `::webhooks/event?`, `::webhooks/batch-key`, `::webhooks/batch-timeout` for the webhook fan-out.
|
||||
|
||||
Backend `create-file` example (simplified):
|
||||
|
||||
```clojure
|
||||
{:name "create-file" :type "action" :source "backend"
|
||||
:profile-id #uuid "…" :ip-addr "203.0.113.7"
|
||||
:props {:project-id #uuid "…" :name "My design"}
|
||||
:context {:version "2.x" :initiator "app" :request-id "…"
|
||||
:client-version "2.x" :client-user-agent "Mozilla/…"}
|
||||
:created-at #inst "…" :tracked-at #inst "…"}
|
||||
```
|
||||
|
||||
Frontend `navigate` example:
|
||||
|
||||
```clojure
|
||||
{:name "navigate" :type "action" :source "frontend"
|
||||
:profile-id #uuid "…" :ip-addr "203.0.113.7"
|
||||
:props {:route "dashboard" :team-id #uuid "…"}
|
||||
:context {:version "2.x" :locale "en" :browser "Chrome" :os "Linux"
|
||||
:event-origin "dashboard" :session-id "…" :session #inst "…"}
|
||||
:created-at #inst "…" :tracked-at #inst "…"}
|
||||
```
|
||||
|
||||
## 5. Backend events: the happy path
|
||||
|
||||
Most backend events need **no manual code**. The `wrap-audit` wrapper in `backend/src/app/rpc.clj` runs after every RPC command when `:webhooks`, `:audit-log`, or `:telemetry` is on (unless the command sets `::audit/skip`). It builds the event with `prepare-rpc-event` and passes it to `audit/submit`.
|
||||
|
||||
Each command can customize through result metadata (`rph/with-meta`):
|
||||
|
||||
- `::audit/replace-props`: replaces props fully (auth commands use `profile->props` so a register event carries the profile, not the password).
|
||||
- `::audit/props`: merges extras over the request params.
|
||||
- `::audit/context`, `::audit/profile-id`, `::audit/name`, `::audit/type`: override the defaults.
|
||||
- `clean-props` always strips nils, namespaced keys, and `:session-id/:password/:old-password/:token/:client-secret` as a last line of defense. **Never put secrets in props.**
|
||||
|
||||
Critical `profile-id` INVARIANT: the event belongs to **whoever made the request**. It resolves as `::audit/profile-id` metadata → `::rpc/profile-id` → `uuid/zero`. It is **never** taken from the response: some results carry someone else's `:profile-id` (for example `get-error-report` returns the report with its content merged; `verify-token` on an invitation returns the inviter), and using it would blame the wrong person. Commands with `::rpc/auth false` (`login-*`, `register-profile`, `create-demo-profile`, `verify-token`, `prepare-register-profile`) have no caller to fall back on, so they **must** set `::audit/profile-id` in the metadata or the event lands on `uuid/zero`.
|
||||
|
||||
The `::audit/profile-id` override goes through `coerce-profile-id`: a string is parsed, anything that cannot become a uuid is dropped (with a warning) and the event falls back to the caller. Do not pass the value raw: `schema:event` needs a uuid and `submit*` swallows the validation error (it only logs, the RPC still succeeds), so a non-uuid override **loses the row silently**. Hand-built events with `event-from-rpc-params` + `submit` (clone template, accept team invitation, `management/nitrate` push-audit-events) do **not** go through that coercion: they must already carry a uuid.
|
||||
|
||||
### Team id and entity ids (projection data)
|
||||
|
||||
The event stream is meant to let a consumer (Nexus) rebuild what it needs **without joining production tables**, so events carry the ids a projection keys on:
|
||||
|
||||
- **Team id enrichment.** After `clean-props`, `prepare-rpc-event` calls `enrich-team-id`: when props have no `:team-id` but carry a `:project-id` or `:file-id`, it resolves the team with a local query and adds `:team-id`. It never resolves from `:id` (read commands put the entity id there, and that would add a query to every read), and it skips read events entirely (`get-*` names plus `has-file-libraries`: their events carry no consumer state). Commands that already loaded the team add `:team-id` themselves in `::audit/props` (`delete-file`, comments, media upload, `delete/update-webhook`, `clone-template`, ...), so the resolver does not run.
|
||||
- **Membership.** `add-profile-to-team!` emits `add-team-member` (props `:team-id`, `:member-id`, `:role`, `:is-owner`, `:is-admin`, `:can-edit`) whenever it actually inserts a row, only with `:audit-log` on, inside the caller's transaction. It covers the flows that add a member with no event of their own (`create-team` owner, invitee added directly when `:email-verification` is off, approved join-request, duplicated team). With `on-conflict-do-nothing?` a conflict returns nil and emits nothing, so an existing membership is reported once. The event belongs to the actor: the two flows where the owner adds somebody else (`create-invitation`, `add-member-to-team`) pass `:actor-id` through a separate `event-opts` arg (never in `params`, which is inserted as-is); everywhere else actor and member coincide. `:role` is derived in the emitter from the flags (inverse of `permissions-for-role`: owner > admin > editor > viewer) because the table has no role column; request context travels in `event-opts` as `:rpc-params`.
|
||||
- **Created entity ids.** `create-project` and `create-file` add the new row's `:id` (and `:project-id`/`:team-id`) to `::audit/props`, so a projection can key it.
|
||||
- **Bulk operations.** `clone-template`, `import-binfile` and `duplicate-project` create entities through internal calls that emit nothing on their own; they emit a `create-file` (and `create-project`) per entity (only with `:audit-log`). The import paths go through `audit/submit-file-created-event`, which reads the light columns (`id`, `name`, `project-id`, timestamps, `revn`) back from the row — the file must really exist, a phantom id emits nothing. `duplicate-project` already holds the rows and uses `audit/submit-create-event` directly. Both binfile versions return the same shape from `import-files!` (`{:file-ids [...]}`).
|
||||
- **Deletes.** `delete-team` still returns nil but carries the deleted row (`:id`, `:name`, `:created-at`, `:modified-at`, `:deleted-at`) in `::audit/props`, like `delete-project`. `delete-profile` carries `:id`, `:email`, `:fullname`, `:created-at`, `:modified-at` and `:deleted-at` (without this its props are empty: every param is namespaced and `clean-props` drops them). `create-team` adds `:created-at` and `:modified-at`. `delete-team-member` needs nothing: the delete is physical and the params already carry `:team-id`/`:member-id`.
|
||||
- **Hard deletes.** `objects-gc` emits one `hard-delete-objects` event per purged chunk, carrying only that chunk's entity key (`:team-ids`, `:project-ids` or `:file-ids`; one chunk transaction purges a single entity, so one event can never span sections) with context `{:triggered-by "objects-gc"}`, only with `:audit-log` on. No other GC path emits (profiles, fonts, thumbnails); those purges stay silent.
|
||||
- **SREPL deletes.** `delete-profile!` carries the subject id in props (it used to emit nothing usable); the event profile stays zero because SREPL has no actor. `delete-team!` props already carried the id. Both travel as actions with a `triggered-by: srepl` context.
|
||||
- **SREPL restores.** `restore-profile!` carries the full row (with id) in props with a zero event profile; `restore-team!` and `restore-project!` already carried their ids. The consumer resolves the subject from props by a uniform rule, with no SREPL special case.
|
||||
- **Cascades.** The `:delete-object` worker task emits one `delete-object-with-cascade` event per execution, type `action`, with `:object`/`:id`/`:deleted-at` plus the affected ids nested under `:cascade` (`:team-ids`, `:project-ids`, `:file-ids`), in the same transaction. The handler submits unconditionally (`submit` fans out per flag, so no `:audit-log` re-check); the `audit_log` row itself is still only written with `:audit-log` on. Direct calls (objects-gc) and snapshot cascades stay silent on purpose.
|
||||
- **Ownership.** `create-team-with-invitations` carries the new team `:id` like `create-team`; `update-team-member-role` names the demoted member in `:prev-owner-id` on owner transfers.
|
||||
|
||||
Entry points:
|
||||
|
||||
- `submit`: the normal one (fills defaults, validates, runs inside `tx-run!`, never breaks the RPC).
|
||||
- `insert`: low level, for CLI helpers and the webhook subsystem; direct write, no webhook/telemetry fan-out, silent unless `:audit-log` is on.
|
||||
- Boot emits `trigger/instance-start` from `setup/props` so every restart stays visible.
|
||||
|
||||
Cases already covered by tests, do not break: `accept-organization-invitation` tells its two flows apart **with props, not with origin**: `:organization-member-add-source` is `"direct-organization-invitation"` or `"team-invitation"`, and `:belongs-to-team-on-add` repeats it as a boolean. Do not add a third prop. The `context` `:event-origin` belongs to the browser (`make-data-event` sets it and it is on the frontend allowlist; `safe-backend-context-keys` has no `:event-origin`, so a backend event has nowhere to put one).
|
||||
|
||||
## 6. Penpot frontend events: what is collected, with what guarantees
|
||||
|
||||
The browser **never writes to the table**; it POSTs transit batches to `push-audit-events` (`backend/src/app/rpc/commands/audit.clj`), which stamps server `id`, session `profile-id`, request IP, server `created-at`, and server-side `:initiator` (from the `x-client` header, see section 8; anything the client sends as `initiator` is overwritten), and distrusts the client clock (future or >1h-late `tracked-at` is reset, the original kept in `context.original-tracked-at`). Without `:audit-log`/`:telemetry`, or on a read-only pool, the endpoint is a no-op.
|
||||
|
||||
Valid types (`schema:frontend-event`): `"action"`, `"identify"`, `"trigger"`. Names must match `#"[\d\w-]{1,50}"` (max 250). `props` is a free map with no schema: **tests are the only guard of the contract** (see section 13).
|
||||
|
||||
What they are for: usage metrics (navigation, editing), workspace file stats, Nitrate membership changes, crash reports (`unhandled-exception` / `exception-page`). **Not proof for audit**: collection is best-effort and includes `PerformanceObserver` noise (`performance-*`, `user-input`, `long-task`). So backend tests must never assert exact frontend event counts.
|
||||
|
||||
How the browser collects (`frontend/src/app/main/data/event.cljs`), most to least important:
|
||||
|
||||
1. **On-demand start**: `initialize` asks the backend with `get-environment-data` (old `get-enabled-flags` is deprecated) and only starts on `:audit-log` or `:telemetry`. On RPC failure it still starts in telemetry mode (the backend will reject if needed).
|
||||
2. **Potok translation**: `make-proto-event` (events with the `Event` protocol) and `make-data-event` (data-events with `::name`) turn the action into `{type, name, props, context}`. `simplify-props` flattens complex values (maps → `:placeholder/map`, and so on) and drops namespaced keys and nils. Context gains `:event-origin`, `:event-namespace`, `:event-symbol`.
|
||||
3. **Capped buffer**: queue of max 1024 events (`append-to-buffer` drops beyond that); every 2s debounce (or on logout, or `::force-persist`) a chunk of max 100 is taken, filtered to the current profile, and sent fire-and-forget to `api/main/methods/push-audit-events` as transit. On confirm, the chunk leaves the buffer (`::chunk-persisted`).
|
||||
4. **Performance noise**: `PerformanceObserver` (`event`/`longtask`) plus blocking measurement with `scheduler.postTask` create `trigger` events (`performance-blocking-event`, `user-input`, `long-task`) with 1s debounce and 1s threshold. Info only, not audit.
|
||||
5. **`skip-audit?`**: UI flag (for example resuming a pending dashboard action after Nitrate SSO, or moving a team into an organization) so one user gesture is not counted twice.
|
||||
|
||||
## 7. Nitrate admin-console events: the third producer
|
||||
|
||||
The admin-console is the third producer. It sends organization and subscription events through **two channels** that look the same once stored, but have different trust levels. All the sending code (server and browser helpers) lives in the private Nitrate repo: it is not Penpot code and no file paths are cited here. What this section documents is only the Penpot-visible side: endpoints, payloads, and which event names arrive.
|
||||
|
||||
### 7a. Server-side channel (high trust, like backend events)
|
||||
|
||||
Code: the admin-console server has a helper that POSTs to the Penpot **management** API:
|
||||
|
||||
```
|
||||
POST api/management/methods/push-audit-events
|
||||
headers: x-shared-key: "admin-console <key>", Cookie: <user cookies>
|
||||
body: {events: [{name, props, profileId, context?}]}
|
||||
```
|
||||
|
||||
Auth happens at the HTTP layer (`backend/src/app/http/middleware.clj` `wrap-shared-key-auth`): the `x-shared-key` header carries `"<key-id> <secret>"` (`admin-console` here), checked in constant time against the configured keys; the key-id lands in the event context as `:initiator`. The call goes server-to-server (ky client, internal URI, user cookies forwarded).
|
||||
|
||||
Who uses it:
|
||||
|
||||
- Organization management (best-effort wrapper): `create-organization`, `rename-organization`, `delete-organization`, member add/remove, and more. It optionally passes the UI origin in the context.
|
||||
- Subscriptions: `create-subscription`, renewals, cancellations, and more, including deployment lookup (saas/selfhost).
|
||||
|
||||
Failure policy: every send is best-effort. A failure never rolls back the business action; it is only recorded for debugging.
|
||||
|
||||
Backend side (`backend/src/app/rpc/management/nitrate.clj` `::push-audit-events`): `::audit/skip`, `::rpc/auth false` (trust comes from the shared key, not from a user session; only the `admin-console` key id is allowed). It builds context with `audit/prepare-context-from-request` + `request-id`, takes the given `profile-id` as-is (no coercion here, must already be a uuid), sets `tracked-at = request-at`, defaults `type` to `"action"`, and calls `audit/submit` (which defaults `source` to `"backend"`). So these rows land as **`source = backend`**: they are audited like backend events because a trusted service set the author.
|
||||
|
||||
### 7b. Browser channel (best-effort, like frontend events)
|
||||
|
||||
Code: the admin-console browser has a helper that POSTs to the Penpot **public** API:
|
||||
|
||||
```
|
||||
POST api/main/methods/push-audit-events
|
||||
credentials: include, keepalive: true
|
||||
headers: x-client: "penpot-admin-console/<version>"
|
||||
body: {events: [{name, type: "action", timestamp, props, context: {eventOrigin?}}]}
|
||||
```
|
||||
|
||||
Dozens of helpers cover the admin-console UI: `create-organization`, `delete-organization-member`, `create-subscription` / `create-trial-subscription`, `open-subscription-modal`, `start-nitrate-checkout`, `open-current-subscription`, SSO settings flows (`open/apply/success/fail/discard-organization-sso-changes`, activate/deactivate modals), `change-organization-advanced-permission`, `onboarding-step` / `onboarding-finish`, `close-subscription-modal`. These helpers live in the private Nitrate repo: they are not Penpot code, you will not find them in this repository. Only the event names above (what arrives at Penpot) are listed here on purpose.
|
||||
|
||||
These rows land as **`source = frontend`** with the session `profile-id`: same guarantees as Penpot frontend events (best-effort, usage analytics). Every browser helper sends an `eventOrigin` in the context: plain `"admin-console"`, with a suffix only when the same event is emitted from two places in the app (for example the SSO discard event distinguishes the config form from the unsaved-changes modal).
|
||||
|
||||
The Nitrate repo also has a shared, generic push client for the management endpoint (not tied to the admin-console server): any other Nitrate-side service that needs to record something in the Penpot audit log can reuse it instead of writing its own HTTP call. Same endpoint, same payload shape as the server channel above.
|
||||
|
||||
## 8. How the frontend helps the backend audit: HTTP headers
|
||||
|
||||
Besides explicit events, **every RPC request from the frontend carries headers** so the backend can fill its own event `context`. This is how the backend learns "which browser/version/screen" made the request:
|
||||
|
||||
- `x-frontend-version` and `x-client: penpot-frontend/<version>`: added by `default-headers` in `frontend/src/app/util/http.cljs` to **all** fetch requests (unless `omit-default-headers`). The admin-console browser channel sends `penpot-admin-console/<version>` instead (`x-frontend-version` is ignored on the event-ingest path, only `x-client` matters there).
|
||||
- `user-agent`: set by the browser alone; the backend trims it to 500 chars (`get-client-user-agent`).
|
||||
- `x-external-session-id`: external SaaS session via `cf/external-session-id`; added by `repo.cljs` (`send!`, special `cmd!`, `multipart-upload`, export...) and also stored by `persist-events` in the frontend event context. The backend checks it (max 256, ignores `"null"`/empty).
|
||||
- `x-session-id`: `cf/session-id` (tab session), only in the main `send!`.
|
||||
- `x-event-origin`: UI origin (`::ev/origin` from param metadata, for example `"dashboard"`, `"workspace"`); sent by `send!` and the OIDC/export/upload `cmd!`. The backend trims it to 200 chars and stores it as `:client-event-origin` (different from the `:event-origin` the frontend puts in its own context!).
|
||||
|
||||
On the backend, `prepare-context-from-request` (`backend/src/app/loggers/audit.clj`) joins everything into the event context: `:external-session-id`, `:initiator` (`auth-key-id` or `"app"`), `:access-token-id/type`, `:client-event-origin`, `:client-user-agent`, `:client-version`, `:version` (backend), plus `:request-id`. Only `safe-backend-context-keys` (`:version`, `:initiator`, `:client-version`, `:client-user-agent`) survive in the telemetry shadow copy; the rest (`:client-event-origin`, `:external-session-id`, `:access-token-id/type`, `:request-id`...) is stored fully in the normal row and sent to Nexus, but dropped from telemetry.
|
||||
|
||||
### Context field guide: `initiator`
|
||||
|
||||
`initiator` labels the application that started a request. The server sets it on both channels, but only the shared-key channel authenticates the sending application:
|
||||
|
||||
- **Backend channel** (RPC + management API): it comes from the shared-key auth layer (`wrap-shared-key-auth` in `backend/src/app/http/middleware.clj`): when a trusted service calls the management API with `x-shared-key: "<key-id> <secret>"`, the key-id (lowercased) is stored on the request as `::http/auth-key-id`, and `prepare-context-from-request` copies it into `:initiator`. When there is no shared key (a normal user RPC), it falls back to `"app"`, which means Penpot itself.
|
||||
- **Frontend channel** (`push-audit-events`): `get-client-initiator` maps the `x-client` header product to an initiator: `penpot-frontend` → `"app"`, `penpot-admin-console` → `"admin-console"`, legacy `penpot-nitrate` (old admin-console releases, remove once redeployed) → `"admin-console"`, missing or unknown → `"app"`. It overwrites any `initiator` in the event context, but the caller controls the header. Any authenticated caller can claim `"admin-console"` by sending that product in `x-client`. Use this label for usage analytics; it does not prove which application sent the event or grant permissions. The session still determines `profile-id`.
|
||||
|
||||
Values you will see:
|
||||
|
||||
- `"app"`: normal Penpot traffic (browser → public RPC or event ingest, or backend internal work). This is the vast majority.
|
||||
- `"admin-console"`: the authenticated Nitrate admin-console on management API calls (section 7a), or a caller claiming that product on browser events (section 7b, via the `x-client` mapping).
|
||||
- `"nexus"`, `"exporter"`, `"media-processor"`: other first-party services using their own shared keys (key ids come from `::setup/shared-keys` in `backend/src/app/setup.clj`, derived from the instance secret unless overridden).
|
||||
|
||||
Why it matters: it supports per-application usage counts, with the trust level described above. It survives into telemetry on both channels (one of the four backend keys, and on the frontend allowlist too), so per-initiator counts stay available even on anonymized rows.
|
||||
|
||||
More metadata the Penpot frontend adds inside each event (not as headers): `collect-context` with `ua-parser` (browser, engine, OS, device, screen, CPU arch), `locale` (updated on language change), `:session` / `:session-id` / `:external-session-id`, and SaaS host extras (`add-external-context-info`).
|
||||
|
||||
## 9. Consumers I: webhooks
|
||||
|
||||
Webhooks let an external URL get notified when something happens in a team (for example "a file was created"). The flow has three steps:
|
||||
|
||||
1. **Match.** Some audit events are marked as webhook-relevant when they are created. For each of those, Penpot looks up the active webhooks of the affected team: it reads the team straight from the event, or finds it through the project or the file in the event. Each matching webhook gets its own delivery job. If several quick events belong together, they are grouped into one delivery (the grouping rule travels with the event itself).
|
||||
2. **Deliver.** Penpot POSTs the event to the webhook URL in the format that webhook asked for: JSON (with camelCase keys), transit, or form-encoded. Every attempt is written to a delivery log with the request, the response, and the error if any, so failures can be inspected later.
|
||||
3. **Protect.** Each webhook counts its consecutive failures. After 3 in a row, Penpot switches the webhook off automatically, so a dead URL stops receiving traffic until someone fixes it. A successful delivery resets the counter to zero.
|
||||
|
||||
One guarantee: sending webhooks never changes the audit event itself. If delivery fails, only the webhook (and its delivery log) is affected; the audit row stays untouched and still gets archived normally.
|
||||
|
||||
Every match also writes a `webhook` / `trigger` row to `audit_log` (even with zero matches): its props are `{:name <origin event name>, :event-id <origin row id>, :total-affected <matched webhook count>}`. `process-event` assigns the origin row id up-front so the trigger can reference it; the telemetry shadow row always gets its own id (reusing it would collide on the PK).
|
||||
|
||||
## 10. Consumers II: error reporters
|
||||
|
||||
Both `app.loggers.database` and `app.loggers.mattermost` listen to backend `:error` logs **and** frontend crash events (found by the `::audit/event` marker), through a sliding-buffer channel so a flood cannot stall the app. The database one stores them in `server-error-report` (source 3 = backend log, 4 = audit-log origin, 5 = rate-limit); the Mattermost one forwards a short note to `:error-report-webhook` when set.
|
||||
|
||||
Both reporters only start when the `:error-reporting` flag is on (off by default). The database reporter starts on the flag alone; the Mattermost reporter needs the flag **plus** its webhook URL configured. Without the flag, nothing is stored and `emit` calls from `push-audit-events` are silent no-ops. Each reporter also keeps a runtime on/off switch for emergencies via REPL.
|
||||
|
||||
Result: crash reports only exist if the frontend collector runs and `push-audit-events` accepts `unhandled-exception`/`exception-page`. Turning off the whole pipeline also blinds frontend error reporting.
|
||||
|
||||
## 11. Consumers III: telemetry (shadow rows)
|
||||
|
||||
Telemetry reuses the same table with **anonymized shadow rows** (`source LIKE 'telemetry:%'`): day-truncated times, `0.0.0.0` IPs, props reduced to uuid/boolean/number values plus a few allowed fields (`source`, `lang`, `auth-backend`, `activation`, derived `email-domain` — never raw emails), and a minimal context allowlist. Full + shadow rows can exist together per event when both flags are on; that duplication is on purpose.
|
||||
|
||||
Filters (`filter-telemetry-props` / `filter-telemetry-context`): by default only uuid/boolean/number with a simple key; exceptions: login/register/update-profile and frontend `identify` keep `lang`/`auth-backend`/`source`/`activation`/`email-domain`; `organization-sso-auth-failed` keeps `failure-reason` if it is a known one; `instance-start` passes whole; frontend `navigate` keeps `[:route :file-id :team-id :page-id]`. Subscription actions have event-specific contracts: backend and frontend `create-subscription` / `create-trial-subscription` preserve the default uuid/boolean/number props plus `type`, `deployment`, `quantity` and `subscription-mode`; backend `update-subscription` preserves only `type`, `deployment`, `previous-status`, `status`, `previous-cancel-at-period-end`, `cancel-at-period-end`, `previous-cancel-at`, `cancel-at` and `changed-fields`; backend `subscription-summary` preserves only `type`, `deployment`, `instance-id`, `snapshot-date`, the per-status `*-count` values, `total-count`, `sync-succeeded`, `sync-failed` and `snapshot-complete`. These subscription exceptions apply only to action events. Other free text drops from telemetry but **stays in the full row and in Nexus**. In `register-profile`, `activation` is a prop (it is what reaches PostHog). `action` exists only in the `context` of the full `register-profile-retry` rows; `filter-telemetry-context` drops it from the anonymised copy, which keeps only `:version`, `:initiator`, `:client-version` and `:client-user-agent`.
|
||||
|
||||
Shipping (`backend/src/app/tasks/telemetry.clj`): the cron sends shadows to `:telemetry-uri` as JSON in 10k batches and **deletes** them on success; it purges leftovers older than 7 days. It also sends a legacy report (counters, teams, email domains...). On official hosts nothing is collected or sent (`telemetry-excluded?` covers `penpot.app`/`penpot.dev`).
|
||||
|
||||
### Where telemetry events go: `telemetry.penpot.app` is Nexus
|
||||
|
||||
The default `:telemetry-uri` (`https://telemetry.penpot.app/`) points at the **Nexus telemetry endpoint**, so telemetry is a second road into Nexus, separate from the archive cron. The payload is plain JSON:
|
||||
|
||||
```
|
||||
POST https://telemetry.penpot.app/
|
||||
{type: "telemetry-events", version, instance-id, events: [...]}
|
||||
```
|
||||
|
||||
Nexus dispatches on `:type`: `"telemetry-events"` batches are coerced and inserted into its own `audit_log`; anything else (old flat payloads, `"telemetry-legacy-report"`) is upserted into `telemetry_report` per instance + day. Three details matter:
|
||||
|
||||
- **Anonymization happens in Penpot, before sending.** By the time events leave, times are day-truncated, IPs are already `0.0.0.0`, props/context are filtered (see above), and emails are gone (only derived `email-domain` survives). Nexus additionally forces the outcome: it prefixes any non-`telemetry:` source, sets `received-at = tracked-at` (already truncated), pins IP to `0.0.0.0`, and stamps the sending `instance-id` into every event context.
|
||||
- **No personal data is expected.** Profile ids are random UUIDs with no PII, so the telemetry consumer needs no login guard; grouping is by `instance`, not by person (see posthog-telemetry below).
|
||||
- **The endpoint always answers 200.** Failures are swallowed and only counted (`nexus_telemetry_errors_total`), so a bad batch never breaks the sending instance; successes are counted (`nexus_telemetry_events_total`, `nexus_telemetry_reports_total`). A body the parser cannot read never reaches the handler: it answers 400 and counts in `nexus_http_malformed_requests_total`. Old instances may send a fressian+zstd base64 blob instead of a plain vector; Nexus decodes both (50 MiB cap against decompression bombs).
|
||||
|
||||
## 12. Archive to Nexus and to the admin console
|
||||
|
||||
### How it works today
|
||||
|
||||
Every 5 min each archive job drains its own queue (`:audit-log-consumer-nexus` on flag `:nexus`, cron `0 */5 * * * ?`; `:audit-log-consumer-admin-console` on flag `:admin-console`, cron `30 */5 * * * ?`, 30s offset so the two drains never start on the same second), in 128-row chunks taken oldest-first (`ORDER BY created_at ASC, id ASC`) with `FOR UPDATE OF audit_log` (`SET LOCAL lock_timeout = '30s'`) inside the same transaction that sends and completes them, with 100 ms pauses between chunks. Batches arrive at Nexus in chronological order (it numbers by arrival for order-sensitive consumers). Nexus receives transit `{:events [...]}` with `x-shared-key: "nexus <key>"` (`:nexus-shared-key`, or derived from the instance secret) and `origin` = public-uri; flag on but no URI raises `:task-not-configured`. The admin console receives only its allowlisted event names (enriched with team names, props emails, and optional top-level `:profile-email` / `:profile-name` from the actor `:profile-id`); anything filtered out counts as done for the admin-console queue. A chunk whose destination refuses it (Nexus non-204 raises `:consumer-chunk-refused`) or throws (transport failure, admin-console error) rolls back and stays for the next tick, and the job aborts as failed: it archives what it can, and the cron retries the rest. Committed chunks stay sent. The send fns return counts or throw carrying the attempted count as `:events`; `run-single-consumer` records the `failed` metric from it, outside every chunk transaction (a local DB failure carries no `:events` and stays uncounted — it is not a destination failure). No `warn` before the throw: the runner reports the aborted job. Each job touches only its own queue, so a fallen Nexus never starves the admin console (nor vice versa).
|
||||
|
||||
A chunk is **one POST** and the task advances only on 204, so a single oversized event wedges the whole queue: a crash report whose message embeds a whole payload (a serialized file, for example) can push the request past the proxy body limit and the rejection blocks every later event until it is pruned by hand. Frontend crash reports are therefore size-bounded at the source (`mem:frontend/routing-app-shell-subtleties`): `app.common.exceptions/get-hint` and the CLJS `format-throwable` clamp exception messages (`max-message-length`).
|
||||
|
||||
On the other side, Nexus checks `origin` against its allowlist (403 if not allowed), coerces the events, and inserts them into its own `audit_log` (partitioned by `received_at`, with an incrementing sequence number and `source`). Watch the rename: Penpot's `created-at` is stored as **`received-at`** in Nexus; Nexus's `created-at` is when it was created there. `tracked-at` keeps the origin time. Telemetry batches arriving at `telemetry.penpot.app` land in the same table through the telemetry endpoint above.
|
||||
|
||||
Concurrent jobs are safe: both lock rows in `created_at ASC, id ASC` order (one total order, deliberately blocking — no `SKIP LOCKED`, so archive arrival stays chronological), so the second waits for the first to commit and then sees whether it was last. Only acknowledged work deletes pendings; an admin-console failure never blocks the Nexus send of the same rows (each job owns its queue).
|
||||
|
||||
Observability (`backend/src/app/loggers/audit/consumer_task.clj`, definitions in `backend/src/app/main.clj:default-metrics`): one `dbg` summary per job run (`:consumer` label, sent/skipped/chunks, reaped disabled pendings, swept orphans, `:elapsed` as `"1.5s"` via `ct/format-duration`, frozen in a strict binding first because log props evaluate lazily in the logger thread), one `dbg` per admin-console send call (`ready` with totals, `sent`/`failed` with counts), plus `dbg` when the queue is empty or the job's send flag is off. Queue depth is Grafana's business: it queries the database directly, so the run performs no `GROUP BY` for the logs. No `warn` before a throw: the runner reports the aborted job (refusal details travel in the error data). Metrics, all recorded with bounded labels only: `penpot_audit_consumer_events_total{ destination = nexus|admin-console, outcome = archived|failed|skipped }` (`skipped` = filtered by the admin-console allowlist) and `penpot_audit_consumer_discarded_total{ reason = reaped|swept }` (entries discarded without archiving: `reaped` counts pendings dropped for disabled consumers, `swept` counts rows deleted while nobody waited for them — the silent R3 loss, now dashboard-visible instead of log-only). Timing needs no custom histogram: the jobs substrate already records execution time per job (`jobs-execution-timing`, split by job name), so each archive job is timed as itself. No queue-depth gauge: pending depth is read off the replica database directly. The bulk frontend ingest counts `penpot_audit_ingest_events_total{ kind = full|shadow, outcome = inserted|failed }` (both faces share one transaction, so a failure drops the batch on both). Archive success counts are recorded by `run-single-consumer` on the drain totals: a chunk that returns normally committed its transaction (a rollback always throws instead), so they cannot inflate. Logs never carry event data (`props`, `context`, `profile-id`, `ip-addr`, `team-name`, emails): only counts, statuses and the uri host.
|
||||
|
||||
### How Nexus consumes the data (and forwards to PostHog)
|
||||
|
||||
Stored rows are not the end: a consumer pipeline reads them back in sequence order and processes them. Each consumer remembers its place (sequence number), fetches the next chunk, runs its handler, then saves the new position. Three consumers run in order:
|
||||
|
||||
1. **`local`**: keeps a mirror of user profiles. It listens to profile events (`register-profile`, `update-profile`, `login-with-*`, `delete-profile`, ...) and upserts email, name, props, and delete markers. It runs first because the next consumer needs profile data.
|
||||
2. **`posthog`** (runs after `local`): forwards our own SaaS archived events to PostHog. It sends an allowlist of backend track events (teams, members, projects, files, comments, Nitrate/SSO, ...) plus frontend events minus a noise exclusion list, grouped by **team**, with person identify data from the local profile mirror and snake_case props. Both paths include the `initiator` property when the stored context has one, so PostHog can split Penpot vs admin-console traffic.
|
||||
3. **`posthog-telemetry`**: forwards events with `source = telemetry:frontend|telemetry:backend` to a separate PostHog project. Because these rows are already anonymous (no IPs, truncated times, filtered props), grouping is by **instance** (the self-hosted server, from `context.instance-id`), with one `$groupidentify` event per instance per chunk so PostHog learns per-instance properties (version, URI, flags). Each consumer has its own PostHog credentials.
|
||||
|
||||
So the full journey is: Penpot buffer → Nexus `audit_log` (via either archive cron or the telemetry endpoint) → consumers → PostHog (SaaS events grouped by team, telemetry grouped by instance) plus local analytics views and materialized KPIs.
|
||||
|
||||
### Admin console: the second consumer
|
||||
|
||||
Penpot → admin-console archiving lives in its own job (`:audit-log-consumer-admin-console`, consumer `2`, flag `:admin-console`, params closed empty): it drains its own pending queue and ingests allowlisted events through `nitrate/call :ingest-audit-log`, enriched as described above. What also exists (do not confuse with archiving):
|
||||
|
||||
- **Nitrate → Penpot**: `POST /api/management/methods/push-audit-events` (`backend/src/app/rpc/management/nitrate.clj`) lets the trusted Nitrate backend inject events into the Penpot audit log (no RPC auth, `::audit/skip`, request context + `submit`; only the `admin-console` shared-key id can call it). Section 7a is the main user.
|
||||
- **Admin-console browser → collector**: usage events from the admin-console UI traveling through the normal public `push-audit-events` (section 7b).
|
||||
- **Penpot frontend Nitrate bits**: membership/subscription events (`frontend/src/app/main/data/nitrate.cljs`, `nitrate-audit/*`) through the same public endpoint.
|
||||
|
||||
## 13. Flags and config
|
||||
|
||||
Flags (`common/src/app/common/flags.cljc`, varia; enabled with `PENPOT_FLAGS=enable-<name>`):
|
||||
|
||||
| Flag | What it does |
|
||||
|---|---|
|
||||
| `:audit-log` | Stores full rows; turns on `wrap-audit` and frontend ingest |
|
||||
| `:nexus` | Send flag of the Nexus archive job `:audit-log-consumer-nexus` (cron `0 */5 * * * ?`; needs `:nexus-uri` (ingest path `api/audit` appended by the job)) |
|
||||
| `:admin-console` | Send flag of the admin-console archive job `:audit-log-consumer-admin-console` (cron `30 */5 * * * ?`, 30s after Nexus); also the module flag for the admin-console integration |
|
||||
| `:audit-log-logger` | Structured `app.audit` log per event |
|
||||
| `:error-reporting` | Starts the error reporters (off by default); database reporter on flag alone, Mattermost on flag plus webhook URL |
|
||||
| `:telemetry` | Stores shadow rows and turns on shipping (auto with `:telemetry-enabled`) |
|
||||
|
||||
Config (`backend/src/app/config.clj`): `:nexus-uri`, `:telemetry-uri` (default `https://telemetry.penpot.app/`), `:nexus-shared-key` / instance secret, `:public-uri` (sent as `origin` to Nexus).
|
||||
|
||||
## 14. Practical guide: add or change events
|
||||
|
||||
- **New backend command that must (or must not) emit**: `wrap-audit` does it alone; to exclude it set `::audit/skip`. To customize props/name/type use result metadata. Add a test mocking `app.loggers.audit/submit` (nil; `helpers.clj` stubs it globally) and assert `:call-args-list` — see `backend_tests/rpc-audit-test.clj` (full-row, telemetry-only, dual, no-op without flags, `insert`, `prepare-rpc-event`) and `rpc-team-test/accept-organization-invitation-audit-event` (three cases, exact props; counts matter: asserting with `first` passes even when a command emits the same name twice).
|
||||
- **New Penpot frontend event**: create the data-event or implement `Event`/`PerformanceEvent` in `app.main.data.event`, with `::name`/`::type`/`::origin`. Honor `skip-audit?` if the action can resume. Remember: with no props schema, **your asserts are the contract**.
|
||||
- **New admin-console event**: server action → add the backend-log call next to the mutation, best-effort so it never breaks the action; browser action → add a new event helper plus its test.
|
||||
- **Command that creates an entity or already knows its team**: put the new row's `:id` in `::audit/props` (`create-project`/`create-file`) and set `:team-id` when the team is loaded, so the consumer never has to join production tables. For entities created inside an internal call (`clone-template`, `import-binfile`, `duplicate-project`), emit one event per entity: `audit/submit-file-created-event` when only the id is known (it reads the row back), `audit/submit-create-event` when the row is already in hand.
|
||||
- **Never** put secrets in `props`; `clean-props` is a last defense, not an excuse.
|
||||
- **Do not duplicate events**: if the backend already emits an event for an action, do not add a frontend (or admin-console browser) event for the same action — the backend one wins because it is reliable. A second event for the same action is only acceptable with a strong reason (for example the backend cannot see it at all), and then it must be clearly justified.
|
||||
- **Follow the naming policy**: use kebab-case names (`"create-organization"`, not `"createOrganization"`), and before inventing prop keys, look at similar existing events and reuse their key names (`organization-id`, `team-id`, `file-id`, `event-origin`...). Same concept, same key spelling — that is what keeps the data queryable later.
|
||||
|
||||
## 15. Code map
|
||||
|
||||
- Backend producer: `backend/src/app/loggers/audit.clj` (`submit`, `submit*`, `insert`, `prepare-rpc-event`, `enrich-team-id`, `submit-create-event`, `event-from-rpc-params`, telemetry filters), `backend/src/app/rpc.clj` (`wrap-audit`), `backend/src/app/rpc/commands/audit.clj` (`push-audit-events`, `get-environment-data`, deprecated `get-enabled-flags`), `backend/src/app/rpc/management/nitrate.clj` (`push-audit-events` Nitrate→Penpot), `backend/src/app/http/middleware.clj` + `backend/src/app/http/management.clj` (shared-key auth).
|
||||
- Membership emitter: `backend/src/app/rpc/commands/teams.clj` (`add-profile-to-team!` → `add-team-member`).
|
||||
- Penpot frontend producer: `frontend/src/app/main/data/event.cljs` (collector, buffer, send), `frontend/src/app/main/repo.cljs` + `frontend/src/app/util/http.cljs` (headers), `frontend/src/app/main/data/nitrate.cljs` (Nitrate membership events).
|
||||
- Admin-console producer (private repo, no paths cited here): server-side organization/subscription events through the management `push-audit-events`, browser-side UI events through the public `push-audit-events`.
|
||||
- Consumers: `backend/src/app/loggers/webhooks.clj`, `backend/src/app/loggers/database.clj`, `backend/src/app/loggers/mattermost.clj`, `backend/src/app/tasks/telemetry.clj`.
|
||||
- Archive (two jobs sharing one namespace + pending queue, no GC): `backend/src/app/loggers/audit/consumer_task.clj` (`:audit-log-consumer-nexus` / `:audit-log-consumer-admin-console` job-defs, shared drain/reap/sweep helpers), producer hook in `backend/src/app/loggers/audit.clj` (`append-audit-entry`, single `queue-pendings` helper) plus the bulk ingest in `backend/src/app/rpc/commands/audit.clj` (`handle-events`), metrics/cron wiring in `backend/src/app/main.clj` (one cron entry per job, Nexus at second 0 and admin console at second 30, each gated by its own send flag), single migration `0157` (queue, drop of `archived_at`, covering `(created_at, id)` index for the chronological scan; no backfill — pre-existing rows are swept unsent, an accepted loss).
|
||||
- Nexus (private repo, no paths cited here): archive ingest, ordered consumers (local profile mirror, PostHog SaaS events, PostHog telemetry), telemetry endpoint, analytics views.
|
||||
- Tests: `backend_tests/rpc-audit-test.clj`, `rpc-team-test/accept-organization-invitation-audit-event`.
|
||||
@ -1,3 +0,0 @@
|
||||
# Backend Audit Log
|
||||
|
||||
Superseded by `mem:audit-log`: the full audit-log and event-processing guide (backend center, frontend collection, headers, webhooks, error reporters, telemetry, Nexus archive and planned Nitrate archive, table-as-buffer lifecycle, event anatomy, flags, tests). Read that memory instead; keep this stub so old references do not rot.
|
||||
@ -7,37 +7,23 @@
|
||||
- Basic Penpot registration is token staged: prepare/register creates or verifies temporary tokens, then profile creation/session setup is reused by other auth backends. The frontend `/auth/verify-token` flow is a hub for registration confirmation, email change, and invitation tokens.
|
||||
- OIDC-compatible providers share a generic flow: redirect to provider, validate callback/request token, fetch identity data, then login an existing profile or register a new one. Known providers may have hardcoded endpoints; generic OIDC can use discovery/configured endpoints.
|
||||
- LDAP login validates credentials against the external directory, fetches identity data, then logs in or registers a matching Penpot profile. LDAP registration is not a separate Penpot signup flow.
|
||||
- LDAP session identity MUST come from the directory-returned email (`info.email`): the profile matching the typed email can differ (aliases, UPNs, multi-valued `mail` attributes) and is used only for lockout checks, never to bind the session.
|
||||
- Account lockout (flag `:account-lockout`, `app.auth.login-lockout`) is Redis-backed and keyed per profile id. Password and LDAP flows check/increment on the profile derived from the typed email and clear on the profile that actually logs in.
|
||||
- Lockout and RPC rate limits share the `app.http.errors/handle-error :rate-limit` HTTP path: status 429, body `{:type :rate-limit :code ... :hint ... :ttl ...}`, and any supplied `::http/headers` preserved. Account lockout raises `:code :account-locked` and, when it carries a non-nil `:ttl` (seconds), the handler adds `retry-after`. The RPC limiter (`app.rpc.rlimit`) raises `:code :request-blocked` and sets `retry-after` itself in `::http/headers` (seconds until the longest rejecting limit resets), alongside `x-rate-limit-remaining`/`x-rate-limit-reset`. CORS (`app.http.middleware/with-cors-headers`) exposes `content-type`, `retry-after`, and both `x-rate-limit-*` headers. External flags: `enable-account-lockout`, `enable-rpc-rlimit`.
|
||||
- Logout may return an OIDC provider redirect URI when the session claims include provider/session data and the provider has a logout URI.
|
||||
- Organization SSO (Nitrate): the RPC `check-nitrate-sso` takes the route's **query string proper** (`:query-string`, e.g. `screen=workspace&team-id=…`), never a URL and never with the `?` separator: the client strips it (`rt/token->query-string`) and the server puts it back. The server builds the destination with `app.auth.oidc/build-dest-url` (`public-uri` + `?` + the params, parsed and rendered back), so the redirect can never leave the public URI; a value whose parsed keys are not param names is not a route: a URL parses into a single key holding the whole URL, and a key carrying `:`, `/`, `?` or `#` is rejected with a `:warn` and the user lands on the public URI (the RPC never fails because of it). Do not reason from `u/map->query-string`: it keeps a valueless key as `key=`, so rendering is not what filters a URL out — the key shape is. A leading `?` is still trimmed server-side, so a client bundled before the trim keeps its destination. Never require `screen`: the root of the app carries its params instead of a screen (`?template=…`), and `check-nitrate-sso` is called from that branch too (`dtm/resolve-login-team-id`, which today reads only `:authorized` and never follows the redirect, so a wrong destination there is a silent `:warn` and not a visible break); the callback re-checks the destination before redirecting, which covers state tokens minted before that. The state token also carries `:profile-id`: only that session is granted the `props :sso` entry (`session-mismatch` otherwise, and the redirect still happens).
|
||||
- Organization SSO needs `app.nitrate/organization-access?` whenever the RPC receives an `:organization-id` (that id always comes from the client): the profile must be a member of the organization or its owner; otherwise the RPC answers `{:authorized true :reason :no-organization-access}`, the same short-circuit shape as `:no-team-access`.
|
||||
- Invitation tokens are verified through token issuers and only accepted when the token member id/email matches the authenticated profile; otherwise login proceeds without consuming the invitation.
|
||||
- When public registration is disabled, invitation-based password registration also requires a matching, non-expired `team_invitation` row; registration revalidates it under a row lock before creating the profile. SSO registration (OIDC and compatible providers) follows the OIDC callback gate: it requires `:registration` or `:oidc-registration`, or a team invitation token issued for the registering email (an invitation substitutes the flags only when `member-email` matches the OIDC/`prepared-register` email via `clean-email`, as in the password flow, which raises `:email-does-not-match-invitation` otherwise). The OIDC callback verifies the invitation JWT and the email before redirecting to register, and `register-profile` revalidates the invitation under a row lock when no SSO flag is set; it never requires password login. Only OIDC-typed backends (`oidc-provider-types` in `app.auth.oidc`) take the SSO registration gate; any other `:backend` in a prepared-register token is rejected as `:invalid-token`.
|
||||
- Access tokens (`app.http.access-token`): personal tokens have JWT issuer `access-token` and DB type other than `mcp`; MCP tokens have issuer `urn:penpot:mcp-token` (`actoken/mcp-token-issuer`) and DB type `mcp`. Authentication requires issuer and type to agree, and the claim `uid` to match the row's profile; other issuers and undecryptable tokens are treated as anonymous.
|
||||
- Management RPC (`/api/management/methods/:name`) is shared-key only. The middleware authenticates `<id> <secret>`; the RPC wrapper `wrap-shared-key-authorization` authorizes the key id against the method's `::rpc/allowed-shared-keys` set. Absent metadata accepts any configured key; all current methods declare one (`exporter`/`media-processor` for `upload-tempfile`, `nexus` for file analysis, `exporter` for jobs, `admin-console` for Nitrate). A valid key outside the set answers 403 `:shared-key-not-allowed`; denied calls do not reach the audit wrapper.
|
||||
- Personal tokens need flag `:access-tokens`. MCP tokens need `:mcp` or `:access-tokens`, and only on routes whose `actoken/authz` gets `::actoken/allow-mcp true` (the RPC routes); elsewhere (e.g. assets) they are anonymous.
|
||||
- On RPC, an MCP token reaches only commands with `::rpc/token-type "mcp"` (`wrap-authentication` in `app.rpc`); every other command answers 403 `:token-auth-required`. A command with that metadata also rejects sessions and personal tokens. Currently only `send-mcp-feedback`. Blocking a profile invalidates sessions, not tokens, so such commands check `:is-blocked`/`:is-active` themselves.
|
||||
- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/subtleties`.
|
||||
- HTTP/session parsing details such as cookie/header precedence, JWT session token versions, and SameSite behavior are in `mem:backend/http-storage-filedata-subtleties`.
|
||||
|
||||
## Permission model
|
||||
|
||||
- `app.rpc.permissions` provides predicate/check factories. Failed permission checks intentionally raise `:not-found` / `:object-not-found`, not an authorization-specific error, to avoid leaking object existence.
|
||||
- Team role flags are normalized as owner > admin > editor > viewer. Owner/admin imply edit; any membership row implies read.
|
||||
- File/project role rows only count with a live team membership: the file/project permission queries require a matching `team_profile_rel` row, and leaving or removing a member deletes their team-scoped `file`/`project`/`team-project-profile-rel` rows (GHSA-v9r9-h77c-55m2).
|
||||
- The organization-owner fallback (`get-organization-owner-permissions` via Nitrate over HTTP) grants read-only access for a non-member organization owner. It resolves the owning team through `resolve-team-id`, which refuses soft-deleted file/project/team: the file and project lookups filter `deleted_at`, and a final `db/get*` check covers the team row itself — `nil` means no access. `db/get*` only filters soft-deleted rows when `deleted_at` is in the projection. The re-checks of open websocket subscriptions rely on this (`mem:backend/websocket-subscriptions`).
|
||||
- File/project/comment checks are implemented in the owning command namespaces, often via helpers imported from `files`, `teams`, or `projects`; do not bypass those helpers with direct DB lookups unless preserving their not-found semantics.
|
||||
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries. RPCs that accept `share-id` enforce the link's page scope before mutating (`share-link-pages` + `check-page-scope`): a thread or page outside the link raises `:not-found`, and that check runs before the ownership checks. `get-profiles-for-file-comments` filters its query by the same pages. `mark-all-threads-as-read` does not take `share-id`.
|
||||
- Comment permission includes both logged-in state and the file/team comment policy. Shared viewer paths may pass `share-id`; preserve that path when changing comment queries.
|
||||
|
||||
## Teams, projects, and invitations
|
||||
|
||||
- Team/project commands mix DB changes, email, message bus notifications, media/storage cleanup, feature flags, quotas, and audit metadata. Keep mutations transactional when the existing command does so.
|
||||
- Invitation flows validate muted/bounced emails before sending and use tokenized invitation state. Accepting an invitation is tied to the invited member identity, not just possession of a token.
|
||||
- The whole invitation family is team-admin only: creating, updating and deleting invitations, minting the invitation token (`get-team-invitation-token`) and listing the pending ones (`get-team-invitations`, which carries the invited email and the offered role). The token of an invitation that grants the `owner` role is restricted to the team owner, because an admin cannot create nor promote an owner invitation (`cant-promote-to-owner`). This family is the exception to the rule above: it answers `:validation` / `:insufficient-permissions` instead of `:not-found`, so the five commands speak one language.
|
||||
- With the `:admin-console` flag, the organization `send-invitations` policy (role value plus Nitrate `restrictions-met`, through `cto/allowed?`) gates every invitation command that hands out or manages access: create (and resend, which reuses it), `update-team-invitation-role`, `delete-team-invitation` and `get-team-invitation-token`. Only `get-team-invitations` stays on team admin alone, so a blocked admin still sees the list. The shared helper `check-send-invitations-policy` in `teams_invitations.clj` does a strict Nitrate lookup (fails closed; 404 means no organization) and runs outside a DB transaction, which is why the role and delete commands carry no `::db/transaction`.
|
||||
- Logical deletion is used for many product objects; prefer existing logical-deletion helpers over hard deletes unless the command already performs permanent cleanup.
|
||||
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv Mailpit is the normal local path for registration/email-flow testing.
|
||||
- Bounced/spam-complaint emails can mute/block a profile for login/registration and email sending. Devenv MailCatcher is the normal local path for registration/email-flow testing.
|
||||
|
||||
## Comments, webhooks, and audit
|
||||
|
||||
@ -51,4 +37,4 @@
|
||||
- Enable LDAP login locally with frontend flag `enable-login-with-ldap`; the devenv includes a configured test LDAP service.
|
||||
- OIDC testing requires external provider app credentials plus matching backend/frontend config.
|
||||
- Backend domain tests usually live under `backend/test/backend_tests/rpc/commands/*_test.clj` or nearby backend test namespaces. Use focused `clojure -M:dev:test --focus ...` from `backend/` when possible.
|
||||
- For auth/session or HTTP behavior, combine backend tests with the HTTP/session notes in `mem:backend/subtleties` because RPC-level tests may not exercise cookie/header transforms.
|
||||
- For auth/session or HTTP behavior, combine backend tests with the HTTP/session notes in `mem:backend/http-storage-filedata-subtleties` because RPC-level tests may not exercise cookie/header transforms.
|
||||
@ -4,18 +4,14 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
|
||||
|
||||
## Focused memories
|
||||
|
||||
- Cross-cutting backend subtleties (RPC, DB helpers, workers, cron, CLIMIT, config and HTTP/session middleware, http self-metrics, media processing, and file data persistence): `mem:backend/subtleties`
|
||||
- Storage abstraction, logical buckets, object lifecycle, deduplication, access, and garbage collection: `mem:backend/storage`.
|
||||
- Embedded Ladybug graph experiment, projection, incremental sync, console, and risks: `mem:backend/graph-experiment`
|
||||
- Session lifetime config, token `:exp`, and idle/absolute GC: `mem:backend/session-expiration`.
|
||||
- RPC, DB helpers, workers, cron: `mem:backend/rpc-db-worker-subtleties`
|
||||
- HTTP sessions, config, storage, media, file data persistence: `mem:backend/http-storage-filedata-subtleties`
|
||||
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
|
||||
- WebSocket subscriptions, the connection registry (local per backend, unlike the bus), and how a lost permission gets revoked: `mem:backend/websocket-subscriptions`.
|
||||
- Audit-log event collection (RPC wrapper, frontend ingestion), telemetry duality, webhook fan-out, error reporters, Nexus archival and retention: `mem:audit-log` (guía completa, también para perfiles junior; el stub `mem:backend/audit-log` solo redirige aquí)
|
||||
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
|
||||
|
||||
## Stable namespace map
|
||||
|
||||
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/main/methods/<method-name>`; `/api/rpc/command/<method-name>` remains as a backward-compatible route.
|
||||
- `app.rpc.commands.*`: RPC command implementations exposed under `/api/rpc/command/<cmd-name>`.
|
||||
- `app.rpc.permissions`: permission predicate/check helper factories.
|
||||
- `app.http.*`: HTTP routes and middleware.
|
||||
- `app.auth.*`: provider-specific authentication helpers such as LDAP/OIDC.
|
||||
@ -25,7 +21,6 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
|
||||
- `app.worker`: task execution/cron plumbing.
|
||||
- `app.main`: Integrant system map and component wiring.
|
||||
- `app.config`: `PENPOT_*` env config and feature flags.
|
||||
- `app.health`: per-process availability flag; `/readyz` answers 503 while offline. Flipped from `app.srepl.main` (`set-offline` / `set-online`).
|
||||
- `app.srepl.*`: development REPL helpers for manual backend operations (data inspection, migration helpers, one-off admin tasks).
|
||||
- `app.nitrate`, `app.rpc.commands.nitrate`, and `app.rpc.management.nitrate`: external Nitrate subscription/organization integration, gated by the `:nitrate` feature flag and shared-key HTTP calls.
|
||||
|
||||
@ -33,65 +28,24 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
|
||||
|
||||
RPC commands are defined with `app.util.services/defmethod` and schemas. Use `get-` prefixes for read operations. Command metadata usually includes auth, docs version, params schema, and result schema. Return plain maps/vectors or raise structured exceptions from `app.common.exceptions`.
|
||||
|
||||
Backend RPC command areas without focused memories include access tokens (auth rules, incl. MCP tokens, in `mem:backend/auth-permissions-product-domains`), binfile, demo, feedback, file snapshots, fonts, management, Nitrate, and webhooks beyond the notes in `mem:backend/auth-permissions-product-domains`; inspect nearby command tests and command metadata before changing them.
|
||||
Backend RPC command areas without focused memories include access tokens, binfile, demo, feedback, file snapshots, fonts, management, Nitrate, and webhooks beyond the notes in `mem:backend/auth-permissions-product-domains`; inspect nearby command tests and command metadata before changing them.
|
||||
|
||||
## DB conventions
|
||||
|
||||
`app.db` helpers accept cfg, pool, or conn in most places and convert kebab-case to snake_case:
|
||||
- `db/get`, `db/get*`, `db/query`, `db/insert!`, `db/update!`, `db/delete!`.
|
||||
- `db/get*`/`db/query` signature is `(ds table where-params & {:as opts})`: trailing keywords become an opts MAP (ignored by the select builder — use `{::db/remove-deleted false}` map as the single vararg to also see rows with `deleted_at` set), while a trailing ODD number of keywords crashes with "Don't know how to create ISeq from: Keyword". Extra keywords are NOT a column filter: rows always come back with all columns.
|
||||
- Next.js/JDBC caveat: where-maps don't support value vectors (no `IN`); use `status = ANY(?)` with `db/create-array` inside `db/tx-run!`.
|
||||
- Use `db/run!` for multiple operations on one connection.
|
||||
- Use `db/tx-run!` for transactions.
|
||||
- `job`-substrate specifics (unified jobs): the only writers of a job status are `claim`, `retry-job`, `complete`, `fail` and `cancel` in `app.jobs`; each one is a conditional update plus its `job_event` row in the same transaction, so history can never contradict the row. A writer that affects no row writes no event and no metric. They never join the caller's transaction, because a lifecycle write reports on work that already happened: a rolled back caller cannot undo a transition, and an independent connection cannot see the caller's uncommitted job row, so a job must be committed before it can be claimed. GC deletes return resource ids via `RETURNING` inside one transaction.
|
||||
- Every public function of `app.jobs` takes a cfg map. A bare pool or connection is a caller bug and is not coerced: the substrate has no conversion of its own, and the ones in `app.db` are the db layer's business.
|
||||
- Every new foreign key of the substrate is `DEFERRABLE`, `job_event.job_id` included.
|
||||
- Some tables carry a SQL deletion guard (`raise_deletion_protection` + a `deletion_protection__tgr` statement trigger): `profile`, `team`, `team_font_variant`, `file_media_object`, `file_thumbnail`, `file_tagged_object_thumbnail` and `job`. A `DELETE` raises unless the transaction runs `SET LOCAL rules.deletion_protection TO off` (the guard is on when the GUC is unset, empty or `on`); see `app.tasks.objects-gc`, `app.jobs.gc` and `th/db-force-delete`.
|
||||
|
||||
Database migrations live in `backend/src/app/migrations/`; pure SQL migrations are under `backend/src/app/migrations/sql/`. SQL filenames conventionally start with a sequence and verb/table description, e.g. `0026-mod-profile-table-add-is-active-field`. Applied migrations are tracked in the `migrations` table.
|
||||
|
||||
Migrations must stay within PostgreSQL 15 syntax: the official docker images (`docker/images/docker-compose.yaml`) pin `postgres:15`, so anything newer breaks those deployments (`SET STORAGE DEFAULT` is PostgreSQL 16, for instance). The documented floor is >=15.
|
||||
For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/rpc-db-worker-subtleties`.
|
||||
|
||||
For interactive PostgreSQL access with correct dev defaults, use `scripts/psql`; to dump the current DDL schema, use `scripts/db-schema` (see `mem:scripts/psql`).
|
||||
## Background tasks
|
||||
|
||||
For deeper details on transaction semantics, advisory locks, Transit vs JSON helpers, and dev/test DB URLs: `mem:backend/subtleties`.
|
||||
|
||||
## Background tasks (unified jobs)
|
||||
|
||||
Every background job is a job-def: a plain `(defn execute-X [cfg params] ...)` in its namespace + a malli params schema + an `ig/init-key` that returns the job-def map `{::jobs/name, ::jobs/schema, ::jobs/handler, ::jobs/decoder, ::jobs/validator}` (decoder/validator precompiled at init), optionally with `::jobs/result-schema` when the result carries types plain JSON does not keep (a uuid, whether a value or a map key): the substrate compiles the decoder from that schema on registry init and checks a result against the schema before storing it. The registry is the `::jobs/defs` wiring in `app.main`, which also populates a module-level registry used by `jobs/submit` as fallback — job-def components cannot ig/ref `::jobs/defs` (wiring cycle). Submit from RPC code passes its RPC cfg (it carries `::jobs/defs` via ig/ref).
|
||||
|
||||
- `::jobs/handler` is `(fn [context params] ...)`. It is NOT given the runner cfg: a job-def closes over its own dependencies, and handing it the runner cfg would let a handler reach the pool or any other component. A wrapper passes the context to its implementation only when the implementation takes it; most do not, and their implementation stays `[cfg params]`.
|
||||
- The context is built by `jobs/make-context` and has exactly five keys: `id`, `name`, `label`, `resource-id`, `profile-id`, behind a closed schema. `profile-id` is the owner of a user-facing job and nil for an internal one, and it is there so a handler can revalidate permissions and check who owns its resource without a second query. `queue` is absent on purpose (a routing detail of the dispatcher, not a property of the job) and so are `retry-num`/`max-retries`/`attempt` (retry policy belongs to the runner, and an attempt number cannot be durable because the `noop` strategy re-runs without incrementing the counter).
|
||||
- `jobs/invoke` is the in-process escape hatch: it delivers a nil context without a row, validates and delivers `::jobs/context` when given, and keeps `::jobs/job-id` independent. `::jobs/context` and `::jobs/job-id` are cfg options, not params.
|
||||
- Business params live in the `params` column (plain JSON, decoded with the job-def decoder). `submit` takes optional `::jobs/profile-id`, `::jobs/resource-id` (both validated as UUIDs) and `::jobs/expires-at` (an instant: the retention the jobs GC sweeps by), all stored as columns; nothing is inferred from `params`. The binfile creation commands are the callers that pass a profile-id and a resource.
|
||||
- A job-def may carry optional metadata keys read from the registry by its consumers, never by the substrate: `::jobs/family` (e.g. `:export`) groups the jobs of a family for quotes, `::jobs/resource-role` (`:input` or `:output`) says whether the job's `resource_id` is what it consumes or what it produces, and `::jobs/queue-name` names the queue the creation command routes the job to (`submit-job` falls back to `:binfile` when the def does not set it).
|
||||
- Job resources (`app.jobs.storage`) are the objects a job owns through `job.resource_id`: they live in the `job-resource` bucket, are written with the profile of the job as owner (only that profile can read them back from `/assets/by-id`) and are never deduplicated. They carry no expiry of their own, because `expired-at` is stored as `deleted_at` and an object with it is unreadable from birth: the retention of the artifact is the retention of its job row. The writer marks the object as touched right away, so an artifact that no row references (a crash between the upload and the completion) is reclaimed by `storage-gc-touched`, and a row that does reference it freezes it until the jobs GC deletes the row and touches the object.
|
||||
- The binfile jobs (`:export-binfile` and `:import-binfile`) run the core of `bf.v1`/`bf.v3` through `app.binfile.jobs`, which turns the `:progress` taps of the core into `progress` events of the job with `jobs/heartbeat` (the `modified_at` touch at most once per second, progress events at up to ~200ms). Each event is a self-contained milestone: a `:stage` keyword of one vocabulary shared by the two formats (the same work has a different name in v1 and v3) and an optional `:counters` map, with the counter of the activity named after its own stage and the outer scopes in course as context, which the tap carries itself (the reader puts `::outer-counters` in every tap inside them — today only `:files`, extendable to `:teams`/`:projects` for multi-scope exports — so the adapter remembers nothing between taps and a tap that happens outside any scope has no context); the ids and names the taps carry never reach the event. Every tap is reported, so a tap with no units to count still keeps the job alive, and a milestone the throttle drops loses nothing because the next one carries the same state. The adapter checks that the job is still active before the run, and the beat raises an `:interrupt` right there on the run thread when the job is no longer active, so the work stops at its next event point inside the transaction of its handler, which rolls back. A run whose last tap predates the interruption is discarded by the check once it returns. The adapter passes the job id from the handler context. Their job-def components carry pool, storage, metrics and msgbus, because a job with a `profile_id` cannot store an event without a msgbus on its cfg. Both readers answer with the same shape (`:file-ids` and, in version 3, the resolution of the libraries), because the version 1 reader returns a bare set of file ids.
|
||||
- Both binfile handlers revalidate their permission at run time, not only when the job is created, because it can be revoked while the job waits: the export checks the read permission of every file before reading anything and stores the package in `job-resource` as the resource of the job (answering with the completion envelope), and the import checks the edition permission of the destination and runs the core inside a transaction, which is what makes a cancelled import roll back instead of leaving half of a file. The import releases its package (marks it as deleted and drops the reference from the job row in the same transaction) and deletes its temporary copy once the job is terminal, whatever the outcome. Both handlers check their own result against a schema before returning it (the import job-def declares that same schema as its result schema, which is what types the resolution when the row is read).
|
||||
- The v3 and v1 imports only materialize a library relation when both ids belong to the package and the link would neither be a self link nor close a dependency cycle (the same `bfc/library-link-status` check that `link-file-to-library` uses to raise); relations are processed in sorted `(file_id, library_file_id)` order, which `get-files-rels` also returns, so the first edge wins deterministically, and every edge it ignores is logged with a warning. The v2 import stays out of that guard on purpose: it is only reachable from the developer REPL (`app.srepl.binfile`) and keeps inserting the relations as they come. Migration `0158-del-forbidden-file-library-rels` deletes the legacy cross-team and self rows from `file_library_rel` and their `file_library_sync` rows, sync rows first.
|
||||
- A version 1 package lists its files in the metadata and the importer creates one file per entry. The v1 export (gone from the product; only tests and the developer REPL use it) unioned the requested files with their libraries without deduplicating, so a package exported from a file inside a library cycle lists that file twice. The importer reads the pair of objects of every entry to keep the stream in sync, imports the first occurrence only, warns about the repeats, and answers with one id per imported file.
|
||||
- The domain cfg of a binfile request lives in `app.binfile.common` (`export-cfg`, `import-cfg`), so the legacy RPC and the jobs freeze the same ids, resolve the same team features and apply the same anti-zip-bomb limits. `resolve-export-type` is the translation the legacy RPC needs for its deprecated boolean flags; the creation command asks the caller for the type instead of guessing a default.
|
||||
- Two retention knobs, easy to confuse: `:jobs-retention` sweeps terminal internal rows (the ones without a profile), while `:jobs-user-ttl` is the default expiry a user-facing job gets when it is created; the jobs GC deletes the row once `expires_at` passes and touches its resource. The heavy work of a user job runs on its own `:binfile` queue (`:worker-binfile-parallelism`, default 1) so it does not block the default runner.
|
||||
- The quote of a family of jobs (`::export-jobs-per-profile`, `::import-jobs-per-profile`) counts the jobs of the profile that are still on their way (`new`/`scheduled`/`running`/`retry`) whose name the registry gives for that family, so a new job of the family is covered without touching the quote.
|
||||
- A user-facing job is created through `app.rpc.commands.jobs`. One command per job type: `create-export-binfile-job` and `create-import-binfile-job` name the job they create and their body carries only `:params`, whose schema the command imports from the job namespace (`::sm/params`), so the RPC validates the business params up front, before the command runs; the job-def is still the place that owns them and validates them again at run time. The commands check the permission and the quote of the family before the expensive part, submit with `max-retries 0`, an expiry from `:jobs-user-ttl` and the queue the job-def routes to through `::jobs/queue-name` (`:binfile` when the def says nothing; read by the command, never by the substrate), and answer with a summary of the row. The import command has its own creation schema (`import-binfile/schema:create-params`, version optional) because the request is not the stored params: the command fills the version (from the caller or the package header) and the manifest metadata kept for audit. It assembles the upload, reads the manifest of a version 3 package for the audit (never its content) and stores the package in `job-resource` as the resource of the job, releasing it when the job never comes to own it.
|
||||
- `get-job` is the read surface a profile has over its own jobs: it answers with the same summary the creation commands return plus the `result` of a finished job or its public error (through `decode-job-error`, so the keywords come back), and a job of another profile is not distinguishable from one that does not exist: both answer `not-found`. `decode-row` decodes `params`, `result` and `error`, and the result is then decoded with the decoder the substrate compiled from the result schema the job-def the row names declares (`decode-result`; a def without one, or a row whose job-def is gone, keeps the plain JSON the row holds: a uuid as the text it is on disk, a uuid used as a map key as a keyword). The event log still has no read surface: a client that follows a job takes its progress from the websocket.
|
||||
- `heartbeat` uses named options `[cfg & {:keys [job-id progress] :as options}]` and returns the number of durable writes, so 0 means there was nothing to write: no job context, or the throttle did not allow it. A write that finds the job no longer active (terminal or gone) raises an `:interrupt` (`code :job-interrupted`, carrying the job id and the status it found) through the shared `jobs/check-active` instead of counting zero, so a task in course stops at its next beat; the interrupt is internal and never reaches HTTP (the management progress report answers `:skip` for it). The job id falls back to `::jobs/job-id` on the cfg and then to the runner-bound `*job-id*`. The `modified_at` touch is throttled to ~1s and the progress event to ~200ms, so the check fires at most once per second with no extra query (the read of the row happens only when a write already found nothing). A progress report is a milestone (`schema:progress`): a required `:stage` keyword plus an optional `:counters` map of `scope -> {:current, :total?}`, and no other key. Counters are non-negative, a `total` is positive and never lower than its `current`, and stage and counter keys are names of at most 64 characters. A milestone without counters is valid (a point of the run with no units to count), the stage vocabulary belongs to the worker and not to the substrate, and a client must degrade for a stage it does not know. Read a stored payload back with `jobs/decode-progress`: the database holds the keywords as strings. The 1s beat (down from 60s, so a cancelled heavy job is seen as gone at its next beat) costs one `modified_at` touch per beating job per second: cheap while the only heavy beater runs alone on its queue, revisit if concurrent beating jobs ever grow.
|
||||
- `job.error` conforms to `app.jobs/schema:job-error` (`type` and `code` keywords, `hint` text, extra details allowed). Read it back with `jobs/decode-job-error`; a plain `db/decode-json-pgobject` leaves the keywords as strings. `complete` takes named options and an optional `resource-id`, which is only ever set, never replaced: it reads the row locked inside its own transaction and applies the rules in a fixed order, skipping a job that is no longer completable before refusing one for its resource, so the answer is a function of the row and not of the order two callers arrive in. It checks the result against the schema the job-def of the row declares before it writes (a violation raises `:invalid-result` and nothing is stored), and only for a transition that is going to write: a terminal job answers `:skip` without validating. `complete` is the single door of a result, so the check covers the runner and the external worker that completes a job through the management RPC (`::complete-job`), which stores a result the substrate would otherwise take blind.
|
||||
|
||||
For worker dispatch, cron, retry semantics (`ex/raise :type ::wrk/retry` with `:delay`/`:strategy`), deduplication, job events, and queue internals: `mem:backend/subtleties`.
|
||||
|
||||
## Jobs metrics
|
||||
|
||||
- `app.jobs.metrics` owns the jobs metric names, bounded labels and the periodic backlog sampler.
|
||||
- Metrics use only `name`, `queue`, `outcome`, `reason`, `stage` and `kind` labels. Job IDs, profile IDs, params, reply bodies and exception text must never be labels. Every label goes as-is (lowercased where case varies): values come from fixed call sites, so a new value in Prometheus means new code, not noise. Only a missing value falls back (`other`, or `failed` for outcomes).
|
||||
- The canonical job name is a keyword (`app.jobs/schema:job-name`): `submit` and `invoke` reject strings. The row stores text, so `jobs/get-job-def` still coerces (it is the runner's boundary), cron entries carry the job under `:job` with `d/name` only at the SQL/log boundary, and test helpers keyword-ize row names before invoking.
|
||||
- The durable lifecycle metrics are submitted, dispatched, completed, retries, orphaned and rescheduled counters; queue-wait, execution and total-time histograms; dispatcher and backlog state; GC, cron and ephemeral request metrics.
|
||||
- `app.metrics/run!` is safe for recording failures, but metrics components still require a valid metrics instance. Durable submit call sites carry `::mtx/metrics` through RPC or job-def configuration. The jobs substrate takes the **cfg**, not the instance, and resolves it with `app.metrics/instance`, which raises `:missing-metrics` when absent: `app.jobs`, `app.jobs.metrics`, the runner, the dispatcher, cron and the jobs GC all follow that. `app.http`, `app.rpc.climit` and `app.storage.s3.metrics` still take the instance and were left out of scope on purpose, so do not read the codebase as uniform. The six job writers resolve at the top of their body, before writing, because their metric lands in a `db/after-commit` callback that swallows exceptions. The GC, the dispatcher and cron resolve inside that callback instead, so their guarantee rests on the component schema instead. A job that calls `heartbeat` from a component cfg must carry `::mtx/metrics` on it, and the event helper records the counter from the cfg the transaction hands it.
|
||||
- SQL-backed job metrics are registered with `app.db/after-commit` from INSIDE the transaction that changed the row, so a rollback cannot inflate submitted, terminal, retry, event or dispatcher counters. Every lifecycle writer owns that transaction (`app.jobs/own-transaction` drops `::db/conn`), and `db/transact!` gives a transaction that joins no other its own callback context, so the write and its metric commit or vanish together. `submit` is the exception and joins the caller on purpose: creating a job is part of the caller's business work.
|
||||
- `penpot_jobs_events_total` counts stored job events with no labels: the interesting split is `kind`, and grouping a growing append-only table to build a metric is not worth the query on the write path.
|
||||
- The backlog sampler is `::jobs-metrics/sampler`; it runs every 30 seconds on worker-enabled, writable systems, groups by status and publishes the oldest pending age, and does not start on a read-only database.
|
||||
- `penpot_tasks_timing` remains exported for compatibility while the jobs-specific histograms are adopted.
|
||||
A task handler is an Integrant component with `ig/assert-key`, `ig/expand-key`, and `ig/init-key`, returning the function run by the worker. New tasks also need wiring in `app.main`: handler config, worker registry entry, and cron entry if scheduled.
|
||||
|
||||
For worker dispatch, cron, retry semantics, deduplication, and queue internals: `mem:backend/rpc-db-worker-subtleties`.
|
||||
|
||||
## REPL
|
||||
|
||||
@ -99,14 +53,14 @@ In devenv, backend nREPL is exposed on port 6064.
|
||||
|
||||
### Non-interactive eval (preferred for agents)
|
||||
|
||||
`./scripts/nrepl-eval.mjs` connects to an already-running nREPL server and evaluates code. Session state (defs, `in-ns`) persists across invocations via a stored session ID in `/tmp/penpot-nrepl-session-<host>-<port>`.
|
||||
`./tools/nrepl-eval.mjs` connects to an already-running nREPL server and evaluates code. Session state (defs, `in-ns`) persists across invocations via a stored session ID in `/tmp/penpot-nrepl-session-<host>-<port>`.
|
||||
|
||||
```bash
|
||||
./scripts/nrepl-eval.mjs '(+ 1 2)' # single expression
|
||||
./scripts/nrepl-eval.mjs "(require '[my.ns :as ns] :reload)" # reload after edits
|
||||
./scripts/nrepl-eval.mjs -e # inspect last exception (*e)
|
||||
./scripts/nrepl-eval.mjs --reset-session '(def x 0)' # discard session, start fresh
|
||||
./scripts/nrepl-eval.mjs <<'EOF' # multi-expression heredoc
|
||||
./tools/nrepl-eval.mjs '(+ 1 2)' # single expression
|
||||
./tools/nrepl-eval.mjs "(require '[my.ns :as ns] :reload)" # reload after edits
|
||||
./tools/nrepl-eval.mjs -e # inspect last exception (*e)
|
||||
./tools/nrepl-eval.mjs --reset-session '(def x 0)' # discard session, start fresh
|
||||
./tools/nrepl-eval.mjs <<'EOF' # multi-expression heredoc
|
||||
(def x 10)
|
||||
(+ x 20)
|
||||
EOF
|
||||
@ -128,19 +82,20 @@ Fixtures can populate local data for manual testing/perf work. From the backend
|
||||
|
||||
* **Type Hinting:** Use explicit JVM type hints (e.g. `^String`, `^long`) in performance-critical paths to avoid reflection overhead.
|
||||
* **Batch inserts:** Use `db/insert-many!` for bulk row inserts — generates a single SQL with multiple parameter tuples. Avoid on very large datasets (SQL length / parameter count limits).
|
||||
* **Server-side cursors:** Use `db/plan` (fetch-size 1000, forward-only, read-only) for large result sets. Never fetch large collections into memory at once. `db/plan` is an `IReduceInit`: `reduce`/`transduce` over it (`sequence` does not work on it), and with `partition-all` the transducer's completing arity is what flushes the last partial chunk. A plan row is a **view over the open result set**, valid only inside the reducing function: read the columns you need right there — buffering rows to read them later (a `partition-all` buffer, a lazy seq) silently yields nil columns. `db/cursor` (manual `DECLARE`/`FETCH`) is a leftover with no callers left.
|
||||
* **Server-side cursors:** Use `db/plan` (fetch-size 1000, forward-only, read-only) or `db/cursor` for large result sets. Never fetch large collections into memory at once.
|
||||
* **Transaction discipline:** Use `tx-run!` for writes (opens a transaction), `run!` for reads (single connection, no transaction). Set `:read-only` on `tx-run!` when applicable to let PostgreSQL optimize.
|
||||
|
||||
## Lint and Format
|
||||
|
||||
IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory.
|
||||
|
||||
* **Linting:** `pnpm run lint:clj`.
|
||||
* **Formatting:** `pnpm run check-fmt:clj` to check, `pnpm run fmt:clj` to fix. After running `fmt:clj`, `check-fmt:clj` is redundant. Avoid unrelated whitespace diffs.
|
||||
|
||||
**Before linting:** if delimiter errors are suspected (after LLM edits), run `scripts/paren-repair` on the affected files first. Delimiter errors produce misleading linter/compiler output. See `mem:scripts/paren-repair`.
|
||||
* **Linting:** `pnpm run lint` from the repository root.
|
||||
* **Formatting:** `pnpm run check-fmt`. Use `pnpm run fmt` to fix. Avoid unrelated whitespace diffs.
|
||||
|
||||
## Testing
|
||||
|
||||
Backend test commands, coverage rules, and conventions: `mem:backend/testing`.
|
||||
Cross-cutting testing principles, anti-patterns, and verification checklist: `mem:testing`.
|
||||
IMPORTANT: all CLI commands must be executed from the `backend/` subdirectory.
|
||||
|
||||
* **Coverage:** If code is added or modified in `src/`, corresponding tests in `test/backend_tests/` must be added or updated.
|
||||
* **Isolated run:** `clojure -M:dev:test --focus backend-tests.my-ns-test` for a specific test namespace.
|
||||
* **Regression run:** `clojure -M:dev:test` to ensure no regressions in related functional areas.
|
||||
|
||||
@ -1,633 +0,0 @@
|
||||
# Graph Experiment
|
||||
|
||||
## Scope
|
||||
|
||||
- Purpose: project Penpot file data into an embedded Ladybug graph database.
|
||||
- Purpose: keep the graph current with Penpot file changes.
|
||||
- Purpose: expose a read-only graph console for backend debugging.
|
||||
- This is an experiment, not a replacement for PostgreSQL file storage.
|
||||
- The graph subsystem is off unless `:graph` is in the backend flags.
|
||||
- The main Penpot frontend has no graph feature code for this subsystem.
|
||||
- The graph console is a backend-served HTML template with JavaScript.
|
||||
|
||||
## Memory Links
|
||||
|
||||
- Read `mem:backend/core` for backend architecture, HTTP routes, DB rules, and test commands.
|
||||
- Read `mem:backend/subtleties` for RPC and message bus behavior, and for file data loading and realization.
|
||||
- Read `mem:common/changes-architecture` for the change record vocabulary.
|
||||
- Read `mem:frontend/routing-app-shell-subtleties` for the existing notification WebSocket.
|
||||
- Read `mem:prod-infra/core` for Redis or Valkey message bus topology.
|
||||
|
||||
## Branch Surface
|
||||
|
||||
- The graph experiment adds about 6,336 lines and changes about 27 files.
|
||||
- The graph implementation lives under `backend/src/app/graph/`.
|
||||
- The graph console lives at `backend/resources/app/templates/graph-console.tmpl`.
|
||||
- The existing debug page gains graph links in `backend/resources/app/templates/debug.tmpl`.
|
||||
- The existing debug HTTP routes gain graph handlers in `backend/src/app/http/debug.clj`.
|
||||
- The backend system passes the message bus to the debug route component in `backend/src/app/main.clj`.
|
||||
- The backend adds Ladybug and Arrow dependencies in `backend/deps.edn`.
|
||||
- The backend adds JVM options for Ladybug and Arrow native access.
|
||||
- The common flag registry adds `:graph` in `common/src/app/common/flags.cljc`.
|
||||
- The graph experiment adds `graph_sync_parity_test.clj` and `graph_binder_gate_test.clj`.
|
||||
|
||||
## System Model
|
||||
|
||||
### Storage layers
|
||||
|
||||
- PostgreSQL remains the source of truth for Penpot files.
|
||||
- The graph database stores a projection of one file.
|
||||
- A persistent graph uses a `.lbug` path under `PENPOT_GRAPH_DIR`.
|
||||
- The default graph directory is `/tmp/penpot-graph`.
|
||||
- A debug session uses a Ladybug `:memory:` database.
|
||||
- A debug session database lives inside the backend JVM process.
|
||||
- A debug session does not survive a backend restart.
|
||||
- A debug session does not store file data back to PostgreSQL.
|
||||
|
||||
### Two graph update paths
|
||||
|
||||
- Cold projection reads the complete file and rebuilds the graph.
|
||||
- Incremental sync reads file change records and updates the open graph.
|
||||
- Both paths must produce the same graph for the same file state.
|
||||
- The parity test treats cold projection as the reference path.
|
||||
- A reload discards the session graph and uses cold projection again.
|
||||
|
||||
## Main Namespaces
|
||||
|
||||
### `app.graph.ladybug`
|
||||
|
||||
- Opens and closes Ladybug `Database` and `Connection` objects.
|
||||
- Installs and loads the Ladybug JSON extension.
|
||||
- Executes Cypher statements.
|
||||
- Executes prepared statements.
|
||||
- Binds scalar parameters.
|
||||
- Formats UUID, string, integer, number, JSON, and timestamp values.
|
||||
- Formats compound values such as arrays, maps, and structs.
|
||||
- Converts Ladybug values back to Clojure values.
|
||||
- Limits normal query results to 200 rows by default.
|
||||
- Detects result truncation with `:truncated?`.
|
||||
- Uses query timeout `0` by default.
|
||||
- Query timeout `0` disables the timeout.
|
||||
- Provides `validate-on-connection!` for parse, bind, and read-only checks.
|
||||
- `exec-prepared-on-connection!` prepares every statement before the first execution.
|
||||
- A prepare failure stops the batch before a mutation runs.
|
||||
|
||||
### `app.graph.schema`
|
||||
|
||||
- Provides the public schema facade.
|
||||
- Exposes schema version `penpot-graph-slice-4`.
|
||||
- Delegates node and relationship definitions to `app.graph.schema.nodes`.
|
||||
|
||||
### `app.graph.schema.nodes`
|
||||
|
||||
- Holds the single registry for graph node tables.
|
||||
- Generates node DDL.
|
||||
- Generates relationship DDL.
|
||||
- Maps Penpot shape types to graph tables.
|
||||
- Projects source attributes into graph attributes.
|
||||
- Formats graph column values.
|
||||
- Quotes reserved graph labels such as `Group` and `Boolean`.
|
||||
- Defines container tables and shape tables.
|
||||
- Defines `IsChildOf`, `IsInstanceOf`, `RefersTo`, and `FillsSwapSlot`.
|
||||
|
||||
### `app.graph.schema.contract`
|
||||
|
||||
- Records deliberate graph contract decisions.
|
||||
- Renames graph columns such as `:revn` to `revision`.
|
||||
- Drops attributes that do not belong in this graph slice.
|
||||
- Records attributes that the graph does not project.
|
||||
- Applies per-table dropped attributes.
|
||||
- Defines type overrides for vectors, transforms, colors, maps, and JSON arrays.
|
||||
- Maps selected map keys to the frontend JSON naming convention.
|
||||
- `:background-blur` remains a declared unprojected attribute.
|
||||
|
||||
### `app.graph.schema.projection`
|
||||
|
||||
- Derives projected schemas from canonical Malli schemas.
|
||||
- Builds the projected document schema.
|
||||
- Builds projected shape schemas.
|
||||
- Selects the schema for each shape type.
|
||||
|
||||
### `app.graph.schema.types`
|
||||
|
||||
- Maps Malli types to Ladybug types.
|
||||
- Maps matrices to `DOUBLE[6]`.
|
||||
- Maps points to `DOUBLE[2]`.
|
||||
- Maps rectangles to `DOUBLE[4]`.
|
||||
- Maps colors to `UINT32`.
|
||||
- Maps collections to Ladybug arrays.
|
||||
- Maps `:map-of` schemas to `MAP`.
|
||||
- Maps closed scalar maps to `STRUCT`.
|
||||
- Maps other complex values to `JSON`.
|
||||
|
||||
### `app.graph.schema.values`
|
||||
|
||||
- Coerces source values to graph column values.
|
||||
- Writes fixed vectors with deterministic order.
|
||||
- Packs colors into the graph color representation.
|
||||
- Sorts set values when deterministic output is needed.
|
||||
|
||||
### `app.graph.arrow`
|
||||
|
||||
- Loads projection rows with Apache Arrow.
|
||||
- Creates temporary staged node and relationship tables.
|
||||
- Uses `COPY ... FROM (MATCH ...)` for bulk loading.
|
||||
- Groups relationship loads by source and target table pair.
|
||||
- Resolves relationship endpoints with joins.
|
||||
- Does not use `createArrowRelTable` for UUID relationship endpoints.
|
||||
- The Arrow `RootAllocator` nests outside the Ladybug connection and closes after the connection and database close sequence; Ladybug keeps staged buffers until connection close, so an inner allocator leaks.
|
||||
- `with-allocator!` keeps the body exception when the allocator close also throws, attaching the close failure as suppressed.
|
||||
|
||||
### `app.graph.ingest`
|
||||
|
||||
- Fetches a complete file with `bfc/get-file` and `:realize? true`.
|
||||
- Rejects missing files.
|
||||
- Rejects files without file data.
|
||||
- Can run file data validation before projection.
|
||||
- Creates the DDL.
|
||||
- Loads nodes and edges through Arrow.
|
||||
- Executes post-load transforms.
|
||||
- Writes graph metadata last.
|
||||
- Treats the final metadata write as the complete-build marker.
|
||||
- Supports a persistent database path and an open connection.
|
||||
- `ingest-on-connection!` requires a caller-owned `:arrow-alloc` that outlives the connection; it raises `:missing-arrow-allocator` without one.
|
||||
|
||||
### `app.graph.projection.document`
|
||||
|
||||
- Projects `Document`, `Page`, `Component`, and supported shape nodes.
|
||||
- Skips the page root frame.
|
||||
- Creates `IsChildOf` edges from shapes to parents.
|
||||
- Creates page edges to the document.
|
||||
- Creates component edges to the document.
|
||||
- Stores page order in `Page.index` and edge `position`.
|
||||
- Reverses the stored `:shapes` list for Penpot z-order.
|
||||
- Adds `page-id` to every projected shape.
|
||||
- Propagates an instance head `component-id` to descendants.
|
||||
- Stops component inheritance at a non-Frame shape with its own component ID.
|
||||
- Skips deleted components during cold projection.
|
||||
- Logs unsupported shape types and missing shape records.
|
||||
|
||||
### `app.graph.projection.transforms`
|
||||
|
||||
- Runs after the base nodes and edges load.
|
||||
- `link-component-instances` creates `IsInstanceOf` edges.
|
||||
- A Frame needs `component-file` to qualify as an instance head.
|
||||
- `link-shape-refs` creates `RefersTo` edges from `shape-ref`.
|
||||
- Ladybug limits multi-label relationship `MERGE` statements.
|
||||
- The transform emits one statement for each shape-table pair.
|
||||
- `link-swap-slots` creates `FillsSwapSlot` edges.
|
||||
- Swap slot IDs come from `swap-slot-<uuid>` entries in `touched`.
|
||||
- The transform removes swap slot entries from `touched` after edge creation.
|
||||
- The transform order matters because it reads and then changes `touched`.
|
||||
|
||||
### `app.graph.meta`
|
||||
|
||||
- Stores graph provenance in `GraphMeta`.
|
||||
- Stores schema version, source revision, producer, and build time.
|
||||
- The source revision identifies the file revision used for cold projection.
|
||||
|
||||
### `app.graph.stats` and `app.graph.report`
|
||||
|
||||
- `app.graph.stats` counts graph nodes and relationships from the live catalog.
|
||||
- `app.graph.report` prints ingest information for REPL use.
|
||||
|
||||
## Cold Projection Flow
|
||||
|
||||
1. Get the file row and realized file data from PostgreSQL.
|
||||
2. Read the file revision from the file row.
|
||||
3. Build the node and edge projection.
|
||||
4. Create all graph tables from the graph schema.
|
||||
5. Load node rows with Arrow.
|
||||
6. Load relationship rows with Arrow.
|
||||
7. Run `CHECKPOINT;`.
|
||||
8. Run the registered derived transforms.
|
||||
9. Write `GraphMeta` as the final build step.
|
||||
10. Return file ID, file revision, database path, projection stats, and transform stats.
|
||||
|
||||
### Projection node groups
|
||||
|
||||
- `Document` contains file-level attributes without the file data blob.
|
||||
- `Document.options` receives file-level options from the data blob.
|
||||
- `Page` contains page attributes without the page object map.
|
||||
- `Component` contains component attributes without component object maps.
|
||||
- Shape tables contain the supported shape attributes.
|
||||
- The graph stores selected derived attributes such as `page-id`.
|
||||
|
||||
### Projection relationship groups
|
||||
|
||||
- Structural edges use `IsChildOf`.
|
||||
- Page and component edges point to `Document`.
|
||||
- Derived edges come from the post-load transform registry.
|
||||
|
||||
## Incremental Sync
|
||||
|
||||
### Change source
|
||||
|
||||
- `app.rpc.commands.files-update` persists the file update first.
|
||||
- The same command publishes a `:file-change` message to the file topic.
|
||||
- The topic key is the file UUID.
|
||||
- The message contains the file ID, profile ID, session ID, revision, version, and changes.
|
||||
- Library changes also publish a team-topic message.
|
||||
- The graph session only consumes the file-topic `:file-change` messages.
|
||||
|
||||
### Session subscription
|
||||
|
||||
- `app.graph.debug/start-sync-loop!` creates a channel with a dropping buffer of 64.
|
||||
- The session subscribes the channel to the file UUID topic.
|
||||
- The loop reads one message at a time.
|
||||
- The loop ignores message types other than `:file-change`.
|
||||
- The loop stops when the channel closes.
|
||||
- `destroy-session!` closes the channel and purges its message bus subscription.
|
||||
|
||||
### Session state
|
||||
|
||||
- Sessions are stored in a global `defonce` atom.
|
||||
- The map key is the string form of `profile-id`.
|
||||
- One profile has one graph session.
|
||||
- Loading another file first destroys the old session.
|
||||
- A session stores the Ladybug database, connection, and Arrow allocator; the allocator closes after the connection and database.
|
||||
- A session stores a shared lock for graph access.
|
||||
- A session stores file metadata.
|
||||
- A session stores the incremental sync index.
|
||||
- A session stores the message bus channel.
|
||||
- A session stores load time and profile ID.
|
||||
- The session keeps projection statistics but drops full projection rows after index creation.
|
||||
|
||||
### Sync index
|
||||
|
||||
- `build-index` starts from the complete cold projection.
|
||||
- The index stores the graph file ID and document ID.
|
||||
- The index stores the current graph revision.
|
||||
- The index stores page IDs, names, and positions.
|
||||
- The index stores component IDs, names, and deleted state.
|
||||
- The index stores shape table, parent, position, frame, page, and component context.
|
||||
- The index stores child IDs by parent ID.
|
||||
- The index supports later change application without another PostgreSQL file read.
|
||||
|
||||
### Change application
|
||||
|
||||
- `apply-changes!` processes the change list in source order.
|
||||
- Each supported change returns a new index and a list of Cypher statements.
|
||||
- Unsupported changes enter the `:skipped` result.
|
||||
- Supported changes enter the `:applied` result.
|
||||
- The function collects all statements before it executes them.
|
||||
- The function appends a document revision statement when at least one change applies.
|
||||
- The index revision advances only when at least one change applies.
|
||||
- A larger incoming revision than the index revision creates a warning.
|
||||
- A revision gap does not trigger catch-up.
|
||||
|
||||
### Shape change rules
|
||||
|
||||
- `:add-obj` reuses `projection.document/denormalized-shape`.
|
||||
- `:add-obj` creates the shape node and its parent edge.
|
||||
- `:mod-obj` applies supported `:set` operations to graph columns.
|
||||
- `:mod-obj` keeps false and zero values as values.
|
||||
- `:del-obj` deletes shapes in deep post-order.
|
||||
- `:mov-objects` detaches shapes from the old parent.
|
||||
- `:mov-objects` closes the old sibling position gap.
|
||||
- `:mov-objects` inserts shapes at the new position.
|
||||
- `:mov-objects` updates `parent_id` and `frame_id`.
|
||||
- `:mov-objects` rewrites container `shapes` values.
|
||||
- The parent columns and child lists must match a cold projection.
|
||||
|
||||
### Page and component change rules
|
||||
|
||||
- Page add creates a projected page node and a document edge.
|
||||
- Page delete removes the page subtree.
|
||||
- Page modification updates supported page attributes.
|
||||
- Component add creates a component node and document edge.
|
||||
- Component modification updates supported component attributes.
|
||||
- Component delete uses a soft-delete state.
|
||||
- Component restore removes the soft-delete state.
|
||||
- Component purge removes the component node and document edge.
|
||||
- Component sync paths need more parity coverage than the current tests provide.
|
||||
|
||||
## Session Locking
|
||||
|
||||
- The sync loop and HTTP handlers share one lock per session.
|
||||
- The lock protects one Ladybug connection from concurrent access.
|
||||
- Queries acquire the lock before binder validation and execution.
|
||||
- Graph data export acquires the lock before catalog reads.
|
||||
- Session export acquires the lock before `EXPORT DATABASE`.
|
||||
- A long query blocks sync for the same session.
|
||||
- A sync batch blocks queries for the same session.
|
||||
- Ladybug connection thread safety is not assumed.
|
||||
|
||||
## Graph Query Rules
|
||||
|
||||
- The console accepts Cypher text.
|
||||
- Blank query text raises a validation error.
|
||||
- The query first passes Ladybug prepare and bind checks.
|
||||
- The query must pass the engine read-only analysis.
|
||||
- A mutating query is rejected.
|
||||
- The graph console does not provide a write path.
|
||||
- A session graph is rebuilt from the file by Reload.
|
||||
- Normal query results have a 200-row limit.
|
||||
- Query results use string values for the HTML console representation.
|
||||
- JSON requests receive a Transit JSON response with the query and result.
|
||||
- HTML requests receive the rendered console with the result.
|
||||
|
||||
## Graph Data Export
|
||||
|
||||
### G6 data
|
||||
|
||||
- `/dbg/actions/graph-data` reads the live Ladybug database.
|
||||
- It does not read the sync index for nodes and edges.
|
||||
- It therefore shows database drift if a batch fails after index update.
|
||||
- Node export covers all registered node tables.
|
||||
- Relationship export reads the Ladybug relationship catalog.
|
||||
- Relationship export includes source, target, relationship name, and position.
|
||||
- Node and relationship export uses a 100,000-row limit.
|
||||
- The response reports `truncated` when a limit cuts the result.
|
||||
- The response reports buffer-manager memory usage.
|
||||
|
||||
### `.lbug` export
|
||||
|
||||
- `source=file` rebuilds the persistent graph from PostgreSQL file data.
|
||||
- `source=file` runs a synchronous full ingest for each request.
|
||||
- `source=session` exports the caller profile's live in-memory graph.
|
||||
- Session export uses Ladybug `EXPORT DATABASE` to Parquet files.
|
||||
- Session export creates a new `.lbug` database with `IMPORT DATABASE`.
|
||||
- The temporary Parquet staging directory is deleted after import.
|
||||
- The final session `.lbug` file remains in the system temporary directory.
|
||||
- The HTTP response streams the database file to the caller.
|
||||
|
||||
## HTTP Routes and Access
|
||||
|
||||
- The graph routes live in `backend/src/app/http/debug.clj`.
|
||||
- The graph route list is added only when `:graph` is enabled.
|
||||
- `/dbg/graph` serves the graph console page.
|
||||
- `/dbg/actions/graph-files` returns the profile file tree.
|
||||
- `/dbg/actions/graph-load` loads a file into the profile session.
|
||||
- `/dbg/actions/graph-unload` closes the profile session.
|
||||
- `/dbg/actions/graph-reload` rebuilds the loaded file graph.
|
||||
- `/dbg/actions/graph-query` runs a read-only Cypher query.
|
||||
- `/dbg/actions/graph-sync-status` returns the sync state.
|
||||
- `/dbg/actions/graph-data` returns nodes and edges for G6.
|
||||
- `/dbg/actions/graph-export` streams a `.lbug` database.
|
||||
- The `/dbg` session middleware remains active.
|
||||
- The `/dbg` admin middleware remains active.
|
||||
- A devenv host with a profile ID passes the debug authorization rule.
|
||||
- Other hosts need a profile email in the configured admin set.
|
||||
- `/dbg/actions/graph-files` lists reachable teams, projects, and files.
|
||||
- The file tree query has a 500-file limit.
|
||||
- The graph handlers resolve graph namespaces at call time.
|
||||
- The backend requires `app.graph.debug` and `app.graph.ingest` when the flag is on.
|
||||
- Ladybug native loading then fails during route initialization instead of first use.
|
||||
|
||||
## Console Frontend
|
||||
|
||||
### Page type
|
||||
|
||||
- `graph-console.tmpl` is a backend resource template.
|
||||
- It is not a Rumext component.
|
||||
- It is not part of the main frontend route table.
|
||||
- The page uses browser `fetch` calls and a browser WebSocket.
|
||||
- The page loads G6 version `5.1.1` from jsDelivr.
|
||||
|
||||
### File tree
|
||||
|
||||
- The page fetches `/dbg/actions/graph-files`.
|
||||
- The response contains team, project, and file groups.
|
||||
- The page creates the tree with DOM APIs.
|
||||
- A file click submits the graph load form.
|
||||
- The page shows a message when no file exists.
|
||||
|
||||
### Graph rendering
|
||||
|
||||
- The page fetches `/dbg/actions/graph-data`.
|
||||
- The page converts graph nodes and edges to G6 data.
|
||||
- The page skips repaint when the node and edge signature does not change.
|
||||
- The page marks added, removed, and changed graph entities.
|
||||
- The page supports tree, dagre, circular, force, and combo layouts.
|
||||
- The page supports collapsed container combos.
|
||||
- The page has render guards at 4,000 nodes and 8,000 edges.
|
||||
- The `?safe` query option bypasses the render guard.
|
||||
- The page shows graph size by node count and relationship count.
|
||||
- The page shows buffer-manager memory in MiB.
|
||||
- The page reports a CDN failure when G6 is undefined.
|
||||
|
||||
### Query result filtering
|
||||
|
||||
- A query can return `filter_*` columns with node IDs.
|
||||
- The HTML result table hides columns with the `filter_` prefix.
|
||||
- The JSON result keeps the full result.
|
||||
- The graph view uses the hidden IDs to select matching nodes.
|
||||
- The graph view re-runs the query after graph refresh.
|
||||
- This keeps the query filter aligned with the current graph.
|
||||
- A user column named `filter_*` follows the same hiding rule.
|
||||
|
||||
### Node inspector
|
||||
|
||||
- A node click creates a query for that node.
|
||||
- The inspector calls `/dbg/actions/graph-query` with JSON negotiation.
|
||||
- The inspector displays the full projected row.
|
||||
- The inspector uses table and ID values from the graph data.
|
||||
|
||||
## WebSocket Data Flow
|
||||
|
||||
1. The page opens `/ws/notifications` with a random `session-id` query value.
|
||||
2. The page sends `:subscribe-file` with a Transit UUID value.
|
||||
3. The server makes sure that the file exists and that the profile has read permission.
|
||||
4. The server subscribes the connection to the file topic.
|
||||
5. `files_update` publishes `:file-change` to the same topic.
|
||||
6. The graph session consumes the message from its message bus subscription.
|
||||
7. The WebSocket server sends the message to the browser connection.
|
||||
8. The browser adds the change to the changelog.
|
||||
9. The browser fetches sync status after 150 milliseconds.
|
||||
10. The browser fetches graph data after a 400-millisecond debounce.
|
||||
11. The browser repaints the G6 graph when the graph data changes.
|
||||
|
||||
### WebSocket reconnect behavior
|
||||
|
||||
- The page reconnects after three seconds when the socket closes.
|
||||
- The page resubscribes to the file after the socket opens.
|
||||
- The page refreshes sync status after reconnect.
|
||||
- The page refreshes graph data after reconnect.
|
||||
- Reconnect does not recover dropped message-bus changes.
|
||||
- The page shows the sync error or skipped-change state when the status reports it.
|
||||
|
||||
## Feature Flag and Runtime Dependencies
|
||||
|
||||
- `:graph` is defined in `common/src/app/common/flags.cljc`.
|
||||
- The flag is off by default.
|
||||
- `com.ladybugdb/lbug` version `0.20.4` is a backend dependency.
|
||||
- `org.apache.arrow/arrow-memory-netty` version `19.0.0` supports Arrow `RootAllocator`.
|
||||
- The JVM uses `--enable-native-access=ALL-UNNAMED`.
|
||||
- The JVM uses `--add-opens=java.base/java.nio=ALL-UNNAMED`.
|
||||
- The JVM uses `--sun-misc-unsafe-memory-access=allow`.
|
||||
- The JVM options appear in the development alias and backend launch scripts.
|
||||
- A Ladybug version change needs new binder and parity tests.
|
||||
- A JDK version change needs a startup test with the graph flag enabled.
|
||||
|
||||
## Tests
|
||||
|
||||
### `backend-tests.graph-sync-parity-test`
|
||||
|
||||
- Uses two Ladybug `:memory:` databases under one Arrow allocator outer to both; an allocator closed before its connections leaks.
|
||||
- Does not use PostgreSQL or a live graph session.
|
||||
- Projects initial file data into database A.
|
||||
- Applies changes to database A through incremental sync.
|
||||
- Applies the same changes to file data.
|
||||
- Projects the changed file data into database B.
|
||||
- Compares every node row and relationship row.
|
||||
- Reports differences by table, row key, and column.
|
||||
- Covers shape add, shape modification, shape deletion, movement, and page changes.
|
||||
- Contains a test that injects a sync defect and expects a graph difference.
|
||||
- Contains a test that `ingest-on-connection!` without `:arrow-alloc` raises `:missing-arrow-allocator`.
|
||||
- Does not cover all component change variants.
|
||||
- Does not cover every movement insertion mode.
|
||||
|
||||
### `backend-tests.graph-binder-gate-test`
|
||||
|
||||
- Creates the live graph DDL in a Ladybug `:memory:` database.
|
||||
- Prepares each sync statement template without executing it.
|
||||
- Detects parse errors and missing tables.
|
||||
- Detects missing columns and bad label quoting.
|
||||
- Reports the expected read-only classification.
|
||||
- Covers reserved node labels across the node registry.
|
||||
- Reports an error result for an invalid statement.
|
||||
|
||||
### Test gaps
|
||||
|
||||
- No automated HTTP handler tests cover graph routes.
|
||||
- No automated session lifecycle tests cover load and unload.
|
||||
- No automated WebSocket tests cover graph subscription.
|
||||
- No automated export tests cover persistent and session sources.
|
||||
- Component add, modify, delete, restore, and purge need parity tests.
|
||||
- Page delete needs parity coverage.
|
||||
- Movement with `:after-shape` needs parity coverage.
|
||||
- Buffer overflow and revision gap behavior need tests.
|
||||
- Partial batch failure and recovery need tests.
|
||||
- Query timeout and long-query behavior need tests.
|
||||
|
||||
## Known Risks and Limits
|
||||
|
||||
### Dropped changes
|
||||
|
||||
- The sync channel uses a dropping buffer of 64.
|
||||
- A burst can discard file-change messages.
|
||||
- The sync loop logs a revision gap when it sees a larger revision.
|
||||
- The sync loop does not fetch missing rows from `file_change`.
|
||||
- Reload is the only built-in recovery path.
|
||||
|
||||
### Partial batch state
|
||||
|
||||
- `apply-changes!` does not provide Ladybug transaction atomicity.
|
||||
- A statement failure can leave a partly changed graph.
|
||||
- The in-memory index can advance before the database state is complete.
|
||||
- `/dbg/actions/graph-data` reads the database and exposes this drift.
|
||||
- Reload rebuilds the graph from PostgreSQL file data.
|
||||
|
||||
### Query resource use
|
||||
|
||||
- The default session query timeout is zero.
|
||||
- A costly query can hold the session lock for a long time.
|
||||
- The same lock blocks incremental sync.
|
||||
- The graph export also holds the same lock during catalog reads.
|
||||
- The graph schema has a high memory floor.
|
||||
- The console reports about 115 MiB for the wide slice before file data.
|
||||
|
||||
### Session lifecycle
|
||||
|
||||
- Sessions have no TTL.
|
||||
- Sessions remain until unload, replacement, or process shutdown.
|
||||
- Each session owns native Ladybug memory and its Arrow allocator.
|
||||
- Many profiles can create many native databases.
|
||||
- A profile load replaces its previous session.
|
||||
- Two browser tabs for one profile share one graph session.
|
||||
|
||||
### Temporary files
|
||||
|
||||
- Session export leaves the final `.lbug` file in the system temporary directory.
|
||||
- Long-lived servers can accumulate exported session databases.
|
||||
- The staging directory is deleted after import.
|
||||
|
||||
### Browser dependency
|
||||
|
||||
- The graph view depends on a runtime CDN request.
|
||||
- A network restriction can remove the G6 view.
|
||||
- Queries and session status still use backend endpoints without G6.
|
||||
|
||||
### Data exposure
|
||||
|
||||
- The graph console can list many files available to the profile.
|
||||
- The console can load complete projected file data.
|
||||
- The console can export a graph database.
|
||||
- The console can inspect all projected node attributes.
|
||||
- The console is safe only when the `/dbg` access boundary is correct.
|
||||
- The graph flag must remain off for deployments that do not need this tool.
|
||||
|
||||
### Contract drift
|
||||
|
||||
- The graph schema is a deliberate slice of the Penpot file model.
|
||||
- New source attributes do not enter the graph automatically in all cases.
|
||||
- Dropped and unprojected attributes need an explicit contract decision.
|
||||
- `applied_tokens` key mapping depends on the JSON naming function.
|
||||
- `filter_*` is a frontend convention, not a graph schema guarantee.
|
||||
|
||||
### Ladybug dialect coupling
|
||||
|
||||
- Cypher strings contain Ladybug-specific syntax.
|
||||
- Label quoting handles reserved labels explicitly.
|
||||
- Relationship transforms depend on Ladybug relationship limits.
|
||||
- Arrow loading depends on Ladybug `COPY FROM (MATCH ...)` behavior.
|
||||
- A dependency upgrade needs schema, binder, Arrow, and parity checks.
|
||||
|
||||
## REPL Helpers
|
||||
|
||||
- `app.srepl.main` resolves graph functions only when a helper runs.
|
||||
- `graph-smoke-test!` runs a basic Ladybug operation.
|
||||
- `graph-query-test!` runs a graph query test.
|
||||
- `ingest-file-to-graph!` projects a file into a graph database.
|
||||
- These helpers use `requiring-resolve` to keep the graph dependency lazy.
|
||||
|
||||
## Operational Invariants
|
||||
|
||||
- PostgreSQL file data remains authoritative.
|
||||
- Cold projection and incremental sync must produce equal graph state.
|
||||
- The graph revision must identify the last applied file revision.
|
||||
- The document revision must update when a sync batch applies.
|
||||
- A missing or skipped change must remain visible in sync status.
|
||||
- A graph query from the console must be read-only.
|
||||
- A graph session must serialize connection access.
|
||||
- Graph routes must remain behind the `:graph` flag and `/dbg` access control.
|
||||
- The Arrow allocator nests outside the Ladybug connection and must outlive all Ladybug operations that use its buffers; a session owns its allocator until unload.
|
||||
- `GraphMeta` must be written after the full ingest and transforms finish.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `backend/src/app/graph/ladybug.clj`: Ladybug API and query gates.
|
||||
- `backend/src/app/graph/arrow.clj`: Arrow bulk load.
|
||||
- `backend/src/app/graph/ingest.clj`: Complete file ingest.
|
||||
- `backend/src/app/graph/debug.clj`: Session lifecycle, sync loop, query, and export.
|
||||
- `backend/src/app/graph/sync.clj`: Incremental change application.
|
||||
- `backend/src/app/graph/meta.clj`: Graph provenance.
|
||||
- `backend/src/app/graph/stats.clj`: Graph counts.
|
||||
- `backend/src/app/graph/report.clj`: REPL ingest report.
|
||||
- `backend/src/app/graph/projection/document.clj`: Base document projection.
|
||||
- `backend/src/app/graph/projection/transforms.clj`: Derived relationship transforms.
|
||||
- `backend/src/app/graph/schema/nodes.clj`: Node and relationship registry.
|
||||
- `backend/src/app/graph/schema/contract.clj`: Projection contract decisions.
|
||||
- `backend/src/app/graph/schema/projection.clj`: Malli projection schemas.
|
||||
- `backend/src/app/graph/schema/types.clj`: Malli-to-Ladybug type mapping.
|
||||
- `backend/src/app/graph/schema/values.clj`: Value coercion.
|
||||
- `backend/src/app/http/debug.clj`: Graph route registration and handlers.
|
||||
- `backend/src/app/http/websocket.clj`: File WebSocket subscription handlers.
|
||||
- `backend/src/app/rpc/commands/files_update.clj`: File-change publication.
|
||||
- `backend/src/app/main.clj`: Integrant message bus wiring.
|
||||
- `backend/resources/app/templates/graph-console.tmpl`: Graph console browser code.
|
||||
- `backend/resources/app/templates/debug.tmpl`: Debug page graph links.
|
||||
- `common/src/app/common/flags.cljc`: `:graph` feature flag.
|
||||
- `backend/test/backend_tests/graph_sync_parity_test.clj`: Cold versus sync parity.
|
||||
- `backend/test/backend_tests/graph_binder_gate_test.clj`: Cypher binder gate.
|
||||
|
||||
## Development Commands
|
||||
|
||||
- Run backend commands from the `backend/` directory.
|
||||
- Run focused parity tests with `clojure -M:dev:test --focus backend-tests.graph-sync-parity-test`.
|
||||
- Run focused binder tests with `clojure -M:dev:test --focus backend-tests.graph-binder-gate-test`.
|
||||
- Run the backend test suite with `clojure -M:dev:test`.
|
||||
- Examine Clojure formatting with `pnpm run check-fmt:clj`.
|
||||
- Run backend Clojure lint with `pnpm run lint:clj`.
|
||||
- Write test output to a file before reading or filtering it.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user