mirror of
https://github.com/penpot/penpot.git
synced 2026-06-16 12:22:22 +00:00
✨ Document and stabilise the parallel-workspace CLI; wire AI agents
Improve parallel-workspaces developer CLI,
and add an opt-in layer that lets four AI
coding agents (Claude Code, opencode, VS Code Copilot, OpenAI Codex CLI)
drive a specific workspace through a single launcher command.
Parallel-workspace semantics
----------------------------
each run-devenv-agentic call brings up one wsN;
--ws N (integer; default 0) targets a specific workspace and auto-starts
ws0 first when N>=1 so the worker invariant holds. --sync is forbidden on
ws0 and re-seeds the workspace from the live repo for ws1+. Stop semantics
mirror the start invariant -- ws0 is the last to stop, shared infra stops
with it, --all walks every instance highest-first. The worker policy
section explains why workers run only on ws0 (Postgres FOR UPDATE
SKIP LOCKED is safe across many workers but the cron dedup primitive is
best-effort, and :telemetry / :audit-log-archive are not idempotent).
Per-instance Valkey Pub/Sub isolation, msgbus topology, and the
"async task notifications miss ws1+ tabs" caveat are stated explicitly.
The mem:prod-infra/core memory captures the same external-services and
task-queue / Pub-Sub topology in agent-readable form, and
mem:backend/core and mem:critical-info now cross-link it so backend work
surfaces the horizontal-scaling constraints from the start.
AI coding agent integration
---------------------------
New top-level .devenv/ directory holds committed templates
(templates/{claude-code,opencode,vscode}.json and templates/codex.toml,
each with \${PENPOT_MCP_PORT} and \${SERENA_MCP_PORT} placeholders) plus
committed shared entries (matching shared/* files for Playwright, the
only workspace-independent server we ship today).
./manage.sh start-coding-agent <claude|opencode|vscode|codex> [--ws N]
launches the chosen client against one workspace. It cd's into the
target's directory (the live repo for ws0; workspace-path "wsN" for ws1+)
and refuses to launch unless (a) the binary is on PATH, (b) the
workspace directory exists for ws1+, and (c) the instance is up
(devenv-main-running) -- the MCP servers only exist while the devenv is
running. The agentic-devenv guide is restructured around this Quick
start path, with a per-client table and a Manual configuration fallback
for clients we don't cover.
Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
6fc03f633a
commit
fccec19243
109
.devenv/README.md
Normal file
109
.devenv/README.md
Normal file
@ -0,0 +1,109 @@
|
||||
# `.devenv/` — Per-Workspace AI-Client MCP Configs
|
||||
|
||||
This directory carries the pieces needed to point an AI coding agent
|
||||
(currently Claude Code, opencode, VS Code Copilot, and the OpenAI Codex CLI)
|
||||
at the MCP servers running inside the parallel devenv instance the developer
|
||||
is currently working in. Every parallel workspace (`ws0`, `ws1`, …) has its
|
||||
own copy because the Penpot MCP and Serena MCP host ports are
|
||||
workspace-specific.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
.devenv/
|
||||
README.md
|
||||
scripts/
|
||||
merge-mcp-config.py # generator helper invoked by manage.sh
|
||||
shared/ # committed; workspace-independent entries
|
||||
claude-code.json # Playwright — same for every workspace
|
||||
opencode.json
|
||||
vscode.json
|
||||
codex.toml
|
||||
templates/ # committed; entries with ${...} port placeholders
|
||||
claude-code.json # Penpot MCP, Serena MCP — port is the only diff
|
||||
opencode.json
|
||||
vscode.json
|
||||
codex.toml
|
||||
mcp/ # gitignored; written by manage.sh per workspace
|
||||
claude-code.json # loaded via Claude Code's --mcp-config flag
|
||||
opencode.json # loaded via OPENCODE_CONFIG env var
|
||||
```
|
||||
|
||||
Two more generated files live outside `.devenv/`, in the directories the
|
||||
clients themselves auto-discover (both gitignored):
|
||||
|
||||
```
|
||||
.vscode/mcp.json # auto-loaded by GitHub Copilot in VS Code
|
||||
.codex/config.toml # auto-loaded by Codex CLI; "trusted project" required
|
||||
```
|
||||
|
||||
* **`shared/`** holds MCP entries that don't depend on the workspace — the
|
||||
browser-driving Playwright server today, plus any other workspace-independent
|
||||
servers we add later. Same content in every workspace, so it's a static
|
||||
checked-in file.
|
||||
* **`templates/`** holds the workspace-specific entries (Penpot MCP, Serena
|
||||
MCP) with `${PENPOT_MCP_PORT}` and `${SERENA_MCP_PORT}` placeholders. The
|
||||
placeholders are resolved per-workspace from the port-base constants in
|
||||
`manage.sh`.
|
||||
* **`mcp/`** plus the two tool-expected paths (`.vscode/mcp.json`,
|
||||
`.codex/config.toml`) are the result of merging `shared/` with the
|
||||
port-substituted `templates/`. `manage.sh` writes them on every
|
||||
`run-devenv-agentic` pass. Gitignored — never edit by hand, your edits will
|
||||
be overwritten on the next reconcile.
|
||||
* **`scripts/merge-mcp-config.py`** is the generator that does the merge.
|
||||
`manage.sh`'s `_merge-mcp-config-{json,toml}` helpers are thin shims over
|
||||
it. Run `python3 .devenv/scripts/merge-mcp-config.py --help` for the CLI;
|
||||
edit the script if you need to change merge semantics, add a new format,
|
||||
or support a new template shape.
|
||||
|
||||
## Launching a coding agent
|
||||
|
||||
The easiest path is the wrapper command, which knows the right flags per
|
||||
client, `cd`'s into the target workspace, and refuses to launch unless the
|
||||
target instance is running and its MCP config has been generated:
|
||||
|
||||
```bash
|
||||
# Default target is ws0 (the live repo).
|
||||
./manage.sh start-coding-agent claude [...args to forward]
|
||||
./manage.sh start-coding-agent opencode [...args to forward]
|
||||
./manage.sh start-coding-agent vscode [...args to forward to 'code']
|
||||
./manage.sh start-coding-agent codex [...args to forward]
|
||||
|
||||
# Target a parallel workspace with --ws N. N is an integer (non-negative);
|
||||
# 'main', 'ws1' and similar spellings are rejected.
|
||||
./manage.sh start-coding-agent claude --ws 1
|
||||
./manage.sh start-coding-agent opencode --ws 2
|
||||
```
|
||||
|
||||
Equivalents by hand (run from inside the workspace directory):
|
||||
|
||||
```bash
|
||||
claude --mcp-config .devenv/mcp/claude-code.json
|
||||
OPENCODE_CONFIG=.devenv/mcp/opencode.json opencode
|
||||
code "$PWD" # VS Code auto-discovers .vscode/mcp.json
|
||||
codex # Codex auto-discovers .codex/config.toml
|
||||
```
|
||||
|
||||
The first `codex` invocation in a workspace will prompt you to **trust the
|
||||
project** — Codex only loads `.codex/config.toml` from trusted projects.
|
||||
|
||||
## Overriding our entries
|
||||
|
||||
Both the auto-discovered configs and the launcher-loaded configs sit *on top
|
||||
of* the developer's global config (with varying precedence rules). All four
|
||||
clients offer escape hatches for shadowing entries we ship:
|
||||
|
||||
* **Claude Code** — `claude mcp add --scope local …` installs a private entry
|
||||
that overrides the one in `mcp/claude-code.json`. Local scope wins.
|
||||
* **opencode** — drop an `opencode.json` at the repo root with the override
|
||||
entries you need. opencode's precedence chain is *global → `OPENCODE_CONFIG`
|
||||
→ project*, so the project file always wins. The root `opencode.json` is
|
||||
gitignored on purpose, since these overrides are personal.
|
||||
* **VS Code Copilot** — VS Code's user-profile MCP config and `.vscode/mcp.json`
|
||||
are both loaded; same-name entries can be shadowed in the user profile.
|
||||
* **Codex CLI** — entries in `~/.codex/config.toml` override the project file
|
||||
for the same `[mcp_servers.<name>]` table.
|
||||
|
||||
See `docs/technical-guide/developer/agentic-devenv.md` for the broader
|
||||
client-configuration story (browser remote debugging, AI-client config
|
||||
schemas, manual setup for unsupported clients).
|
||||
113
.devenv/scripts/merge-mcp-config.py
Executable file
113
.devenv/scripts/merge-mcp-config.py
Executable file
@ -0,0 +1,113 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Merge a shared MCP-server config with a port-substituted template into one
|
||||
output file for one AI coding-agent client.
|
||||
|
||||
Invoked per workspace by manage.sh's `write-instance-mcp-configs`. Each
|
||||
supported client (Claude Code, opencode, VS Code Copilot, OpenAI Codex CLI)
|
||||
ships a `.devenv/shared/<tool>.{json,toml}` (workspace-independent entries,
|
||||
e.g. Playwright) and a `.devenv/templates/<tool>.{json,toml}` (per-workspace
|
||||
entries with `${PENPOT_MCP_PORT}` / `${SERENA_MCP_PORT}` placeholders). This
|
||||
script combines the two into the final config that the client loads.
|
||||
|
||||
Two formats are supported:
|
||||
|
||||
json Deep-merge two JSON documents under a configurable top-level key
|
||||
(`mcpServers` for Claude Code, `mcp` for opencode, `servers` for VS
|
||||
Code Copilot). Same-name entries in the template override entries
|
||||
in shared.
|
||||
toml Concatenate two TOML chunks. Codex's `[mcp_servers.<name>]` layout
|
||||
puts every server in its own top-level table, so a textual concat
|
||||
is equivalent to a structural merge -- and avoids pulling in a
|
||||
third-party TOML writer.
|
||||
|
||||
In both modes, `${VAR}` placeholders inside *either* chunk are resolved
|
||||
from the current environment (only template chunks carry placeholders in
|
||||
practice, but the substitution is uniform either way) using Python's
|
||||
`os.path.expandvars`. Undefined placeholders are left as `${VAR}` literal
|
||||
text -- callers (i.e. manage.sh) are responsible for exporting the
|
||||
variables before invoking the script.
|
||||
|
||||
Usage:
|
||||
merge-mcp-config.py --format json --key <key> <shared> <template> <out>
|
||||
merge-mcp-config.py --format toml <shared> <template> <out>
|
||||
|
||||
Exit codes:
|
||||
0 success
|
||||
2 argparse error (missing required option, bad value, unreadable input)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def merge_json(shared_path: Path, tpl_path: Path, out_path: Path, key: str) -> None:
|
||||
"""Deep-merge two JSON documents under a single top-level dict key.
|
||||
|
||||
Both files must be valid JSON. Entries from the template under `key` take
|
||||
precedence over entries from the shared chunk with the same name. Keys
|
||||
other than `key` are taken from shared (the template is only expected to
|
||||
contribute MCP-server entries; everything else lives in shared).
|
||||
"""
|
||||
shared = json.loads(shared_path.read_text())
|
||||
tpl = json.loads(os.path.expandvars(tpl_path.read_text()))
|
||||
|
||||
merged: dict = {**shared}
|
||||
merged[key] = {**shared.get(key, {}), **tpl.get(key, {})}
|
||||
|
||||
out_path.write_text(json.dumps(merged, indent=2) + "\n")
|
||||
|
||||
|
||||
def concat_toml(shared_path: Path, tpl_path: Path, out_path: Path) -> None:
|
||||
"""Concatenate two TOML chunks separated by a blank line.
|
||||
|
||||
Relies on the convention that each MCP server occupies its own
|
||||
`[mcp_servers.<name>]` table at the top level; no two chunks may
|
||||
declare the same server name (the resulting file would parse but
|
||||
contain duplicate tables, which Codex would reject).
|
||||
"""
|
||||
chunks = [
|
||||
os.path.expandvars(p.read_text()).rstrip("\n")
|
||||
for p in (shared_path, tpl_path)
|
||||
]
|
||||
out_path.write_text("\n\n".join(chunks) + "\n")
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__.split("\n\n", 1)[0],
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--format",
|
||||
choices=("json", "toml"),
|
||||
required=True,
|
||||
help="Format of the inputs and the output.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--key",
|
||||
help="Top-level JSON key under which MCP entries live (required for --format json).",
|
||||
)
|
||||
parser.add_argument("shared", type=Path, help="Path to the shared chunk.")
|
||||
parser.add_argument("template", type=Path, help="Path to the port-placeholder template chunk.")
|
||||
parser.add_argument("out", type=Path, help="Path the merged result is written to.")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.format == "json":
|
||||
if not args.key:
|
||||
parser.error("--key is required when --format json")
|
||||
merge_json(args.shared, args.template, args.out, args.key)
|
||||
else:
|
||||
if args.key:
|
||||
parser.error("--key is not accepted when --format toml")
|
||||
concat_toml(args.shared, args.template, args.out)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
8
.devenv/shared/claude-code.json
Normal file
8
.devenv/shared/claude-code.json
Normal file
@ -0,0 +1,8 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"playwright": {
|
||||
"command": "npx",
|
||||
"args": ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
|
||||
}
|
||||
}
|
||||
}
|
||||
8
.devenv/shared/codex.toml
Normal file
8
.devenv/shared/codex.toml
Normal file
@ -0,0 +1,8 @@
|
||||
# Workspace-independent MCP servers for the OpenAI Codex CLI.
|
||||
# This block is concatenated with the port-substituted templates/codex.toml
|
||||
# by manage.sh's write-instance-mcp-configs to produce .codex/config.toml at
|
||||
# the workspace root.
|
||||
|
||||
[mcp_servers.playwright]
|
||||
command = "npx"
|
||||
args = ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
|
||||
9
.devenv/shared/opencode.json
Normal file
9
.devenv/shared/opencode.json
Normal file
@ -0,0 +1,9 @@
|
||||
{
|
||||
"mcp": {
|
||||
"playwright": {
|
||||
"type": "local",
|
||||
"command": ["npx", "@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"],
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
9
.devenv/shared/vscode.json
Normal file
9
.devenv/shared/vscode.json
Normal file
@ -0,0 +1,9 @@
|
||||
{
|
||||
"servers": {
|
||||
"playwright": {
|
||||
"type": "stdio",
|
||||
"command": "npx",
|
||||
"args": ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
|
||||
}
|
||||
}
|
||||
}
|
||||
12
.devenv/templates/claude-code.json
Normal file
12
.devenv/templates/claude-code.json
Normal file
@ -0,0 +1,12 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"penpot": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "mcp-remote", "http://localhost:${PENPOT_MCP_PORT}/mcp", "--allow-http"]
|
||||
},
|
||||
"serena-devenv": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "mcp-remote", "http://localhost:${SERENA_MCP_PORT}/mcp", "--allow-http"]
|
||||
}
|
||||
}
|
||||
}
|
||||
10
.devenv/templates/codex.toml
Normal file
10
.devenv/templates/codex.toml
Normal file
@ -0,0 +1,10 @@
|
||||
# Workspace-specific MCP servers for the OpenAI Codex CLI. The PENPOT_MCP_PORT
|
||||
# and SERENA_MCP_PORT placeholders below are filled in per workspace by
|
||||
# manage.sh's write-instance-mcp-configs, then the result is concatenated
|
||||
# with shared/codex.toml to produce .codex/config.toml.
|
||||
|
||||
[mcp_servers.penpot]
|
||||
url = "http://localhost:${PENPOT_MCP_PORT}/mcp"
|
||||
|
||||
[mcp_servers.serena-devenv]
|
||||
url = "http://localhost:${SERENA_MCP_PORT}/mcp"
|
||||
14
.devenv/templates/opencode.json
Normal file
14
.devenv/templates/opencode.json
Normal file
@ -0,0 +1,14 @@
|
||||
{
|
||||
"mcp": {
|
||||
"penpot": {
|
||||
"type": "remote",
|
||||
"url": "http://localhost:${PENPOT_MCP_PORT}/mcp",
|
||||
"enabled": true
|
||||
},
|
||||
"serena-devenv": {
|
||||
"type": "remote",
|
||||
"url": "http://localhost:${SERENA_MCP_PORT}/mcp",
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
}
|
||||
12
.devenv/templates/vscode.json
Normal file
12
.devenv/templates/vscode.json
Normal file
@ -0,0 +1,12 @@
|
||||
{
|
||||
"servers": {
|
||||
"penpot": {
|
||||
"type": "http",
|
||||
"url": "http://localhost:${PENPOT_MCP_PORT}/mcp"
|
||||
},
|
||||
"serena-devenv": {
|
||||
"type": "http",
|
||||
"url": "http://localhost:${SERENA_MCP_PORT}/mcp"
|
||||
}
|
||||
}
|
||||
}
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@ -96,4 +96,7 @@
|
||||
/.claude
|
||||
/.playwright-mcp
|
||||
/docker/devenv/instances/
|
||||
/.devenv/mcp/
|
||||
/opencode.json
|
||||
/.codex/
|
||||
/tools/__pycache__
|
||||
|
||||
@ -7,6 +7,7 @@ Backend: JVM Clojure; Integrant; PostgreSQL; Redis/Valkey; RPC; HTTP; storage; m
|
||||
- RPC, DB helpers, workers, cron: `mem:backend/rpc-db-worker-subtleties`
|
||||
- HTTP sessions, config, storage, media, file data persistence: `mem:backend/http-storage-filedata-subtleties`
|
||||
- Auth flows, permission model, teams, projects, invitations, comments, webhooks, audit: `mem:backend/auth-permissions-product-domains`
|
||||
- Services, task-queue/Pub-Sub topology constraints -> `mem:prod-infra/core`.
|
||||
|
||||
## Stable namespace map
|
||||
|
||||
|
||||
@ -21,7 +21,7 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
|
||||
This is a monorepo. Principles that apply to one module do *not* generally apply to others. Do not make assumptions.
|
||||
|
||||
- `frontend/`: ClojureScript + SCSS SPA/design editor.
|
||||
- `backend/`: JVM Clojure HTTP/RPC server with PostgreSQL, Redis, storage, mail, and workers.
|
||||
- `backend/`: JVM Clojure HTTP/RPC server with PostgreSQL, Redis, storage, mail, and workers.Runtime services and the task-queue vs Pub/Sub topology that constrains horizontal scaling: `mem:prod-infra/core`.
|
||||
- `common/`: shared CLJC data types, geometry, schemas, file/change logic, and utilities.
|
||||
- `render-wasm/`: Rust -> WebAssembly Skia renderer consumed by frontend.
|
||||
- `exporter/`: ClojureScript/Node headless Playwright SVG/PDF export.
|
||||
|
||||
@ -5,13 +5,13 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
|
||||
## Compose project layout
|
||||
|
||||
- `penpotdev-infra`: shared `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. File: `docker-compose.infra.yml`.
|
||||
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 binds `$PWD`; ws1+ bind clones at `~/.penpot/penpot_workspaces/wsN/`.
|
||||
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 (a.k.a. `main`) binds `$PWD`; ws1+ bind clones at `${PENPOT_WORKSPACES_DIR}/wsN/` (default `~/.penpot/penpot_workspaces/`), maintained by the developer.
|
||||
- All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands.
|
||||
|
||||
## Source-of-truth files
|
||||
|
||||
- `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable.
|
||||
- `docker/devenv/instances/wsN.env` (N≥1): auto-generated per reconciler pass. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored.
|
||||
- `docker/devenv/instances/wsN.env` (N≥1): regenerated by `write-instance-env` on every ws1+ start. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored.
|
||||
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`.
|
||||
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
|
||||
|
||||
@ -25,7 +25,7 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
|
||||
|
||||
## Worker policy
|
||||
|
||||
Backend workers run only on ws0. Task queue is shared (one Postgres DB) but Pub/Sub is per-instance Valkey: a task triggered from ws0's UI must complete on ws0 so its notification reaches the originating WebSocket. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. Known consequence: async tasks triggered from a ws1+ tab won't see completion notifications.
|
||||
Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. ws0 must be running whenever any ws1+ is running, and is the last instance to stop — `run-devenv-agentic --ws N` (N≥1) auto-starts ws0 first; `stop-devenv` refuses to stop ws0 while any ws1+ is up. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`.
|
||||
|
||||
## Port layout
|
||||
|
||||
@ -44,26 +44,30 @@ Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin d
|
||||
|
||||
## Tmux + MCP routing
|
||||
|
||||
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus optional `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). The conditional windows are added only on create — to switch from non-agentic to agentic, kill the session first.
|
||||
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv-agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv-agentic` — the conditional windows are only added at session create time.
|
||||
|
||||
MCP plugin routing is same-origin: frontend uses `<public-uri>/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`.
|
||||
|
||||
## Workspace orchestration (ws1+)
|
||||
|
||||
Workspace directories are user-maintained at `${PENPOT_WORKSPACES_DIR}/wsN`. `run-devenv-agentic --ws i` syncs only when `--sync` is passed, with one exception: if the workspace directory is missing on first use, sync runs implicitly to seed it.
|
||||
|
||||
`sync-workspace wsN`:
|
||||
1. `assert-clean-git-state` — refuses on `.git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}`.
|
||||
1. `assert-clean-git-state` — refuses on `.git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}`. No `--sync-force` escape.
|
||||
2. `rsync -a --delete $PWD/.git/ $workspace/.git/`.
|
||||
3. `git ls-files -z --cached --others --exclude-standard` → `rsync --files-from` (Git is the authority on tracked files; rsync's gitignore filter would drop committed files under gitignored parents like `.clj-kondo/config.edn`).
|
||||
4. `git switch -C "wsN/<current-branch>"` inside the workspace.
|
||||
4. Initial-only copy of `frontend/resources/public/js/config.js` (gitignored, but agentic mode needs it). After the first sync the workspace's copy belongs to the developer — subsequent syncs leave it alone.
|
||||
5. `git switch -C "wsN/<current-branch>"` inside the workspace.
|
||||
|
||||
No `--delete` on the working-tree pass: gitignored caches in the workspace survive. Workspace dir + named volumes survive `compose down`.
|
||||
|
||||
## CLI surface
|
||||
|
||||
- `run-devenv-agentic [--n-instances N] [--no-mcp] [--no-serena] [--serena-context CTX]`: desired-state reconciler. Brings the running set to exactly `{ws0..ws(N-1)}`. Missing → sync + env-file + `compose up` + detached tmux. Extra → `compose down` highest-first (never `-v`). Running-in-target → left alone. `--n-instances 0` is rejected.
|
||||
- `run-devenv-agentic [--ws main|0|wsN|N] [--sync] [--serena-context CTX]`: bring one instance up. Agentic only — MCP and Serena windows are always created. Default target main. Errors out if the target is already running. `--sync` is rejected on main; on ws1+ it's optional (forced only when the workspace dir does not exist yet). Auto-starts ws0 first when the target is ws1+ and ws0 is not yet up.
|
||||
- `stop-devenv [--ws main|0|wsN|N] [--all]`: stop instances. Flags mutually exclusive. `--ws N` (N≥1) stops just that workspace. `--ws 0` or no flag stops ws0 + shared infra, refused while any ws1+ is running. `--all` stops every ws highest-first then ws0, then infra.
|
||||
- `run-devenv`: legacy alias, ws0 non-agentic attached.
|
||||
- `attach-devenv [--instance 0|wsN|N]`: pure attach. Fails fast if instance/session missing.
|
||||
- `run-devenv-shell [--instance 0|wsN|N] [cmd...]`: bash in target instance.
|
||||
- `start-devenv` / `stop-devenv` / `log-devenv` / `drop-devenv`: operate on infra + all parallel instances. `drop-devenv` never removes volumes.
|
||||
- `attach-devenv [--ws main|0|wsN|N]`: pure attach. Fails fast if instance/session missing.
|
||||
- `run-devenv-shell [--instance 0|wsN|N] [cmd...]`: bash in target instance. (`--instance` flag not yet renamed to `--ws`.)
|
||||
- `start-devenv` / `log-devenv` / `drop-devenv`: legacy paths around ws0 + shared infra. `drop-devenv` never removes volumes.
|
||||
|
||||
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.
|
||||
|
||||
33
.serena/memories/prod-infra/core.md
Normal file
33
.serena/memories/prod-infra/core.md
Normal file
@ -0,0 +1,33 @@
|
||||
# Production infrastructure (services Penpot depends on)
|
||||
|
||||
Backend (`app.config`, `PENPOT_*` env vars) is parameterized; deployments choose providers.
|
||||
|
||||
## Services
|
||||
|
||||
- **PostgreSQL**: durable store. Profiles, teams, files, sessions, audit, `storage_object` metadata, the `task` queue, `scheduled_task` cron registry, migrations. File-data also lives here when the file-data backend is `legacy-db`/`db`. One shared DB across all backends.
|
||||
- **Redis (Valkey-compatible)**: per-backend message bus and cache. Concrete uses: msgbus Pub/Sub for collaborative-editing broadcasts and team/profile-org notifications fired by RPC handlers (`app.rpc.notifications`, `files_update`, `teams`, `websocket`); file-summary cache gated by `enable-redis-cache`; rate-limit counters; and the dispatcher→runner work hand-off list `penpot.worker.queue:<tenant>:<queue>`. `PENPOT_REDIS_URI`.
|
||||
- **Object storage**: backends `:s3` and `:fs`. S3 in prod; devenv uses MinIO. Holds uploaded media, file-data when the file-data backend is `storage`, exports. Backend-side details (resolve, dedup, bucket set, file-data backends): `mem:backend/http-storage-filedata-subtleties`.
|
||||
- **SMTP mailer**: invitations, password resets, email verification (sent via the `:sendmail` worker task).
|
||||
- **LDAP** (optional auth provider): helpers in `app.auth.*`, gated by `enable-login-with-ldap`.
|
||||
|
||||
## Task queue and worker model
|
||||
|
||||
Async tasks are enqueued via `wrk/submit!` (`app.worker`), which inserts a row into the shared Postgres `task` table tagged with `queue = "<tenant>:<queue-name>"`. Submission is **fire-and-forget** — RPC handlers never poll, never wait, and workers never publish to msgbus. The only completion signal is the `task` row's `status` / `completed_at` columns, which nothing in `rpc/` reads. Soft-delete RPCs return immediately after marking the top-level row, leaving the cascade and reaping to workers.
|
||||
|
||||
Workers run on backends with `enable-backend-worker` in `PENPOT_FLAGS`. Each worker-enabled backend has a `dispatcher` (polls `task` with `FOR UPDATE SKIP LOCKED`, marks status='scheduled', RPUSHes claimed task IDs into **its own** Redis list) and one or more `runner`s per queue (BLPOP from that same local list, execute, update the Postgres row). The Redis hand-off list is purely intra-backend — cross-backend coordination happens at the Postgres row level.
|
||||
|
||||
## Cross-backend safety
|
||||
|
||||
Postgres row locking is the only correctness primitive: `task` claims via `FOR UPDATE SKIP LOCKED`, cron firing via `FOR UPDATE SKIP LOCKED` on the `scheduled_task` row, plus task-handler-internal locks (e.g. `file_gc_scheduler` locks candidate file rows). This makes the work-claim path safe across any number of worker-enabled backends.
|
||||
|
||||
Two known race patterns survive multi-backend operation:
|
||||
|
||||
- **Cron dedup is best-effort.** The lock on `scheduled_task` is released when the task body finishes. If two backends' cron timers fire for the same scheduled instant with a gap larger than the task body's runtime, both execute it. Penpot's cron entries are idempotent (`session-gc`, `objects-gc`, `storage-gc-*`, `tasks-gc`, `upload-session-gc`, `file-gc-scheduler`); the exceptions are `:telemetry` (would double-report) and `:audit-log-archive` (depends on archive target idempotency).
|
||||
- **`wrk/submit! ::dedupe true`** does a non-atomic `DELETE` then `INSERT`. Concurrent cross-backend submits can both bypass the `DELETE` (each sees the other's uncommitted insert as absent) and end up with duplicate `'new'` rows. Each row claims and runs once independently, so the underlying work is fine; the "at most one pending" guarantee weakens.
|
||||
|
||||
Penpot in production lives with both: horizontal-scale deployments accept "exactly-once" as "essentially-once for idempotent operations." Devenv parallel instances handle it by running workers only on ws0 (see `mem:devenv/core`).
|
||||
|
||||
## See also
|
||||
|
||||
- Devenv composition and the ws0-only worker placement: `mem:devenv/core`.
|
||||
- Storage backend resolution, dedup, file-data lifecycle: `mem:backend/http-storage-filedata-subtleties`.
|
||||
@ -1,11 +1,10 @@
|
||||
# Single source of truth for instance-specific devenv configuration.
|
||||
# Loaded by docker compose via --env-file (see manage.sh).
|
||||
# Loaded by docker compose via --env-file and also sourced by manage.sh
|
||||
# (see manage.sh). Per-instance overlay files (e.g. instances/ws1.env) are
|
||||
# layered on top; variables not overridden fall back to the values here.
|
||||
#
|
||||
# Stage 2 adds per-instance overlay files (e.g. instances/ws1.env) loaded
|
||||
# after this one; variables not overridden fall back to the values here.
|
||||
#
|
||||
# This file is consumed by compose only. Backend runtime defaults that
|
||||
# compose does not care about live in backend/scripts/_env.
|
||||
# Backend runtime defaults that compose does not care about live in
|
||||
# backend/scripts/_env.
|
||||
|
||||
# Compose project name (replaces the -p flag). Set per instance; ws0 default
|
||||
# here. Infra is launched under penpotdev-infra by manage.sh's infra-compose
|
||||
@ -63,3 +62,8 @@ PENPOT_BACKEND_WORKER=true
|
||||
# Tmux session inside the main container.
|
||||
PENPOT_TMUX_SESSION=penpot
|
||||
PENPOT_TMUX_ATTACH=true
|
||||
|
||||
# Base directory holding non-main workspace clones (one subdir per wsN, N>=1).
|
||||
# Consumed by manage.sh only. Default lives in manage.sh ($HOME expansion is
|
||||
# not applied to values in this file). Export PENPOT_WORKSPACES_DIR to
|
||||
# override.
|
||||
|
||||
@ -32,7 +32,6 @@ with a comprehensive toolbox for Penpot development:
|
||||
* **Serena MCP Server** provides code intelligence tools with support for Clojure and TypeScript.
|
||||
Its memory system is used to organise project knowledge in a context-efficient manner.
|
||||
* **Playwright MCP Server** provides tools for browser remote control.
|
||||
* (optional) **GitHub MCP Server** provides tools for interacting with GitHub (issue, PRs, etc.)
|
||||
|
||||
Equipped with the tools provided by these MCP servers, the agent can fully close the development loop,
|
||||
i.e. it can ...
|
||||
@ -67,24 +66,34 @@ creating it if it does not exist, and make sure the `penpotFlags` variable conta
|
||||
var penpotFlags = "enable-mcp";
|
||||
```
|
||||
|
||||
**Running the DevEnv in Agentic Mode.** Start the DevEnv in agentic mode with:
|
||||
**Running the DevEnv in Agentic Mode.** Each invocation starts one instance.
|
||||
`--ws N` (N≥1) auto-starts ws0 first if it's not already up — ws0 is the
|
||||
worker-bearer and must be running whenever any ws1+ is:
|
||||
|
||||
```bash
|
||||
./manage.sh run-devenv-agentic # ws0 only
|
||||
./manage.sh run-devenv-agentic --n-instances 3 # ws0, ws1, ws2
|
||||
./manage.sh run-devenv-agentic # main (ws0)
|
||||
./manage.sh run-devenv-agentic --ws 1 # ws0 if needed, then ws1
|
||||
```
|
||||
|
||||
Per-instance ports are offset by `10000 × N` (ws1's MCP at
|
||||
`http://localhost:14401/mcp`, Serena at `http://localhost:24281`, etc.). See
|
||||
the *Parallel workspaces* section in the [Dev environment guide](./devenv.md).
|
||||
Starting an instance that is already running is an error. Per-instance ports
|
||||
are offset by `10000 × N` (ws1's MCP at `http://localhost:14401/mcp`, Serena
|
||||
MCP at `http://localhost:24181`, Serena dashboard at `http://localhost:24182`,
|
||||
etc.). `manage.sh` prints the full per-instance URL set (Penpot UI, MCP
|
||||
stream, Serena MCP, Serena dashboard, attach command) every time it brings an
|
||||
instance up, so you don't need to compute the offsets by hand. See the
|
||||
*Parallel workspaces* section in the [Dev environment guide](./devenv.md) for
|
||||
the workspace and lifecycle details (including the `--sync` flag and shutdown
|
||||
shape).
|
||||
|
||||
> **Note:** the MCP and Serena tmux windows are only added when the session is
|
||||
> first created. If you've already run `./manage.sh run-devenv` (non-agentic)
|
||||
> in an instance, `run-devenv-agentic` just reattaches without starting them.
|
||||
> Kill the session first to recreate with the agentic windows:
|
||||
> in an instance, `run-devenv-agentic` errors out because the instance is
|
||||
> already running. Kill the session first to recreate with the agentic
|
||||
> windows:
|
||||
>
|
||||
> ```bash
|
||||
> docker exec penpot-devenv-ws0-main sudo -u penpot tmux kill-session -t penpot
|
||||
> ./manage.sh stop-devenv
|
||||
> ./manage.sh run-devenv-agentic
|
||||
> ```
|
||||
|
||||
@ -114,36 +123,140 @@ Note: You do not need to use the generated key (or the provided URL), as the MCP
|
||||
|
||||
## Configuring Your AI Client
|
||||
|
||||
Your AI client needs to be configured to connect to the MCP servers that collectively provide the agent with the necessary tools for Penpot development.
|
||||
A given AI client session drives **exactly one workspace**. The Penpot and
|
||||
Serena MCP servers it talks to live inside that workspace's `main` container
|
||||
on workspace-specific ports — a client configured against ws0 cannot drive
|
||||
ws1, and vice versa. Running N parallel workspaces therefore means
|
||||
configuring (or launching) N AI clients, each pointed at a different
|
||||
workspace's ports.
|
||||
|
||||
Below, we exemplarily provide a JSON-based configuration snippet, using `mcp-remote` to wrap HTTP-based servers.
|
||||
There are two ways to wire this up:
|
||||
|
||||
Most clients using JSON-based configuration (e.g. Copilot, JetBrains AI Assistant, Claude Desktop, Antigravity)
|
||||
will work when inserting the server entries below into the client's configuration file.
|
||||
If your client uses a different configuration format, extract the relevant information (i.e. server URLs or launch commands)
|
||||
and configure the servers appropriately, referring to the documentation of your client.
|
||||
* **For Claude Code, opencode, VS Code Copilot, and the Codex CLI**, use
|
||||
`./manage.sh start-coding-agent`, which loads a per-workspace MCP config
|
||||
generated by the devenv tooling. See *Quick start* below.
|
||||
* **For any other client** (JetBrains AI Assistant, Claude Desktop,
|
||||
Antigravity, …) — or if you'd rather configure the supported clients
|
||||
yourself — see *Manual configuration*.
|
||||
|
||||
### Quick start: `start-coding-agent`
|
||||
|
||||
Every time you bring a workspace up with `run-devenv-agentic`, `manage.sh`
|
||||
regenerates four MCP config files with that workspace's ports baked in:
|
||||
|
||||
| Client | File | Loaded how |
|
||||
| ------ | ---- | ---------- |
|
||||
| Claude Code | `<workspace>/.devenv/mcp/claude-code.json` | `--mcp-config <file>` flag |
|
||||
| opencode | `<workspace>/.devenv/mcp/opencode.json` | `OPENCODE_CONFIG=<file>` env var |
|
||||
| VS Code Copilot | `<workspace>/.vscode/mcp.json` | Auto-discovered when opening the workspace |
|
||||
| Codex CLI | `<workspace>/.codex/config.toml` | Auto-discovered from the project root (trusted projects only) |
|
||||
|
||||
The files are committed templates + an envsubst pass; see
|
||||
[`.devenv/README.md`](../../../.devenv/README.md) for the full layout.
|
||||
|
||||
To launch a coding agent:
|
||||
|
||||
```bash
|
||||
# Default target is ws0 (the live repo).
|
||||
./manage.sh start-coding-agent claude [...args forwarded to claude]
|
||||
./manage.sh start-coding-agent opencode [...args forwarded to opencode]
|
||||
./manage.sh start-coding-agent vscode [...args forwarded to 'code']
|
||||
./manage.sh start-coding-agent codex [...args forwarded to codex]
|
||||
|
||||
# Target a specific parallel workspace with --ws N (integer only).
|
||||
./manage.sh start-coding-agent claude --ws 1 # drives ws1
|
||||
./manage.sh start-coding-agent opencode --ws 2 # drives ws2
|
||||
```
|
||||
|
||||
The launcher `cd`'s into the target workspace before exec'ing the client, so
|
||||
config files are resolved from the right directory regardless of where you
|
||||
invoke `manage.sh` from. It refuses to launch if the target instance's
|
||||
`main` container is not running — the Penpot and Serena MCP servers only
|
||||
exist while the devenv is up. Bring the instance up first with
|
||||
`./manage.sh run-devenv-agentic --ws N`, then retry.
|
||||
|
||||
`--ws` accepts a non-negative integer only (`--ws 0`, `--ws 1`, …). Spellings
|
||||
like `--ws main` or `--ws ws1` are rejected so the flag shape stays uniform
|
||||
across `attach-devenv`, `run-devenv-agentic`, `stop-devenv`, and
|
||||
`start-coding-agent`.
|
||||
|
||||
What each launcher does:
|
||||
|
||||
* **Claude Code** is started with `--mcp-config .devenv/mcp/claude-code.json`,
|
||||
which is **additive** — your existing global Claude Code MCP entries stay
|
||||
available alongside the three entries we ship (Penpot MCP, Serena MCP,
|
||||
Playwright). To shadow one of our entries with a private one, install it
|
||||
under Claude Code's local scope (`claude mcp add --scope local …`); local
|
||||
wins over `--mcp-config`.
|
||||
* **opencode** is started with `OPENCODE_CONFIG=.devenv/mcp/opencode.json`.
|
||||
opencode's precedence chain is *global → `OPENCODE_CONFIG` → project*, so
|
||||
the file we generate **overrides** entries with the same name in your
|
||||
global config. To override our entries in turn, drop a personal
|
||||
`opencode.json` at the repo root — it's gitignored on purpose.
|
||||
* **VS Code Copilot** — `code "$PWD"` opens VS Code on the current workspace.
|
||||
Copilot loads both your user-profile MCP config and the workspace's
|
||||
`.vscode/mcp.json`, so our entries land alongside whatever you have
|
||||
globally. Same-name entries can be shadowed via your user profile.
|
||||
* **Codex CLI** — `codex` is exec'd from `$PWD`. Codex auto-discovers
|
||||
`.codex/config.toml` at the project root, but **only for "trusted"
|
||||
projects**; the first launch in a workspace will prompt you to trust it.
|
||||
Entries in `~/.codex/config.toml` override the project file on name
|
||||
conflict.
|
||||
|
||||
### Manual configuration
|
||||
|
||||
The `start-coding-agent` launcher covers Claude Code and opencode. For any
|
||||
other client, or if you prefer to wire things up yourself, configure the
|
||||
MCP servers in your client's native config format using the URLs below.
|
||||
|
||||
The Penpot and Serena URLs for the workspace you want to target are printed
|
||||
by `manage.sh` every time it brings an instance up; copy them straight from
|
||||
that output. The mechanical rule is `port = base + 10000 × N` for `wsN`, with
|
||||
bases `4401` (Penpot MCP) and `14181` (Serena MCP). Playwright is not
|
||||
workspace-scoped — it connects to your local browser, so the same entry works
|
||||
for every client.
|
||||
|
||||
#### Project-level MCP config files (Claude Code, opencode — manual path)
|
||||
|
||||
If you'd rather not go through the launcher, both Claude Code and opencode
|
||||
support project-level MCP config files that **merge with** the developer's
|
||||
global config:
|
||||
|
||||
* **Claude Code** reads `.mcp.json` at the project root. Local scope
|
||||
(`claude mcp add --scope local …`) overrides project scope.
|
||||
* **opencode** reads `opencode.json` at the project root (or any ancestor
|
||||
Git directory). Configs are merged in the order *global → `OPENCODE_CONFIG`
|
||||
→ project*.
|
||||
|
||||
The schemas differ between the two, so a workspace supporting both ships
|
||||
both files. For multi-workspace work, edit the port numbers in each
|
||||
workspace's copy once to match its offset.
|
||||
|
||||
#### Example configuration
|
||||
|
||||
Below is a JSON-based configuration snippet in Claude Code's `mcpServers`
|
||||
schema, targeting `ws0` (main), using `mcp-remote` to wrap HTTP-based
|
||||
servers. To target a different workspace, substitute the Penpot MCP and
|
||||
Serena MCP URLs with the ones for that workspace.
|
||||
|
||||
For clients using a different configuration format, extract the relevant
|
||||
information (server URLs or launch commands) and configure the servers
|
||||
appropriately, referring to your client's documentation.
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"penpot": {
|
||||
"penpot-ws0": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "mcp-remote", "http://localhost:4401/mcp", "--allow-http" ]
|
||||
},
|
||||
"serena-devenv": {
|
||||
"serena-ws0-devenv": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "mcp-remote", "http://localhost:14281/mcp", "--allow-http"]
|
||||
"args": ["-y", "mcp-remote", "http://localhost:14181/mcp", "--allow-http"]
|
||||
},
|
||||
"playwright": {
|
||||
"command": "npx",
|
||||
"args": ["@playwright/mcp@latest", "--cdp-endpoint=http://127.0.0.1:9222"]
|
||||
},
|
||||
"github": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "@modelcontextprotocol/server-github"],
|
||||
"env": {
|
||||
"GITHUB_PERSONAL_ACCESS_TOKEN": "TODO_your_token"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -154,20 +267,12 @@ and configure the servers appropriately, referring to the documentation of your
|
||||
You do not need to use the proxied URL or the user token that is provided by the Penpot UI.
|
||||
|
||||
**Serena MCP Server**
|
||||
* You can access Serena's dashboard at [http://localhost:14282](http://localhost:14282)
|
||||
|
||||
**GitHub MCP Server**
|
||||
* The use of this MCP server is optional. (Direct shell access to GitHub CLI can be used alternatively.)
|
||||
* You need to provide a personal access token (PAT) with appropriate permissions:
|
||||
* Create a token in your GitHub account settings [here](https://github.com/settings/personal-access-tokens).
|
||||
* Choose the right resource owner: As a member of the `penpot` organisation, be sure to create a token where the resource owner is the organisation.
|
||||
Otherwise, you will not be able to create pull requests or issues in the `penpot/penpot` repository.
|
||||
* Grant the necessary permissions, e.g. read and write access to issues and pull requests.
|
||||
* The matching Serena dashboard lives on the next port (`14182` for ws0, `24182` for ws1, …) and is also printed by `manage.sh` on startup.
|
||||
|
||||
## Working on Development Tasks
|
||||
|
||||
After having made the configuration changes, restart your AI client.
|
||||
All four MCP servers should now be running and accessible to your client.
|
||||
The configured MCP servers should now be running and accessible to your client.
|
||||
|
||||
The agent's entrypoint for development is an activation of the `penpot` project with Serena.
|
||||
Start by instructing your agent as follows,
|
||||
|
||||
@ -45,31 +45,53 @@ This is an incomplete list of devenv related subcommands found on
|
||||
manage.sh script:
|
||||
|
||||
```bash
|
||||
./manage.sh build-devenv-local # builds the local devenv docker image
|
||||
./manage.sh start-devenv # brings up the shared infra + ws0 in background
|
||||
./manage.sh run-devenv # ws0 with non-agentic tmux, attached (legacy alias)
|
||||
./manage.sh run-devenv-agentic # ws0 (default) with MCP + Serena enabled; see below
|
||||
./manage.sh attach-devenv # re-attaches to the tmux session of a running instance
|
||||
./manage.sh stop-devenv # stops infra and all running parallel instances
|
||||
./manage.sh drop-devenv # removes containers (data volumes preserved)
|
||||
./manage.sh build-devenv --local # builds the local devenv docker image
|
||||
./manage.sh start-devenv # brings up the shared infra + ws0 in background
|
||||
./manage.sh run-devenv # ws0 with non-agentic tmux, attached (legacy alias)
|
||||
./manage.sh run-devenv-agentic # one agentic instance; --ws to target ws1+; see below
|
||||
./manage.sh attach-devenv # re-attaches to the tmux session of a running instance
|
||||
./manage.sh stop-devenv # stops one instance (or --all); infra stops with the last
|
||||
./manage.sh drop-devenv # removes containers (data volumes preserved)
|
||||
```
|
||||
|
||||
### Parallel workspaces
|
||||
|
||||
The devenv runs as separate compose projects: shared infra (`penpotdev-infra`:
|
||||
Postgres, MinIO, mailer, LDAP) plus one `penpotdev-wsN` project per runtime
|
||||
instance. `ws0` binds the live repo; `ws1..wsN-1` are disposable clones under
|
||||
`~/.penpot/penpot_workspaces/` seeded from the current working tree on each
|
||||
startup.
|
||||
instance. `ws0` (a.k.a. `main`) binds the live repo; `ws1+` bind clones the
|
||||
developer maintains explicitly under `${PENPOT_WORKSPACES_DIR}/wsN/`
|
||||
(default `~/.penpot/penpot_workspaces/`).
|
||||
|
||||
Each call to `run-devenv-agentic` brings up one instance, and ws0 is always
|
||||
running whenever any ws1+ is — `--ws N` (N≥1) auto-starts ws0 first if it
|
||||
isn't already up:
|
||||
|
||||
```bash
|
||||
./manage.sh run-devenv-agentic --n-instances 3
|
||||
./manage.sh run-devenv-agentic # main (ws0)
|
||||
./manage.sh run-devenv-agentic --ws 1 # ws0 if needed, then ws1
|
||||
./manage.sh run-devenv-agentic --ws 2 --sync # ws2, re-seeding from the live repo
|
||||
```
|
||||
|
||||
is a desired-state reconciler: it brings the running set to exactly
|
||||
`{ws0, ws1, ws2}`. Missing instances are created; surplus instances
|
||||
(highest-numbered first) are stopped; instances already at their target index
|
||||
are left alone. Stopping never removes data volumes or workspace directories.
|
||||
Starting an instance that is already running is an error. `--sync` is only
|
||||
valid for `ws1+`; on ws0 it errors out. When a `ws1+` workspace directory
|
||||
does not exist yet, the first start syncs it implicitly from the live repo.
|
||||
Otherwise the workspace contents are left untouched unless `--sync` is passed
|
||||
again. Live-repo Git in a fragile state (rebase / merge / cherry-pick /
|
||||
`index.lock`) blocks all syncs.
|
||||
|
||||
`frontend/resources/public/js/config.js` (which is gitignored and configures
|
||||
the frontend's MCP flag) is copied into each workspace on its initial sync
|
||||
only. After that the developer maintains it in each workspace; subsequent
|
||||
`--sync` runs leave the workspace copy alone.
|
||||
|
||||
Stopping mirrors the start invariant — ws0 is the last to stop, and shared
|
||||
infra stops with it:
|
||||
|
||||
```bash
|
||||
./manage.sh stop-devenv --ws 1 # stops ws1; ws0 + infra stay up
|
||||
./manage.sh stop-devenv # stops ws0 + infra; errors if ws1+ still running
|
||||
./manage.sh stop-devenv --all # stops every ws1+ first, then ws0 + infra
|
||||
```
|
||||
|
||||
Host ports are offset by `10000 × N`:
|
||||
|
||||
@ -80,18 +102,28 @@ Host ports are offset by `10000 × N`:
|
||||
| Serena MCP | `http://localhost:14281` | `http://localhost:24281` | `http://localhost:34281` |
|
||||
|
||||
Container-internal ports stay fixed. Target a specific instance with
|
||||
`--instance ws1` on `attach-devenv` / `run-devenv-shell`. `run-devenv-agentic`
|
||||
accepts `--no-mcp`, `--no-serena`, and `--serena-context CTX`.
|
||||
`--ws 1` on `attach-devenv`, `run-devenv-agentic`, `stop-devenv`, and
|
||||
`start-coding-agent` (`--instance ws1` on `run-devenv-shell` /
|
||||
`run-devenv`). The `--ws` flag accepts a **non-negative integer only** —
|
||||
`--ws main` or `--ws ws1` is rejected, keeping the flag shape uniform across
|
||||
commands. `run-devenv-agentic` also accepts `--serena-context CTX`.
|
||||
|
||||
### Shared state and workers
|
||||
|
||||
All instances share one Penpot database and one MinIO bucket; users, teams,
|
||||
files, and MCP tokens are visible from every instance. Per-instance Valkey
|
||||
keeps WebSocket Pub/Sub channels isolated. Background workers
|
||||
(`enable-backend-worker`) run only on ws0 — ws1+ overlays disable it so async
|
||||
task notifications stay bound to a single Pub/Sub. Trade-off: async tasks
|
||||
triggered from a ws1+ tab execute (on ws0) but their completion notifications
|
||||
never reach the originating tab.
|
||||
keeps msgbus Pub/Sub channels (collab broadcasts, team-org notifications,
|
||||
file-summary cache, rate-limit counters) isolated.
|
||||
|
||||
Background workers (`enable-backend-worker`) run only on ws0 — ws1+ overlays
|
||||
disable it. ws1+ RPC handlers still enqueue tasks into the shared Postgres
|
||||
`task` table; ws0's dispatcher claims them via `FOR UPDATE SKIP LOCKED` and
|
||||
runs them against the shared DB and MinIO. The "ws0 always up when ws1+ is
|
||||
up" invariant exists for this reason: it keeps a single worker-bearer and
|
||||
avoids the multi-instance cron-dedup race (the lock on `scheduled_task` is
|
||||
released when the task body finishes, so two cron timers firing the same
|
||||
scheduled instant with a gap larger than the body's runtime can both
|
||||
execute it).
|
||||
|
||||
### Upgrading from a pre-parallel devenv
|
||||
|
||||
@ -128,7 +160,7 @@ docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
|
||||
docker network rm penpotdev_default 2>/dev/null
|
||||
|
||||
# Bring up infra + ws0 under the new project layout.
|
||||
./manage.sh run-devenv-agentic --n-instances 1
|
||||
./manage.sh run-devenv-agentic
|
||||
```
|
||||
|
||||
After the cleanup, normal `./manage.sh start-devenv` / `run-devenv` /
|
||||
|
||||
546
manage.sh
546
manage.sh
@ -29,6 +29,22 @@ unset __key __value
|
||||
# ws0 binds the live repo at $PWD; ws1+ override this in their overlay env file.
|
||||
export PENPOT_SOURCE_PATH="${PENPOT_SOURCE_PATH:-$PWD}"
|
||||
|
||||
# Base directory under which non-main workspace clones live (one subdir per
|
||||
# wsN, N>=1). Documented in defaults.env; default lives here so $HOME expands.
|
||||
export PENPOT_WORKSPACES_DIR="${PENPOT_WORKSPACES_DIR:-$HOME/.penpot/penpot_workspaces}"
|
||||
|
||||
# Port allocation for parallel instances. Each wsN reserves a stride-wide port
|
||||
# block starting at N*stride; per-service base ports sit at well-known offsets
|
||||
# inside ws0's block. Single source of truth for both write-instance-env (the
|
||||
# values baked into the per-instance compose env file) and print-instance-info
|
||||
# (the URLs printed at startup).
|
||||
PENPOT_INSTANCE_PORT_STRIDE=10000
|
||||
PENPOT_PORT_BASE_PUBLIC=3449
|
||||
PENPOT_PORT_BASE_MCP=4401
|
||||
PENPOT_PORT_BASE_MCP_REPL=4403
|
||||
PENPOT_PORT_BASE_SERENA=14181
|
||||
PENPOT_PORT_BASE_SERENA_DASHBOARD=14182
|
||||
|
||||
# Per-instance values like PENPOT_REDIS_URI must live in each instance's env
|
||||
# file (not in this shell), because docker compose's --env-file mechanism
|
||||
# lets a per-instance overlay override the baseline while the shell env
|
||||
@ -68,9 +84,13 @@ set -e
|
||||
# Devenv interactive entry points (all operate on the running 'main' container)
|
||||
# run-devenv-tmux starts 'main' if needed and execs start-tmux.sh
|
||||
# interactively (this is what 'run-devenv' resolves to)
|
||||
# run-devenv-agentic same as run-devenv-tmux but enables MCP + Serena
|
||||
# run-devenv-agentic same as run-devenv-tmux but enables MCP + Serena;
|
||||
# also regenerates .devenv/mcp/<tool>.json for the
|
||||
# target workspace (via write-instance-mcp-configs)
|
||||
# attach-devenv pure attach to the existing tmux session; fails
|
||||
# fast if the devenv or session is missing
|
||||
# start-coding-agent launches Claude Code or opencode against the
|
||||
# current workspace's generated MCP config
|
||||
# run-devenv-shell starts 'main' if needed and execs a bash shell
|
||||
# run-devenv-isolated-shell one-shot 'docker run' (NOT compose) against the
|
||||
# project user_data volume and the current PWD; used
|
||||
@ -204,7 +224,12 @@ function list-running-instances {
|
||||
|
||||
function devenv-main-container {
|
||||
local instance="${1:-ws0}"
|
||||
instance-compose "$instance" ps -q main
|
||||
# For ws1+, skip compose if the env-file hasn't been generated yet — the
|
||||
# instance has never been brought up so there is no container to find.
|
||||
if [[ "$instance" != "ws0" && ! -f "docker/devenv/instances/${instance}.env" ]]; then
|
||||
return 0
|
||||
fi
|
||||
instance-compose "$instance" ps -q main 2>/dev/null
|
||||
}
|
||||
|
||||
function devenv-main-running {
|
||||
@ -246,7 +271,16 @@ function assert-clean-git-state {
|
||||
# Absolute path of the workspace directory for a non-ws0 instance.
|
||||
function workspace-path {
|
||||
local instance="$1"
|
||||
echo "$HOME/.penpot/penpot_workspaces/$instance"
|
||||
echo "${PENPOT_WORKSPACES_DIR}/${instance}"
|
||||
}
|
||||
|
||||
# Echo the host port that <base> maps to for <instance>.
|
||||
function instance-port {
|
||||
local instance="$1"
|
||||
local base="$2"
|
||||
local n=0
|
||||
[[ "$instance" =~ ^ws([0-9]+)$ ]] && n="${BASH_REMATCH[1]}"
|
||||
echo $(( base + n * PENPOT_INSTANCE_PORT_STRIDE ))
|
||||
}
|
||||
|
||||
# Generate (or refresh) the per-instance Compose env-file overlay. Idempotent;
|
||||
@ -261,13 +295,17 @@ function write-instance-env {
|
||||
echo "write-instance-env: invalid instance '$instance'" >&2
|
||||
return 1
|
||||
fi
|
||||
local n="${BASH_REMATCH[1]}"
|
||||
local offset=$(( n * 10000 ))
|
||||
|
||||
local file="docker/devenv/instances/${instance}.env"
|
||||
mkdir -p docker/devenv/instances
|
||||
local workspace
|
||||
workspace=$(workspace-path "$instance")
|
||||
local public mcp mcp_repl serena serena_dash
|
||||
public=$(instance-port "$instance" "$PENPOT_PORT_BASE_PUBLIC")
|
||||
mcp=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP")
|
||||
mcp_repl=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP_REPL")
|
||||
serena=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA")
|
||||
serena_dash=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA_DASHBOARD")
|
||||
cat >"$file" <<EOF
|
||||
# Auto-generated by manage.sh for instance '$instance'.
|
||||
# Edits are overwritten on the next reconciler pass.
|
||||
@ -279,15 +317,15 @@ PENPOT_VALKEY_HOSTNAME=penpot-devenv-${instance}-valkey
|
||||
PENPOT_USER_DATA_VOLUME=penpotdev_${instance}_user_data
|
||||
PENPOT_VALKEY_DATA_VOLUME=penpotdev_${instance}_valkey_data
|
||||
|
||||
PENPOT_PUBLIC_URI=https://localhost:$(( 3449 + offset ))
|
||||
PENPOT_PUBLIC_URI=https://localhost:${public}
|
||||
PENPOT_REDIS_URI=redis://penpot-devenv-${instance}-valkey/0
|
||||
PENPOT_TMUX_SESSION=penpot
|
||||
|
||||
PENPOT_PUBLIC_HTTP_PORT=$(( 3449 + offset ))
|
||||
PENPOT_MCP_SERVER_PORT=$(( 4401 + offset ))
|
||||
PENPOT_MCP_REPL_PORT=$(( 4403 + offset ))
|
||||
SERENA_EXTERNAL_PORT=$(( 14281 + offset ))
|
||||
SERENA_DASHBOARD_EXTERNAL_PORT=$(( 14282 + offset ))
|
||||
PENPOT_PUBLIC_HTTP_PORT=${public}
|
||||
PENPOT_MCP_SERVER_PORT=${mcp}
|
||||
PENPOT_MCP_REPL_PORT=${mcp_repl}
|
||||
SERENA_EXTERNAL_PORT=${serena}
|
||||
SERENA_DASHBOARD_EXTERNAL_PORT=${serena_dash}
|
||||
|
||||
# Background workers run only on ws0 to keep async-task notifications bound
|
||||
# to a single Valkey Pub/Sub. See mem:devenv/core.
|
||||
@ -299,6 +337,81 @@ PENPOT_SOURCE_PATH=${workspace}
|
||||
EOF
|
||||
}
|
||||
|
||||
# Thin wrappers around .devenv/scripts/merge-mcp-config.py. The script does
|
||||
# the actual envsubst + JSON deep-merge or TOML concat; see its docstring for
|
||||
# the contract. ${PENPOT_MCP_PORT} / ${SERENA_MCP_PORT} placeholders in the
|
||||
# template are resolved from the caller's environment.
|
||||
function _merge-mcp-config-json {
|
||||
local shared="$1" tpl="$2" out="$3" key="$4"
|
||||
python3 .devenv/scripts/merge-mcp-config.py \
|
||||
--format json --key "$key" \
|
||||
"$shared" "$tpl" "$out"
|
||||
}
|
||||
|
||||
function _merge-mcp-config-toml {
|
||||
local shared="$1" tpl="$2" out="$3"
|
||||
python3 .devenv/scripts/merge-mcp-config.py \
|
||||
--format toml \
|
||||
"$shared" "$tpl" "$out"
|
||||
}
|
||||
|
||||
# Generate the per-workspace AI-client MCP config files by merging the
|
||||
# committed .devenv/shared/<tool>.* with the port-substituted
|
||||
# .devenv/templates/<tool>.*. Idempotent; overwrites on every call (the
|
||||
# generated files are gitignored and not meant for hand-editing — developers
|
||||
# who want to override entries should use the client's own override mechanism:
|
||||
# Claude Code's local scope, a project-level opencode.json, a personal
|
||||
# .vscode/mcp.json entry, or a user-level ~/.codex/config.toml).
|
||||
#
|
||||
# Generated paths per tool:
|
||||
# <workspace>/.devenv/mcp/claude-code.json loaded via --mcp-config
|
||||
# <workspace>/.devenv/mcp/opencode.json loaded via OPENCODE_CONFIG=
|
||||
# <workspace>/.vscode/mcp.json auto-loaded by VS Code Copilot
|
||||
# <workspace>/.codex/config.toml auto-loaded by Codex CLI
|
||||
# (project must be "trusted")
|
||||
function write-instance-mcp-configs {
|
||||
local instance="$1"
|
||||
local workspace
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
workspace="$PWD"
|
||||
else
|
||||
workspace=$(workspace-path "$instance")
|
||||
fi
|
||||
|
||||
local src_dir="$workspace/.devenv"
|
||||
if [[ ! -d "$src_dir/shared" || ! -d "$src_dir/templates" ]]; then
|
||||
echo "[$instance] .devenv/shared or .devenv/templates missing under $workspace; skipping MCP config generation." >&2
|
||||
return 0
|
||||
fi
|
||||
|
||||
local mcp_dir="$src_dir/mcp"
|
||||
mkdir -p "$mcp_dir" "$workspace/.vscode" "$workspace/.codex"
|
||||
|
||||
PENPOT_MCP_PORT=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP")
|
||||
SERENA_MCP_PORT=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA")
|
||||
export PENPOT_MCP_PORT SERENA_MCP_PORT
|
||||
|
||||
_merge-mcp-config-json \
|
||||
"$src_dir/shared/claude-code.json" \
|
||||
"$src_dir/templates/claude-code.json" \
|
||||
"$mcp_dir/claude-code.json" \
|
||||
mcpServers
|
||||
_merge-mcp-config-json \
|
||||
"$src_dir/shared/opencode.json" \
|
||||
"$src_dir/templates/opencode.json" \
|
||||
"$mcp_dir/opencode.json" \
|
||||
mcp
|
||||
_merge-mcp-config-json \
|
||||
"$src_dir/shared/vscode.json" \
|
||||
"$src_dir/templates/vscode.json" \
|
||||
"$workspace/.vscode/mcp.json" \
|
||||
servers
|
||||
_merge-mcp-config-toml \
|
||||
"$src_dir/shared/codex.toml" \
|
||||
"$src_dir/templates/codex.toml" \
|
||||
"$workspace/.codex/config.toml"
|
||||
}
|
||||
|
||||
# Seed (or re-seed) a workspace from the live repo, then switch it onto a
|
||||
# unique branch. Two-step sync:
|
||||
# 1. .git directory is rsync'd directly (so the workspace has its own
|
||||
@ -335,6 +448,15 @@ function sync-workspace {
|
||||
rsync -a --files-from="$files" --from0 "$PWD/" "$workspace/"
|
||||
rm -f "$files"
|
||||
|
||||
# Initial seed of frontend/resources/public/js/config.js. The file is
|
||||
# gitignored, so git ls-files would not list it, yet the agentic devenv
|
||||
# needs it (enable-mcp flag). After the first sync the workspace copy
|
||||
# belongs to the user — subsequent syncs leave it untouched.
|
||||
local cfg="frontend/resources/public/js/config.js"
|
||||
if [[ -f "$PWD/$cfg" && ! -f "$workspace/$cfg" ]]; then
|
||||
install -D "$PWD/$cfg" "$workspace/$cfg"
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$workspace"
|
||||
git switch -C "${instance}/${CURRENT_BRANCH}" >/dev/null
|
||||
@ -357,12 +479,74 @@ function create-devenv {
|
||||
instance-compose ws0 create
|
||||
}
|
||||
|
||||
# Stop instances. ws0 is the worker-bearer and must be the last one to stop;
|
||||
# shared infra is shut down together with ws0. Flags are mutually exclusive.
|
||||
#
|
||||
# --ws N (N>=1) stop just that workspace. Leaves ws0 and infra alone.
|
||||
# --ws 0 | (no flag) stop ws0 + shared infra. Refused if any ws1+ is running.
|
||||
# --all stop every wsN highest-first, then ws0, then infra.
|
||||
function stop-devenv {
|
||||
local ws
|
||||
for ws in $(list-running-instances); do
|
||||
instance-compose "$ws" stop -t 2
|
||||
local target=""
|
||||
local all=false
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--ws)
|
||||
target="$(parse-ws-integer "$2")" || return 1; shift 2;;
|
||||
--all)
|
||||
all=true; shift;;
|
||||
*)
|
||||
echo "stop-devenv: unknown argument '$1'" >&2
|
||||
return 1;;
|
||||
esac
|
||||
done
|
||||
infra-compose stop -t 2
|
||||
if [[ -n "$target" && "$all" == "true" ]]; then
|
||||
echo "stop-devenv: --ws and --all are mutually exclusive." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
local running ws
|
||||
running=$(list-running-instances)
|
||||
|
||||
if [[ "$all" == "true" ]]; then
|
||||
# Highest wsN first, then ws0, then infra. ws0 stop also brings infra.
|
||||
for ws in $(printf '%s\n' $running | grep -v '^ws0$' | sed 's/^ws//' | sort -rn | sed 's/^/ws/'); do
|
||||
stop-instance "$ws"
|
||||
done
|
||||
if printf '%s\n' $running | grep -qx ws0; then
|
||||
stop-instance ws0
|
||||
fi
|
||||
infra-compose down -t 2
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Default target: ws0 (which also stops infra).
|
||||
[[ -z "$target" ]] && target="ws0"
|
||||
|
||||
if [[ "$target" == "ws0" ]]; then
|
||||
local non_main=""
|
||||
for ws in $running; do
|
||||
[[ "$ws" != "ws0" ]] && non_main="$non_main $ws"
|
||||
done
|
||||
if [[ -n "$non_main" ]]; then
|
||||
echo "stop-devenv: cannot stop ws0 while other instances are running:${non_main}." >&2
|
||||
echo "Stop them first (--ws N) or use --all." >&2
|
||||
return 1
|
||||
fi
|
||||
if printf '%s\n' $running | grep -qx ws0; then
|
||||
stop-instance ws0
|
||||
else
|
||||
echo "[ws0] not running."
|
||||
fi
|
||||
infra-compose down -t 2
|
||||
return 0
|
||||
fi
|
||||
|
||||
# --ws N (N>=1): stop just that instance, leave ws0 + infra up.
|
||||
if printf '%s\n' $running | grep -qx "$target"; then
|
||||
stop-instance "$target"
|
||||
else
|
||||
echo "[$target] not running."
|
||||
fi
|
||||
}
|
||||
|
||||
function drop-devenv {
|
||||
@ -418,33 +602,49 @@ function run-devenv-tmux {
|
||||
"$container" sudo -EH -u penpot PENPOT_PLUGIN_DEV=$PENPOT_PLUGIN_DEV /home/start-tmux.sh
|
||||
}
|
||||
|
||||
# Normalize an instance specifier ("0", "ws0", "1", "ws3", ...) to "wsN".
|
||||
# Normalize an instance specifier ("main", "0", "ws0", "1", "ws3", ...) to "wsN".
|
||||
function normalize-instance {
|
||||
local raw="$1"
|
||||
if [[ "$raw" =~ ^ws[0-9]+$ ]]; then
|
||||
if [[ "$raw" == "main" ]]; then
|
||||
echo "ws0"
|
||||
elif [[ "$raw" =~ ^ws[0-9]+$ ]]; then
|
||||
echo "$raw"
|
||||
elif [[ "$raw" =~ ^[0-9]+$ ]]; then
|
||||
echo "ws$raw"
|
||||
else
|
||||
echo "Invalid --instance value: '$raw' (expected 0|ws0|1|ws1|...)" >&2
|
||||
echo "Invalid instance value: '$raw' (expected main|0|ws0|1|ws1|...)" >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Strict parser for --ws values. Accepts a bare integer in the supported
|
||||
# range (0..PENPOT_MAX_WS_INDEX) and returns the canonical "wsN" form.
|
||||
# Anything else fails fast. The upper bound exists because the host ports
|
||||
# computed for higher N overflow the 16-bit TCP port range:
|
||||
# port = base + N * PENPOT_INSTANCE_PORT_STRIDE (10000)
|
||||
# With the current Serena bases (14181/14182), N = 5 still fits inside
|
||||
# 65535 (64181/64182) but N = 6 overflows (74181/74182), so the cap is 5.
|
||||
PENPOT_MAX_WS_INDEX=5
|
||||
function parse-ws-integer {
|
||||
local raw="$1"
|
||||
if [[ ! "$raw" =~ ^[0-9]+$ ]]; then
|
||||
echo "Invalid --ws value: '$raw' (expected a non-negative integer, e.g. --ws 0, --ws 1, --ws 2)" >&2
|
||||
return 1
|
||||
fi
|
||||
if (( raw > PENPOT_MAX_WS_INDEX )); then
|
||||
echo "Invalid --ws value: '$raw' (max supported is --ws $PENPOT_MAX_WS_INDEX; higher indexes would overflow the 16-bit TCP port range)" >&2
|
||||
return 1
|
||||
fi
|
||||
echo "ws$raw"
|
||||
}
|
||||
|
||||
# Bring a single instance up: workspace sync (skipped for ws0), env-file
|
||||
# write (skipped for ws0), compose up, and detached tmux start with the
|
||||
# requested feature flags.
|
||||
|
||||
# Bring a single agentic instance up: compose up + detached tmux start with
|
||||
# MCP and Serena enabled. Workspace sync and env-file generation are the
|
||||
# caller's responsibility (run-devenv-agentic handles them for ws1+).
|
||||
function start-instance {
|
||||
local instance="$1"
|
||||
local enable_mcp="$2"
|
||||
local enable_serena="$3"
|
||||
local serena_context="$4"
|
||||
|
||||
if [[ "$instance" != "ws0" ]]; then
|
||||
sync-workspace "$instance"
|
||||
write-instance-env "$instance"
|
||||
fi
|
||||
local serena_context="$2"
|
||||
|
||||
instance-compose "$instance" up -d --no-deps main redis
|
||||
|
||||
@ -460,16 +660,14 @@ function start-instance {
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Start the tmux session detached so the reconciler can proceed to the
|
||||
# next instance without blocking on an interactive attach.
|
||||
local tmux_env=(-e PENPOT_TMUX_ATTACH=false)
|
||||
if [[ "$enable_mcp" == "true" ]]; then
|
||||
tmux_env+=(-e PENPOT_FLAGS="${PENPOT_FLAGS:-} enable-mcp")
|
||||
fi
|
||||
if [[ "$enable_serena" == "true" ]]; then
|
||||
tmux_env+=(-e SERENA_ENABLED=true -e SERENA_CONTEXT="$serena_context")
|
||||
fi
|
||||
docker exec -d "${tmux_env[@]}" "$container" \
|
||||
# Detached tmux so callers don't block on attach. Agentic mode is the only
|
||||
# mode here, so MCP and Serena are always on.
|
||||
docker exec -d \
|
||||
-e PENPOT_TMUX_ATTACH=false \
|
||||
-e PENPOT_FLAGS="${PENPOT_FLAGS:-} enable-mcp" \
|
||||
-e SERENA_ENABLED=true \
|
||||
-e SERENA_CONTEXT="$serena_context" \
|
||||
"$container" \
|
||||
sudo -EH -u penpot PENPOT_PLUGIN_DEV="${PENPOT_PLUGIN_DEV:-}" /home/start-tmux.sh
|
||||
}
|
||||
|
||||
@ -484,52 +682,64 @@ function stop-instance {
|
||||
# command) for one instance.
|
||||
function print-instance-info {
|
||||
local instance="$1"
|
||||
local enable_mcp="$2"
|
||||
local enable_serena="$3"
|
||||
local n=0
|
||||
[[ "$instance" =~ ^ws([0-9]+)$ ]] && n="${BASH_REMATCH[1]}"
|
||||
local offset=$(( n * 10000 ))
|
||||
local public=$(( 3449 + offset ))
|
||||
local mcp=$(( 4401 + offset ))
|
||||
local serena=$(( 14281 + offset ))
|
||||
local public mcp serena serena_dash
|
||||
public=$(instance-port "$instance" "$PENPOT_PORT_BASE_PUBLIC")
|
||||
mcp=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP")
|
||||
serena=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA")
|
||||
serena_dash=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA_DASHBOARD")
|
||||
|
||||
# --ws takes a bare integer; ws0 is the default, so its flag is elided.
|
||||
local n="${instance#ws}"
|
||||
local ws_flag=""
|
||||
[[ "$instance" != "ws0" ]] && ws_flag=" --ws ${n}"
|
||||
|
||||
echo
|
||||
echo "[$instance]"
|
||||
echo " Penpot UI: https://localhost:${public}"
|
||||
if [[ "$enable_mcp" == "true" ]]; then
|
||||
echo " MCP stream: http://localhost:${mcp}/mcp"
|
||||
fi
|
||||
if [[ "$enable_serena" == "true" ]]; then
|
||||
echo " Serena MCP: http://localhost:${serena}"
|
||||
fi
|
||||
echo " Attach: ./manage.sh attach-devenv --instance ${instance}"
|
||||
echo " MCP stream: http://localhost:${mcp}/mcp"
|
||||
echo " Serena MCP: http://localhost:${serena}"
|
||||
echo " Serena dashboard: http://localhost:${serena_dash}"
|
||||
echo " Attach: ./manage.sh attach-devenv${ws_flag}"
|
||||
echo " Coding agent: ./manage.sh start-coding-agent claude${ws_flag} (or: opencode|vscode|codex)"
|
||||
}
|
||||
|
||||
# Reconcile the running parallel set to exactly {ws0..ws(N-1)}.
|
||||
# Bring up a single agentic instance (always with MCP + Serena).
|
||||
#
|
||||
# --ws N target instance (non-negative integer). Default: 0 (ws0).
|
||||
# --sync re-seed the workspace from the live repo. Forbidden on main;
|
||||
# ws1+ also sync implicitly the first time when their workspace
|
||||
# directory does not exist yet.
|
||||
# --serena-context CTX passed to Serena (default: desktop-app).
|
||||
#
|
||||
# ws0 is the worker-bearer and must be running whenever any ws1+ is up. When
|
||||
# starting ws1+, ws0 is brought up first automatically if it is not already
|
||||
# running. Errors out if the requested target itself is already running.
|
||||
function run-devenv-agentic {
|
||||
local n_instances=1
|
||||
local enable_mcp=true
|
||||
local enable_serena=true
|
||||
local target="ws0"
|
||||
local do_sync=false
|
||||
local serena_context="desktop-app"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--n-instances)
|
||||
n_instances="$2"; shift 2;;
|
||||
--ws)
|
||||
target="$(parse-ws-integer "$2")" || return 1; shift 2;;
|
||||
--sync)
|
||||
do_sync=true; shift;;
|
||||
--serena-context)
|
||||
serena_context="$2"; shift 2;;
|
||||
--no-mcp)
|
||||
enable_mcp=false; shift;;
|
||||
--no-serena)
|
||||
enable_serena=false; shift;;
|
||||
*)
|
||||
echo "run-devenv-agentic: unknown argument '$1'" >&2
|
||||
return 1;;
|
||||
esac
|
||||
done
|
||||
|
||||
if ! [[ "$n_instances" =~ ^[1-9][0-9]*$ ]]; then
|
||||
echo "run-devenv-agentic: --n-instances must be a positive integer (got '$n_instances')" >&2
|
||||
if [[ "$target" == "ws0" && "$do_sync" == "true" ]]; then
|
||||
echo "run-devenv-agentic: --sync is not allowed on main (ws0)." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if devenv-main-running "$target"; then
|
||||
echo "run-devenv-agentic: instance '$target' is already running." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
@ -537,47 +747,33 @@ function run-devenv-agentic {
|
||||
ensure-devenv-network
|
||||
ensure-infra-up
|
||||
|
||||
# Compute target and running sets.
|
||||
local target=()
|
||||
local i
|
||||
for (( i=0; i < n_instances; i++ )); do
|
||||
target+=("ws$i")
|
||||
done
|
||||
local running
|
||||
running=$(list-running-instances)
|
||||
|
||||
# Stop extras, highest-numbered first.
|
||||
local to_stop=()
|
||||
for ws in $running; do
|
||||
if ! printf '%s\n' "${target[@]}" | grep -qx "$ws"; then
|
||||
to_stop+=("$ws")
|
||||
fi
|
||||
done
|
||||
if [[ ${#to_stop[@]} -gt 0 ]]; then
|
||||
# Sort numerically descending.
|
||||
IFS=$'\n' to_stop=($(printf '%s\n' "${to_stop[@]}" \
|
||||
| sed 's/^ws//' | sort -rn | sed 's/^/ws/'))
|
||||
unset IFS
|
||||
for ws in "${to_stop[@]}"; do
|
||||
echo "Stopping $ws..."
|
||||
stop-instance "$ws"
|
||||
done
|
||||
# ws0 invariant: must be up whenever any ws1+ runs. Bring it up first if a
|
||||
# non-main instance is being requested and ws0 is not yet running.
|
||||
if [[ "$target" != "ws0" ]] && ! devenv-main-running "ws0"; then
|
||||
echo "[ws0] not running; starting it first (workers run only on ws0)."
|
||||
echo "Starting ws0..."
|
||||
write-instance-mcp-configs "ws0"
|
||||
start-instance "ws0" "$serena_context"
|
||||
print-instance-info "ws0"
|
||||
fi
|
||||
|
||||
# Start missing instances.
|
||||
for ws in "${target[@]}"; do
|
||||
if printf '%s\n' "$running" | grep -qx "$ws"; then
|
||||
echo "[$ws] already running; leaving alone"
|
||||
continue
|
||||
if [[ "$target" != "ws0" ]]; then
|
||||
local workspace
|
||||
workspace=$(workspace-path "$target")
|
||||
if [[ ! -d "$workspace" ]]; then
|
||||
echo "[$target] workspace at $workspace does not exist; performing initial sync."
|
||||
do_sync=true
|
||||
fi
|
||||
echo "Starting $ws..."
|
||||
start-instance "$ws" "$enable_mcp" "$enable_serena" "$serena_context"
|
||||
done
|
||||
if [[ "$do_sync" == "true" ]]; then
|
||||
sync-workspace "$target"
|
||||
fi
|
||||
write-instance-env "$target"
|
||||
fi
|
||||
|
||||
# Per-instance startup info.
|
||||
for ws in "${target[@]}"; do
|
||||
print-instance-info "$ws" "$enable_mcp" "$enable_serena"
|
||||
done
|
||||
echo "Starting $target..."
|
||||
write-instance-mcp-configs "$target"
|
||||
start-instance "$target" "$serena_context"
|
||||
print-instance-info "$target"
|
||||
}
|
||||
|
||||
function run-devenv-shell {
|
||||
@ -612,8 +808,8 @@ function attach-devenv {
|
||||
local instance="ws0"
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--instance)
|
||||
instance="$(normalize-instance "$2")"; shift 2;;
|
||||
--ws)
|
||||
instance="$(parse-ws-integer "$2")" || return 1; shift 2;;
|
||||
*)
|
||||
echo "attach-devenv: unknown argument '$1'" >&2
|
||||
return 1;;
|
||||
@ -622,7 +818,7 @@ function attach-devenv {
|
||||
|
||||
if ! devenv-main-running "$instance"; then
|
||||
echo "Instance '$instance' is not running." >&2
|
||||
echo "Start it first with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic --n-instances N' (parallel)." >&2
|
||||
echo "Start it first with './manage.sh run-devenv-agentic [--ws N]'." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
@ -632,13 +828,126 @@ function attach-devenv {
|
||||
|
||||
if ! docker exec "$container" sudo -EH -u penpot tmux has-session -t "$session" 2>/dev/null; then
|
||||
echo "No tmux session '$session' inside instance '$instance'." >&2
|
||||
echo "Start it with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic'." >&2
|
||||
echo "Start it with './manage.sh run-devenv-agentic [--ws N]'." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
docker exec -ti "$container" sudo -EH -u penpot tmux attach -t "$session"
|
||||
}
|
||||
|
||||
# Launch an AI coding agent against one parallel devenv workspace with the
|
||||
# right MCP config wired in. The generated config enhances rather than
|
||||
# replaces the developer's global client config; see .devenv/README.md for
|
||||
# the precedence rules and override paths.
|
||||
#
|
||||
# Target selection:
|
||||
# no flag → ws0 (the live repo at $PWD).
|
||||
# --ws N → wsN's workspace clone at ${PENPOT_WORKSPACES_DIR}/wsN
|
||||
# (N is an integer; non-integer values are rejected).
|
||||
# Before launching, the function cd's into the resolved workspace and refuses
|
||||
# to start unless the target instance's 'main' container is up — the Penpot
|
||||
# and Serena MCP servers only exist while the devenv is running.
|
||||
#
|
||||
# Per-client launch behaviour:
|
||||
# claude exec'd with --mcp-config <workspace>/.devenv/mcp/claude-code.json
|
||||
# opencode exec'd with OPENCODE_CONFIG=<workspace>/.devenv/mcp/opencode.json
|
||||
# vscode 'code' launched on the workspace; .vscode/mcp.json is
|
||||
# auto-discovered by GitHub Copilot
|
||||
# codex 'codex' exec'd from the workspace; .codex/config.toml is
|
||||
# auto-discovered (the project must be marked trusted in Codex
|
||||
# on first run)
|
||||
#
|
||||
# Usage: ./manage.sh start-coding-agent <claude|opencode|vscode|codex> [--ws N] [...passthrough]
|
||||
function start-coding-agent {
|
||||
local client="${1:-}"
|
||||
[[ $# -gt 0 ]] && shift
|
||||
|
||||
case "$client" in
|
||||
""|-h|--help)
|
||||
echo "Usage: $0 start-coding-agent <claude|opencode|vscode|codex> [--ws N] [...passthrough]" >&2
|
||||
return 1
|
||||
;;
|
||||
claude|opencode|vscode|codex)
|
||||
;;
|
||||
*)
|
||||
echo "start-coding-agent: unknown client '$client' (expected one of claude, opencode, vscode, codex)." >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
|
||||
local instance="ws0"
|
||||
if [[ $# -gt 0 && "$1" == "--ws" ]]; then
|
||||
instance="$(parse-ws-integer "$2")" || return 1
|
||||
shift 2
|
||||
fi
|
||||
|
||||
# --ws is the default-elided flag: only emit it in suggestion strings for
|
||||
# ws1+; ws0 is the default target so 'run-devenv-agentic' is the right hint.
|
||||
local ws_flag=""
|
||||
[[ "$instance" != "ws0" ]] && ws_flag=" --ws ${instance#ws}"
|
||||
|
||||
# Resolve the workspace directory for the target instance. ws0 binds
|
||||
# the live repo; ws1+ are clones under PENPOT_WORKSPACES_DIR.
|
||||
local workspace
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
workspace="$PWD"
|
||||
else
|
||||
workspace="$(workspace-path "$instance")"
|
||||
if [[ ! -d "$workspace" ]]; then
|
||||
echo "start-coding-agent: workspace for $instance not found at $workspace." >&2
|
||||
echo "Bring '$instance' up first with './manage.sh run-devenv-agentic${ws_flag}'." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# The MCP servers the agent talks to only exist while 'main' is up.
|
||||
# Refuse rather than launch an agent that would error on every tool call.
|
||||
if ! devenv-main-running "$instance"; then
|
||||
echo "start-coding-agent: instance '$instance' is not running." >&2
|
||||
echo "Start it first with './manage.sh run-devenv-agentic${ws_flag}'." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Per-client binary + config path (relative to the workspace dir so the
|
||||
# launch line in error messages and `exec` is short and stable).
|
||||
local bin cfg_rel
|
||||
case "$client" in
|
||||
claude) bin="claude"; cfg_rel=".devenv/mcp/claude-code.json" ;;
|
||||
opencode) bin="opencode"; cfg_rel=".devenv/mcp/opencode.json" ;;
|
||||
vscode) bin="code"; cfg_rel=".vscode/mcp.json" ;;
|
||||
codex) bin="codex"; cfg_rel=".codex/config.toml" ;;
|
||||
esac
|
||||
|
||||
if ! command -v "$bin" >/dev/null 2>&1; then
|
||||
echo "start-coding-agent: '$bin' is not on PATH." >&2
|
||||
echo "Install it first, then retry." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [[ ! -f "$workspace/$cfg_rel" ]]; then
|
||||
echo "start-coding-agent: $workspace/$cfg_rel not found." >&2
|
||||
echo "Bring '$instance' up with './manage.sh run-devenv-agentic${ws_flag}'," >&2
|
||||
echo "which (re)generates the per-workspace MCP config." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
cd "$workspace" || return 1
|
||||
case "$client" in
|
||||
claude)
|
||||
exec claude --mcp-config "$cfg_rel" "$@"
|
||||
;;
|
||||
opencode)
|
||||
OPENCODE_CONFIG="$cfg_rel" exec opencode "$@"
|
||||
;;
|
||||
vscode)
|
||||
exec code "$workspace" "$@"
|
||||
;;
|
||||
codex)
|
||||
exec codex "$@"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
function run-devenv-isolated-shell {
|
||||
docker volume create ${PENPOT_USER_DATA_VOLUME};
|
||||
docker run -ti --rm \
|
||||
@ -872,14 +1181,24 @@ function usage {
|
||||
echo "- run-devenv Brings ws0 up and attaches to its tmux session (no MCP, no Serena)."
|
||||
echo " Optional --instance <wsN> targets a different instance."
|
||||
echo " Optional -e flags are forwarded to 'docker exec' (e.g. -e MY_VAR=value)."
|
||||
echo "- run-devenv-agentic Desired-state reconciler. Brings the running parallel set to exactly"
|
||||
echo " {ws0..ws(N-1)} with MCP and Serena enabled on each."
|
||||
echo " Options: --n-instances N (default: 1), --serena-context CONTEXT (default: desktop-app),"
|
||||
echo " --no-mcp, --no-serena"
|
||||
echo "- run-devenv-agentic Brings one agentic instance (MCP + Serena) up. Errors out if it is already running."
|
||||
echo " Auto-starts ws0 first when --ws N (N>=1) is requested (workers run only on ws0)."
|
||||
echo " Options: --ws N (default: 0; non-negative integer only),"
|
||||
echo " --sync (re-seed workspace from live repo; ws1+ only),"
|
||||
echo " --serena-context CONTEXT (default: desktop-app)"
|
||||
echo "- attach-devenv Attaches to the tmux session inside a running instance."
|
||||
echo " Options: --instance 0|wsN|N (default: 0)"
|
||||
echo " Options: --ws N (default: 0; non-negative integer only)"
|
||||
echo "- start-coding-agent <client> Launches an AI coding agent against one workspace with the right MCP config wired in."
|
||||
echo " client: claude | opencode | vscode | codex"
|
||||
echo " Options: --ws N (default: 0; non-negative integer only)."
|
||||
echo " cd's into the target workspace and refuses to launch if the instance is not running."
|
||||
echo " Extra args after --ws (or after the client name) are forwarded to the underlying client."
|
||||
echo " See docs/technical-guide/developer/agentic-devenv.md and .devenv/README.md"
|
||||
echo " for per-client setup, override paths, and the Codex 'trusted project' caveat."
|
||||
echo "- run-devenv-shell Opens a bash shell inside a running instance."
|
||||
echo " Options: --instance 0|wsN|N (default: 0)"
|
||||
echo "- stop-devenv Stops instances. ws0 must be the last to stop; shared infra stops with ws0."
|
||||
echo " Options: --ws N (stop one ws1+) | --ws 0 | (none) (stop ws0 + infra) | --all"
|
||||
echo "- isolated-shell Starts a bash shell in a new devenv container."
|
||||
echo "- log-devenv Show logs of the running devenv docker compose service."
|
||||
echo ""
|
||||
@ -932,6 +1251,9 @@ case $1 in
|
||||
attach-devenv)
|
||||
attach-devenv ${@:2}
|
||||
;;
|
||||
start-coding-agent)
|
||||
start-coding-agent "${@:2}"
|
||||
;;
|
||||
run-devenv-shell)
|
||||
run-devenv-shell ${@:2}
|
||||
;;
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user