🐳 Run parallel devenv instances against shared infra

Add support for running N parallel devenv instances under separate compose
projects sharing Postgres, MinIO, mailer, and LDAP. Each instance has its
own main container, Valkey, source checkout, tmux session, and host port
range offset by 10000 (3449 -> 13449 -> 23449, etc.).

./manage.sh run-devenv-agentic --n-instances N reconciles the running set
to exactly {ws0..ws(N-1)}: missing instances are created (workspace sync
from the live repo via git ls-files + per-instance env-file generation
under docker/devenv/instances/ + detached tmux startup), surplus instances
are stopped highest-first via compose down (never -v), already-running
instances are left untouched. ws0 binds the live repo at PWD; ws1+ are
scratch clones under ~/.penpot/penpot_workspaces/.

Backend workers (enable-backend-worker) are gated on PENPOT_BACKEND_WORKER
in backend/scripts/_env; ws1+ overlays disable them so async-task
notifications stay bound to a single Valkey Pub/Sub instance.

Compose helpers wrap docker compose with env -i so per-instance overlay
--env-file actually overrides defaults.env -- without the strip, the shell
env from sourcing defaults.env at startup would shadow the overlay (Compose
gives shell precedence over --env-file).

Other:
- Drop network aliases (- main, - redis); use container_name for
  cross-container DNS so multiple instances on the shared network don't
  fight over the same DNS name.
- Pin volume names via name: (PENPOT_*_VOLUME) so volumes survive project
  renames; ws0 keeps the pre-existing physical names (penpotdev_*).
- Remove cross-project depends_on from main.yml (postgres/minio-setup now
  live in penpotdev-infra); manage.sh ensure-infra-up docker-waits on the
  minio-setup one-shot.
- Strict arg parsing in run-devenv / run-devenv-agentic; --n-instances 0
  rejected.
- Remove unused Host-matched server block from the Caddyfile.

Memory mem:devenv/core and developer docs updated.

Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
Codex 2026-05-21 20:44:51 +02:00 committed by alonso.torres
parent 0e390010fa
commit ac78a0015b
9 changed files with 592 additions and 139 deletions

1
.gitignore vendored
View File

@ -95,4 +95,5 @@
/.idea
/.claude
/.playwright-mcp
/docker/devenv/instances/
/tools/__pycache__

View File

@ -1,48 +1,69 @@
# Devenv startup and configuration
Compose-based development environment under `docker/devenv/`, driven by `manage.sh`.
Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Parallel instances share infra + Postgres + MinIO; each instance has its own `main` container, Valkey, source checkout, tmux session.
## Source-of-truth layout
## Compose project layout
- `docker/devenv/defaults.env`: single source of truth for devenv config. Loaded by `manage.sh`'s simple env-file parser and by `docker compose --env-file`. Holds `COMPOSE_PROJECT_NAME`, container names (`PENPOT_MAIN_CONTAINER_NAME`, `PENPOT_VALKEY_CONTAINER_NAME`, `PENPOT_VALKEY_HOSTNAME`), runtime config that is passed into the container env, every published host port, Serena host ports, tmux session/attach defaults. `manage.sh` aborts if the file is unreadable.
- `backend/scripts/_env`: backend-internal defaults only — `PENPOT_*_SHARED_KEY`, `PENPOT_SECRET_KEY`, `PENPOT_FLAGS`, deletion/upload sizes, `PENPOT_NITRATE_BACKEND_URI`, `JAVA_OPTS`, the `setup_minio` function. Never duplicates anything in `defaults.env`.
- `docker/devenv/docker-compose.infra.yml`: shared services — `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. Attached to external network `penpot_shared`.
- `docker/devenv/docker-compose.main.yml`: main devenv container plus `redis` (valkey). Same network. Pure `${VAR}` references (no inline `:-` defaults) — missing var = compose fails.
- `penpotdev-infra`: shared `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. File: `docker-compose.infra.yml`.
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 binds `$PWD`; ws1+ bind clones at `~/.penpot/penpot_workspaces/wsN/`.
- All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands.
## Source-of-truth files
- `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable.
- `docker/devenv/instances/wsN.env` (N≥1): auto-generated per reconciler pass. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants
- Published ports are host-side only. Compose maps `${PENPOT_*_PORT}:<fixed internal port>` so parallel instances can offset host ports while container-local services keep their normal devenv ports. Do not pass host-side port offsets into processes that expect container-local ports.
- Volume keys in compose are literal (`user_data`, `valkey_data`). Docker prefixes them with `COMPOSE_PROJECT_NAME` to form the actual volume names.
- External network `penpot_shared` is created idempotently by `manage.sh ensure-devenv-network`; `drop-devenv` does **not** remove it.
- `PENPOT_SOURCE_PATH` is set by `manage.sh` to `$PWD` and bind-mounted as `/home/penpot/penpot`. Not in `defaults.env` because its value is dynamic.
- `CURRENT_USER_ID=$(id -u)` is exported by `manage.sh` and passed as `EXTERNAL_UID` so file ownership inside the container matches the host.
- `JAVA_OPTS` exported at the top of `manage.sh` (line ~28) is **shadowed inside the container** by `_env`, which reassigns it unconditionally to a much larger JVM config. The `-e JAVA_OPTS=$JAVA_OPTS` flag that `run-devenv-shell` / `run-devenv-isolated-shell` / `build` pass into `docker run`/`exec` only matters for processes that do not source `_env`.
- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`. Without it, sourcing `defaults.env` into the shell at startup would shadow per-instance overlay `--env-file` (Compose gives shell precedence over `--env-file`).
- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ overlays set distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration.
- Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead.
- No cross-project `depends_on`. `manage.sh ensure-infra-up` `docker wait`s on the `minio-setup` one-shot.
- `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`.
## MinIO provisioning split
## Worker policy
- Shared user/policy: provisioned once by the `minio-setup` one-shot service in the infra compose file. Alias-set loop bounded to 30 attempts. `main` depends on `service_completed_successfully`.
- Per-process bucket creation: `setup_minio()` in `_env`. Idempotent (`mc mb -p`). Short-circuits if `PENPOT_OBJECTS_STORAGE_BACKEND != s3`.
Backend workers run only on ws0. Task queue is shared (one Postgres DB) but Pub/Sub is per-instance Valkey: a task triggered from ws0's UI must complete on ws0 so its notification reaches the originating WebSocket. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. Known consequence: async tasks triggered from a ws1+ tab won't see completion notifications.
## Tmux session lifecycle
## Port layout
- `docker/devenv/files/start-tmux.sh` is idempotent at the session level. Reads `PENPOT_TMUX_SESSION` (default `penpot`) and `PENPOT_TMUX_ATTACH` (default `true`). If the session exists it attaches or exits depending on `PENPOT_TMUX_ATTACH`; otherwise runs `./scripts/setup` for frontend/exporter and creates the session with frontend-watch / storybook / exporter / backend / optional MCP / optional Serena windows.
- MCP and Serena windows are added only on session create (gated by `enable-mcp` in `PENPOT_FLAGS` and `SERENA_ENABLED=true`). `run-devenv-agentic` against an existing non-agentic session attaches without adding them — kill the session first to recreate.
- `manage.sh run-devenv`: ensures containers, then invokes start-tmux.sh interactively (attaches).
- `manage.sh attach-devenv`: pure attach — fails fast if devenv isn't running or session doesn't exist. Never starts containers. Takes no arguments.
Container-internal ports fixed; host side offset `10000·N`.
## Lifecycle commands
| ws0 | ws1 | wsN | container | role |
|---|---|---|---|---|
| 3449 | 13449 | 3449+10000·N | 3449 | public HTTPS (Caddy; `/mcp/ws` same-origin) |
| 3449/udp | 13449/udp | … | 3449/udp | HTTP/3 |
| 4401 | 14401 | … | 4401 | MCP HTTP stream |
| 4403 | 14403 | … | 4403 | MCP REPL |
| 14281 | 24281 | … | 14281 | Serena MCP |
| 14282 | 24282 | … | 24282 | Serena dashboard |
`manage.sh` thin wrappers around `devenv-compose` (which adds `--env-file defaults.env` and both compose files):
- `start-devenv` / `create-devenv`: pull image if missing, ensure network, `up -d` / `create`.
- `stop-devenv`, `log-devenv`: as expected.
- `drop-devenv`: `down -v` (removes containers + named volumes) and prunes the devenv image. Preserves `penpot_shared`.
- `run-devenv-shell`: starts containers if needed, `docker exec -ti` as `penpot`.
- `run-devenv-isolated-shell` / `build`: one-shot `docker run` against `${COMPOSE_PROJECT_NAME}_user_data` volume + repo bind mount. Not driven by compose.
Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin dev, MCP inspector/WebSocket) is in-process or same-origin via Caddy/nginx. Infra publishes: mailer 1080, ldap 10389/10636 (singletons, not offset).
## Exporter env
## Tmux + MCP routing
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env`, then `_env.local` if present. Backend-style env reaches the exporter via that chain.
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus optional `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). The conditional windows are added only on create — to switch from non-agentic to agentic, kill the session first.
## MCP routing in parallel devenvs
MCP plugin routing is same-origin: frontend uses `<public-uri>/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`.
The normal MCP path is same-origin: frontend computes `<public-uri>/mcp/ws`, the plugin opens that WebSocket, and the instance-local nginx proxies it to the MCP server inside the same main container. This depends on fixed internal ports; per-instance overlays should only change the published host ports and `PENPOT_PUBLIC_URI`.
## Workspace orchestration (ws1+)
`sync-workspace wsN`:
1. `assert-clean-git-state` — refuses on `.git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}`.
2. `rsync -a --delete $PWD/.git/ $workspace/.git/`.
3. `git ls-files -z --cached --others --exclude-standard``rsync --files-from` (Git is the authority on tracked files; rsync's gitignore filter would drop committed files under gitignored parents like `.clj-kondo/config.edn`).
4. `git switch -C "wsN/<current-branch>"` inside the workspace.
No `--delete` on the working-tree pass: gitignored caches in the workspace survive. Workspace dir + named volumes survive `compose down`.
## CLI surface
- `run-devenv-agentic [--n-instances N] [--no-mcp] [--no-serena] [--serena-context CTX]`: desired-state reconciler. Brings the running set to exactly `{ws0..ws(N-1)}`. Missing → sync + env-file + `compose up` + detached tmux. Extra → `compose down` highest-first (never `-v`). Running-in-target → left alone. `--n-instances 0` is rejected.
- `run-devenv`: legacy alias, ws0 non-agentic attached.
- `attach-devenv [--instance 0|wsN|N]`: pure attach. Fails fast if instance/session missing.
- `run-devenv-shell [--instance 0|wsN|N] [cmd...]`: bash in target instance.
- `start-devenv` / `stop-devenv` / `log-devenv` / `drop-devenv`: operate on infra + all parallel instances. `drop-devenv` never removes volumes.
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.

View File

@ -12,6 +12,15 @@ export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
# PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
# docker/devenv/defaults.env and injected via the main service's env block.
# Background worker flag is per-instance. Defaults to enabled (ws0); ws1+
# overlays set PENPOT_BACKEND_WORKER=false so scheduled and async tasks only
# run on ws0, keeping notification Pub/Sub bound to a single Valkey. See
# mem:devenv/core for the rationale.
__worker_flag=""
if [[ "${PENPOT_BACKEND_WORKER:-true}" == "true" ]]; then
__worker_flag="enable-backend-worker"
fi
export PENPOT_FLAGS="\
$PENPOT_FLAGS \
enable-login-with-password \
@ -21,7 +30,7 @@ export PENPOT_FLAGS="\
disable-login-with-github \
disable-login-with-gitlab \
disable-telemetry \
enable-backend-worker \
$__worker_flag \
enable-backend-asserts \
disable-feature-fdata-pointer-map \
enable-feature-fdata-objects-map \

View File

@ -7,28 +7,33 @@
# This file is consumed by compose only. Backend runtime defaults that
# compose does not care about live in backend/scripts/_env.
# Compose project name (replaces the -p flag).
COMPOSE_PROJECT_NAME=penpotdev
# Compose project name (replaces the -p flag). Set per instance; ws0 default
# here. Infra is launched under penpotdev-infra by manage.sh's infra-compose
# helper; ws1+ overlays override this to penpotdev-wsN.
COMPOSE_PROJECT_NAME=penpotdev-ws0
# Container names and volume names. Volumes are pinned by explicit name
# (rather than relying on COMPOSE_PROJECT_NAME prefixing) so the physical
# volumes can survive a future project rename without a data-migration step.
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-main
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-valkey
PENPOT_VALKEY_HOSTNAME=penpot-devenv-valkey
# volumes survive project renames without a data migration. ws0 reuses the
# pre-Stage-2 physical volume names (penpotdev_*).
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-ws0-main
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-ws0-valkey
PENPOT_VALKEY_HOSTNAME=penpot-devenv-ws0-valkey
PENPOT_POSTGRES_DATA_VOLUME=penpotdev_postgres_data_pg16
PENPOT_MINIO_DATA_VOLUME=penpotdev_minio_data
PENPOT_USER_DATA_VOLUME=penpotdev_user_data
PENPOT_VALKEY_DATA_VOLUME=penpotdev_valkey_data
# Backend runtime config (passed to the container env block).
# Backend runtime config (passed to the container env block). PENPOT_REDIS_URI
# is set explicitly per instance to match the per-instance Valkey container
# name; ws1+ overlays override this.
PENPOT_HOST=devenv
PENPOT_PUBLIC_URI=https://localhost:3449
PENPOT_DATABASE_URI=postgresql://postgres/penpot
PENPOT_DATABASE_USERNAME=penpot
PENPOT_DATABASE_PASSWORD=penpot
PENPOT_DATABASE_MAX_POOL_SIZE=60
PENPOT_REDIS_URI=redis://redis/0
PENPOT_DATABASE_MAX_POOL_SIZE=20
PENPOT_REDIS_URI=redis://penpot-devenv-ws0-valkey/0
# Object storage (MinIO user/policy are provisioned by the infra compose file).
PENPOT_OBJECTS_STORAGE_BACKEND=s3
@ -51,6 +56,10 @@ PENPOT_MCP_REPL_PORT=4403
SERENA_EXTERNAL_PORT=14281
SERENA_DASHBOARD_EXTERNAL_PORT=14282
# Backend worker (scheduled + async tasks). ws0 only; per-instance overlays
# for ws1+ override this to false. See mem:devenv/core.
PENPOT_BACKEND_WORKER=true
# Tmux session inside the main container.
PENPOT_TMUX_SESSION=penpot
PENPOT_TMUX_ATTACH=true

View File

@ -18,13 +18,12 @@ services:
container_name: "${PENPOT_MAIN_CONTAINER_NAME}"
stop_signal: SIGINT
# postgres / minio / minio-setup live in the penpotdev-infra compose
# project and cannot be referenced via depends_on across projects.
# manage.sh waits for infra readiness before bringing main up.
depends_on:
postgres:
condition: service_started
redis:
condition: service_started
minio-setup:
condition: service_completed_successfully
volumes:
- "user_data:/home/penpot/"
@ -83,6 +82,7 @@ services:
- PENPOT_OBJECTS_STORAGE_S3_BUCKET=${PENPOT_OBJECTS_STORAGE_S3_BUCKET}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- PENPOT_BACKEND_WORKER=${PENPOT_BACKEND_WORKER}
- PENPOT_TMUX_SESSION=${PENPOT_TMUX_SESSION}
- PENPOT_TMUX_ATTACH=${PENPOT_TMUX_ATTACH}
@ -91,9 +91,7 @@ services:
- SERENA_UPDATE_VERSION=1.5.0
networks:
default:
aliases:
- main
- default
redis:
image: valkey/valkey:8.1
@ -105,6 +103,4 @@ services:
- "valkey_data:/data"
networks:
default:
aliases:
- redis
- default

View File

@ -15,7 +15,3 @@ localhost:3449 {
# }
reverse_proxy localhost:4449
}
http://penpot-devenv-main:3450 {
reverse_proxy localhost:4449
}

View File

@ -70,18 +70,21 @@ var penpotFlags = "enable-mcp";
**Running the DevEnv in Agentic Mode.** Start the DevEnv in agentic mode with:
```bash
./manage.sh run-devenv-agentic
./manage.sh run-devenv-agentic # ws0 only
./manage.sh run-devenv-agentic --n-instances 3 # ws0, ws1, ws2
```
> **Note:** the MCP and Serena tmux windows are only added when the tmux
> session is created, not when an existing session is reattached. If you have
> already run `./manage.sh run-devenv` (non-agentic) in the current devenv
> container, the agentic command will just attach you to that session without
> starting MCP or Serena. To switch to agentic mode, kill the existing session
> first and rerun:
Per-instance ports are offset by `10000 × N` (ws1's MCP at
`http://localhost:14401/mcp`, Serena at `http://localhost:24281`, etc.). See
the *Parallel workspaces* section in the [Dev environment guide](./devenv.md).
> **Note:** the MCP and Serena tmux windows are only added when the session is
> first created. If you've already run `./manage.sh run-devenv` (non-agentic)
> in an instance, `run-devenv-agentic` just reattaches without starting them.
> Kill the session first to recreate with the agentic windows:
>
> ```bash
> docker exec penpot-devenv-main sudo -u penpot tmux kill-session -t penpot
> docker exec penpot-devenv-ws0-main sudo -u penpot tmux kill-session -t penpot
> ./manage.sh run-devenv-agentic
> ```

View File

@ -45,47 +45,95 @@ This is an incomplete list of devenv related subcommands found on
manage.sh script:
```bash
./manage.sh build-devenv-local # builds the local devenv docker image (called by run-devenv automatically when needed)
./manage.sh start-devenv # starts background running containers
./manage.sh run-devenv # enters to new tmux session inside of one of the running containers
./manage.sh attach-devenv # re-attaches to the tmux session inside an already-running devenv container
./manage.sh stop-devenv # stops background running containers
./manage.sh drop-devenv # removes all the containers, volumes and networks used by the devenv
./manage.sh build-devenv-local # builds the local devenv docker image
./manage.sh start-devenv # brings up the shared infra + ws0 in background
./manage.sh run-devenv # ws0 with non-agentic tmux, attached (legacy alias)
./manage.sh run-devenv-agentic # ws0 (default) with MCP + Serena enabled; see below
./manage.sh attach-devenv # re-attaches to the tmux session of a running instance
./manage.sh stop-devenv # stops infra and all running parallel instances
./manage.sh drop-devenv # removes containers (data volumes preserved)
```
### Upgrading from a pre-split devenv
### Parallel workspaces
The devenv compose configuration was split into two files,
`docker/devenv/docker-compose.infra.yml` (Postgres, MinIO, mailer, LDAP) and
`docker/devenv/docker-compose.main.yml` (the main devenv container and its
Valkey), joined by an external Docker network called `penpot_shared`.
Per-instance defaults live in `docker/devenv/defaults.env`.
If you had the devenv running on the previous single-file compose setup, the
first `./manage.sh start-devenv` after pulling the new code may fail with
`minio-setup: gave up waiting for MinIO after 30 attempts`. The cause is that
docker compose silently reuses the old infra containers (`penpotdev-postgres-1`,
`penpotdev-minio-1`, `penpotdev-mailer-1`, `penpotdev-ldap-1`), which are still
attached to the auto-generated `penpotdev_default` network from the old project
layout. The freshly created `minio-setup` joins the new `penpot_shared`
network instead, so it cannot resolve the `minio` hostname.
Migration steps (data in the named volumes is preserved):
The devenv runs as separate compose projects: shared infra (`penpotdev-infra`:
Postgres, MinIO, mailer, LDAP) plus one `penpotdev-wsN` project per runtime
instance. `ws0` binds the live repo; `ws1..wsN-1` are disposable clones under
`~/.penpot/penpot_workspaces/` seeded from the current working tree on each
startup.
```bash
./manage.sh stop-devenv
docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
penpotdev-mailer-1 penpotdev-ldap-1 \
penpot-devenv-main penpot-devenv-valkey 2>/dev/null
docker network rm penpotdev_default 2>/dev/null
./manage.sh start-devenv
./manage.sh run-devenv-agentic --n-instances 3
```
After this one-time cleanup the new compose files create fresh containers on
`penpot_shared` and `start-devenv` works normally. The named volumes
(`penpotdev_postgres_data_pg16`, `penpotdev_minio_data`,
`penpotdev_user_data`, `penpotdev_valkey_data`) are not touched, so existing
Penpot state survives.
is a desired-state reconciler: it brings the running set to exactly
`{ws0, ws1, ws2}`. Missing instances are created; surplus instances
(highest-numbered first) are stopped; instances already at their target index
are left alone. Stopping never removes data volumes or workspace directories.
Host ports are offset by `10000 × N`:
| Service | ws0 | ws1 | ws2 |
|---|---|---|---|
| Penpot UI (HTTPS) | `https://localhost:3449` | `https://localhost:13449` | `https://localhost:23449` |
| MCP HTTP stream | `http://localhost:4401/mcp` | `http://localhost:14401/mcp` | `http://localhost:24401/mcp` |
| Serena MCP | `http://localhost:14281` | `http://localhost:24281` | `http://localhost:34281` |
Container-internal ports stay fixed. Target a specific instance with
`--instance ws1` on `attach-devenv` / `run-devenv-shell`. `run-devenv-agentic`
accepts `--no-mcp`, `--no-serena`, and `--serena-context CTX`.
### Shared state and workers
All instances share one Penpot database and one MinIO bucket; users, teams,
files, and MCP tokens are visible from every instance. Per-instance Valkey
keeps WebSocket Pub/Sub channels isolated. Background workers
(`enable-backend-worker`) run only on ws0 — ws1+ overlays disable it so async
task notifications stay bound to a single Pub/Sub. Trade-off: async tasks
triggered from a ws1+ tab execute (on ws0) but their completion notifications
never reach the originating tab.
### Upgrading from a pre-parallel devenv
The devenv compose configuration has been split into two files and reorganized
into separate compose projects per runtime instance:
- `docker/devenv/docker-compose.infra.yml` (Postgres, MinIO, mailer, LDAP)
runs under the compose project `penpotdev-infra`.
- `docker/devenv/docker-compose.main.yml` (one main container + its Valkey)
runs once per runtime instance under `penpotdev-ws0`, `penpotdev-ws1`, ….
- Both projects join the external Docker network `penpot_shared`, created
idempotently by `manage.sh`.
- Per-instance configuration lives in `docker/devenv/defaults.env` (ws0
baseline) plus generated overlays under `docker/devenv/instances/`.
If you had the devenv running on the previous single-project (`penpotdev`)
layout, leftover containers and the auto-generated `penpotdev_default`
network must be removed before bringing the new ws0 instance up. The named
data volumes (`penpotdev_postgres_data_pg16`, `penpotdev_minio_data`,
`penpotdev_user_data`, `penpotdev_valkey_data`) are pinned by explicit
`name:` entries in the new compose files and are preserved through the
transition — your Postgres DB, MinIO objects, and home cache survive.
One-time cleanup, then bring up ws0:
```bash
# Stop and remove the old single-project containers (data volumes stay).
docker stop penpot-devenv-main penpot-devenv-valkey 2>/dev/null
docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
penpotdev-mailer-1 penpotdev-ldap-1 \
penpot-devenv-main penpot-devenv-valkey 2>/dev/null
# Remove the orphaned auto-generated network.
docker network rm penpotdev_default 2>/dev/null
# Bring up infra + ws0 under the new project layout.
./manage.sh run-devenv-agentic --n-instances 1
```
After the cleanup, normal `./manage.sh start-devenv` / `run-devenv` /
`run-devenv-agentic` commands work against the new layout. The legacy
`penpotdev` compose project is no longer used.
Having the container running and tmux opened inside the container,
you are free to execute commands and open as many shells as you want.

454
manage.sh
View File

@ -26,8 +26,14 @@ done < "$DEVENV_DEFAULTS_FILE"
unset __key __value
# Source path for the workspace bind mount; consumed by docker-compose.main.yml.
# ws0 binds the live repo at $PWD; ws1+ override this in their overlay env file.
export PENPOT_SOURCE_PATH="${PENPOT_SOURCE_PATH:-$PWD}"
# Per-instance values like PENPOT_REDIS_URI must live in each instance's env
# file (not in this shell), because docker compose's --env-file mechanism
# lets a per-instance overlay override the baseline while the shell env
# would otherwise shadow both for every project.
export CURRENT_USER_ID=$(id -u);
export CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD);
@ -145,57 +151,245 @@ function ensure-devenv-network {
docker network inspect "$DEVENV_NETWORK" >/dev/null 2>&1 || docker network create "$DEVENV_NETWORK" >/dev/null
}
function devenv-compose {
docker compose \
--env-file "$DEVENV_DEFAULTS_FILE" \
-f docker/devenv/docker-compose.infra.yml \
-f docker/devenv/docker-compose.main.yml \
"$@"
# Compose-project plumbing for the parallel-workspaces layout.
#
# - Shared infrastructure (postgres, minio, mailer, ldap, minio-setup) runs
# under project `penpotdev-infra`.
# - Each runtime instance (ws0, ws1, ...) runs its own main + valkey under
# project `penpotdev-wsN`. ws0 uses only `defaults.env`; ws1+ additionally
# layer a generated overlay file under `docker/devenv/instances/`.
# `env -i` strips the shell env before invoking docker compose so the
# per-instance overlay --env-file actually overrides defaults.env. Without
# stripping, the shell would still hold whatever values defaults.env was
# sourced into at startup (PENPOT_MAIN_CONTAINER_NAME, etc.), and Docker
# Compose's substitution gives the shell precedence over --env-file.
# Only the values that genuinely need to be per-call (HOME/PATH for tooling,
# CURRENT_USER_ID/PENPOT_SOURCE_PATH for the compose substitution) are
# re-exported.
function infra-compose {
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
docker compose -p penpotdev-infra \
--env-file "$DEVENV_DEFAULTS_FILE" \
-f docker/devenv/docker-compose.infra.yml \
"$@"
}
function instance-compose {
local instance="$1"; shift
local source_path env_files
env_files=(--env-file "$DEVENV_DEFAULTS_FILE")
if [[ "$instance" == "ws0" ]]; then
source_path="$PWD"
else
source_path="$(workspace-path "$instance")"
env_files+=(--env-file "docker/devenv/instances/${instance}.env")
fi
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
CURRENT_USER_ID="${CURRENT_USER_ID:-$(id -u)}" \
PENPOT_SOURCE_PATH="$source_path" \
docker compose -p "penpotdev-${instance}" \
"${env_files[@]}" \
-f docker/devenv/docker-compose.main.yml \
"$@"
}
# Names of currently-running parallel instances (ws0, ws1, ...).
function list-running-instances {
docker ps --format '{{.Label "com.docker.compose.project"}}' 2>/dev/null \
| sort -u \
| grep -oE '^penpotdev-ws[0-9]+$' \
| sed 's/^penpotdev-//' \
|| true
}
function devenv-main-container {
devenv-compose ps -q main
local instance="${1:-ws0}"
instance-compose "$instance" ps -q main
}
function devenv-main-running {
local container=$(devenv-main-container)
local instance="${1:-ws0}"
local container
container=$(devenv-main-container "$instance")
[[ -n "$container" ]] && [[ "$(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null)" = "true" ]]
}
# Bring shared infra up and block until minio-setup has provisioned the
# shared MinIO user/policy. Idempotent: a second call when everything is
# already up returns immediately.
function ensure-infra-up {
infra-compose up -d
local setup_container
setup_container=$(infra-compose ps -aq minio-setup 2>/dev/null)
if [[ -n "$setup_container" ]]; then
docker wait "$setup_container" >/dev/null 2>&1 || true
fi
}
# Refuse to sync workspaces if the live repo is in a fragile Git state.
# Copying a partial rebase/merge/cherry-pick into all workspaces would leave
# every instance in the same broken state.
function assert-clean-git-state {
local fragile=""
[ -d .git/rebase-apply ] && fragile="$fragile rebase-apply"
[ -d .git/rebase-merge ] && fragile="$fragile rebase-merge"
[ -f .git/MERGE_HEAD ] && fragile="$fragile merge"
[ -f .git/CHERRY_PICK_HEAD ] && fragile="$fragile cherry-pick"
[ -f .git/index.lock ] && fragile="$fragile index.lock"
if [[ -n "$fragile" ]]; then
echo "Live repo Git state is unsafe to copy into workspaces:$fragile" >&2
echo "Finish or abort the in-progress operation, then retry." >&2
return 1
fi
}
# Absolute path of the workspace directory for a non-ws0 instance.
function workspace-path {
local instance="$1"
echo "$HOME/.penpot/penpot_workspaces/$instance"
}
# Generate (or refresh) the per-instance Compose env-file overlay. Idempotent;
# safe to call on every reconciler pass.
function write-instance-env {
local instance="$1"
if [[ "$instance" == "ws0" ]]; then
return 0
fi
if [[ ! "$instance" =~ ^ws([0-9]+)$ ]]; then
echo "write-instance-env: invalid instance '$instance'" >&2
return 1
fi
local n="${BASH_REMATCH[1]}"
local offset=$(( n * 10000 ))
local file="docker/devenv/instances/${instance}.env"
mkdir -p docker/devenv/instances
local workspace
workspace=$(workspace-path "$instance")
cat >"$file" <<EOF
# Auto-generated by manage.sh for instance '$instance'.
# Edits are overwritten on the next reconciler pass.
COMPOSE_PROJECT_NAME=penpotdev-${instance}
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-${instance}-main
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-${instance}-valkey
PENPOT_VALKEY_HOSTNAME=penpot-devenv-${instance}-valkey
PENPOT_USER_DATA_VOLUME=penpotdev_${instance}_user_data
PENPOT_VALKEY_DATA_VOLUME=penpotdev_${instance}_valkey_data
PENPOT_PUBLIC_URI=https://localhost:$(( 3449 + offset ))
PENPOT_REDIS_URI=redis://penpot-devenv-${instance}-valkey/0
PENPOT_TMUX_SESSION=penpot
PENPOT_PUBLIC_HTTP_PORT=$(( 3449 + offset ))
PENPOT_MCP_SERVER_PORT=$(( 4401 + offset ))
PENPOT_MCP_REPL_PORT=$(( 4403 + offset ))
SERENA_EXTERNAL_PORT=$(( 14281 + offset ))
SERENA_DASHBOARD_EXTERNAL_PORT=$(( 14282 + offset ))
# Background workers run only on ws0 to keep async-task notifications bound
# to a single Valkey Pub/Sub. See mem:devenv/core.
PENPOT_BACKEND_WORKER=false
# Workspace bind mount (computed in manage.sh too, but recorded here for
# clarity when inspecting the env file).
PENPOT_SOURCE_PATH=${workspace}
EOF
}
# Seed (or re-seed) a workspace from the live repo, then switch it onto a
# unique branch. Two-step sync:
# 1. .git directory is rsync'd directly (so the workspace has its own
# clone with the developer's current commits / index).
# 2. Working-tree files are enumerated by `git ls-files`, which is the
# only authoritative source for "what files belong in the working
# tree" (Git tracks files even when their parent directory matches
# a gitignore pattern, e.g. .clj-kondo/config.edn). Using rsync's
# gitignore filter directly misses those.
# Gitignored caches already in the workspace (node_modules, target, etc.)
# are left in place: no --delete on the working-tree pass.
function sync-workspace {
local instance="$1"
if [[ "$instance" == "ws0" ]]; then
return 0
fi
assert-clean-git-state || return 1
local workspace
workspace=$(workspace-path "$instance")
mkdir -p "$workspace"
echo "[$instance] syncing workspace at $workspace ..."
# .git directory — direct mirror, including index, refs, hooks, etc.
rsync -a --delete "$PWD/.git/" "$workspace/.git/"
# Working-tree files: tracked + untracked-not-ignored. git ls-files
# speaks Git's actual semantics, including the "tracked overrides
# gitignore" rule. --files-from feeds the path list to rsync verbatim.
local files
files=$(mktemp)
git -C "$PWD" ls-files -z --cached --others --exclude-standard >"$files"
rsync -a --files-from="$files" --from0 "$PWD/" "$workspace/"
rm -f "$files"
(
cd "$workspace"
git switch -C "${instance}/${CURRENT_BRANCH}" >/dev/null
)
}
function start-devenv {
pull-devenv-if-not-exists $@;
ensure-devenv-network;
devenv-compose up -d;
ensure-infra-up
instance-compose ws0 up -d
}
function create-devenv {
pull-devenv-if-not-exists $@;
ensure-devenv-network;
devenv-compose create;
infra-compose create
instance-compose ws0 create
}
function stop-devenv {
devenv-compose stop -t 2;
local ws
for ws in $(list-running-instances); do
instance-compose "$ws" stop -t 2
done
infra-compose stop -t 2
}
function drop-devenv {
devenv-compose down -t 2 -v;
local ws
for ws in $(list-running-instances); do
# Never -v: data preservation rule.
instance-compose "$ws" down -t 2
done
infra-compose down -t 2
echo "Clean old development image $DEVENV_IMGNAME..."
docker images $DEVENV_IMGNAME -q | awk '{print $3}' | xargs --no-run-if-empty docker rmi
docker images $DEVENV_IMGNAME -q | xargs --no-run-if-empty docker rmi
}
function log-devenv {
devenv-compose logs -f --tail=50
# Tail ws0 by default; for multi-instance dev, attach explicitly per project.
instance-compose ws0 logs -f --tail=50
}
function run-devenv-tmux {
local extra_env_args=()
local instance="ws0"
while [[ $# -gt 0 ]]; do
case "$1" in
--instance)
instance="$(normalize-instance "$2")"; shift 2;;
-e)
extra_env_args+=(-e "$2"); shift 2;;
-e*)
@ -206,68 +400,239 @@ function run-devenv-tmux {
esac
done
if ! devenv-main-running; then
start-devenv
echo "Waiting for containers fully start (5s)..."
sleep 5;
if ! devenv-main-running "$instance"; then
if [[ "$instance" == "ws0" ]]; then
start-devenv
echo "Waiting for containers fully start (5s)..."
sleep 5
else
echo "Instance '$instance' is not running; bring it up first with './manage.sh run-devenv-agentic --n-instances N'." >&2
return 1
fi
fi
local container=$(devenv-main-container)
local container
container=$(devenv-main-container "$instance")
docker exec -ti \
"${extra_env_args[@]}" \
"$container" sudo -EH -u penpot PENPOT_PLUGIN_DEV=$PENPOT_PLUGIN_DEV /home/start-tmux.sh
}
# Normalize an instance specifier ("0", "ws0", "1", "ws3", ...) to "wsN".
function normalize-instance {
local raw="$1"
if [[ "$raw" =~ ^ws[0-9]+$ ]]; then
echo "$raw"
elif [[ "$raw" =~ ^[0-9]+$ ]]; then
echo "ws$raw"
else
echo "Invalid --instance value: '$raw' (expected 0|ws0|1|ws1|...)" >&2
return 1
fi
}
# Bring a single instance up: workspace sync (skipped for ws0), env-file
# write (skipped for ws0), compose up, and detached tmux start with the
# requested feature flags.
function start-instance {
local instance="$1"
local enable_mcp="$2"
local enable_serena="$3"
local serena_context="$4"
if [[ "$instance" != "ws0" ]]; then
sync-workspace "$instance"
write-instance-env "$instance"
fi
instance-compose "$instance" up -d --no-deps main redis
# Wait briefly for main to be reachable; the tmux session lives inside.
local container deadline
container=$(devenv-main-container "$instance")
deadline=$(( SECONDS + 30 ))
while ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; do
[[ $SECONDS -ge $deadline ]] && {
echo "[${instance}] main container did not reach Running within 30s" >&2
return 1
}
sleep 1
done
# Start the tmux session detached so the reconciler can proceed to the
# next instance without blocking on an interactive attach.
local tmux_env=(-e PENPOT_TMUX_ATTACH=false)
if [[ "$enable_mcp" == "true" ]]; then
tmux_env+=(-e PENPOT_FLAGS="${PENPOT_FLAGS:-} enable-mcp")
fi
if [[ "$enable_serena" == "true" ]]; then
tmux_env+=(-e SERENA_ENABLED=true -e SERENA_CONTEXT="$serena_context")
fi
docker exec -d "${tmux_env[@]}" "$container" \
sudo -EH -u penpot PENPOT_PLUGIN_DEV="${PENPOT_PLUGIN_DEV:-}" /home/start-tmux.sh
}
# Stop and remove one instance's containers without touching its volumes or
# its on-disk workspace directory (rule: never wipe data).
function stop-instance {
local instance="$1"
instance-compose "$instance" down -t 2
}
# Print per-instance URLs (Penpot UI, MCP stream endpoint, Serena, attach
# command) for one instance.
function print-instance-info {
local instance="$1"
local enable_mcp="$2"
local enable_serena="$3"
local n=0
[[ "$instance" =~ ^ws([0-9]+)$ ]] && n="${BASH_REMATCH[1]}"
local offset=$(( n * 10000 ))
local public=$(( 3449 + offset ))
local mcp=$(( 4401 + offset ))
local serena=$(( 14281 + offset ))
echo
echo "[$instance]"
echo " Penpot UI: https://localhost:${public}"
if [[ "$enable_mcp" == "true" ]]; then
echo " MCP stream: http://localhost:${mcp}/mcp"
fi
if [[ "$enable_serena" == "true" ]]; then
echo " Serena MCP: http://localhost:${serena}"
fi
echo " Attach: ./manage.sh attach-devenv --instance ${instance}"
}
# Reconcile the running parallel set to exactly {ws0..ws(N-1)}.
function run-devenv-agentic {
local n_instances=1
local enable_mcp=true
local enable_serena=true
local serena_context="desktop-app"
while [[ $# -gt 0 ]]; do
case "$1" in
--n-instances)
n_instances="$2"; shift 2;;
--serena-context)
serena_context="$2"; shift 2;;
--no-mcp)
enable_mcp=false; shift;;
--no-serena)
enable_serena=false; shift;;
*)
echo "run-devenv-agentic: unknown argument '$1'" >&2
return 1;;
esac
done
if ! devenv-main-running; then
start-devenv
echo "Waiting for containers fully start (5s)..."
sleep 5;
if ! [[ "$n_instances" =~ ^[1-9][0-9]*$ ]]; then
echo "run-devenv-agentic: --n-instances must be a positive integer (got '$n_instances')" >&2
return 1
fi
run-devenv-tmux \
-e SERENA_ENABLED=true \
-e SERENA_CONTEXT="$serena_context" \
-e PENPOT_FLAGS="${PENPOT_FLAGS} enable-mcp"
pull-devenv-if-not-exists
ensure-devenv-network
ensure-infra-up
# Compute target and running sets.
local target=()
local i
for (( i=0; i < n_instances; i++ )); do
target+=("ws$i")
done
local running
running=$(list-running-instances)
# Stop extras, highest-numbered first.
local to_stop=()
for ws in $running; do
if ! printf '%s\n' "${target[@]}" | grep -qx "$ws"; then
to_stop+=("$ws")
fi
done
if [[ ${#to_stop[@]} -gt 0 ]]; then
# Sort numerically descending.
IFS=$'\n' to_stop=($(printf '%s\n' "${to_stop[@]}" \
| sed 's/^ws//' | sort -rn | sed 's/^/ws/'))
unset IFS
for ws in "${to_stop[@]}"; do
echo "Stopping $ws..."
stop-instance "$ws"
done
fi
# Start missing instances.
for ws in "${target[@]}"; do
if printf '%s\n' "$running" | grep -qx "$ws"; then
echo "[$ws] already running; leaving alone"
continue
fi
echo "Starting $ws..."
start-instance "$ws" "$enable_mcp" "$enable_serena" "$serena_context"
done
# Per-instance startup info.
for ws in "${target[@]}"; do
print-instance-info "$ws" "$enable_mcp" "$enable_serena"
done
}
function run-devenv-shell {
if ! devenv-main-running; then
start-devenv
local instance="ws0"
local positional=()
while [[ $# -gt 0 ]]; do
case "$1" in
--instance)
instance="$(normalize-instance "$2")"; shift 2;;
*)
positional+=("$1"); shift;;
esac
done
if ! devenv-main-running "$instance"; then
if [[ "$instance" == "ws0" ]]; then
start-devenv
else
echo "Instance '$instance' is not running." >&2
return 1
fi
fi
local container=$(devenv-main-container)
local container
container=$(devenv-main-container "$instance")
docker exec -ti \
-e JAVA_OPTS="$JAVA_OPTS" \
-e EXTERNAL_UID=$CURRENT_USER_ID \
"$container" sudo -EH -u penpot $@
"$container" sudo -EH -u penpot "${positional[@]}"
}
function attach-devenv {
if ! devenv-main-running; then
echo "devenv is not running." >&2
echo "Start it first with './manage.sh run-devenv' (or './manage.sh start-devenv' for containers only)." >&2
local instance="ws0"
while [[ $# -gt 0 ]]; do
case "$1" in
--instance)
instance="$(normalize-instance "$2")"; shift 2;;
*)
echo "attach-devenv: unknown argument '$1'" >&2
return 1;;
esac
done
if ! devenv-main-running "$instance"; then
echo "Instance '$instance' is not running." >&2
echo "Start it first with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic --n-instances N' (parallel)." >&2
return 1
fi
local session="${PENPOT_TMUX_SESSION:-penpot}"
local container=$(devenv-main-container)
local container
container=$(devenv-main-container "$instance")
if ! docker exec "$container" sudo -EH -u penpot tmux has-session -t "$session" 2>/dev/null; then
echo "No tmux session '$session' inside the devenv container." >&2
echo "Start it with './manage.sh run-devenv'." >&2
echo "No tmux session '$session' inside instance '$instance'." >&2
echo "Start it with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic'." >&2
return 1
fi
@ -504,12 +869,17 @@ function usage {
echo "- start-devenv Start the development oriented docker compose service."
echo "- stop-devenv Stops the development oriented docker compose service."
echo "- drop-devenv Remove the development oriented docker compose containers, volumes and clean images."
echo "- run-devenv Attaches to the running devenv container and starts development environment"
echo "- run-devenv Brings ws0 up and attaches to its tmux session (no MCP, no Serena)."
echo " Optional --instance <wsN> targets a different instance."
echo " Optional -e flags are forwarded to 'docker exec' (e.g. -e MY_VAR=value)."
echo "- run-devenv-agentic Like run-devenv but with additional processes for agentic development enabled."
echo " Options: --serena-context CONTEXT (default: desktop-app)"
echo "- attach-devenv Attaches to the tmux session inside the running devenv container."
echo "- run-devenv-shell Attaches to the running devenv container and starts a bash shell."
echo "- run-devenv-agentic Desired-state reconciler. Brings the running parallel set to exactly"
echo " {ws0..ws(N-1)} with MCP and Serena enabled on each."
echo " Options: --n-instances N (default: 1), --serena-context CONTEXT (default: desktop-app),"
echo " --no-mcp, --no-serena"
echo "- attach-devenv Attaches to the tmux session inside a running instance."
echo " Options: --instance 0|wsN|N (default: 0)"
echo "- run-devenv-shell Opens a bash shell inside a running instance."
echo " Options: --instance 0|wsN|N (default: 0)"
echo "- isolated-shell Starts a bash shell in a new devenv container."
echo "- log-devenv Show logs of the running devenv docker compose service."
echo ""