mirror of
https://github.com/penpot/penpot.git
synced 2026-06-16 04:12:03 +00:00
🐳 Run parallel devenv instances against shared infra
Add support for running N parallel devenv instances under separate compose
projects sharing Postgres, MinIO, mailer, and LDAP. Each instance has its
own main container, Valkey, source checkout, tmux session, and host port
range offset by 10000 (3449 -> 13449 -> 23449, etc.).
./manage.sh run-devenv-agentic --n-instances N reconciles the running set
to exactly {ws0..ws(N-1)}: missing instances are created (workspace sync
from the live repo via git ls-files + per-instance env-file generation
under docker/devenv/instances/ + detached tmux startup), surplus instances
are stopped highest-first via compose down (never -v), already-running
instances are left untouched. ws0 binds the live repo at PWD; ws1+ are
scratch clones under ~/.penpot/penpot_workspaces/.
Backend workers (enable-backend-worker) are gated on PENPOT_BACKEND_WORKER
in backend/scripts/_env; ws1+ overlays disable them so async-task
notifications stay bound to a single Valkey Pub/Sub instance.
Compose helpers wrap docker compose with env -i so per-instance overlay
--env-file actually overrides defaults.env -- without the strip, the shell
env from sourcing defaults.env at startup would shadow the overlay (Compose
gives shell precedence over --env-file).
Other:
- Drop network aliases (- main, - redis); use container_name for
cross-container DNS so multiple instances on the shared network don't
fight over the same DNS name.
- Pin volume names via name: (PENPOT_*_VOLUME) so volumes survive project
renames; ws0 keeps the pre-existing physical names (penpotdev_*).
- Remove cross-project depends_on from main.yml (postgres/minio-setup now
live in penpotdev-infra); manage.sh ensure-infra-up docker-waits on the
minio-setup one-shot.
- Strict arg parsing in run-devenv / run-devenv-agentic; --n-instances 0
rejected.
- Remove unused Host-matched server block from the Caddyfile.
Memory mem:devenv/core and developer docs updated.
Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
parent
0e390010fa
commit
ac78a0015b
1
.gitignore
vendored
1
.gitignore
vendored
@ -95,4 +95,5 @@
|
||||
/.idea
|
||||
/.claude
|
||||
/.playwright-mcp
|
||||
/docker/devenv/instances/
|
||||
/tools/__pycache__
|
||||
|
||||
@ -1,48 +1,69 @@
|
||||
# Devenv startup and configuration
|
||||
|
||||
Compose-based development environment under `docker/devenv/`, driven by `manage.sh`.
|
||||
Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Parallel instances share infra + Postgres + MinIO; each instance has its own `main` container, Valkey, source checkout, tmux session.
|
||||
|
||||
## Source-of-truth layout
|
||||
## Compose project layout
|
||||
|
||||
- `docker/devenv/defaults.env`: single source of truth for devenv config. Loaded by `manage.sh`'s simple env-file parser and by `docker compose --env-file`. Holds `COMPOSE_PROJECT_NAME`, container names (`PENPOT_MAIN_CONTAINER_NAME`, `PENPOT_VALKEY_CONTAINER_NAME`, `PENPOT_VALKEY_HOSTNAME`), runtime config that is passed into the container env, every published host port, Serena host ports, tmux session/attach defaults. `manage.sh` aborts if the file is unreadable.
|
||||
- `backend/scripts/_env`: backend-internal defaults only — `PENPOT_*_SHARED_KEY`, `PENPOT_SECRET_KEY`, `PENPOT_FLAGS`, deletion/upload sizes, `PENPOT_NITRATE_BACKEND_URI`, `JAVA_OPTS`, the `setup_minio` function. Never duplicates anything in `defaults.env`.
|
||||
- `docker/devenv/docker-compose.infra.yml`: shared services — `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. Attached to external network `penpot_shared`.
|
||||
- `docker/devenv/docker-compose.main.yml`: main devenv container plus `redis` (valkey). Same network. Pure `${VAR}` references (no inline `:-` defaults) — missing var = compose fails.
|
||||
- `penpotdev-infra`: shared `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. File: `docker-compose.infra.yml`.
|
||||
- `penpotdev-wsN` (N=0,1,…): per-instance `main` + `redis` (Valkey). File: `docker-compose.main.yml`. ws0 binds `$PWD`; ws1+ bind clones at `~/.penpot/penpot_workspaces/wsN/`.
|
||||
- All projects join external network `penpot_shared`. Created idempotently by `ensure-devenv-network`, never removed by lifecycle commands.
|
||||
|
||||
## Source-of-truth files
|
||||
|
||||
- `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable.
|
||||
- `docker/devenv/instances/wsN.env` (N≥1): auto-generated per reconciler pass. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored.
|
||||
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`.
|
||||
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Published ports are host-side only. Compose maps `${PENPOT_*_PORT}:<fixed internal port>` so parallel instances can offset host ports while container-local services keep their normal devenv ports. Do not pass host-side port offsets into processes that expect container-local ports.
|
||||
- Volume keys in compose are literal (`user_data`, `valkey_data`). Docker prefixes them with `COMPOSE_PROJECT_NAME` to form the actual volume names.
|
||||
- External network `penpot_shared` is created idempotently by `manage.sh ensure-devenv-network`; `drop-devenv` does **not** remove it.
|
||||
- `PENPOT_SOURCE_PATH` is set by `manage.sh` to `$PWD` and bind-mounted as `/home/penpot/penpot`. Not in `defaults.env` because its value is dynamic.
|
||||
- `CURRENT_USER_ID=$(id -u)` is exported by `manage.sh` and passed as `EXTERNAL_UID` so file ownership inside the container matches the host.
|
||||
- `JAVA_OPTS` exported at the top of `manage.sh` (line ~28) is **shadowed inside the container** by `_env`, which reassigns it unconditionally to a much larger JVM config. The `-e JAVA_OPTS=$JAVA_OPTS` flag that `run-devenv-shell` / `run-devenv-isolated-shell` / `build` pass into `docker run`/`exec` only matters for processes that do not source `_env`.
|
||||
- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`. Without it, sourcing `defaults.env` into the shell at startup would shadow per-instance overlay `--env-file` (Compose gives shell precedence over `--env-file`).
|
||||
- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ overlays set distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration.
|
||||
- Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead.
|
||||
- No cross-project `depends_on`. `manage.sh ensure-infra-up` `docker wait`s on the `minio-setup` one-shot.
|
||||
- `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`.
|
||||
|
||||
## MinIO provisioning split
|
||||
## Worker policy
|
||||
|
||||
- Shared user/policy: provisioned once by the `minio-setup` one-shot service in the infra compose file. Alias-set loop bounded to 30 attempts. `main` depends on `service_completed_successfully`.
|
||||
- Per-process bucket creation: `setup_minio()` in `_env`. Idempotent (`mc mb -p`). Short-circuits if `PENPOT_OBJECTS_STORAGE_BACKEND != s3`.
|
||||
Backend workers run only on ws0. Task queue is shared (one Postgres DB) but Pub/Sub is per-instance Valkey: a task triggered from ws0's UI must complete on ws0 so its notification reaches the originating WebSocket. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. Known consequence: async tasks triggered from a ws1+ tab won't see completion notifications.
|
||||
|
||||
## Tmux session lifecycle
|
||||
## Port layout
|
||||
|
||||
- `docker/devenv/files/start-tmux.sh` is idempotent at the session level. Reads `PENPOT_TMUX_SESSION` (default `penpot`) and `PENPOT_TMUX_ATTACH` (default `true`). If the session exists it attaches or exits depending on `PENPOT_TMUX_ATTACH`; otherwise runs `./scripts/setup` for frontend/exporter and creates the session with frontend-watch / storybook / exporter / backend / optional MCP / optional Serena windows.
|
||||
- MCP and Serena windows are added only on session create (gated by `enable-mcp` in `PENPOT_FLAGS` and `SERENA_ENABLED=true`). `run-devenv-agentic` against an existing non-agentic session attaches without adding them — kill the session first to recreate.
|
||||
- `manage.sh run-devenv`: ensures containers, then invokes start-tmux.sh interactively (attaches).
|
||||
- `manage.sh attach-devenv`: pure attach — fails fast if devenv isn't running or session doesn't exist. Never starts containers. Takes no arguments.
|
||||
Container-internal ports fixed; host side offset `10000·N`.
|
||||
|
||||
## Lifecycle commands
|
||||
| ws0 | ws1 | wsN | container | role |
|
||||
|---|---|---|---|---|
|
||||
| 3449 | 13449 | 3449+10000·N | 3449 | public HTTPS (Caddy; `/mcp/ws` same-origin) |
|
||||
| 3449/udp | 13449/udp | … | 3449/udp | HTTP/3 |
|
||||
| 4401 | 14401 | … | 4401 | MCP HTTP stream |
|
||||
| 4403 | 14403 | … | 4403 | MCP REPL |
|
||||
| 14281 | 24281 | … | 14281 | Serena MCP |
|
||||
| 14282 | 24282 | … | 24282 | Serena dashboard |
|
||||
|
||||
`manage.sh` thin wrappers around `devenv-compose` (which adds `--env-file defaults.env` and both compose files):
|
||||
- `start-devenv` / `create-devenv`: pull image if missing, ensure network, `up -d` / `create`.
|
||||
- `stop-devenv`, `log-devenv`: as expected.
|
||||
- `drop-devenv`: `down -v` (removes containers + named volumes) and prunes the devenv image. Preserves `penpot_shared`.
|
||||
- `run-devenv-shell`: starts containers if needed, `docker exec -ti` as `penpot`.
|
||||
- `run-devenv-isolated-shell` / `build`: one-shot `docker run` against `${COMPOSE_PROJECT_NAME}_user_data` volume + repo bind mount. Not driven by compose.
|
||||
Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin dev, MCP inspector/WebSocket) is in-process or same-origin via Caddy/nginx. Infra publishes: mailer 1080, ldap 10389/10636 (singletons, not offset).
|
||||
|
||||
## Exporter env
|
||||
## Tmux + MCP routing
|
||||
|
||||
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env`, then `_env.local` if present. Backend-style env reaches the exporter via that chain.
|
||||
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus optional `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). The conditional windows are added only on create — to switch from non-agentic to agentic, kill the session first.
|
||||
|
||||
## MCP routing in parallel devenvs
|
||||
MCP plugin routing is same-origin: frontend uses `<public-uri>/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`.
|
||||
|
||||
The normal MCP path is same-origin: frontend computes `<public-uri>/mcp/ws`, the plugin opens that WebSocket, and the instance-local nginx proxies it to the MCP server inside the same main container. This depends on fixed internal ports; per-instance overlays should only change the published host ports and `PENPOT_PUBLIC_URI`.
|
||||
## Workspace orchestration (ws1+)
|
||||
|
||||
`sync-workspace wsN`:
|
||||
1. `assert-clean-git-state` — refuses on `.git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}`.
|
||||
2. `rsync -a --delete $PWD/.git/ $workspace/.git/`.
|
||||
3. `git ls-files -z --cached --others --exclude-standard` → `rsync --files-from` (Git is the authority on tracked files; rsync's gitignore filter would drop committed files under gitignored parents like `.clj-kondo/config.edn`).
|
||||
4. `git switch -C "wsN/<current-branch>"` inside the workspace.
|
||||
|
||||
No `--delete` on the working-tree pass: gitignored caches in the workspace survive. Workspace dir + named volumes survive `compose down`.
|
||||
|
||||
## CLI surface
|
||||
|
||||
- `run-devenv-agentic [--n-instances N] [--no-mcp] [--no-serena] [--serena-context CTX]`: desired-state reconciler. Brings the running set to exactly `{ws0..ws(N-1)}`. Missing → sync + env-file + `compose up` + detached tmux. Extra → `compose down` highest-first (never `-v`). Running-in-target → left alone. `--n-instances 0` is rejected.
|
||||
- `run-devenv`: legacy alias, ws0 non-agentic attached.
|
||||
- `attach-devenv [--instance 0|wsN|N]`: pure attach. Fails fast if instance/session missing.
|
||||
- `run-devenv-shell [--instance 0|wsN|N] [cmd...]`: bash in target instance.
|
||||
- `start-devenv` / `stop-devenv` / `log-devenv` / `drop-devenv`: operate on infra + all parallel instances. `drop-devenv` never removes volumes.
|
||||
|
||||
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env` then `_env.local` if present.
|
||||
|
||||
@ -12,6 +12,15 @@ export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
|
||||
# PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
|
||||
# docker/devenv/defaults.env and injected via the main service's env block.
|
||||
|
||||
# Background worker flag is per-instance. Defaults to enabled (ws0); ws1+
|
||||
# overlays set PENPOT_BACKEND_WORKER=false so scheduled and async tasks only
|
||||
# run on ws0, keeping notification Pub/Sub bound to a single Valkey. See
|
||||
# mem:devenv/core for the rationale.
|
||||
__worker_flag=""
|
||||
if [[ "${PENPOT_BACKEND_WORKER:-true}" == "true" ]]; then
|
||||
__worker_flag="enable-backend-worker"
|
||||
fi
|
||||
|
||||
export PENPOT_FLAGS="\
|
||||
$PENPOT_FLAGS \
|
||||
enable-login-with-password \
|
||||
@ -21,7 +30,7 @@ export PENPOT_FLAGS="\
|
||||
disable-login-with-github \
|
||||
disable-login-with-gitlab \
|
||||
disable-telemetry \
|
||||
enable-backend-worker \
|
||||
$__worker_flag \
|
||||
enable-backend-asserts \
|
||||
disable-feature-fdata-pointer-map \
|
||||
enable-feature-fdata-objects-map \
|
||||
|
||||
@ -7,28 +7,33 @@
|
||||
# This file is consumed by compose only. Backend runtime defaults that
|
||||
# compose does not care about live in backend/scripts/_env.
|
||||
|
||||
# Compose project name (replaces the -p flag).
|
||||
COMPOSE_PROJECT_NAME=penpotdev
|
||||
# Compose project name (replaces the -p flag). Set per instance; ws0 default
|
||||
# here. Infra is launched under penpotdev-infra by manage.sh's infra-compose
|
||||
# helper; ws1+ overlays override this to penpotdev-wsN.
|
||||
COMPOSE_PROJECT_NAME=penpotdev-ws0
|
||||
|
||||
# Container names and volume names. Volumes are pinned by explicit name
|
||||
# (rather than relying on COMPOSE_PROJECT_NAME prefixing) so the physical
|
||||
# volumes can survive a future project rename without a data-migration step.
|
||||
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-main
|
||||
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-valkey
|
||||
PENPOT_VALKEY_HOSTNAME=penpot-devenv-valkey
|
||||
# volumes survive project renames without a data migration. ws0 reuses the
|
||||
# pre-Stage-2 physical volume names (penpotdev_*).
|
||||
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-ws0-main
|
||||
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-ws0-valkey
|
||||
PENPOT_VALKEY_HOSTNAME=penpot-devenv-ws0-valkey
|
||||
PENPOT_POSTGRES_DATA_VOLUME=penpotdev_postgres_data_pg16
|
||||
PENPOT_MINIO_DATA_VOLUME=penpotdev_minio_data
|
||||
PENPOT_USER_DATA_VOLUME=penpotdev_user_data
|
||||
PENPOT_VALKEY_DATA_VOLUME=penpotdev_valkey_data
|
||||
|
||||
# Backend runtime config (passed to the container env block).
|
||||
# Backend runtime config (passed to the container env block). PENPOT_REDIS_URI
|
||||
# is set explicitly per instance to match the per-instance Valkey container
|
||||
# name; ws1+ overlays override this.
|
||||
PENPOT_HOST=devenv
|
||||
PENPOT_PUBLIC_URI=https://localhost:3449
|
||||
PENPOT_DATABASE_URI=postgresql://postgres/penpot
|
||||
PENPOT_DATABASE_USERNAME=penpot
|
||||
PENPOT_DATABASE_PASSWORD=penpot
|
||||
PENPOT_DATABASE_MAX_POOL_SIZE=60
|
||||
PENPOT_REDIS_URI=redis://redis/0
|
||||
PENPOT_DATABASE_MAX_POOL_SIZE=20
|
||||
PENPOT_REDIS_URI=redis://penpot-devenv-ws0-valkey/0
|
||||
|
||||
# Object storage (MinIO user/policy are provisioned by the infra compose file).
|
||||
PENPOT_OBJECTS_STORAGE_BACKEND=s3
|
||||
@ -51,6 +56,10 @@ PENPOT_MCP_REPL_PORT=4403
|
||||
SERENA_EXTERNAL_PORT=14281
|
||||
SERENA_DASHBOARD_EXTERNAL_PORT=14282
|
||||
|
||||
# Backend worker (scheduled + async tasks). ws0 only; per-instance overlays
|
||||
# for ws1+ override this to false. See mem:devenv/core.
|
||||
PENPOT_BACKEND_WORKER=true
|
||||
|
||||
# Tmux session inside the main container.
|
||||
PENPOT_TMUX_SESSION=penpot
|
||||
PENPOT_TMUX_ATTACH=true
|
||||
|
||||
@ -18,13 +18,12 @@ services:
|
||||
container_name: "${PENPOT_MAIN_CONTAINER_NAME}"
|
||||
stop_signal: SIGINT
|
||||
|
||||
# postgres / minio / minio-setup live in the penpotdev-infra compose
|
||||
# project and cannot be referenced via depends_on across projects.
|
||||
# manage.sh waits for infra readiness before bringing main up.
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_started
|
||||
redis:
|
||||
condition: service_started
|
||||
minio-setup:
|
||||
condition: service_completed_successfully
|
||||
|
||||
volumes:
|
||||
- "user_data:/home/penpot/"
|
||||
@ -83,6 +82,7 @@ services:
|
||||
- PENPOT_OBJECTS_STORAGE_S3_BUCKET=${PENPOT_OBJECTS_STORAGE_S3_BUCKET}
|
||||
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
|
||||
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
|
||||
- PENPOT_BACKEND_WORKER=${PENPOT_BACKEND_WORKER}
|
||||
- PENPOT_TMUX_SESSION=${PENPOT_TMUX_SESSION}
|
||||
- PENPOT_TMUX_ATTACH=${PENPOT_TMUX_ATTACH}
|
||||
|
||||
@ -91,9 +91,7 @@ services:
|
||||
- SERENA_UPDATE_VERSION=1.5.0
|
||||
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- main
|
||||
- default
|
||||
|
||||
redis:
|
||||
image: valkey/valkey:8.1
|
||||
@ -105,6 +103,4 @@ services:
|
||||
- "valkey_data:/data"
|
||||
|
||||
networks:
|
||||
default:
|
||||
aliases:
|
||||
- redis
|
||||
- default
|
||||
|
||||
@ -15,7 +15,3 @@ localhost:3449 {
|
||||
# }
|
||||
reverse_proxy localhost:4449
|
||||
}
|
||||
|
||||
http://penpot-devenv-main:3450 {
|
||||
reverse_proxy localhost:4449
|
||||
}
|
||||
|
||||
@ -70,18 +70,21 @@ var penpotFlags = "enable-mcp";
|
||||
**Running the DevEnv in Agentic Mode.** Start the DevEnv in agentic mode with:
|
||||
|
||||
```bash
|
||||
./manage.sh run-devenv-agentic
|
||||
./manage.sh run-devenv-agentic # ws0 only
|
||||
./manage.sh run-devenv-agentic --n-instances 3 # ws0, ws1, ws2
|
||||
```
|
||||
|
||||
> **Note:** the MCP and Serena tmux windows are only added when the tmux
|
||||
> session is created, not when an existing session is reattached. If you have
|
||||
> already run `./manage.sh run-devenv` (non-agentic) in the current devenv
|
||||
> container, the agentic command will just attach you to that session without
|
||||
> starting MCP or Serena. To switch to agentic mode, kill the existing session
|
||||
> first and rerun:
|
||||
Per-instance ports are offset by `10000 × N` (ws1's MCP at
|
||||
`http://localhost:14401/mcp`, Serena at `http://localhost:24281`, etc.). See
|
||||
the *Parallel workspaces* section in the [Dev environment guide](./devenv.md).
|
||||
|
||||
> **Note:** the MCP and Serena tmux windows are only added when the session is
|
||||
> first created. If you've already run `./manage.sh run-devenv` (non-agentic)
|
||||
> in an instance, `run-devenv-agentic` just reattaches without starting them.
|
||||
> Kill the session first to recreate with the agentic windows:
|
||||
>
|
||||
> ```bash
|
||||
> docker exec penpot-devenv-main sudo -u penpot tmux kill-session -t penpot
|
||||
> docker exec penpot-devenv-ws0-main sudo -u penpot tmux kill-session -t penpot
|
||||
> ./manage.sh run-devenv-agentic
|
||||
> ```
|
||||
|
||||
|
||||
@ -45,47 +45,95 @@ This is an incomplete list of devenv related subcommands found on
|
||||
manage.sh script:
|
||||
|
||||
```bash
|
||||
./manage.sh build-devenv-local # builds the local devenv docker image (called by run-devenv automatically when needed)
|
||||
./manage.sh start-devenv # starts background running containers
|
||||
./manage.sh run-devenv # enters to new tmux session inside of one of the running containers
|
||||
./manage.sh attach-devenv # re-attaches to the tmux session inside an already-running devenv container
|
||||
./manage.sh stop-devenv # stops background running containers
|
||||
./manage.sh drop-devenv # removes all the containers, volumes and networks used by the devenv
|
||||
./manage.sh build-devenv-local # builds the local devenv docker image
|
||||
./manage.sh start-devenv # brings up the shared infra + ws0 in background
|
||||
./manage.sh run-devenv # ws0 with non-agentic tmux, attached (legacy alias)
|
||||
./manage.sh run-devenv-agentic # ws0 (default) with MCP + Serena enabled; see below
|
||||
./manage.sh attach-devenv # re-attaches to the tmux session of a running instance
|
||||
./manage.sh stop-devenv # stops infra and all running parallel instances
|
||||
./manage.sh drop-devenv # removes containers (data volumes preserved)
|
||||
```
|
||||
|
||||
### Upgrading from a pre-split devenv
|
||||
### Parallel workspaces
|
||||
|
||||
The devenv compose configuration was split into two files,
|
||||
`docker/devenv/docker-compose.infra.yml` (Postgres, MinIO, mailer, LDAP) and
|
||||
`docker/devenv/docker-compose.main.yml` (the main devenv container and its
|
||||
Valkey), joined by an external Docker network called `penpot_shared`.
|
||||
Per-instance defaults live in `docker/devenv/defaults.env`.
|
||||
|
||||
If you had the devenv running on the previous single-file compose setup, the
|
||||
first `./manage.sh start-devenv` after pulling the new code may fail with
|
||||
`minio-setup: gave up waiting for MinIO after 30 attempts`. The cause is that
|
||||
docker compose silently reuses the old infra containers (`penpotdev-postgres-1`,
|
||||
`penpotdev-minio-1`, `penpotdev-mailer-1`, `penpotdev-ldap-1`), which are still
|
||||
attached to the auto-generated `penpotdev_default` network from the old project
|
||||
layout. The freshly created `minio-setup` joins the new `penpot_shared`
|
||||
network instead, so it cannot resolve the `minio` hostname.
|
||||
|
||||
Migration steps (data in the named volumes is preserved):
|
||||
The devenv runs as separate compose projects: shared infra (`penpotdev-infra`:
|
||||
Postgres, MinIO, mailer, LDAP) plus one `penpotdev-wsN` project per runtime
|
||||
instance. `ws0` binds the live repo; `ws1..wsN-1` are disposable clones under
|
||||
`~/.penpot/penpot_workspaces/` seeded from the current working tree on each
|
||||
startup.
|
||||
|
||||
```bash
|
||||
./manage.sh stop-devenv
|
||||
docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
|
||||
penpotdev-mailer-1 penpotdev-ldap-1 \
|
||||
penpot-devenv-main penpot-devenv-valkey 2>/dev/null
|
||||
docker network rm penpotdev_default 2>/dev/null
|
||||
./manage.sh start-devenv
|
||||
./manage.sh run-devenv-agentic --n-instances 3
|
||||
```
|
||||
|
||||
After this one-time cleanup the new compose files create fresh containers on
|
||||
`penpot_shared` and `start-devenv` works normally. The named volumes
|
||||
(`penpotdev_postgres_data_pg16`, `penpotdev_minio_data`,
|
||||
`penpotdev_user_data`, `penpotdev_valkey_data`) are not touched, so existing
|
||||
Penpot state survives.
|
||||
is a desired-state reconciler: it brings the running set to exactly
|
||||
`{ws0, ws1, ws2}`. Missing instances are created; surplus instances
|
||||
(highest-numbered first) are stopped; instances already at their target index
|
||||
are left alone. Stopping never removes data volumes or workspace directories.
|
||||
|
||||
Host ports are offset by `10000 × N`:
|
||||
|
||||
| Service | ws0 | ws1 | ws2 |
|
||||
|---|---|---|---|
|
||||
| Penpot UI (HTTPS) | `https://localhost:3449` | `https://localhost:13449` | `https://localhost:23449` |
|
||||
| MCP HTTP stream | `http://localhost:4401/mcp` | `http://localhost:14401/mcp` | `http://localhost:24401/mcp` |
|
||||
| Serena MCP | `http://localhost:14281` | `http://localhost:24281` | `http://localhost:34281` |
|
||||
|
||||
Container-internal ports stay fixed. Target a specific instance with
|
||||
`--instance ws1` on `attach-devenv` / `run-devenv-shell`. `run-devenv-agentic`
|
||||
accepts `--no-mcp`, `--no-serena`, and `--serena-context CTX`.
|
||||
|
||||
### Shared state and workers
|
||||
|
||||
All instances share one Penpot database and one MinIO bucket; users, teams,
|
||||
files, and MCP tokens are visible from every instance. Per-instance Valkey
|
||||
keeps WebSocket Pub/Sub channels isolated. Background workers
|
||||
(`enable-backend-worker`) run only on ws0 — ws1+ overlays disable it so async
|
||||
task notifications stay bound to a single Pub/Sub. Trade-off: async tasks
|
||||
triggered from a ws1+ tab execute (on ws0) but their completion notifications
|
||||
never reach the originating tab.
|
||||
|
||||
### Upgrading from a pre-parallel devenv
|
||||
|
||||
The devenv compose configuration has been split into two files and reorganized
|
||||
into separate compose projects per runtime instance:
|
||||
|
||||
- `docker/devenv/docker-compose.infra.yml` (Postgres, MinIO, mailer, LDAP)
|
||||
runs under the compose project `penpotdev-infra`.
|
||||
- `docker/devenv/docker-compose.main.yml` (one main container + its Valkey)
|
||||
runs once per runtime instance under `penpotdev-ws0`, `penpotdev-ws1`, ….
|
||||
- Both projects join the external Docker network `penpot_shared`, created
|
||||
idempotently by `manage.sh`.
|
||||
- Per-instance configuration lives in `docker/devenv/defaults.env` (ws0
|
||||
baseline) plus generated overlays under `docker/devenv/instances/`.
|
||||
|
||||
If you had the devenv running on the previous single-project (`penpotdev`)
|
||||
layout, leftover containers and the auto-generated `penpotdev_default`
|
||||
network must be removed before bringing the new ws0 instance up. The named
|
||||
data volumes (`penpotdev_postgres_data_pg16`, `penpotdev_minio_data`,
|
||||
`penpotdev_user_data`, `penpotdev_valkey_data`) are pinned by explicit
|
||||
`name:` entries in the new compose files and are preserved through the
|
||||
transition — your Postgres DB, MinIO objects, and home cache survive.
|
||||
|
||||
One-time cleanup, then bring up ws0:
|
||||
|
||||
```bash
|
||||
# Stop and remove the old single-project containers (data volumes stay).
|
||||
docker stop penpot-devenv-main penpot-devenv-valkey 2>/dev/null
|
||||
docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
|
||||
penpotdev-mailer-1 penpotdev-ldap-1 \
|
||||
penpot-devenv-main penpot-devenv-valkey 2>/dev/null
|
||||
|
||||
# Remove the orphaned auto-generated network.
|
||||
docker network rm penpotdev_default 2>/dev/null
|
||||
|
||||
# Bring up infra + ws0 under the new project layout.
|
||||
./manage.sh run-devenv-agentic --n-instances 1
|
||||
```
|
||||
|
||||
After the cleanup, normal `./manage.sh start-devenv` / `run-devenv` /
|
||||
`run-devenv-agentic` commands work against the new layout. The legacy
|
||||
`penpotdev` compose project is no longer used.
|
||||
|
||||
Having the container running and tmux opened inside the container,
|
||||
you are free to execute commands and open as many shells as you want.
|
||||
|
||||
454
manage.sh
454
manage.sh
@ -26,8 +26,14 @@ done < "$DEVENV_DEFAULTS_FILE"
|
||||
unset __key __value
|
||||
|
||||
# Source path for the workspace bind mount; consumed by docker-compose.main.yml.
|
||||
# ws0 binds the live repo at $PWD; ws1+ override this in their overlay env file.
|
||||
export PENPOT_SOURCE_PATH="${PENPOT_SOURCE_PATH:-$PWD}"
|
||||
|
||||
# Per-instance values like PENPOT_REDIS_URI must live in each instance's env
|
||||
# file (not in this shell), because docker compose's --env-file mechanism
|
||||
# lets a per-instance overlay override the baseline while the shell env
|
||||
# would otherwise shadow both for every project.
|
||||
|
||||
export CURRENT_USER_ID=$(id -u);
|
||||
export CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD);
|
||||
|
||||
@ -145,57 +151,245 @@ function ensure-devenv-network {
|
||||
docker network inspect "$DEVENV_NETWORK" >/dev/null 2>&1 || docker network create "$DEVENV_NETWORK" >/dev/null
|
||||
}
|
||||
|
||||
function devenv-compose {
|
||||
docker compose \
|
||||
--env-file "$DEVENV_DEFAULTS_FILE" \
|
||||
-f docker/devenv/docker-compose.infra.yml \
|
||||
-f docker/devenv/docker-compose.main.yml \
|
||||
"$@"
|
||||
# Compose-project plumbing for the parallel-workspaces layout.
|
||||
#
|
||||
# - Shared infrastructure (postgres, minio, mailer, ldap, minio-setup) runs
|
||||
# under project `penpotdev-infra`.
|
||||
# - Each runtime instance (ws0, ws1, ...) runs its own main + valkey under
|
||||
# project `penpotdev-wsN`. ws0 uses only `defaults.env`; ws1+ additionally
|
||||
# layer a generated overlay file under `docker/devenv/instances/`.
|
||||
# `env -i` strips the shell env before invoking docker compose so the
|
||||
# per-instance overlay --env-file actually overrides defaults.env. Without
|
||||
# stripping, the shell would still hold whatever values defaults.env was
|
||||
# sourced into at startup (PENPOT_MAIN_CONTAINER_NAME, etc.), and Docker
|
||||
# Compose's substitution gives the shell precedence over --env-file.
|
||||
# Only the values that genuinely need to be per-call (HOME/PATH for tooling,
|
||||
# CURRENT_USER_ID/PENPOT_SOURCE_PATH for the compose substitution) are
|
||||
# re-exported.
|
||||
function infra-compose {
|
||||
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
|
||||
docker compose -p penpotdev-infra \
|
||||
--env-file "$DEVENV_DEFAULTS_FILE" \
|
||||
-f docker/devenv/docker-compose.infra.yml \
|
||||
"$@"
|
||||
}
|
||||
|
||||
function instance-compose {
|
||||
local instance="$1"; shift
|
||||
local source_path env_files
|
||||
env_files=(--env-file "$DEVENV_DEFAULTS_FILE")
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
source_path="$PWD"
|
||||
else
|
||||
source_path="$(workspace-path "$instance")"
|
||||
env_files+=(--env-file "docker/devenv/instances/${instance}.env")
|
||||
fi
|
||||
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
|
||||
CURRENT_USER_ID="${CURRENT_USER_ID:-$(id -u)}" \
|
||||
PENPOT_SOURCE_PATH="$source_path" \
|
||||
docker compose -p "penpotdev-${instance}" \
|
||||
"${env_files[@]}" \
|
||||
-f docker/devenv/docker-compose.main.yml \
|
||||
"$@"
|
||||
}
|
||||
|
||||
# Names of currently-running parallel instances (ws0, ws1, ...).
|
||||
function list-running-instances {
|
||||
docker ps --format '{{.Label "com.docker.compose.project"}}' 2>/dev/null \
|
||||
| sort -u \
|
||||
| grep -oE '^penpotdev-ws[0-9]+$' \
|
||||
| sed 's/^penpotdev-//' \
|
||||
|| true
|
||||
}
|
||||
|
||||
function devenv-main-container {
|
||||
devenv-compose ps -q main
|
||||
local instance="${1:-ws0}"
|
||||
instance-compose "$instance" ps -q main
|
||||
}
|
||||
|
||||
function devenv-main-running {
|
||||
local container=$(devenv-main-container)
|
||||
local instance="${1:-ws0}"
|
||||
local container
|
||||
container=$(devenv-main-container "$instance")
|
||||
[[ -n "$container" ]] && [[ "$(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null)" = "true" ]]
|
||||
}
|
||||
|
||||
# Bring shared infra up and block until minio-setup has provisioned the
|
||||
# shared MinIO user/policy. Idempotent: a second call when everything is
|
||||
# already up returns immediately.
|
||||
function ensure-infra-up {
|
||||
infra-compose up -d
|
||||
local setup_container
|
||||
setup_container=$(infra-compose ps -aq minio-setup 2>/dev/null)
|
||||
if [[ -n "$setup_container" ]]; then
|
||||
docker wait "$setup_container" >/dev/null 2>&1 || true
|
||||
fi
|
||||
}
|
||||
|
||||
# Refuse to sync workspaces if the live repo is in a fragile Git state.
|
||||
# Copying a partial rebase/merge/cherry-pick into all workspaces would leave
|
||||
# every instance in the same broken state.
|
||||
function assert-clean-git-state {
|
||||
local fragile=""
|
||||
[ -d .git/rebase-apply ] && fragile="$fragile rebase-apply"
|
||||
[ -d .git/rebase-merge ] && fragile="$fragile rebase-merge"
|
||||
[ -f .git/MERGE_HEAD ] && fragile="$fragile merge"
|
||||
[ -f .git/CHERRY_PICK_HEAD ] && fragile="$fragile cherry-pick"
|
||||
[ -f .git/index.lock ] && fragile="$fragile index.lock"
|
||||
if [[ -n "$fragile" ]]; then
|
||||
echo "Live repo Git state is unsafe to copy into workspaces:$fragile" >&2
|
||||
echo "Finish or abort the in-progress operation, then retry." >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Absolute path of the workspace directory for a non-ws0 instance.
|
||||
function workspace-path {
|
||||
local instance="$1"
|
||||
echo "$HOME/.penpot/penpot_workspaces/$instance"
|
||||
}
|
||||
|
||||
# Generate (or refresh) the per-instance Compose env-file overlay. Idempotent;
|
||||
# safe to call on every reconciler pass.
|
||||
function write-instance-env {
|
||||
local instance="$1"
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ ! "$instance" =~ ^ws([0-9]+)$ ]]; then
|
||||
echo "write-instance-env: invalid instance '$instance'" >&2
|
||||
return 1
|
||||
fi
|
||||
local n="${BASH_REMATCH[1]}"
|
||||
local offset=$(( n * 10000 ))
|
||||
|
||||
local file="docker/devenv/instances/${instance}.env"
|
||||
mkdir -p docker/devenv/instances
|
||||
local workspace
|
||||
workspace=$(workspace-path "$instance")
|
||||
cat >"$file" <<EOF
|
||||
# Auto-generated by manage.sh for instance '$instance'.
|
||||
# Edits are overwritten on the next reconciler pass.
|
||||
|
||||
COMPOSE_PROJECT_NAME=penpotdev-${instance}
|
||||
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-${instance}-main
|
||||
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-${instance}-valkey
|
||||
PENPOT_VALKEY_HOSTNAME=penpot-devenv-${instance}-valkey
|
||||
PENPOT_USER_DATA_VOLUME=penpotdev_${instance}_user_data
|
||||
PENPOT_VALKEY_DATA_VOLUME=penpotdev_${instance}_valkey_data
|
||||
|
||||
PENPOT_PUBLIC_URI=https://localhost:$(( 3449 + offset ))
|
||||
PENPOT_REDIS_URI=redis://penpot-devenv-${instance}-valkey/0
|
||||
PENPOT_TMUX_SESSION=penpot
|
||||
|
||||
PENPOT_PUBLIC_HTTP_PORT=$(( 3449 + offset ))
|
||||
PENPOT_MCP_SERVER_PORT=$(( 4401 + offset ))
|
||||
PENPOT_MCP_REPL_PORT=$(( 4403 + offset ))
|
||||
SERENA_EXTERNAL_PORT=$(( 14281 + offset ))
|
||||
SERENA_DASHBOARD_EXTERNAL_PORT=$(( 14282 + offset ))
|
||||
|
||||
# Background workers run only on ws0 to keep async-task notifications bound
|
||||
# to a single Valkey Pub/Sub. See mem:devenv/core.
|
||||
PENPOT_BACKEND_WORKER=false
|
||||
|
||||
# Workspace bind mount (computed in manage.sh too, but recorded here for
|
||||
# clarity when inspecting the env file).
|
||||
PENPOT_SOURCE_PATH=${workspace}
|
||||
EOF
|
||||
}
|
||||
|
||||
# Seed (or re-seed) a workspace from the live repo, then switch it onto a
|
||||
# unique branch. Two-step sync:
|
||||
# 1. .git directory is rsync'd directly (so the workspace has its own
|
||||
# clone with the developer's current commits / index).
|
||||
# 2. Working-tree files are enumerated by `git ls-files`, which is the
|
||||
# only authoritative source for "what files belong in the working
|
||||
# tree" (Git tracks files even when their parent directory matches
|
||||
# a gitignore pattern, e.g. .clj-kondo/config.edn). Using rsync's
|
||||
# gitignore filter directly misses those.
|
||||
# Gitignored caches already in the workspace (node_modules, target, etc.)
|
||||
# are left in place: no --delete on the working-tree pass.
|
||||
function sync-workspace {
|
||||
local instance="$1"
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
return 0
|
||||
fi
|
||||
assert-clean-git-state || return 1
|
||||
|
||||
local workspace
|
||||
workspace=$(workspace-path "$instance")
|
||||
mkdir -p "$workspace"
|
||||
|
||||
echo "[$instance] syncing workspace at $workspace ..."
|
||||
|
||||
# .git directory — direct mirror, including index, refs, hooks, etc.
|
||||
rsync -a --delete "$PWD/.git/" "$workspace/.git/"
|
||||
|
||||
# Working-tree files: tracked + untracked-not-ignored. git ls-files
|
||||
# speaks Git's actual semantics, including the "tracked overrides
|
||||
# gitignore" rule. --files-from feeds the path list to rsync verbatim.
|
||||
local files
|
||||
files=$(mktemp)
|
||||
git -C "$PWD" ls-files -z --cached --others --exclude-standard >"$files"
|
||||
rsync -a --files-from="$files" --from0 "$PWD/" "$workspace/"
|
||||
rm -f "$files"
|
||||
|
||||
(
|
||||
cd "$workspace"
|
||||
git switch -C "${instance}/${CURRENT_BRANCH}" >/dev/null
|
||||
)
|
||||
}
|
||||
|
||||
function start-devenv {
|
||||
pull-devenv-if-not-exists $@;
|
||||
ensure-devenv-network;
|
||||
|
||||
devenv-compose up -d;
|
||||
ensure-infra-up
|
||||
instance-compose ws0 up -d
|
||||
}
|
||||
|
||||
function create-devenv {
|
||||
pull-devenv-if-not-exists $@;
|
||||
ensure-devenv-network;
|
||||
|
||||
devenv-compose create;
|
||||
infra-compose create
|
||||
instance-compose ws0 create
|
||||
}
|
||||
|
||||
function stop-devenv {
|
||||
devenv-compose stop -t 2;
|
||||
local ws
|
||||
for ws in $(list-running-instances); do
|
||||
instance-compose "$ws" stop -t 2
|
||||
done
|
||||
infra-compose stop -t 2
|
||||
}
|
||||
|
||||
function drop-devenv {
|
||||
devenv-compose down -t 2 -v;
|
||||
local ws
|
||||
for ws in $(list-running-instances); do
|
||||
# Never -v: data preservation rule.
|
||||
instance-compose "$ws" down -t 2
|
||||
done
|
||||
infra-compose down -t 2
|
||||
|
||||
echo "Clean old development image $DEVENV_IMGNAME..."
|
||||
docker images $DEVENV_IMGNAME -q | awk '{print $3}' | xargs --no-run-if-empty docker rmi
|
||||
docker images $DEVENV_IMGNAME -q | xargs --no-run-if-empty docker rmi
|
||||
}
|
||||
|
||||
function log-devenv {
|
||||
devenv-compose logs -f --tail=50
|
||||
# Tail ws0 by default; for multi-instance dev, attach explicitly per project.
|
||||
instance-compose ws0 logs -f --tail=50
|
||||
}
|
||||
|
||||
function run-devenv-tmux {
|
||||
local extra_env_args=()
|
||||
local instance="ws0"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--instance)
|
||||
instance="$(normalize-instance "$2")"; shift 2;;
|
||||
-e)
|
||||
extra_env_args+=(-e "$2"); shift 2;;
|
||||
-e*)
|
||||
@ -206,68 +400,239 @@ function run-devenv-tmux {
|
||||
esac
|
||||
done
|
||||
|
||||
if ! devenv-main-running; then
|
||||
start-devenv
|
||||
echo "Waiting for containers fully start (5s)..."
|
||||
sleep 5;
|
||||
if ! devenv-main-running "$instance"; then
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
start-devenv
|
||||
echo "Waiting for containers fully start (5s)..."
|
||||
sleep 5
|
||||
else
|
||||
echo "Instance '$instance' is not running; bring it up first with './manage.sh run-devenv-agentic --n-instances N'." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
local container=$(devenv-main-container)
|
||||
local container
|
||||
container=$(devenv-main-container "$instance")
|
||||
docker exec -ti \
|
||||
"${extra_env_args[@]}" \
|
||||
"$container" sudo -EH -u penpot PENPOT_PLUGIN_DEV=$PENPOT_PLUGIN_DEV /home/start-tmux.sh
|
||||
}
|
||||
|
||||
# Normalize an instance specifier ("0", "ws0", "1", "ws3", ...) to "wsN".
|
||||
function normalize-instance {
|
||||
local raw="$1"
|
||||
if [[ "$raw" =~ ^ws[0-9]+$ ]]; then
|
||||
echo "$raw"
|
||||
elif [[ "$raw" =~ ^[0-9]+$ ]]; then
|
||||
echo "ws$raw"
|
||||
else
|
||||
echo "Invalid --instance value: '$raw' (expected 0|ws0|1|ws1|...)" >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
# Bring a single instance up: workspace sync (skipped for ws0), env-file
|
||||
# write (skipped for ws0), compose up, and detached tmux start with the
|
||||
# requested feature flags.
|
||||
function start-instance {
|
||||
local instance="$1"
|
||||
local enable_mcp="$2"
|
||||
local enable_serena="$3"
|
||||
local serena_context="$4"
|
||||
|
||||
if [[ "$instance" != "ws0" ]]; then
|
||||
sync-workspace "$instance"
|
||||
write-instance-env "$instance"
|
||||
fi
|
||||
|
||||
instance-compose "$instance" up -d --no-deps main redis
|
||||
|
||||
# Wait briefly for main to be reachable; the tmux session lives inside.
|
||||
local container deadline
|
||||
container=$(devenv-main-container "$instance")
|
||||
deadline=$(( SECONDS + 30 ))
|
||||
while ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; do
|
||||
[[ $SECONDS -ge $deadline ]] && {
|
||||
echo "[${instance}] main container did not reach Running within 30s" >&2
|
||||
return 1
|
||||
}
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Start the tmux session detached so the reconciler can proceed to the
|
||||
# next instance without blocking on an interactive attach.
|
||||
local tmux_env=(-e PENPOT_TMUX_ATTACH=false)
|
||||
if [[ "$enable_mcp" == "true" ]]; then
|
||||
tmux_env+=(-e PENPOT_FLAGS="${PENPOT_FLAGS:-} enable-mcp")
|
||||
fi
|
||||
if [[ "$enable_serena" == "true" ]]; then
|
||||
tmux_env+=(-e SERENA_ENABLED=true -e SERENA_CONTEXT="$serena_context")
|
||||
fi
|
||||
docker exec -d "${tmux_env[@]}" "$container" \
|
||||
sudo -EH -u penpot PENPOT_PLUGIN_DEV="${PENPOT_PLUGIN_DEV:-}" /home/start-tmux.sh
|
||||
}
|
||||
|
||||
# Stop and remove one instance's containers without touching its volumes or
|
||||
# its on-disk workspace directory (rule: never wipe data).
|
||||
function stop-instance {
|
||||
local instance="$1"
|
||||
instance-compose "$instance" down -t 2
|
||||
}
|
||||
|
||||
# Print per-instance URLs (Penpot UI, MCP stream endpoint, Serena, attach
|
||||
# command) for one instance.
|
||||
function print-instance-info {
|
||||
local instance="$1"
|
||||
local enable_mcp="$2"
|
||||
local enable_serena="$3"
|
||||
local n=0
|
||||
[[ "$instance" =~ ^ws([0-9]+)$ ]] && n="${BASH_REMATCH[1]}"
|
||||
local offset=$(( n * 10000 ))
|
||||
local public=$(( 3449 + offset ))
|
||||
local mcp=$(( 4401 + offset ))
|
||||
local serena=$(( 14281 + offset ))
|
||||
|
||||
echo
|
||||
echo "[$instance]"
|
||||
echo " Penpot UI: https://localhost:${public}"
|
||||
if [[ "$enable_mcp" == "true" ]]; then
|
||||
echo " MCP stream: http://localhost:${mcp}/mcp"
|
||||
fi
|
||||
if [[ "$enable_serena" == "true" ]]; then
|
||||
echo " Serena MCP: http://localhost:${serena}"
|
||||
fi
|
||||
echo " Attach: ./manage.sh attach-devenv --instance ${instance}"
|
||||
}
|
||||
|
||||
# Reconcile the running parallel set to exactly {ws0..ws(N-1)}.
|
||||
function run-devenv-agentic {
|
||||
local n_instances=1
|
||||
local enable_mcp=true
|
||||
local enable_serena=true
|
||||
local serena_context="desktop-app"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--n-instances)
|
||||
n_instances="$2"; shift 2;;
|
||||
--serena-context)
|
||||
serena_context="$2"; shift 2;;
|
||||
--no-mcp)
|
||||
enable_mcp=false; shift;;
|
||||
--no-serena)
|
||||
enable_serena=false; shift;;
|
||||
*)
|
||||
echo "run-devenv-agentic: unknown argument '$1'" >&2
|
||||
return 1;;
|
||||
esac
|
||||
done
|
||||
|
||||
if ! devenv-main-running; then
|
||||
start-devenv
|
||||
echo "Waiting for containers fully start (5s)..."
|
||||
sleep 5;
|
||||
if ! [[ "$n_instances" =~ ^[1-9][0-9]*$ ]]; then
|
||||
echo "run-devenv-agentic: --n-instances must be a positive integer (got '$n_instances')" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
run-devenv-tmux \
|
||||
-e SERENA_ENABLED=true \
|
||||
-e SERENA_CONTEXT="$serena_context" \
|
||||
-e PENPOT_FLAGS="${PENPOT_FLAGS} enable-mcp"
|
||||
pull-devenv-if-not-exists
|
||||
ensure-devenv-network
|
||||
ensure-infra-up
|
||||
|
||||
# Compute target and running sets.
|
||||
local target=()
|
||||
local i
|
||||
for (( i=0; i < n_instances; i++ )); do
|
||||
target+=("ws$i")
|
||||
done
|
||||
local running
|
||||
running=$(list-running-instances)
|
||||
|
||||
# Stop extras, highest-numbered first.
|
||||
local to_stop=()
|
||||
for ws in $running; do
|
||||
if ! printf '%s\n' "${target[@]}" | grep -qx "$ws"; then
|
||||
to_stop+=("$ws")
|
||||
fi
|
||||
done
|
||||
if [[ ${#to_stop[@]} -gt 0 ]]; then
|
||||
# Sort numerically descending.
|
||||
IFS=$'\n' to_stop=($(printf '%s\n' "${to_stop[@]}" \
|
||||
| sed 's/^ws//' | sort -rn | sed 's/^/ws/'))
|
||||
unset IFS
|
||||
for ws in "${to_stop[@]}"; do
|
||||
echo "Stopping $ws..."
|
||||
stop-instance "$ws"
|
||||
done
|
||||
fi
|
||||
|
||||
# Start missing instances.
|
||||
for ws in "${target[@]}"; do
|
||||
if printf '%s\n' "$running" | grep -qx "$ws"; then
|
||||
echo "[$ws] already running; leaving alone"
|
||||
continue
|
||||
fi
|
||||
echo "Starting $ws..."
|
||||
start-instance "$ws" "$enable_mcp" "$enable_serena" "$serena_context"
|
||||
done
|
||||
|
||||
# Per-instance startup info.
|
||||
for ws in "${target[@]}"; do
|
||||
print-instance-info "$ws" "$enable_mcp" "$enable_serena"
|
||||
done
|
||||
}
|
||||
|
||||
function run-devenv-shell {
|
||||
if ! devenv-main-running; then
|
||||
start-devenv
|
||||
local instance="ws0"
|
||||
local positional=()
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--instance)
|
||||
instance="$(normalize-instance "$2")"; shift 2;;
|
||||
*)
|
||||
positional+=("$1"); shift;;
|
||||
esac
|
||||
done
|
||||
|
||||
if ! devenv-main-running "$instance"; then
|
||||
if [[ "$instance" == "ws0" ]]; then
|
||||
start-devenv
|
||||
else
|
||||
echo "Instance '$instance' is not running." >&2
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
local container=$(devenv-main-container)
|
||||
local container
|
||||
container=$(devenv-main-container "$instance")
|
||||
docker exec -ti \
|
||||
-e JAVA_OPTS="$JAVA_OPTS" \
|
||||
-e EXTERNAL_UID=$CURRENT_USER_ID \
|
||||
"$container" sudo -EH -u penpot $@
|
||||
"$container" sudo -EH -u penpot "${positional[@]}"
|
||||
}
|
||||
|
||||
function attach-devenv {
|
||||
if ! devenv-main-running; then
|
||||
echo "devenv is not running." >&2
|
||||
echo "Start it first with './manage.sh run-devenv' (or './manage.sh start-devenv' for containers only)." >&2
|
||||
local instance="ws0"
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--instance)
|
||||
instance="$(normalize-instance "$2")"; shift 2;;
|
||||
*)
|
||||
echo "attach-devenv: unknown argument '$1'" >&2
|
||||
return 1;;
|
||||
esac
|
||||
done
|
||||
|
||||
if ! devenv-main-running "$instance"; then
|
||||
echo "Instance '$instance' is not running." >&2
|
||||
echo "Start it first with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic --n-instances N' (parallel)." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
local session="${PENPOT_TMUX_SESSION:-penpot}"
|
||||
local container=$(devenv-main-container)
|
||||
local container
|
||||
container=$(devenv-main-container "$instance")
|
||||
|
||||
if ! docker exec "$container" sudo -EH -u penpot tmux has-session -t "$session" 2>/dev/null; then
|
||||
echo "No tmux session '$session' inside the devenv container." >&2
|
||||
echo "Start it with './manage.sh run-devenv'." >&2
|
||||
echo "No tmux session '$session' inside instance '$instance'." >&2
|
||||
echo "Start it with './manage.sh run-devenv' (ws0) or './manage.sh run-devenv-agentic'." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
@ -504,12 +869,17 @@ function usage {
|
||||
echo "- start-devenv Start the development oriented docker compose service."
|
||||
echo "- stop-devenv Stops the development oriented docker compose service."
|
||||
echo "- drop-devenv Remove the development oriented docker compose containers, volumes and clean images."
|
||||
echo "- run-devenv Attaches to the running devenv container and starts development environment"
|
||||
echo "- run-devenv Brings ws0 up and attaches to its tmux session (no MCP, no Serena)."
|
||||
echo " Optional --instance <wsN> targets a different instance."
|
||||
echo " Optional -e flags are forwarded to 'docker exec' (e.g. -e MY_VAR=value)."
|
||||
echo "- run-devenv-agentic Like run-devenv but with additional processes for agentic development enabled."
|
||||
echo " Options: --serena-context CONTEXT (default: desktop-app)"
|
||||
echo "- attach-devenv Attaches to the tmux session inside the running devenv container."
|
||||
echo "- run-devenv-shell Attaches to the running devenv container and starts a bash shell."
|
||||
echo "- run-devenv-agentic Desired-state reconciler. Brings the running parallel set to exactly"
|
||||
echo " {ws0..ws(N-1)} with MCP and Serena enabled on each."
|
||||
echo " Options: --n-instances N (default: 1), --serena-context CONTEXT (default: desktop-app),"
|
||||
echo " --no-mcp, --no-serena"
|
||||
echo "- attach-devenv Attaches to the tmux session inside a running instance."
|
||||
echo " Options: --instance 0|wsN|N (default: 0)"
|
||||
echo "- run-devenv-shell Opens a bash shell inside a running instance."
|
||||
echo " Options: --instance 0|wsN|N (default: 0)"
|
||||
echo "- isolated-shell Starts a bash shell in a new devenv container."
|
||||
echo "- log-devenv Show logs of the running devenv docker compose service."
|
||||
echo ""
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user