♻️ Inject per-instance devenv config instead of overlay files

Also removes unneeded configurability of the tmux session name

Parallel ws1+ instances were configured by generating a gitignored docker/devenv/instances/wsN.env overlay on every start and layering it over defaults.env with a second --env-file. The file looked editable but was overwritten each run and could go stale; every value in it is a pure function of the instance number.

Replace it with instance-env-overrides, which echoes the per-instance KEY=VALUE pairs that instance-compose injects into its env -i docker compose call. Compose gives shell-env precedence over --env-file, so they override the defaults.env baseline; nothing is written to disk and the values are recomputed every call, so they cannot drift. Re-key the is-it-set-up guard to the workspace clone's existence, drop the unused instances/ dir and its .gitignore entry, and trim the per-instance-config docs and memory to current behaviour.

docker compose config renders byte-identical to the old overlay mechanism for ws0 and ws1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Michael Panchenko 2026-06-02 15:25:21 +03:00
parent dfe9f6e93e
commit 5a780f16c5
7 changed files with 67 additions and 125 deletions

1
.gitignore vendored
View File

@ -95,7 +95,6 @@
/.idea
/.claude
/.playwright-mcp
/docker/devenv/instances/
/.devenv/mcp/
/opencode.json
/.codex/

View File

@ -11,21 +11,21 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par
## Source-of-truth files
- `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable.
- `docker/devenv/instances/wsN.env` (N≥1): regenerated by `write-instance-env` on every ws1+ start. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored.
- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is.
- `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`.
- Compose files use pure `${VAR}` substitution; missing var = compose fails.
## Invariants
- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`. Without it, sourcing `defaults.env` into the shell at startup would shadow per-instance overlay `--env-file` (Compose gives shell precedence over `--env-file`).
- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ overlays set distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration.
- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`, then re-inject what compose needs. Stripping is required because `defaults.env` is sourced into manage.sh's shell at startup (stale values would leak); the ws1+ overrides are deliberately re-injected as shell env vars precisely because Compose gives shell precedence over `--env-file`, so they override the `defaults.env` baseline.
- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ inject distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration.
- Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead.
- No cross-project `depends_on`. `manage.sh ensure-infra-up` `docker wait`s on the `minio-setup` one-shot.
- `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`.
## Worker policy
Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. ws0 must be running whenever any ws1+ is running, and is the last instance to stop — `run-devenv-agentic --ws N` (N≥1) auto-starts ws0 first; `stop-devenv` refuses to stop ws0 while any ws1+ is up. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`.
Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ inject it as false. ws0 must be running whenever any ws1+ is running, and is the last instance to stop — `run-devenv-agentic --ws N` (N≥1) auto-starts ws0 first; `stop-devenv` refuses to stop ws0 while any ws1+ is up. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`.
## Port layout
@ -44,7 +44,7 @@ Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin d
## Tmux + MCP routing
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv-agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv-agentic` — the conditional windows are only added at session create time.
`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv-agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv-agentic` — the conditional windows are only added at session create time.
MCP plugin routing is same-origin: frontend uses `<public-uri>/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`.

View File

@ -1,7 +1,9 @@
# Single source of truth for instance-specific devenv configuration.
# Loaded by docker compose via --env-file and also sourced by manage.sh
# (see manage.sh). Per-instance overlay files (e.g. instances/ws1.env) are
# layered on top; variables not overridden fall back to the values here.
# (see manage.sh). This is the ws0 baseline; for ws1+ manage.sh injects the
# per-instance values as environment variables (see instance-env-overrides),
# which override these via Compose's shell-over-env-file precedence. Variables
# not overridden fall back to the values here.
#
# Backend runtime defaults that compose does not care about live in
# backend/scripts/_env.
@ -57,7 +59,6 @@ SERENA_DASHBOARD_EXTERNAL_PORT=14182
PENPOT_BACKEND_WORKER=true
# Tmux session inside the main container.
PENPOT_TMUX_SESSION=penpot
PENPOT_TMUX_ATTACH=true
# Base directory holding non-main workspace clones (one subdir per wsN, N>=1).

View File

@ -83,7 +83,6 @@ services:
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- PENPOT_BACKEND_WORKER=${PENPOT_BACKEND_WORKER}
- PENPOT_TMUX_SESSION=${PENPOT_TMUX_SESSION}
- PENPOT_TMUX_ATTACH=${PENPOT_TMUX_ATTACH}
# Agentic devenv: set to a commit/tag to update Serena on startup,

View File

@ -6,7 +6,7 @@ cd ~;
source ~/.bashrc
PENPOT_TMUX_SESSION="${PENPOT_TMUX_SESSION:-penpot}"
PENPOT_TMUX_SESSION="penpot"
PENPOT_TMUX_ATTACH="${PENPOT_TMUX_ATTACH:-true}"
function attach_or_exit() {

View File

@ -110,32 +110,9 @@ ws0-only and takes no workspace flag. `run-devenv-agentic` also accepts
`--serena-context CTX` and `--git-user-name NAME` / `--git-user-email
EMAIL` (see below).
### Per-instance configuration
Instance configuration is layered, with two sources:
- `docker/devenv/defaults.env` — the baseline, tracked in git. `ws0` uses it
directly, and it supplies every value that does not vary per instance
(database/object-storage settings, shared-infra hostnames, the worker
default, …).
- `docker/devenv/instances/wsN.env` — a generated overlay for each `ws1+`
instance. `manage.sh` writes it on every start/reconciler pass with only the
values that differ per instance: the offset host ports, the per-instance
container/volume names, and the Redis/public URIs. Compose loads it *after*
`defaults.env` (`--env-file defaults.env --env-file instances/wsN.env`), so
these override the baseline; everything else falls through to `defaults.env`.
Two things to keep in mind:
- **The overlays are auto-generated and gitignored** (`docker/devenv/instances/`
is in `.gitignore`). They are regenerated from scratch on each run, so hand
edits do not survive — change `defaults.env` (or the port/naming logic in
`manage.sh`) instead of editing an overlay.
- **The overlay lives in the control checkout, not in the workspace it
configures.** `wsN`'s overlay sits at `docker/devenv/instances/wsN.env` inside
the repo you run `manage.sh` from, while the workspace clone it configures
lives separately at `${PENPOT_WORKSPACES_DIR}/wsN/`. Compose is invoked from
that control checkout, so the relative `--env-file` path has to resolve there.
Configuration lives in one tracked file, `docker/devenv/defaults.env` (the
ws0 baseline); `ws1+` values (offset ports, `wsN` container/volume names) are
derived and injected automatically, so there is no per-instance file to edit.
### Git identity inside the container
@ -184,8 +161,8 @@ into separate compose projects per runtime instance:
runs once per runtime instance under `penpotdev-ws0`, `penpotdev-ws1`, ….
- Both projects join the external Docker network `penpot_shared`, created
idempotently by `manage.sh`.
- Per-instance configuration lives in `docker/devenv/defaults.env` (ws0
baseline) plus generated overlays under `docker/devenv/instances/`.
- Configuration lives in `docker/devenv/defaults.env` (the ws0 baseline);
ws1+ overrides are computed and injected at compose time.
If you had the devenv running on the previous single-project (`penpotdev`)
layout, leftover containers and the auto-generated `penpotdev_default`

138
manage.sh
View File

@ -38,9 +38,9 @@ export PENPOT_WORKSPACES_DIR="${PENPOT_WORKSPACES_DIR:-$HOME/.penpot/penpot_work
# IS ws0's published port. To keep a single source of truth, the bases are
# derived from the ws0 values sourced from defaults.env above rather than
# duplicated here -- this makes it impossible for ws0's compose substitution and
# the ws1+ overlay arithmetic to drift apart. `:?` aborts loudly if defaults.env
# is missing one. Consumed by write-instance-env (the values baked into the
# per-instance compose env file) and print-instance-info (the startup URLs).
# the ws1+ offset arithmetic to drift apart. `:?` aborts loudly if defaults.env
# is missing one. Consumed by instance-env-overrides (the values injected into
# the per-instance compose env) and print-instance-info (the startup URLs).
PENPOT_INSTANCE_PORT_STRIDE=10000
PENPOT_PORT_BASE_PUBLIC=${PENPOT_PUBLIC_HTTP_PORT:?missing in defaults.env}
PENPOT_PORT_BASE_MCP=${PENPOT_MCP_SERVER_PORT:?missing in defaults.env}
@ -77,7 +77,10 @@ set -e
#
# Devenv compose plumbing (used by every *-devenv command below)
# ensure-devenv-network create the external 'penpot_shared' network
# devenv-compose wrap 'docker compose' with --env-file + both files
# infra-compose wrap 'docker compose' for the shared-infra project
# instance-compose wrap 'docker compose' for one instance's main
# project, injecting that instance's overrides
# instance-env-overrides the per-instance KEY=VALUE overrides (ws1+)
# devenv-main-container resolve the 'main' container id via compose ps
# devenv-main-running true if 'main' is up
#
@ -179,16 +182,17 @@ function ensure-devenv-network {
# - Shared infrastructure (postgres, minio, mailer, ldap, minio-setup) runs
# under project `penpotdev-infra`.
# - Each runtime instance (ws0, ws1, ...) runs its own main + valkey under
# project `penpotdev-wsN`. ws0 uses only `defaults.env`; ws1+ additionally
# layer a generated overlay file under `docker/devenv/instances/`.
# `env -i` strips the shell env before invoking docker compose so the
# per-instance overlay --env-file actually overrides defaults.env. Without
# stripping, the shell would still hold whatever values defaults.env was
# sourced into at startup (PENPOT_MAIN_CONTAINER_NAME, etc.), and Docker
# Compose's substitution gives the shell precedence over --env-file.
# Only the values that genuinely need to be per-call (HOME/PATH for tooling,
# CURRENT_USER_ID/PENPOT_SOURCE_PATH for the compose substitution) are
# re-exported.
# project `penpotdev-wsN`. ws0 uses the `defaults.env` baseline as-is; ws1+
# override the per-instance values by injecting them as environment
# variables -- see instance-env-overrides (no files are written).
# `env -i` strips the ambient shell before invoking docker compose, then we
# re-inject exactly what compose needs. The stripping matters because
# defaults.env is sourced into manage.sh's own shell at startup, so otherwise
# those stale values would leak into substitution. And because Docker Compose
# gives shell-env precedence over --env-file, the re-injected per-instance
# overrides cleanly override the defaults.env baseline. Re-injected: HOME/PATH
# (tooling), CURRENT_USER_ID/PENPOT_SOURCE_PATH (always per-call), and for ws1+
# the instance-env-overrides block.
function infra-compose {
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
docker compose -p penpotdev-infra \
@ -199,19 +203,20 @@ function infra-compose {
function instance-compose {
local instance="$1"; shift
local source_path env_files
env_files=(--env-file "$DEVENV_DEFAULTS_FILE")
local source_path
local -a overrides=()
if [[ "$instance" == "ws0" ]]; then
source_path="$PWD"
else
source_path="$(workspace-path "$instance")"
env_files+=(--env-file "docker/devenv/instances/${instance}.env")
mapfile -t overrides < <(instance-env-overrides "$instance")
fi
env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \
CURRENT_USER_ID="${CURRENT_USER_ID:-$(id -u)}" \
PENPOT_SOURCE_PATH="$source_path" \
"${overrides[@]}" \
docker compose -p "penpotdev-${instance}" \
"${env_files[@]}" \
--env-file "$DEVENV_DEFAULTS_FILE" \
-f docker/devenv/docker-compose.main.yml \
"$@"
}
@ -227,9 +232,9 @@ function list-running-instances {
function devenv-main-container {
local instance="${1:-ws0}"
# For ws1+, skip compose if the env-file hasn't been generated yet — the
# instance has never been brought up so there is no container to find.
if [[ "$instance" != "ws0" && ! -f "docker/devenv/instances/${instance}.env" ]]; then
# For ws1+, skip compose if the workspace clone doesn't exist yet — the
# instance has never been set up, so there is no container to find.
if [[ "$instance" != "ws0" && ! -d "$(workspace-path "$instance")" ]]; then
return 0
fi
instance-compose "$instance" ps -q main 2>/dev/null
@ -286,76 +291,38 @@ function instance-port {
echo $(( base + n * PENPOT_INSTANCE_PORT_STRIDE ))
}
# Generate (or refresh) the per-instance Compose env-file overlay. Idempotent;
# safe to call on every reconciler pass.
# For ws0, this is a no-op since it uses the baseline defaults.env directly.
# Echo the per-instance Compose variable overrides for a ws1+ instance, one
# KEY=VALUE per line, for instance-compose to inject into its `env -i` line.
# Compose gives shell-env precedence over --env-file, so these override the
# defaults.env baseline. Every value is a pure function of the instance number,
# so nothing is persisted: they are recomputed on every compose invocation and
# can never drift from this logic. ws0 has no overrides (it uses defaults.env
# as-is) and is never passed here.
#
# The overlay is layered on top of defaults.env by instance-compose's
# `--env-file defaults.env --env-file instances/wsN.env` (compose applies the
# later file last), so this writes only the values that differ per instance --
# ports, container/volume names, the Redis/public URIs, worker flag. Everything
# else falls through to defaults.env.
#
# Note the split: the overlay lives in the *control checkout* at
# docker/devenv/instances/wsN.env (gitignored, regenerated each pass), keyed by
# instance name -- NOT inside the wsN workspace clone it configures (that lives
# at $PENPOT_WORKSPACES_DIR/wsN and holds only source). It has to live here
# because compose runs from $PWD, so the relative --env-file path resolves
# against this repo. Hand edits do not survive a reconciler pass.
function write-instance-env {
# Omitted on purpose: COMPOSE_PROJECT_NAME (set via compose's -p flag),
# PENPOT_SOURCE_PATH (injected directly by instance-compose), and
function instance-env-overrides {
local instance="$1"
if [[ "$instance" == "ws0" ]]; then
return 0
fi
if [[ ! "$instance" =~ ^ws([0-9]+)$ ]]; then
echo "write-instance-env: invalid instance '$instance'" >&2
return 1
fi
local file="docker/devenv/instances/${instance}.env"
mkdir -p docker/devenv/instances
local workspace
workspace=$(workspace-path "$instance")
local public mcp mcp_repl serena serena_dash
public=$(instance-port "$instance" "$PENPOT_PORT_BASE_PUBLIC")
mcp=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP")
mcp_repl=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP_REPL")
serena=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA")
serena_dash=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA_DASHBOARD")
cat >"$file" <<EOF
# Auto-generated by manage.sh for instance '$instance'.
# Edits are overwritten on the next reconciler pass.
COMPOSE_PROJECT_NAME=penpotdev-${instance}
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-${instance}-main
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-${instance}-valkey
PENPOT_VALKEY_HOSTNAME=penpot-devenv-${instance}-valkey
PENPOT_USER_DATA_VOLUME=penpotdev_${instance}_user_data
PENPOT_VALKEY_DATA_VOLUME=penpotdev_${instance}_valkey_data
PENPOT_PUBLIC_URI=https://localhost:${public}
PENPOT_REDIS_URI=redis://penpot-devenv-${instance}-valkey/0
PENPOT_TMUX_SESSION=penpot
PENPOT_PUBLIC_HTTP_PORT=${public}
PENPOT_MCP_SERVER_PORT=${mcp}
PENPOT_MCP_REPL_PORT=${mcp_repl}
SERENA_EXTERNAL_PORT=${serena}
SERENA_DASHBOARD_EXTERNAL_PORT=${serena_dash}
# Background workers run only on ws0 to keep async-task notifications bound
# to a single Valkey Pub/Sub. See mem:devenv/core.
PENPOT_BACKEND_WORKER=false
# shadow-cljs relay proxied through nginx so the browser can reach it without
# a separate published port. URL must match the instance's public HTTPS port.
SHADOW_SERVER_URL=wss://localhost:${public}
# Workspace bind mount (computed in manage.sh too, but recorded here for
# clarity when inspecting the env file).
PENPOT_SOURCE_PATH=${workspace}
EOF
printf '%s\n' \
"PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-${instance}-main" \
"PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-${instance}-valkey" \
"PENPOT_VALKEY_HOSTNAME=penpot-devenv-${instance}-valkey" \
"PENPOT_USER_DATA_VOLUME=penpotdev_${instance}_user_data" \
"PENPOT_VALKEY_DATA_VOLUME=penpotdev_${instance}_valkey_data" \
"PENPOT_PUBLIC_URI=https://localhost:${public}" \
"PENPOT_REDIS_URI=redis://penpot-devenv-${instance}-valkey/0" \
"PENPOT_PUBLIC_HTTP_PORT=${public}" \
"PENPOT_MCP_SERVER_PORT=${mcp}" \
"PENPOT_MCP_REPL_PORT=${mcp_repl}" \
"SERENA_EXTERNAL_PORT=${serena}" \
"SERENA_DASHBOARD_EXTERNAL_PORT=${serena_dash}" \
"PENPOT_BACKEND_WORKER=false"
}
# Thin wrappers around .devenv/scripts/merge-mcp-config.py. The script does
@ -868,7 +835,6 @@ function run-devenv-agentic {
if [[ "$do_sync" == "true" ]]; then
sync-workspace "$target"
fi
write-instance-env "$target"
fi
echo "Starting $target..."
@ -931,7 +897,7 @@ function attach-devenv {
return 1
fi
local session="${PENPOT_TMUX_SESSION:-penpot}"
local session="penpot"
local container
container=$(devenv-main-container "$instance")
@ -1072,7 +1038,7 @@ function run-devenv-isolated-shell {
# Resolve the user_data volume and source tree for the target workspace.
# ws0 uses the baseline volume name from defaults.env; wsN follows the
# write-instance-env naming convention (penpotdev_<instance>_user_data)
# instance-env-overrides naming convention (penpotdev_<instance>_user_data)
# and bind-mounts the workspace clone instead of $PWD.
local user_data_volume source_path
if [[ "$instance" == "ws0" ]]; then