From 5a780f16c5eb058e8236401a6bd972cd170fc933 Mon Sep 17 00:00:00 2001 From: Michael Panchenko Date: Tue, 2 Jun 2026 15:25:21 +0300 Subject: [PATCH] :recycle: Inject per-instance devenv config instead of overlay files Also removes unneeded configurability of the tmux session name Parallel ws1+ instances were configured by generating a gitignored docker/devenv/instances/wsN.env overlay on every start and layering it over defaults.env with a second --env-file. The file looked editable but was overwritten each run and could go stale; every value in it is a pure function of the instance number. Replace it with instance-env-overrides, which echoes the per-instance KEY=VALUE pairs that instance-compose injects into its env -i docker compose call. Compose gives shell-env precedence over --env-file, so they override the defaults.env baseline; nothing is written to disk and the values are recomputed every call, so they cannot drift. Re-key the is-it-set-up guard to the workspace clone's existence, drop the unused instances/ dir and its .gitignore entry, and trim the per-instance-config docs and memory to current behaviour. docker compose config renders byte-identical to the old overlay mechanism for ws0 and ws1. Co-Authored-By: Claude Opus 4.8 --- .gitignore | 1 - .serena/memories/devenv/core.md | 10 +- docker/devenv/defaults.env | 7 +- docker/devenv/docker-compose.main.yml | 1 - docker/devenv/files/start-tmux.sh | 2 +- docs/technical-guide/developer/devenv.md | 33 +----- manage.sh | 138 +++++++++-------------- 7 files changed, 67 insertions(+), 125 deletions(-) diff --git a/.gitignore b/.gitignore index ef69f7a7b7..f800ad7f53 100644 --- a/.gitignore +++ b/.gitignore @@ -95,7 +95,6 @@ /.idea /.claude /.playwright-mcp -/docker/devenv/instances/ /.devenv/mcp/ /opencode.json /.codex/ diff --git a/.serena/memories/devenv/core.md b/.serena/memories/devenv/core.md index 2017155628..e612a5f923 100644 --- a/.serena/memories/devenv/core.md +++ b/.serena/memories/devenv/core.md @@ -11,21 +11,21 @@ Compose-based dev environment under `docker/devenv/`, driven by `manage.sh`. Par ## Source-of-truth files - `docker/devenv/defaults.env`: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. `manage.sh` aborts if unreadable. -- `docker/devenv/instances/wsN.env` (N≥1): regenerated by `write-instance-env` on every ws1+ start. Overrides project name, container names, volume names, host ports (offset `10000·N`), `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`, `PENPOT_SOURCE_PATH`. Gitignored. +- For ws1+, `instance-env-overrides` computes the per-instance overrides (container/volume names, host ports offset `10000·N`, `PENPOT_PUBLIC_URI`, `PENPOT_REDIS_URI`, `PENPOT_BACKEND_WORKER=false`) and `instance-compose` injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses `defaults.env` as-is. - `backend/scripts/_env`: backend-internal only — secret keys, `PENPOT_FLAGS` (with `enable-backend-worker` gated on `PENPOT_BACKEND_WORKER`), `JAVA_OPTS`, `setup_minio()`. Never duplicates `defaults.env`. - Compose files use pure `${VAR}` substitution; missing var = compose fails. ## Invariants -- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`. Without it, sourcing `defaults.env` into the shell at startup would shadow per-instance overlay `--env-file` (Compose gives shell precedence over `--env-file`). -- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ overlays set distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration. +- `infra-compose` / `instance-compose` wrap `docker compose` with `env -i`, then re-inject what compose needs. Stripping is required because `defaults.env` is sourced into manage.sh's shell at startup (stale values would leak); the ws1+ overrides are deliberately re-injected as shell env vars precisely because Compose gives shell precedence over `--env-file`, so they override the `defaults.env` baseline. +- Volume names pinned via `name:` (PENPOT_*_VOLUME), decoupled from the compose project name. ws1+ inject distinct per-instance volume names; ws0 keeps the historical `penpotdev_*` physical names so project renames never require data migration. - Network aliases (`- main`, `- redis`) are not declared in main.yml. Compose's auto-service-alias still registers `redis` on the shared network, so DNS for `redis` is non-deterministic with multiple instances. Backend uses `PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0` (container_name) instead. - No cross-project `depends_on`. `manage.sh ensure-infra-up` `docker wait`s on the `minio-setup` one-shot. - `JAVA_OPTS` in `manage.sh` is shadowed inside the container by `_env`. The `-e JAVA_OPTS=...` flag only matters for processes that don't source `_env`. ## Worker policy -Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ overlays set it to false. ws0 must be running whenever any ws1+ is running, and is the last instance to stop — `run-devenv-agentic --ws N` (N≥1) auto-starts ws0 first; `stop-devenv` refuses to stop ws0 while any ws1+ is up. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`. +Backend workers run only on ws0. `_env` gates `enable-backend-worker` on `PENPOT_BACKEND_WORKER`; ws1+ inject it as false. ws0 must be running whenever any ws1+ is running, and is the last instance to stop — `run-devenv-agentic --ws N` (N≥1) auto-starts ws0 first; `stop-devenv` refuses to stop ws0 while any ws1+ is up. Workers are pure fire-and-forget: `wrk/submit!` inserts a row into the shared Postgres `task` table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, `wrk/submit!` `dedupe` is racy across submitters); details in `mem:prod-infra/core`. ## Port layout @@ -44,7 +44,7 @@ Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin d ## Tmux + MCP routing -`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_SESSION` and `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv-agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv-agentic` — the conditional windows are only added at session create time. +`docker/devenv/files/start-tmux.sh` is session-level idempotent. Reads `PENPOT_TMUX_ATTACH`. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus `mcp` (when `enable-mcp` in `PENPOT_FLAGS`) and `serena` (when `SERENA_ENABLED=true`). `run-devenv-agentic` always sets both env vars. The legacy `run-devenv` alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, `stop-devenv` then `run-devenv-agentic` — the conditional windows are only added at session create time. MCP plugin routing is same-origin: frontend uses `/mcp/ws`, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see `mem:mcp/core`. diff --git a/docker/devenv/defaults.env b/docker/devenv/defaults.env index 9dd4b6c042..a41ae2c577 100644 --- a/docker/devenv/defaults.env +++ b/docker/devenv/defaults.env @@ -1,7 +1,9 @@ # Single source of truth for instance-specific devenv configuration. # Loaded by docker compose via --env-file and also sourced by manage.sh -# (see manage.sh). Per-instance overlay files (e.g. instances/ws1.env) are -# layered on top; variables not overridden fall back to the values here. +# (see manage.sh). This is the ws0 baseline; for ws1+ manage.sh injects the +# per-instance values as environment variables (see instance-env-overrides), +# which override these via Compose's shell-over-env-file precedence. Variables +# not overridden fall back to the values here. # # Backend runtime defaults that compose does not care about live in # backend/scripts/_env. @@ -57,7 +59,6 @@ SERENA_DASHBOARD_EXTERNAL_PORT=14182 PENPOT_BACKEND_WORKER=true # Tmux session inside the main container. -PENPOT_TMUX_SESSION=penpot PENPOT_TMUX_ATTACH=true # Base directory holding non-main workspace clones (one subdir per wsN, N>=1). diff --git a/docker/devenv/docker-compose.main.yml b/docker/devenv/docker-compose.main.yml index a249aea93c..1ea9858c66 100644 --- a/docker/devenv/docker-compose.main.yml +++ b/docker/devenv/docker-compose.main.yml @@ -83,7 +83,6 @@ services: - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} - PENPOT_BACKEND_WORKER=${PENPOT_BACKEND_WORKER} - - PENPOT_TMUX_SESSION=${PENPOT_TMUX_SESSION} - PENPOT_TMUX_ATTACH=${PENPOT_TMUX_ATTACH} # Agentic devenv: set to a commit/tag to update Serena on startup, diff --git a/docker/devenv/files/start-tmux.sh b/docker/devenv/files/start-tmux.sh index 835290941b..a029dbe84a 100755 --- a/docker/devenv/files/start-tmux.sh +++ b/docker/devenv/files/start-tmux.sh @@ -6,7 +6,7 @@ cd ~; source ~/.bashrc -PENPOT_TMUX_SESSION="${PENPOT_TMUX_SESSION:-penpot}" +PENPOT_TMUX_SESSION="penpot" PENPOT_TMUX_ATTACH="${PENPOT_TMUX_ATTACH:-true}" function attach_or_exit() { diff --git a/docs/technical-guide/developer/devenv.md b/docs/technical-guide/developer/devenv.md index 04e66a3630..88ae8db6ca 100644 --- a/docs/technical-guide/developer/devenv.md +++ b/docs/technical-guide/developer/devenv.md @@ -110,32 +110,9 @@ ws0-only and takes no workspace flag. `run-devenv-agentic` also accepts `--serena-context CTX` and `--git-user-name NAME` / `--git-user-email EMAIL` (see below). -### Per-instance configuration - -Instance configuration is layered, with two sources: - -- `docker/devenv/defaults.env` — the baseline, tracked in git. `ws0` uses it - directly, and it supplies every value that does not vary per instance - (database/object-storage settings, shared-infra hostnames, the worker - default, …). -- `docker/devenv/instances/wsN.env` — a generated overlay for each `ws1+` - instance. `manage.sh` writes it on every start/reconciler pass with only the - values that differ per instance: the offset host ports, the per-instance - container/volume names, and the Redis/public URIs. Compose loads it *after* - `defaults.env` (`--env-file defaults.env --env-file instances/wsN.env`), so - these override the baseline; everything else falls through to `defaults.env`. - -Two things to keep in mind: - -- **The overlays are auto-generated and gitignored** (`docker/devenv/instances/` - is in `.gitignore`). They are regenerated from scratch on each run, so hand - edits do not survive — change `defaults.env` (or the port/naming logic in - `manage.sh`) instead of editing an overlay. -- **The overlay lives in the control checkout, not in the workspace it - configures.** `wsN`'s overlay sits at `docker/devenv/instances/wsN.env` inside - the repo you run `manage.sh` from, while the workspace clone it configures - lives separately at `${PENPOT_WORKSPACES_DIR}/wsN/`. Compose is invoked from - that control checkout, so the relative `--env-file` path has to resolve there. +Configuration lives in one tracked file, `docker/devenv/defaults.env` (the +ws0 baseline); `ws1+` values (offset ports, `wsN` container/volume names) are +derived and injected automatically, so there is no per-instance file to edit. ### Git identity inside the container @@ -184,8 +161,8 @@ into separate compose projects per runtime instance: runs once per runtime instance under `penpotdev-ws0`, `penpotdev-ws1`, …. - Both projects join the external Docker network `penpot_shared`, created idempotently by `manage.sh`. -- Per-instance configuration lives in `docker/devenv/defaults.env` (ws0 - baseline) plus generated overlays under `docker/devenv/instances/`. +- Configuration lives in `docker/devenv/defaults.env` (the ws0 baseline); + ws1+ overrides are computed and injected at compose time. If you had the devenv running on the previous single-project (`penpotdev`) layout, leftover containers and the auto-generated `penpotdev_default` diff --git a/manage.sh b/manage.sh index 70987bc06e..605cd5e0ef 100755 --- a/manage.sh +++ b/manage.sh @@ -38,9 +38,9 @@ export PENPOT_WORKSPACES_DIR="${PENPOT_WORKSPACES_DIR:-$HOME/.penpot/penpot_work # IS ws0's published port. To keep a single source of truth, the bases are # derived from the ws0 values sourced from defaults.env above rather than # duplicated here -- this makes it impossible for ws0's compose substitution and -# the ws1+ overlay arithmetic to drift apart. `:?` aborts loudly if defaults.env -# is missing one. Consumed by write-instance-env (the values baked into the -# per-instance compose env file) and print-instance-info (the startup URLs). +# the ws1+ offset arithmetic to drift apart. `:?` aborts loudly if defaults.env +# is missing one. Consumed by instance-env-overrides (the values injected into +# the per-instance compose env) and print-instance-info (the startup URLs). PENPOT_INSTANCE_PORT_STRIDE=10000 PENPOT_PORT_BASE_PUBLIC=${PENPOT_PUBLIC_HTTP_PORT:?missing in defaults.env} PENPOT_PORT_BASE_MCP=${PENPOT_MCP_SERVER_PORT:?missing in defaults.env} @@ -77,7 +77,10 @@ set -e # # Devenv compose plumbing (used by every *-devenv command below) # ensure-devenv-network create the external 'penpot_shared' network -# devenv-compose wrap 'docker compose' with --env-file + both files +# infra-compose wrap 'docker compose' for the shared-infra project +# instance-compose wrap 'docker compose' for one instance's main +# project, injecting that instance's overrides +# instance-env-overrides the per-instance KEY=VALUE overrides (ws1+) # devenv-main-container resolve the 'main' container id via compose ps # devenv-main-running true if 'main' is up # @@ -179,16 +182,17 @@ function ensure-devenv-network { # - Shared infrastructure (postgres, minio, mailer, ldap, minio-setup) runs # under project `penpotdev-infra`. # - Each runtime instance (ws0, ws1, ...) runs its own main + valkey under -# project `penpotdev-wsN`. ws0 uses only `defaults.env`; ws1+ additionally -# layer a generated overlay file under `docker/devenv/instances/`. -# `env -i` strips the shell env before invoking docker compose so the -# per-instance overlay --env-file actually overrides defaults.env. Without -# stripping, the shell would still hold whatever values defaults.env was -# sourced into at startup (PENPOT_MAIN_CONTAINER_NAME, etc.), and Docker -# Compose's substitution gives the shell precedence over --env-file. -# Only the values that genuinely need to be per-call (HOME/PATH for tooling, -# CURRENT_USER_ID/PENPOT_SOURCE_PATH for the compose substitution) are -# re-exported. +# project `penpotdev-wsN`. ws0 uses the `defaults.env` baseline as-is; ws1+ +# override the per-instance values by injecting them as environment +# variables -- see instance-env-overrides (no files are written). +# `env -i` strips the ambient shell before invoking docker compose, then we +# re-inject exactly what compose needs. The stripping matters because +# defaults.env is sourced into manage.sh's own shell at startup, so otherwise +# those stale values would leak into substitution. And because Docker Compose +# gives shell-env precedence over --env-file, the re-injected per-instance +# overrides cleanly override the defaults.env baseline. Re-injected: HOME/PATH +# (tooling), CURRENT_USER_ID/PENPOT_SOURCE_PATH (always per-call), and for ws1+ +# the instance-env-overrides block. function infra-compose { env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \ docker compose -p penpotdev-infra \ @@ -199,19 +203,20 @@ function infra-compose { function instance-compose { local instance="$1"; shift - local source_path env_files - env_files=(--env-file "$DEVENV_DEFAULTS_FILE") + local source_path + local -a overrides=() if [[ "$instance" == "ws0" ]]; then source_path="$PWD" else source_path="$(workspace-path "$instance")" - env_files+=(--env-file "docker/devenv/instances/${instance}.env") + mapfile -t overrides < <(instance-env-overrides "$instance") fi env -i HOME="$HOME" PATH="$PATH" PWD="$PWD" \ CURRENT_USER_ID="${CURRENT_USER_ID:-$(id -u)}" \ PENPOT_SOURCE_PATH="$source_path" \ + "${overrides[@]}" \ docker compose -p "penpotdev-${instance}" \ - "${env_files[@]}" \ + --env-file "$DEVENV_DEFAULTS_FILE" \ -f docker/devenv/docker-compose.main.yml \ "$@" } @@ -227,9 +232,9 @@ function list-running-instances { function devenv-main-container { local instance="${1:-ws0}" - # For ws1+, skip compose if the env-file hasn't been generated yet — the - # instance has never been brought up so there is no container to find. - if [[ "$instance" != "ws0" && ! -f "docker/devenv/instances/${instance}.env" ]]; then + # For ws1+, skip compose if the workspace clone doesn't exist yet — the + # instance has never been set up, so there is no container to find. + if [[ "$instance" != "ws0" && ! -d "$(workspace-path "$instance")" ]]; then return 0 fi instance-compose "$instance" ps -q main 2>/dev/null @@ -286,76 +291,38 @@ function instance-port { echo $(( base + n * PENPOT_INSTANCE_PORT_STRIDE )) } -# Generate (or refresh) the per-instance Compose env-file overlay. Idempotent; -# safe to call on every reconciler pass. -# For ws0, this is a no-op since it uses the baseline defaults.env directly. +# Echo the per-instance Compose variable overrides for a ws1+ instance, one +# KEY=VALUE per line, for instance-compose to inject into its `env -i` line. +# Compose gives shell-env precedence over --env-file, so these override the +# defaults.env baseline. Every value is a pure function of the instance number, +# so nothing is persisted: they are recomputed on every compose invocation and +# can never drift from this logic. ws0 has no overrides (it uses defaults.env +# as-is) and is never passed here. # -# The overlay is layered on top of defaults.env by instance-compose's -# `--env-file defaults.env --env-file instances/wsN.env` (compose applies the -# later file last), so this writes only the values that differ per instance -- -# ports, container/volume names, the Redis/public URIs, worker flag. Everything -# else falls through to defaults.env. -# -# Note the split: the overlay lives in the *control checkout* at -# docker/devenv/instances/wsN.env (gitignored, regenerated each pass), keyed by -# instance name -- NOT inside the wsN workspace clone it configures (that lives -# at $PENPOT_WORKSPACES_DIR/wsN and holds only source). It has to live here -# because compose runs from $PWD, so the relative --env-file path resolves -# against this repo. Hand edits do not survive a reconciler pass. -function write-instance-env { +# Omitted on purpose: COMPOSE_PROJECT_NAME (set via compose's -p flag), +# PENPOT_SOURCE_PATH (injected directly by instance-compose), and +function instance-env-overrides { local instance="$1" - if [[ "$instance" == "ws0" ]]; then - return 0 - fi - - if [[ ! "$instance" =~ ^ws([0-9]+)$ ]]; then - echo "write-instance-env: invalid instance '$instance'" >&2 - return 1 - fi - - local file="docker/devenv/instances/${instance}.env" - mkdir -p docker/devenv/instances - local workspace - workspace=$(workspace-path "$instance") local public mcp mcp_repl serena serena_dash public=$(instance-port "$instance" "$PENPOT_PORT_BASE_PUBLIC") mcp=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP") mcp_repl=$(instance-port "$instance" "$PENPOT_PORT_BASE_MCP_REPL") serena=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA") serena_dash=$(instance-port "$instance" "$PENPOT_PORT_BASE_SERENA_DASHBOARD") - cat >"$file" <_user_data) + # instance-env-overrides naming convention (penpotdev__user_data) # and bind-mounts the workspace clone instead of $PWD. local user_data_volume source_path if [[ "$instance" == "ws0" ]]; then