🐳 Split devenv compose for parallel workspaces

Move shared services into an infra compose file and keep the main devenv container plus Valkey in a separate compose file driven by defaults.env. Parameterize host-side ports, container names, source path, and runtime env while keeping container-internal ports fixed for same-origin proxying.

Make tmux startup idempotent, add attach-devenv for the live instance, move shared MinIO user setup to infra startup, and let exporter scripts load backend _env.local overrides.

Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
Codex 2026-05-21 16:23:23 +00:00 committed by alonso.torres
parent a9c0b5394c
commit 0e390010fa
14 changed files with 565 additions and 247 deletions

View File

@ -12,6 +12,7 @@ You are working on the GitHub project `penpot/penpot`, a monorepo.
- Commit only when explicitly asked. Commit/PR format + changelog: `mem:workflow/creating-commits`, `mem:workflow/creating-prs`.
- You have access to the GitHub CLI `gh` or corresponding MCP tools.
- Issues are also managed on Taiga. Read issues using the `read_taiga_issue` tool.
- Never run anything that destroys data without explicit permission, including `drop-devenv`, `docker compose down -v`, `docker volume rm ...`. The user's real work lives in the volumes of the shared infra.
# Project modules
@ -29,8 +30,10 @@ This is a monorepo. Principles that apply to one module do *not* generally apply
# Low-centrality project paths
- `docker/` contains devenv related code, not needed unless specifically instructed.
More info in docs/technical-guide if instructed to work on this.
- `docker/` contains devenv related code, not needed unless specifically instructed.
When working on devenv startup, compose layout, instance config (`defaults.env`),
tmux session lifecycle, MinIO provisioning, or anything in `manage.sh`'s
`*-devenv` commands, read `mem:devenv/core`.
- `experiments/` contains standalone experimental HTML/JS/scripts; treat it as non-core unless the user explicitly asks about it.
- `sample_media/` contains sample image/icon media and config used as fixtures/demo material; do not infer app behavior from it.

View File

@ -0,0 +1,48 @@
# Devenv startup and configuration
Compose-based development environment under `docker/devenv/`, driven by `manage.sh`.
## Source-of-truth layout
- `docker/devenv/defaults.env`: single source of truth for devenv config. Loaded by `manage.sh`'s simple env-file parser and by `docker compose --env-file`. Holds `COMPOSE_PROJECT_NAME`, container names (`PENPOT_MAIN_CONTAINER_NAME`, `PENPOT_VALKEY_CONTAINER_NAME`, `PENPOT_VALKEY_HOSTNAME`), runtime config that is passed into the container env, every published host port, Serena host ports, tmux session/attach defaults. `manage.sh` aborts if the file is unreadable.
- `backend/scripts/_env`: backend-internal defaults only — `PENPOT_*_SHARED_KEY`, `PENPOT_SECRET_KEY`, `PENPOT_FLAGS`, deletion/upload sizes, `PENPOT_NITRATE_BACKEND_URI`, `JAVA_OPTS`, the `setup_minio` function. Never duplicates anything in `defaults.env`.
- `docker/devenv/docker-compose.infra.yml`: shared services — `postgres`, `minio`, `minio-setup`, `mailer`, `ldap`. Attached to external network `penpot_shared`.
- `docker/devenv/docker-compose.main.yml`: main devenv container plus `redis` (valkey). Same network. Pure `${VAR}` references (no inline `:-` defaults) — missing var = compose fails.
## Invariants
- Published ports are host-side only. Compose maps `${PENPOT_*_PORT}:<fixed internal port>` so parallel instances can offset host ports while container-local services keep their normal devenv ports. Do not pass host-side port offsets into processes that expect container-local ports.
- Volume keys in compose are literal (`user_data`, `valkey_data`). Docker prefixes them with `COMPOSE_PROJECT_NAME` to form the actual volume names.
- External network `penpot_shared` is created idempotently by `manage.sh ensure-devenv-network`; `drop-devenv` does **not** remove it.
- `PENPOT_SOURCE_PATH` is set by `manage.sh` to `$PWD` and bind-mounted as `/home/penpot/penpot`. Not in `defaults.env` because its value is dynamic.
- `CURRENT_USER_ID=$(id -u)` is exported by `manage.sh` and passed as `EXTERNAL_UID` so file ownership inside the container matches the host.
- `JAVA_OPTS` exported at the top of `manage.sh` (line ~28) is **shadowed inside the container** by `_env`, which reassigns it unconditionally to a much larger JVM config. The `-e JAVA_OPTS=$JAVA_OPTS` flag that `run-devenv-shell` / `run-devenv-isolated-shell` / `build` pass into `docker run`/`exec` only matters for processes that do not source `_env`.
## MinIO provisioning split
- Shared user/policy: provisioned once by the `minio-setup` one-shot service in the infra compose file. Alias-set loop bounded to 30 attempts. `main` depends on `service_completed_successfully`.
- Per-process bucket creation: `setup_minio()` in `_env`. Idempotent (`mc mb -p`). Short-circuits if `PENPOT_OBJECTS_STORAGE_BACKEND != s3`.
## Tmux session lifecycle
- `docker/devenv/files/start-tmux.sh` is idempotent at the session level. Reads `PENPOT_TMUX_SESSION` (default `penpot`) and `PENPOT_TMUX_ATTACH` (default `true`). If the session exists it attaches or exits depending on `PENPOT_TMUX_ATTACH`; otherwise runs `./scripts/setup` for frontend/exporter and creates the session with frontend-watch / storybook / exporter / backend / optional MCP / optional Serena windows.
- MCP and Serena windows are added only on session create (gated by `enable-mcp` in `PENPOT_FLAGS` and `SERENA_ENABLED=true`). `run-devenv-agentic` against an existing non-agentic session attaches without adding them — kill the session first to recreate.
- `manage.sh run-devenv`: ensures containers, then invokes start-tmux.sh interactively (attaches).
- `manage.sh attach-devenv`: pure attach — fails fast if devenv isn't running or session doesn't exist. Never starts containers. Takes no arguments.
## Lifecycle commands
`manage.sh` thin wrappers around `devenv-compose` (which adds `--env-file defaults.env` and both compose files):
- `start-devenv` / `create-devenv`: pull image if missing, ensure network, `up -d` / `create`.
- `stop-devenv`, `log-devenv`: as expected.
- `drop-devenv`: `down -v` (removes containers + named volumes) and prunes the devenv image. Preserves `penpot_shared`.
- `run-devenv-shell`: starts containers if needed, `docker exec -ti` as `penpot`.
- `run-devenv-isolated-shell` / `build`: one-shot `docker run` against `${COMPOSE_PROJECT_NAME}_user_data` volume + repo bind mount. Not driven by compose.
## Exporter env
`exporter/scripts/run` and `wait-and-start.sh` source `backend/scripts/_env`, then `_env.local` if present. Backend-style env reaches the exporter via that chain.
## MCP routing in parallel devenvs
The normal MCP path is same-origin: frontend computes `<public-uri>/mcp/ws`, the plugin opens that WebSocket, and the instance-local nginx proxies it to the MCP server inside the same main container. This depends on fixed internal ports; per-instance overlays should only change the published host ports and `PENPOT_PUBLIC_URI`.

View File

@ -85,3 +85,11 @@ From the `mcp/` directory, run
* `pnpm run build` to test the build of all packages
* `pnpm run fmt` to apply the auto-formatter
## Devenv plugin/server wiring
In the normal Penpot devenv MCP path, the browser plugin does not discover or route through Postgres. The frontend provides the plugin extension API with `mcp.getServerUrl()`, currently derived from `frontend/src/app/config.cljs` as `penpotMcpServerURI` if set, otherwise `<public-uri>/mcp/ws`. The MCP plugin opens a direct WebSocket to that URL and appends the current MCP access token as a query parameter.
The live plugin connection registry is in-memory inside each MCP server process (`PluginBridge.connectedClients` / `clientsByToken`). The database only stores MCP access tokens and profile props such as `mcp-enabled`; it does not manage which plugin is connected to which MCP server.
For parallel devenvs, prefer same-origin MCP routing: each Penpot instance should expose `/mcp/ws` through its own nginx/Caddy path to the MCP server running inside the same main container. Keep container-internal ports fixed (MCP defaults `4401/4402/4403`, backend/exporter/frontend defaults, etc.) and only offset host-side published ports per instance. If internal ports are offset, hardcoded local proxy config such as `docker/devenv/files/nginx.conf` will misroute unless templated too.

View File

@ -8,8 +8,9 @@ export PENPOT_SECRET_KEY=super-secret-devenv-key
# DEPRECATED: only used for subscriptions
export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
export PENPOT_HOST=devenv
export PENPOT_PUBLIC_URI=https://localhost:3449
# Runtime config that varies per devenv instance (PENPOT_HOST, PENPOT_PUBLIC_URI,
# PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
# docker/devenv/defaults.env and injected via the main service's env block.
export PENPOT_FLAGS="\
$PENPOT_FLAGS \
@ -60,12 +61,6 @@ export PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE=314572800
export PENPOT_USER_FEEDBACK_DESTINATION="support@example.com"
export AWS_ACCESS_KEY_ID=penpot-devenv
export AWS_SECRET_ACCESS_KEY=penpot-devenv
export PENPOT_OBJECTS_STORAGE_BACKEND=s3
export PENPOT_OBJECTS_STORAGE_S3_ENDPOINT=http://minio:9000
export PENPOT_OBJECTS_STORAGE_S3_BUCKET=penpot
export PENPOT_NITRATE_BACKEND_URI=http://localhost:3000/admin-console
export JAVA_OPTS="\
@ -84,14 +79,14 @@ export JAVA_OPTS="\
--enable-native-access=ALL-UNNAMED";
function setup_minio() {
# Initialize MINIO config
mc alias set penpot-s3/ http://minio:9000 minioadmin minioadmin -q
mc admin user add penpot-s3 penpot-devenv penpot-devenv -q
mc admin user info penpot-s3 penpot-devenv |grep -F -q "readwrite"
if [ "$?" = "1" ]; then
mc admin policy attach penpot-s3 readwrite --user=penpot-devenv -q
if [ "${PENPOT_OBJECTS_STORAGE_BACKEND}" != "s3" ]; then
return 0
fi
mc mb penpot-s3/penpot -p -q
# Shared MinIO user/policy provisioning is handled by docker-compose.infra.yml.
# Per process startup only ensures that the configured bucket exists.
mc alias set penpot-s3/ "${PENPOT_OBJECTS_STORAGE_S3_ENDPOINT}" minioadmin minioadmin -q
mc mb "penpot-s3/${PENPOT_OBJECTS_STORAGE_S3_BUCKET}" -p -q
}

View File

@ -0,0 +1,56 @@
# Single source of truth for instance-specific devenv configuration.
# Loaded by docker compose via --env-file (see manage.sh).
#
# Stage 2 adds per-instance overlay files (e.g. instances/ws1.env) loaded
# after this one; variables not overridden fall back to the values here.
#
# This file is consumed by compose only. Backend runtime defaults that
# compose does not care about live in backend/scripts/_env.
# Compose project name (replaces the -p flag).
COMPOSE_PROJECT_NAME=penpotdev
# Container names and volume names. Volumes are pinned by explicit name
# (rather than relying on COMPOSE_PROJECT_NAME prefixing) so the physical
# volumes can survive a future project rename without a data-migration step.
PENPOT_MAIN_CONTAINER_NAME=penpot-devenv-main
PENPOT_VALKEY_CONTAINER_NAME=penpot-devenv-valkey
PENPOT_VALKEY_HOSTNAME=penpot-devenv-valkey
PENPOT_POSTGRES_DATA_VOLUME=penpotdev_postgres_data_pg16
PENPOT_MINIO_DATA_VOLUME=penpotdev_minio_data
PENPOT_USER_DATA_VOLUME=penpotdev_user_data
PENPOT_VALKEY_DATA_VOLUME=penpotdev_valkey_data
# Backend runtime config (passed to the container env block).
PENPOT_HOST=devenv
PENPOT_PUBLIC_URI=https://localhost:3449
PENPOT_DATABASE_URI=postgresql://postgres/penpot
PENPOT_DATABASE_USERNAME=penpot
PENPOT_DATABASE_PASSWORD=penpot
PENPOT_DATABASE_MAX_POOL_SIZE=60
PENPOT_REDIS_URI=redis://redis/0
# Object storage (MinIO user/policy are provisioned by the infra compose file).
PENPOT_OBJECTS_STORAGE_BACKEND=s3
PENPOT_OBJECTS_STORAGE_S3_ENDPOINT=http://minio:9000
PENPOT_OBJECTS_STORAGE_S3_BUCKET=penpot
AWS_ACCESS_KEY_ID=penpot-devenv
AWS_SECRET_ACCESS_KEY=penpot-devenv
# Published host ports. Only ports that need to be reachable from outside the
# container are exposed; everything else (frontend dev server, backend API,
# storybook, exporter, REPLs, plugins, MCP inspector/websocket, aux) is
# accessed in-process or through the same-origin Caddy/nginx proxy at
# PENPOT_PUBLIC_HTTP_PORT. Container-internal ports remain fixed; per-instance
# overlays may offset these host-side values.
PENPOT_PUBLIC_HTTP_PORT=3449
PENPOT_MCP_SERVER_PORT=4401
PENPOT_MCP_REPL_PORT=4403
# Serena (agentic devenv). Internal ports are fixed by Serena itself.
SERENA_EXTERNAL_PORT=14281
SERENA_DASHBOARD_EXTERNAL_PORT=14282
# Tmux session inside the main container.
PENPOT_TMUX_SESSION=penpot
PENPOT_TMUX_ATTACH=true

View File

@ -0,0 +1,99 @@
networks:
default:
name: penpot_shared
external: true
volumes:
postgres_data_pg16:
name: ${PENPOT_POSTGRES_DATA_VOLUME}
minio_data:
name: ${PENPOT_MINIO_DATA_VOLUME}
services:
minio:
image: "minio/minio:RELEASE.2025-04-03T14-56-28Z"
command: minio server /mnt/data --console-address ":9001"
volumes:
- "minio_data:/mnt/data"
environment:
- MINIO_ROOT_USER=minioadmin
- MINIO_ROOT_PASSWORD=minioadmin
networks:
default:
aliases:
- minio
minio-setup:
image: "minio/mc:latest"
depends_on:
- minio
entrypoint: ["/bin/sh", "-c"]
command:
- |
attempts=0
until mc alias set penpot-s3 http://minio:9000 minioadmin minioadmin -q; do
attempts=$$((attempts + 1))
if [ "$$attempts" -ge 30 ]; then
echo "minio-setup: gave up waiting for MinIO after $$attempts attempts" >&2
exit 1
fi
sleep 1
done
mc admin user info penpot-s3 penpot-devenv >/dev/null 2>&1 || mc admin user add penpot-s3 penpot-devenv penpot-devenv -q
mc admin policy attach penpot-s3 readwrite --user=penpot-devenv -q
networks:
default:
postgres:
image: postgres:16.8
command: postgres -c config_file=/etc/postgresql.conf
restart: always
stop_signal: SIGINT
environment:
- POSTGRES_INITDB_ARGS=--data-checksums
- POSTGRES_DB=penpot
- POSTGRES_USER=penpot
- POSTGRES_PASSWORD=penpot
volumes:
- ./files/postgresql.conf:/etc/postgresql.conf:z
- ./files/postgresql_init.sql:/docker-entrypoint-initdb.d/init.sql:z
- postgres_data_pg16:/var/lib/postgresql/data
networks:
default:
aliases:
- postgres
mailer:
image: sj26/mailcatcher:latest
restart: always
expose:
- '1025'
ports:
- "1080:1080"
networks:
default:
aliases:
- mailer
# https://github.com/rroemhild/docker-test-openldap
ldap:
image: rroemhild/test-openldap:2.1
expose:
- '10389'
- '10636'
ports:
- "10389:10389"
- "10636:10636"
ulimits:
nofile:
soft: 1024
hard: 1024
networks:
default:
aliases:
- ldap

View File

@ -0,0 +1,110 @@
networks:
default:
name: penpot_shared
external: true
volumes:
user_data:
name: ${PENPOT_USER_DATA_VOLUME}
valkey_data:
name: ${PENPOT_VALKEY_DATA_VOLUME}
services:
main:
privileged: true
image: "penpotapp/devenv:latest"
build:
context: "."
container_name: "${PENPOT_MAIN_CONTAINER_NAME}"
stop_signal: SIGINT
depends_on:
postgres:
condition: service_started
redis:
condition: service_started
minio-setup:
condition: service_completed_successfully
volumes:
- "user_data:/home/penpot/"
- "${PENPOT_SOURCE_PATH}:/home/penpot/penpot:z"
ports:
# Host ports are instance-specific; container ports stay fixed.
- ${PENPOT_PUBLIC_HTTP_PORT}:3449
- ${PENPOT_PUBLIC_HTTP_PORT}:3449/udp
# MCP
- ${PENPOT_MCP_SERVER_PORT}:4401
- ${PENPOT_MCP_REPL_PORT}:4403
# Serena MCP server (agentic mode only). Internal ports fixed by Serena.
- ${SERENA_EXTERNAL_PORT}:14281
- ${SERENA_DASHBOARD_EXTERNAL_PORT}:24282
environment:
- EXTERNAL_UID=${CURRENT_USER_ID}
# SMTP setup (shared infra service; identical across instances)
- PENPOT_SMTP_ENABLED=true
- PENPOT_SMTP_DEFAULT_FROM=no-reply@example.com
- PENPOT_SMTP_DEFAULT_REPLY_TO=no-reply@example.com
- PENPOT_SMTP_HOST=mailer
- PENPOT_SMTP_PORT=1025
- PENPOT_SMTP_USERNAME=
- PENPOT_SMTP_PASSWORD=
- PENPOT_SMTP_SSL=false
- PENPOT_SMTP_TLS=false
# LDAP setup (shared infra service; identical across instances)
- PENPOT_LDAP_HOST=ldap
- PENPOT_LDAP_PORT=10389
- PENPOT_LDAP_SSL=false
- PENPOT_LDAP_STARTTLS=false
- PENPOT_LDAP_BASE_DN=ou=people,dc=planetexpress,dc=com
- PENPOT_LDAP_BIND_DN=cn=admin,dc=planetexpress,dc=com
- PENPOT_LDAP_BIND_PASSWORD=GoodNewsEveryone
- PENPOT_LDAP_ATTRS_USERNAME=uid
- PENPOT_LDAP_ATTRS_EMAIL=mail
- PENPOT_LDAP_ATTRS_FULLNAME=cn
- PENPOT_LDAP_ATTRS_PHOTO=jpegPhoto
# Per-instance runtime config. Defaults live in defaults.env.
- PENPOT_HOST=${PENPOT_HOST}
- PENPOT_PUBLIC_URI=${PENPOT_PUBLIC_URI}
- PENPOT_DATABASE_URI=${PENPOT_DATABASE_URI}
- PENPOT_DATABASE_USERNAME=${PENPOT_DATABASE_USERNAME}
- PENPOT_DATABASE_PASSWORD=${PENPOT_DATABASE_PASSWORD}
- PENPOT_DATABASE_MAX_POOL_SIZE=${PENPOT_DATABASE_MAX_POOL_SIZE}
- PENPOT_REDIS_URI=${PENPOT_REDIS_URI}
- PENPOT_OBJECTS_STORAGE_BACKEND=${PENPOT_OBJECTS_STORAGE_BACKEND}
- PENPOT_OBJECTS_STORAGE_S3_ENDPOINT=${PENPOT_OBJECTS_STORAGE_S3_ENDPOINT}
- PENPOT_OBJECTS_STORAGE_S3_BUCKET=${PENPOT_OBJECTS_STORAGE_S3_BUCKET}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- PENPOT_TMUX_SESSION=${PENPOT_TMUX_SESSION}
- PENPOT_TMUX_ATTACH=${PENPOT_TMUX_ATTACH}
# Agentic devenv: set to a commit/tag to update Serena on startup,
# leave empty to skip update and use the version baked into the image.
- SERENA_UPDATE_VERSION=1.5.0
networks:
default:
aliases:
- main
redis:
image: valkey/valkey:8.1
hostname: "${PENPOT_VALKEY_HOSTNAME}"
container_name: "${PENPOT_VALKEY_CONTAINER_NAME}"
restart: always
command: valkey-server --save 120 1 --loglevel warning
volumes:
- "valkey_data:/data"
networks:
default:
aliases:
- redis

View File

@ -1,180 +0,0 @@
networks:
default:
driver: bridge
ipam:
config:
- subnet: 172.177.9.0/24
volumes:
postgres_data_pg16:
user_data:
minio_data:
valkey_data:
services:
main:
privileged: true
image: "penpotapp/devenv:latest"
build:
context: "."
container_name: "penpot-devenv-main"
stop_signal: SIGINT
depends_on:
- postgres
- redis
# - keycloak
volumes:
- "user_data:/home/penpot/"
- "${PWD}:/home/penpot/penpot:z"
ports:
- 3447:3447
- 3448:3448
- 3449:3449
- 3449:3449/udp
- 3450:3450
- 6006:6006
- 6060:6060
- 6061:6061
- 6062:6062
- 6063:6063
- 6064:6064
- 9000:9000
- 9001:9001
- 9090:9090
- 9091:9091
# MCP
- 4400:4400
- 4401:4401
- 4402:4402
- 4403:4403
# Plugins
- 4200:4200
- 4201:4201
- 4202:4202
# Serena MCP server (agentic mode only)
- ${SERENA_EXTERNAL_PORT:-14281}:14281
- ${SERENA_DASHBOARD_EXTERNAL_PORT:-14282}:24282
environment:
- EXTERNAL_UID=${CURRENT_USER_ID}
# SMTP setup
- PENPOT_SMTP_ENABLED=true
- PENPOT_SMTP_DEFAULT_FROM=no-reply@example.com
- PENPOT_SMTP_DEFAULT_REPLY_TO=no-reply@example.com
- PENPOT_SMTP_HOST=mailer
- PENPOT_SMTP_PORT=1025
- PENPOT_SMTP_USERNAME=
- PENPOT_SMTP_PASSWORD=
- PENPOT_SMTP_SSL=false
- PENPOT_SMTP_TLS=false
# LDAP setup
- PENPOT_LDAP_HOST=ldap
- PENPOT_LDAP_PORT=10389
- PENPOT_LDAP_SSL=false
- PENPOT_LDAP_STARTTLS=false
- PENPOT_LDAP_BASE_DN=ou=people,dc=planetexpress,dc=com
- PENPOT_LDAP_BIND_DN=cn=admin,dc=planetexpress,dc=com
- PENPOT_LDAP_BIND_PASSWORD=GoodNewsEveryone
- PENPOT_LDAP_ATTRS_USERNAME=uid
- PENPOT_LDAP_ATTRS_EMAIL=mail
- PENPOT_LDAP_ATTRS_FULLNAME=cn
- PENPOT_LDAP_ATTRS_PHOTO=jpegPhoto
# agentic devenv
# Serena update: set to a commit/tag to update Serena on startup, leave empty to skip update and use the version in the image
- SERENA_UPDATE_VERSION=1.5.0
networks:
default:
aliases:
- main
minio:
image: "minio/minio:RELEASE.2025-04-03T14-56-28Z"
command: minio server /mnt/data --console-address ":9001"
volumes:
- "minio_data:/mnt/data"
environment:
- MINIO_ROOT_USER=minioadmin
- MINIO_ROOT_PASSWORD=minioadmin
networks:
default:
aliases:
- minio
postgres:
image: postgres:16.8
command: postgres -c config_file=/etc/postgresql.conf
restart: always
stop_signal: SIGINT
environment:
- POSTGRES_INITDB_ARGS=--data-checksums
- POSTGRES_DB=penpot
- POSTGRES_USER=penpot
- POSTGRES_PASSWORD=penpot
volumes:
- ./files/postgresql.conf:/etc/postgresql.conf:z
- ./files/postgresql_init.sql:/docker-entrypoint-initdb.d/init.sql:z
- postgres_data_pg16:/var/lib/postgresql/data
networks:
default:
aliases:
- postgres
redis:
image: valkey/valkey:8.1
hostname: "penpot-devenv-valkey"
container_name: "penpot-devenv-valkey"
restart: always
command: valkey-server --save 120 1 --loglevel warning
volumes:
- "valkey_data:/data"
networks:
default:
aliases:
- redis
mailer:
image: sj26/mailcatcher:latest
restart: always
expose:
- '1025'
ports:
- "1080:1080"
networks:
default:
aliases:
- mailer
# https://github.com/rroemhild/docker-test-openldap
ldap:
image: rroemhild/test-openldap:2.1
expose:
- '10389'
- '10636'
ports:
- "10389:10389"
- "10636:10636"
ulimits:
nofile:
soft: 1024
hard: 1024
networks:
default:
aliases:
- ldap

View File

@ -6,6 +6,23 @@ cd ~;
source ~/.bashrc
PENPOT_TMUX_SESSION="${PENPOT_TMUX_SESSION:-penpot}"
PENPOT_TMUX_ATTACH="${PENPOT_TMUX_ATTACH:-true}"
function attach_or_exit() {
if [ "$PENPOT_TMUX_ATTACH" = "true" ]; then
exec tmux -2 attach-session -t "$PENPOT_TMUX_SESSION"
fi
echo "[start-tmux.sh] tmux session '$PENPOT_TMUX_SESSION' is running detached"
exit 0
}
if tmux has-session -t "$PENPOT_TMUX_SESSION" 2>/dev/null; then
echo "[start-tmux.sh] Reusing existing tmux session '$PENPOT_TMUX_SESSION'"
attach_or_exit
fi
echo "[start-tmux.sh] Installing node dependencies"
pushd ~/penpot/frontend/
./scripts/setup;
@ -14,32 +31,32 @@ pushd ~/penpot/exporter/
./scripts/setup;
popd
tmux -2 new-session -d -s penpot
tmux -2 new-session -d -s "$PENPOT_TMUX_SESSION"
tmux rename-window -t penpot:0 'frontend watch'
tmux select-window -t penpot:0
tmux send-keys -t penpot 'cd penpot/frontend' enter C-l
tmux send-keys -t penpot './scripts/watch app' enter
tmux rename-window -t "$PENPOT_TMUX_SESSION:0" 'frontend watch'
tmux select-window -t "$PENPOT_TMUX_SESSION:0"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/frontend' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/watch app' enter
tmux new-window -t penpot:1 -n 'frontend storybook'
tmux select-window -t penpot:1
tmux send-keys -t penpot 'cd penpot/frontend' enter C-l
tmux send-keys -t penpot './scripts/watch storybook' enter
tmux new-window -t "$PENPOT_TMUX_SESSION:1" -n 'frontend storybook'
tmux select-window -t "$PENPOT_TMUX_SESSION:1"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/frontend' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/watch storybook' enter
tmux new-window -t penpot:2 -n 'exporter'
tmux select-window -t penpot:2
tmux send-keys -t penpot 'cd penpot/exporter' enter C-l
tmux send-keys -t penpot 'rm -f target/app.js*' enter C-l
tmux send-keys -t penpot './scripts/watch' enter
tmux new-window -t "$PENPOT_TMUX_SESSION:2" -n 'exporter'
tmux select-window -t "$PENPOT_TMUX_SESSION:2"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/exporter' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'rm -f target/app.js*' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/watch' enter
tmux split-window -v
tmux send-keys -t penpot 'cd penpot/exporter' enter C-l
tmux send-keys -t penpot './scripts/wait-and-start.sh' enter
tmux split-window -v -t "$PENPOT_TMUX_SESSION"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/exporter' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/wait-and-start.sh' enter
tmux new-window -t penpot:3 -n 'backend'
tmux select-window -t penpot:3
tmux send-keys -t penpot 'cd penpot/backend' enter C-l
tmux send-keys -t penpot './scripts/start-dev' enter
tmux new-window -t "$PENPOT_TMUX_SESSION:3" -n 'backend'
tmux select-window -t "$PENPOT_TMUX_SESSION:3"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/backend' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/start-dev' enter
if echo "$PENPOT_FLAGS" | grep -q "enable-mcp"; then
pushd ~/penpot/mcp/
@ -47,10 +64,10 @@ if echo "$PENPOT_FLAGS" | grep -q "enable-mcp"; then
pnpm run build;
popd
tmux new-window -t penpot:4 -n 'mcp'
tmux select-window -t penpot:4
tmux send-keys -t penpot 'cd penpot/mcp' enter C-l
tmux send-keys -t penpot './scripts/start-mcp-devenv' enter
tmux new-window -t "$PENPOT_TMUX_SESSION:4" -n 'mcp'
tmux select-window -t "$PENPOT_TMUX_SESSION:4"
tmux send-keys -t "$PENPOT_TMUX_SESSION" 'cd penpot/mcp' enter C-l
tmux send-keys -t "$PENPOT_TMUX_SESSION" './scripts/start-mcp-devenv' enter
fi
if [ "${SERENA_ENABLED:-false}" = "true" ]; then
@ -58,9 +75,9 @@ if [ "${SERENA_ENABLED:-false}" = "true" ]; then
# update Serena (use sudo since the initial Serena installation is global; see Dockerfile)
sudo -E uv tool install -p 3.13 serena-agent@${SERENA_UPDATE_VERSION} --prerelease=allow
fi
tmux new-window -t penpot:5 -n 'serena'
tmux select-window -t penpot:5
tmux send-keys -t penpot "serena start-mcp-server --transport streamable-http --port 14281 --project penpot --context ${SERENA_CONTEXT} --host 0.0.0.0" enter
tmux new-window -t "$PENPOT_TMUX_SESSION:5" -n 'serena'
tmux select-window -t "$PENPOT_TMUX_SESSION:5"
tmux send-keys -t "$PENPOT_TMUX_SESSION" "serena start-mcp-server --transport streamable-http --port 14281 --project penpot --context ${SERENA_CONTEXT} --host 0.0.0.0" enter
fi
tmux -2 attach-session -t penpot
attach_or_exit

View File

@ -73,6 +73,18 @@ var penpotFlags = "enable-mcp";
./manage.sh run-devenv-agentic
```
> **Note:** the MCP and Serena tmux windows are only added when the tmux
> session is created, not when an existing session is reattached. If you have
> already run `./manage.sh run-devenv` (non-agentic) in the current devenv
> container, the agentic command will just attach you to that session without
> starting MCP or Serena. To switch to agentic mode, kill the existing session
> first and rerun:
>
> ```bash
> docker exec penpot-devenv-main sudo -u penpot tmux kill-session -t penpot
> ./manage.sh run-devenv-agentic
> ```
## Opening Penpot with Remote Debugging & MCP Enabled
**Enable Remote Debugging in Your Browser.**

View File

@ -48,10 +48,45 @@ manage.sh script:
./manage.sh build-devenv-local # builds the local devenv docker image (called by run-devenv automatically when needed)
./manage.sh start-devenv # starts background running containers
./manage.sh run-devenv # enters to new tmux session inside of one of the running containers
./manage.sh attach-devenv # re-attaches to the tmux session inside an already-running devenv container
./manage.sh stop-devenv # stops background running containers
./manage.sh drop-devenv # removes all the containers, volumes and networks used by the devenv
```
### Upgrading from a pre-split devenv
The devenv compose configuration was split into two files,
`docker/devenv/docker-compose.infra.yml` (Postgres, MinIO, mailer, LDAP) and
`docker/devenv/docker-compose.main.yml` (the main devenv container and its
Valkey), joined by an external Docker network called `penpot_shared`.
Per-instance defaults live in `docker/devenv/defaults.env`.
If you had the devenv running on the previous single-file compose setup, the
first `./manage.sh start-devenv` after pulling the new code may fail with
`minio-setup: gave up waiting for MinIO after 30 attempts`. The cause is that
docker compose silently reuses the old infra containers (`penpotdev-postgres-1`,
`penpotdev-minio-1`, `penpotdev-mailer-1`, `penpotdev-ldap-1`), which are still
attached to the auto-generated `penpotdev_default` network from the old project
layout. The freshly created `minio-setup` joins the new `penpot_shared`
network instead, so it cannot resolve the `minio` hostname.
Migration steps (data in the named volumes is preserved):
```bash
./manage.sh stop-devenv
docker rm penpotdev-postgres-1 penpotdev-minio-1 penpotdev-minio-setup-1 \
penpotdev-mailer-1 penpotdev-ldap-1 \
penpot-devenv-main penpot-devenv-valkey 2>/dev/null
docker network rm penpotdev_default 2>/dev/null
./manage.sh start-devenv
```
After this one-time cleanup the new compose files create fresh containers on
`penpot_shared` and `start-devenv` works normally. The named volumes
(`penpotdev_postgres_data_pg16`, `penpotdev_minio_data`,
`penpotdev_user_data`, `penpotdev_valkey_data`) are not touched, so existing
Penpot state survives.
Having the container running and tmux opened inside the container,
you are free to execute commands and open as many shells as you want.

View File

@ -3,4 +3,8 @@
SCRIPT_DIR=$(dirname $0);
source $SCRIPT_DIR/../../backend/scripts/_env;
if [ -f $SCRIPT_DIR/../../backend/scripts/_env.local ]; then
source $SCRIPT_DIR/../../backend/scripts/_env.local;
fi
exec node target/app.js

View File

@ -3,6 +3,10 @@
SCRIPT_DIR=$(dirname $0);
source $SCRIPT_DIR/../../backend/scripts/_env;
if [ -f $SCRIPT_DIR/../../backend/scripts/_env.local ]; then
source $SCRIPT_DIR/../../backend/scripts/_env.local;
fi
bb -i '(babashka.wait/wait-for-port "localhost" 9630)';
bb -i '(babashka.wait/wait-for-path "target/app.js")';
sleep 2;

149
manage.sh
View File

@ -2,7 +2,31 @@
export ORGANIZATION="penpotapp";
export DEVENV_IMGNAME="$ORGANIZATION/devenv";
export DEVENV_PNAME="penpotdev";
export DEVENV_NETWORK="penpot_shared";
export DEVENV_DEFAULTS_FILE="docker/devenv/defaults.env";
# Load instance configuration (project name, container names, ports, runtime
# config). Single source of truth for the devenv; consumed by both docker
# compose (via --env-file) and the shell logic below. Hard dependency — abort
# loudly if it's missing or unreadable.
#
# Host-shell env wins over file values: a value already set in the parent
# environment is preserved. This matches docker compose's own precedence rule
# for --env-file (so substitution-time and shell-time agree).
if [ ! -r "$DEVENV_DEFAULTS_FILE" ]; then
echo "manage.sh: cannot read $DEVENV_DEFAULTS_FILE" >&2
exit 1
fi
while IFS='=' read -r __key __value; do
[[ -z "$__key" || "$__key" =~ ^[[:space:]]*# ]] && continue
if [ -z "${!__key+x}" ]; then
export "$__key=$__value"
fi
done < "$DEVENV_DEFAULTS_FILE"
unset __key __value
# Source path for the workspace bind mount; consumed by docker-compose.main.yml.
export PENPOT_SOURCE_PATH="${PENPOT_SOURCE_PATH:-$PWD}"
export CURRENT_USER_ID=$(id -u);
export CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD);
@ -17,6 +41,43 @@ export JAVA_OPTS=${JAVA_OPTS:-"-Xmx1000m -Xms50m"};
set -e
# ----------------------------------------------------------------------------
# Function map
#
# Utility helpers
# print-current-version, setup-buildx, put-license-file
#
# Devenv image lifecycle
# build-devenv, pull-devenv, pull-devenv-if-not-exists
#
# Devenv compose plumbing (used by every *-devenv command below)
# ensure-devenv-network create the external 'penpot_shared' network
# devenv-compose wrap 'docker compose' with --env-file + both files
# devenv-main-container resolve the 'main' container id via compose ps
# devenv-main-running true if 'main' is up
#
# Devenv lifecycle (operate on the whole compose project)
# start-devenv, create-devenv, stop-devenv, drop-devenv, log-devenv
#
# Devenv interactive entry points (all operate on the running 'main' container)
# run-devenv-tmux starts 'main' if needed and execs start-tmux.sh
# interactively (this is what 'run-devenv' resolves to)
# run-devenv-agentic same as run-devenv-tmux but enables MCP + Serena
# attach-devenv pure attach to the existing tmux session; fails
# fast if the devenv or session is missing
# run-devenv-shell starts 'main' if needed and execs a bash shell
# run-devenv-isolated-shell one-shot 'docker run' (NOT compose) against the
# project user_data volume and the current PWD; used
# for ad-hoc operations that should not touch a
# running devenv
#
# Production build pipeline
# build one-shot 'docker run' that invokes a per-module
# build script inside the devenv image
# build-<mod>-bundle project a module's build output into ./bundles/
# build-<mod>-docker-image package a bundle into a release docker image
# ----------------------------------------------------------------------------
ARCH=$(uname -m)
if [[ "$ARCH" == "x86_64" || "$ARCH" == "amd64" || "$ARCH" == "i386" || "$ARCH" == "i686" ]]; then
@ -80,31 +141,54 @@ function pull-devenv-if-not-exists {
fi
}
function ensure-devenv-network {
docker network inspect "$DEVENV_NETWORK" >/dev/null 2>&1 || docker network create "$DEVENV_NETWORK" >/dev/null
}
function devenv-compose {
docker compose \
--env-file "$DEVENV_DEFAULTS_FILE" \
-f docker/devenv/docker-compose.infra.yml \
-f docker/devenv/docker-compose.main.yml \
"$@"
}
function devenv-main-container {
devenv-compose ps -q main
}
function devenv-main-running {
local container=$(devenv-main-container)
[[ -n "$container" ]] && [[ "$(docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null)" = "true" ]]
}
function start-devenv {
pull-devenv-if-not-exists $@;
ensure-devenv-network;
docker compose -p $DEVENV_PNAME -f docker/devenv/docker-compose.yaml up -d;
devenv-compose up -d;
}
function create-devenv {
pull-devenv-if-not-exists $@;
ensure-devenv-network;
docker compose -p $DEVENV_PNAME -f docker/devenv/docker-compose.yaml create;
devenv-compose create;
}
function stop-devenv {
docker compose -p $DEVENV_PNAME -f docker/devenv/docker-compose.yaml stop -t 2;
devenv-compose stop -t 2;
}
function drop-devenv {
docker compose -p $DEVENV_PNAME -f docker/devenv/docker-compose.yaml down -t 2 -v;
devenv-compose down -t 2 -v;
echo "Clean old development image $DEVENV_IMGNAME..."
docker images $DEVENV_IMGNAME -q | awk '{print $3}' | xargs --no-run-if-empty docker rmi
}
function log-devenv {
docker compose -p $DEVENV_PNAME -f docker/devenv/docker-compose.yaml logs -f --tail=50
devenv-compose logs -f --tail=50
}
function run-devenv-tmux {
@ -117,39 +201,38 @@ function run-devenv-tmux {
-e*)
extra_env_args+=(-e "${1#-e}"); shift;;
*)
shift;;
echo "run-devenv: unknown argument '$1'" >&2
return 1;;
esac
done
if [[ ! $(docker ps -f "name=penpot-devenv-main" -q) ]]; then
if ! devenv-main-running; then
start-devenv
echo "Waiting for containers fully start (5s)..."
sleep 5;
fi
local container=$(devenv-main-container)
docker exec -ti \
"${extra_env_args[@]}" \
penpot-devenv-main sudo -EH -u penpot PENPOT_PLUGIN_DEV=$PENPOT_PLUGIN_DEV /home/start-tmux.sh
"$container" sudo -EH -u penpot PENPOT_PLUGIN_DEV=$PENPOT_PLUGIN_DEV /home/start-tmux.sh
}
function run-devenv-agentic {
local serena_context="desktop-app"
local serena_external_port="14281"
local serena_dashboard_external_port="14282"
while [[ $# -gt 0 ]]; do
case "$1" in
--serena-context)
serena_context="$2"; shift 2;;
*)
shift;;
echo "run-devenv-agentic: unknown argument '$1'" >&2
return 1;;
esac
done
if [[ ! $(docker ps -f "name=penpot-devenv-main" -q) ]]; then
SERENA_EXTERNAL_PORT="$serena_external_port" \
SERENA_DASHBOARD_EXTERNAL_PORT="$serena_dashboard_external_port" \
if ! devenv-main-running; then
start-devenv
echo "Waiting for containers fully start (5s)..."
sleep 5;
@ -162,19 +245,39 @@ function run-devenv-agentic {
}
function run-devenv-shell {
if [[ ! $(docker ps -f "name=penpot-devenv-main" -q) ]]; then
if ! devenv-main-running; then
start-devenv
fi
local container=$(devenv-main-container)
docker exec -ti \
-e JAVA_OPTS="$JAVA_OPTS" \
-e EXTERNAL_UID=$CURRENT_USER_ID \
penpot-devenv-main sudo -EH -u penpot $@
"$container" sudo -EH -u penpot $@
}
function attach-devenv {
if ! devenv-main-running; then
echo "devenv is not running." >&2
echo "Start it first with './manage.sh run-devenv' (or './manage.sh start-devenv' for containers only)." >&2
return 1
fi
local session="${PENPOT_TMUX_SESSION:-penpot}"
local container=$(devenv-main-container)
if ! docker exec "$container" sudo -EH -u penpot tmux has-session -t "$session" 2>/dev/null; then
echo "No tmux session '$session' inside the devenv container." >&2
echo "Start it with './manage.sh run-devenv'." >&2
return 1
fi
docker exec -ti "$container" sudo -EH -u penpot tmux attach -t "$session"
}
function run-devenv-isolated-shell {
docker volume create ${DEVENV_PNAME}_user_data;
docker volume create ${PENPOT_USER_DATA_VOLUME};
docker run -ti --rm \
--mount source=${DEVENV_PNAME}_user_data,type=volume,target=/home/penpot/ \
--mount source=${PENPOT_USER_DATA_VOLUME},type=volume,target=/home/penpot/ \
--mount source=`pwd`,type=bind,target=/home/penpot/penpot \
-e EXTERNAL_UID=$CURRENT_USER_ID \
-e BUILD_STORYBOOK=$BUILD_STORYBOOK \
@ -217,9 +320,9 @@ function build {
local script=${2:-build}
pull-devenv-if-not-exists;
docker volume create ${DEVENV_PNAME}_user_data;
docker volume create ${PENPOT_USER_DATA_VOLUME};
docker run -t --rm \
--mount source=${DEVENV_PNAME}_user_data,type=volume,target=/home/penpot/ \
--mount source=${PENPOT_USER_DATA_VOLUME},type=volume,target=/home/penpot/ \
--mount source=`pwd`,type=bind,target=/home/penpot/penpot \
-e EXTERNAL_UID=$CURRENT_USER_ID \
-e BUILD_STORYBOOK=$BUILD_STORYBOOK \
@ -405,6 +508,7 @@ function usage {
echo " Optional -e flags are forwarded to 'docker exec' (e.g. -e MY_VAR=value)."
echo "- run-devenv-agentic Like run-devenv but with additional processes for agentic development enabled."
echo " Options: --serena-context CONTEXT (default: desktop-app)"
echo "- attach-devenv Attaches to the tmux session inside the running devenv container."
echo "- run-devenv-shell Attaches to the running devenv container and starts a bash shell."
echo "- isolated-shell Starts a bash shell in a new devenv container."
echo "- log-devenv Show logs of the running devenv docker compose service."
@ -455,6 +559,9 @@ case $1 in
run-devenv-agentic)
run-devenv-agentic ${@:2}
;;
attach-devenv)
attach-devenv ${@:2}
;;
run-devenv-shell)
run-devenv-shell ${@:2}
;;