Codex ac78a0015b 🐳 Run parallel devenv instances against shared infra
Add support for running N parallel devenv instances under separate compose
projects sharing Postgres, MinIO, mailer, and LDAP. Each instance has its
own main container, Valkey, source checkout, tmux session, and host port
range offset by 10000 (3449 -> 13449 -> 23449, etc.).

./manage.sh run-devenv-agentic --n-instances N reconciles the running set
to exactly {ws0..ws(N-1)}: missing instances are created (workspace sync
from the live repo via git ls-files + per-instance env-file generation
under docker/devenv/instances/ + detached tmux startup), surplus instances
are stopped highest-first via compose down (never -v), already-running
instances are left untouched. ws0 binds the live repo at PWD; ws1+ are
scratch clones under ~/.penpot/penpot_workspaces/.

Backend workers (enable-backend-worker) are gated on PENPOT_BACKEND_WORKER
in backend/scripts/_env; ws1+ overlays disable them so async-task
notifications stay bound to a single Valkey Pub/Sub instance.

Compose helpers wrap docker compose with env -i so per-instance overlay
--env-file actually overrides defaults.env -- without the strip, the shell
env from sourcing defaults.env at startup would shadow the overlay (Compose
gives shell precedence over --env-file).

Other:
- Drop network aliases (- main, - redis); use container_name for
  cross-container DNS so multiple instances on the shared network don't
  fight over the same DNS name.
- Pin volume names via name: (PENPOT_*_VOLUME) so volumes survive project
  renames; ws0 keeps the pre-existing physical names (penpotdev_*).
- Remove cross-project depends_on from main.yml (postgres/minio-setup now
  live in penpotdev-infra); manage.sh ensure-infra-up docker-waits on the
  minio-setup one-shot.
- Strict arg parsing in run-devenv / run-devenv-agentic; --n-instances 0
  rejected.
- Remove unused Host-matched server block from the Caddyfile.

Memory mem:devenv/core and developer docs updated.

Co-authored-by: Codex <codex@openai.com>
2026-06-01 10:39:03 +02:00

102 lines
3.5 KiB
Bash

#!/usr/bin/env bash
export PENPOT_NITRATE_SHARED_KEY=super-secret-nitrate-api-key
export PENPOT_EXPORTER_SHARED_KEY=super-secret-exporter-api-key
export PENPOT_NEXUS_SHARED_KEY=super-secret-nexus-api-key
export PENPOT_SECRET_KEY=super-secret-devenv-key
# DEPRECATED: only used for subscriptions
export PENPOT_MANAGEMENT_API_KEY=super-secret-management-api-key
# Runtime config that varies per devenv instance (PENPOT_HOST, PENPOT_PUBLIC_URI,
# PENPOT_DATABASE_*, PENPOT_REDIS_URI, PENPOT_OBJECTS_STORAGE_*, AWS_*) is owned by
# docker/devenv/defaults.env and injected via the main service's env block.
# Background worker flag is per-instance. Defaults to enabled (ws0); ws1+
# overlays set PENPOT_BACKEND_WORKER=false so scheduled and async tasks only
# run on ws0, keeping notification Pub/Sub bound to a single Valkey. See
# mem:devenv/core for the rationale.
__worker_flag=""
if [[ "${PENPOT_BACKEND_WORKER:-true}" == "true" ]]; then
__worker_flag="enable-backend-worker"
fi
export PENPOT_FLAGS="\
$PENPOT_FLAGS \
enable-login-with-password \
disable-login-with-ldap \
disable-login-with-oidc \
disable-login-with-google \
disable-login-with-github \
disable-login-with-gitlab \
disable-telemetry \
$__worker_flag \
enable-backend-asserts \
disable-feature-fdata-pointer-map \
enable-feature-fdata-objects-map \
enable-audit-log \
enable-transit-readable-response \
enable-demo-users \
enable-user-feedback \
disable-secure-session-cookies \
enable-smtp \
enable-prepl-server \
enable-urepl-server \
enable-rpc-climit \
enable-rpc-rlimit \
enable-quotes \
enable-soft-rpc-rlimit \
enable-auto-file-snapshot \
enable-webhooks \
enable-access-tokens \
disable-tiered-file-data-storage \
enable-file-validation \
enable-file-schema-validation \
enable-redis-cache \
enable-subscriptions";
# Uncomment for nexus integration testing
# export PENPOT_FLAGS="$PENPOT_FLAGS enable-audit-log-archive";
# export PENPOT_AUDIT_LOG_ARCHIVE_URI="http://localhost:6070/api/audit";
# Default deletion delay for devenv
export PENPOT_DELETION_DELAY="24h"
# Setup default upload media file size to 100MiB
export PENPOT_MEDIA_MAX_FILE_SIZE=104857600
# Setup default multipart upload size to 300MiB
export PENPOT_HTTP_SERVER_MAX_MULTIPART_BODY_SIZE=314572800
export PENPOT_USER_FEEDBACK_DESTINATION="support@example.com"
export PENPOT_NITRATE_BACKEND_URI=http://localhost:3000/admin-console
export JAVA_OPTS="\
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager \
-Djdk.attach.allowAttachSelf \
-Dlog4j2.configurationFile=log4j2-devenv.xml \
-Djdk.tracePinnedThreads=full \
-Dim4java.useV7=true \
-XX:+UnlockExperimentalVMOptions \
-XX:+UseShenandoahGC \
-XX:+UseCompactObjectHeaders \
-XX:ShenandoahGCMode=generational \
-XX:-OmitStackTraceInFastThrow \
--sun-misc-unsafe-memory-access=allow \
--enable-preview \
--enable-native-access=ALL-UNNAMED";
function setup_minio() {
if [ "${PENPOT_OBJECTS_STORAGE_BACKEND}" != "s3" ]; then
return 0
fi
# Shared MinIO user/policy provisioning is handled by docker-compose.infra.yml.
# Per process startup only ensures that the configured bucket exists.
mc alias set penpot-s3/ "${PENPOT_OBJECTS_STORAGE_S3_ENDPOINT}" minioadmin minioadmin -q
mc mb "penpot-s3/${PENPOT_OBJECTS_STORAGE_S3_BUCKET}" -p -q
}