mirror of https://github.com/penpot/penpot.git synced 2026-10-02 08:46:15 +00:00

* ⬆️ Update devenv dependencies

Update Node.js, OpenCode, clj-kondo, Babashka, Pixi, GitHub CLI, uv,
and Serena to their current stable releases.

AI-assisted-by: gpt-5.6-sol

* ⬆️ Update devenv to Java 27

Use Zulu JDK 27 in the development image for compatibility testing.
Update the official checksums for both supported architectures.

AI-assisted-by: gpt-5.6-sol

* 🐳 Replace MinIO with RustFS in devenv

Run RustFS as the development S3 service and wait for its health check.
Install a pinned AWS CLI with checksums and use it to create the bucket
idempotently from each backend entry point.

Keep the old MinIO volume untouched and use a new RustFS volume.

AI-assisted-by: gpt-5.6-sol

* 🐳 Replace MailCatcher with persistent Mailpit

Run Mailpit as the devenv SMTP sink while preserving mailer:1025 and the
localhost:1080 UI.

Store its SQLite inbox in a named volume and wait for the readiness
endpoint before starting runtime containers. Bind the web UI to loopback so
development emails stay local.

AI-assisted-by: gpt-5.6-sol

* ⬆️ Update Node.js to 24.21.0

Align the host NVM version with the Node.js version used by devenv.

AI-assisted-by: gpt-5.6-sol

* ⬆️ Update devenv to PostgreSQL 18.6

Run PostgreSQL 18 with its versioned volume layout and a TCP readiness
check that ignores the temporary initialization server.

Install the matching client, create penpot_nexus, and preserve the old
PostgreSQL 16 volume for rollback or logical migration.

AI-assisted-by: gpt-5.6-sol

* 🐳 Expose RustFS ports in devenv

Publish the RustFS S3 API and management console on localhost port 9000
and 9001.

Keep both bindings on loopback so object storage is not exposed to the local
network.

AI-assisted-by: gpt-5.6-sol

* 🐳 Install standalone pnpm in devenv

Install pnpm 12.5.0 from architecture-specific release archives and
verify their published checksums.

Remove the Corepack setup while allowing pnpm to honor the project
packageManager pins.

AI-assisted-by: gpt-5.6-sol

* 🔥 Remove corepack, use system pnpm everywhere

Corepack is gone from Node 25+, so every `corepack enable` call
fails. pnpm now ships as a system binary (devenv, CI runners and
Docker images install it directly) and auto-downloads the version
pinned in `packageManager` on mismatch.

Scripts, workflows and Dockerfiles call `pnpm` straight away; the
three deploy workflows use a single `pnpm/setup@v2` step; and the
new `scripts/sync-pnpm-version` stamps all 35 `packageManager`
fields from the system pnpm, replacing the `corepack use` sweep.

AI-assisted-by: muse-spark-1.3-contributor

* 🐛 Fix exporter watch missing render-wasm build step

The exporter watch compiled CLJS requiring the generated
src/app/wasm/shared.js, which only render-wasm/build export
produces. Without it shadow-cljs failed with a cryptic missing
./shared.js dependency. Run build:wasm before watching, as
the frontend watch:app and exporter scripts/build already do.

AI-assisted-by: muse-spark-1.3-contributor

* 🔧 Add opencode V2 support and adapt plugins

Register the penpot tools for both opencode V1 (server())
and V2 (setup() with JSON Schema inputs) from a single
dependency-free plugin file, sharing the psql and
paren-repair runners between both paths.

Install the opencode2 binary side-by-side with V1 in the
devenv image and document the dual registration in the
paren-repair and psql memories.

AI-assisted-by: muse-spark-1.3-contributor

* ⬆️ Update pnpm and opencode

2026-09-22 10:22:31 +02:00

8.3 KiB

Raw Blame History

Devenv startup and configuration

Compose-based dev environment under docker/devenv/, driven by manage.sh. Parallel instances share infra + Postgres + RustFS; each instance has its own main container, Valkey, source checkout, tmux session.

Compose project layout

penpotdev-infra: shared postgres, rustfs, valkey, mailer, ldap. File: docker-compose.infra.yml.
penpotdev-wsN (N=0,1,…): per-instance main + redis (Valkey). File: docker-compose.main.yml. ws0 (a.k.a. main) binds $PWD; ws1+ bind clones at ${PENPOT_WORKSPACES_DIR}/wsN/ (default ~/.penpot/penpot_workspaces/), maintained by the developer.
Optional overlay docker-compose.opencode.yml: added by instance-compose as an extra -f only when PENPOT_OPENCODE_CONFIG_DIR is set (i.e. run-devenv --opencode-config-dir DIR ran in this process). Bind-mounts the host dir at /home/penpot/.config/opencode (:z). Flag-only, per-call; not read from ambient env. Parser parse-opencode-config-dir absolutizes (~, realpath) because compose resolves relative bind sources against the compose file's dir. Only instances brought up with the flag get the mount.
All projects join external network penpot_shared. Created idempotently by ensure-devenv-network, never removed by lifecycle commands.

Source-of-truth files

docker/devenv/defaults.env: ws0 baseline — container/volume names, runtime env, published host ports, tmux defaults. manage.sh aborts if unreadable.
For ws1+, instance-env-overrides computes the per-instance overrides (container/volume names, host ports offset 10000·N, PENPOT_PUBLIC_URI, PENPOT_REDIS_URI, PENPOT_BACKEND_WORKER=false) and instance-compose injects them as env vars at compose time — never written to disk, recomputed each call so they can't drift. ws0 uses defaults.env as-is.
backend/scripts/_env: backend-internal only — secret keys, PENPOT_FLAGS (with enable-backend-worker gated on PENPOT_BACKEND_WORKER), JAVA_OPTS, setup_s3_bucket(). Never duplicates defaults.env.
Compose files use pure ${VAR} substitution; missing var = compose fails.

Invariants

infra-compose / instance-compose wrap docker compose with env -i, then re-inject what compose needs. Stripping is required because defaults.env is sourced into manage.sh's shell at startup (stale values would leak); the ws1+ overrides are deliberately re-injected as shell env vars precisely because Compose gives shell precedence over --env-file, so they override the defaults.env baseline.
Volume names pinned via name: (PENPOT_*_VOLUME), decoupled from the compose project name. Replacement services use new volumes and leave old volumes untouched. PostgreSQL 18 mounts penpotdev_postgres_data_pg18 at /var/lib/postgresql; the old PG16 volume is not migrated automatically.
Network aliases (- main, - redis) are not declared in main.yml. Compose's auto-service-alias still registers redis on the shared network, so DNS for redis is non-deterministic with multiple instances. Backend uses PENPOT_REDIS_URI=redis://penpot-devenv-wsN-valkey/0 (container_name) instead.
No cross-project depends_on. manage.sh ensure-infra-up uses Compose --wait; PostgreSQL, RustFS, and Mailpit expose healthchecks. PostgreSQL checks TCP so its temporary init server cannot report ready.
JAVA_OPTS in manage.sh is shadowed inside the container by _env. The -e JAVA_OPTS=... flag only matters for processes that don't source _env.

Worker policy

Backend workers run only on ws0. _env gates enable-backend-worker on PENPOT_BACKEND_WORKER; ws1+ inject it as false. Workers are pure fire-and-forget: wrk/submit! inserts a row into the shared Postgres task table and returns; RPC handlers never wait on completion and workers never publish to msgbus. The reason for "ws0 only" is avoiding multi-instance worker races (cron dedup is best-effort across instances, wrk/submit! dedupe is racy across submitters); details in mem:prod-infra/core.

Each workspace is independent and can be started/stopped in any order. Shared infra (Postgres, RustFS, etc.) is shut down only when no instances remain running.

Port layout

Container-internal ports fixed; host side offset 10000·N.

ws0	ws1	wsN	container	role
3449	13449	3449+10000·N	3449	public HTTPS (Caddy; `/mcp/ws` same-origin)
3449/udp	13449/udp	…	3449/udp	HTTP/3
4401	14401	…	4401	MCP HTTP stream
4403	14403	…	4403	MCP REPL
14181	24181	…	14281	Serena MCP
14182	24182	…	24282	Serena dashboard

Everything else (frontend dev, backend API, exporter, storybook, REPLs, plugin dev, MCP inspector/WebSocket) is in-process or same-origin via Caddy/nginx. Infra publishes RustFS S3 API 9000, RustFS console 9001, and Mailpit UI 1080 on loopback; ldap 10389/10636 remains a singleton without offsets. Mailpit stores its SQLite database in penpotdev_mailpit_data.

Tmux + MCP routing

docker/devenv/files/start-tmux.sh is session-level idempotent. Reads PENPOT_TMUX_ATTACH. If the session exists it attaches or exits; otherwise creates 4 base windows (frontend watch / storybook / exporter / backend) plus mcp (when enable-mcp in PENPOT_FLAGS) and serena (when SERENA_ENABLED=true). run-devenv --agentic always sets both env vars. The legacy run-devenv alias doesn't, hence its 4-window-only session. To switch from a legacy session to agentic, stop-devenv then run-devenv --agentic — the conditional windows are only added at session create time.

MCP plugin routing is same-origin: frontend uses <public-uri>/mcp/ws, per-instance nginx proxies to MCP port 4401 in-container. For the plugin↔MCP server wiring (how the browser plugin discovers the URL, the in-memory connection registry, why DB-mediated routing isn't needed), see mem:mcp/core.

Workspace orchestration (ws1+)

Workspace directories are user-maintained at ${PENPOT_WORKSPACES_DIR}/wsN. run-devenv --agentic --ws i syncs only when --sync is passed, with one exception: if the workspace directory is missing on first use, sync runs implicitly to seed it.

sync-workspace wsN:

assert-clean-git-state — refuses on .git/{rebase-apply,rebase-merge,MERGE_HEAD,CHERRY_PICK_HEAD,index.lock}. No --sync-force escape.
rsync -a --delete $PWD/.git/ $workspace/.git/.
git ls-files -z --cached --others --exclude-standard → rsync --files-from (Git is the authority on tracked files; rsync's gitignore filter would drop committed files under gitignored parents like .clj-kondo/config.edn).
Initial-only copy of frontend/resources/public/js/config.js (gitignored, but agentic mode needs it). After the first sync the workspace's copy belongs to the developer — subsequent syncs leave it alone.
git switch -C "wsN/<current-branch>" inside the workspace.

No --delete on the working-tree pass: gitignored caches in the workspace survive. Workspace dir + named volumes survive compose down.

CLI surface

run-devenv --agentic [--ws main|0|wsN|N] [--sync] [--serena-context CTX] [--opencode-config-dir DIR]: bring one instance up. Agentic only — MCP and Serena windows are always created. Default target main. Errors out if the target is already running. --sync is rejected on main; on ws1+ it's optional (forced only when the workspace dir does not exist yet). --opencode-config-dir DIR bind-mounts DIR at ~/.config/opencode in-container via the optional overlay above; mount applies at container creation, so changing it requires stop + re-run.
stop-devenv [--ws main|0|wsN|N] [--all]: stop instances. Flags mutually exclusive. --ws N stops just that workspace. --ws 0 or no flag stops ws0; shared infra shuts down only if no other instances remain. --all stops every ws highest-first then ws0, then infra.
run-devenv: legacy alias, ws0 non-agentic attached.
attach-devenv [--ws main|0|wsN|N]: pure attach. Fails fast if instance/session missing.
run-devenv-shell [--instance 0|wsN|N] [cmd...]: bash in target instance. (--instance flag not yet renamed to --ws.)
start-devenv / log-devenv / drop-devenv: legacy paths around ws0 + shared infra. drop-devenv never removes volumes.

exporter/scripts/run and wait-and-start.sh source backend/scripts/_env then _env.local if present.

8.3 KiB Raw Blame History