145 Commits

Author SHA1 Message Date
Michael Panchenko
16dc83616a
Add the ability to launch parallel devenv instances (#9906)
* 🐳 Split devenv compose for parallel workspaces

Move shared services into an infra compose file and keep the main devenv container plus Valkey in a separate compose file driven by defaults.env. Parameterize host-side ports, container names, source path, and runtime env while keeping container-internal ports fixed for same-origin proxying.

Make tmux startup idempotent, add attach-devenv for the live instance, move shared MinIO user setup to infra startup, and let exporter scripts load backend _env.local overrides.

Co-authored-by: Codex <codex@openai.com>

* 🐳 Run parallel devenv instances against shared infra

Add support for running N parallel devenv instances under separate compose
projects sharing Postgres, MinIO, mailer, and LDAP. Each instance has its
own main container, Valkey, source checkout, tmux session, and host port
range offset by 10000 (3449 -> 13449 -> 23449, etc.).

./manage.sh run-devenv-agentic --n-instances N reconciles the running set
to exactly {ws0..ws(N-1)}: missing instances are created (workspace sync
from the live repo via git ls-files + per-instance env-file generation
under docker/devenv/instances/ + detached tmux startup), surplus instances
are stopped highest-first via compose down (never -v), already-running
instances are left untouched. ws0 binds the live repo at PWD; ws1+ are
scratch clones under ~/.penpot/penpot_workspaces/.

Backend workers (enable-backend-worker) are gated on PENPOT_BACKEND_WORKER
in backend/scripts/_env; ws1+ overlays disable them so async-task
notifications stay bound to a single Valkey Pub/Sub instance.

Compose helpers wrap docker compose with env -i so per-instance overlay
--env-file actually overrides defaults.env -- without the strip, the shell
env from sourcing defaults.env at startup would shadow the overlay (Compose
gives shell precedence over --env-file).

Other:
- Drop network aliases (- main, - redis); use container_name for
  cross-container DNS so multiple instances on the shared network don't
  fight over the same DNS name.
- Pin volume names via name: (PENPOT_*_VOLUME) so volumes survive project
  renames; ws0 keeps the pre-existing physical names (penpotdev_*).
- Remove cross-project depends_on from main.yml (postgres/minio-setup now
  live in penpotdev-infra); manage.sh ensure-infra-up docker-waits on the
  minio-setup one-shot.
- Strict arg parsing in run-devenv / run-devenv-agentic; --n-instances 0
  rejected.
- Remove unused Host-matched server block from the Caddyfile.

Memory mem:devenv/core and developer docs updated.

Co-authored-by: Codex <codex@openai.com>

*  Document and stabilise the parallel-workspace CLI; wire AI agents

Improve parallel-workspaces developer CLI,
and add an opt-in layer that lets four AI
coding agents (Claude Code, opencode, VS Code Copilot, OpenAI Codex CLI)
drive a specific workspace through a single launcher command.

Parallel-workspace semantics
----------------------------

each run-devenv-agentic call brings up one wsN;
--ws N (integer; default 0) targets a specific workspace and auto-starts
ws0 first when N>=1 so the worker invariant holds. --sync is forbidden on
ws0 and re-seeds the workspace from the live repo for ws1+. Stop semantics
mirror the start invariant -- ws0 is the last to stop, shared infra stops
with it, --all walks every instance highest-first. The worker policy
section explains why workers run only on ws0 (Postgres FOR UPDATE
SKIP LOCKED is safe across many workers but the cron dedup primitive is
best-effort, and :telemetry / :audit-log-archive are not idempotent).
Per-instance Valkey Pub/Sub isolation, msgbus topology, and the
"async task notifications miss ws1+ tabs" caveat are stated explicitly.

The mem:prod-infra/core memory captures the same external-services and
task-queue / Pub-Sub topology in agent-readable form, and
mem:backend/core and mem:critical-info now cross-link it so backend work
surfaces the horizontal-scaling constraints from the start.

AI coding agent integration
---------------------------

New top-level .devenv/ directory holds committed templates
(templates/{claude-code,opencode,vscode}.json and templates/codex.toml,
each with \${PENPOT_MCP_PORT} and \${SERENA_MCP_PORT} placeholders) plus
committed shared entries (matching shared/* files for Playwright, the
only workspace-independent server we ship today).

./manage.sh start-coding-agent <claude|opencode|vscode|codex> [--ws N]
launches the chosen client against one workspace. It cd's into the
target's directory (the live repo for ws0; workspace-path "wsN" for ws1+)
and refuses to launch unless (a) the binary is on PATH, (b) the
workspace directory exists for ws1+, and (c) the instance is up
(devenv-main-running) -- the MCP servers only exist while the devenv is
running. The agentic-devenv guide is restructured around this Quick
start path, with a per-client table and a Manual configuration fallback
for clients we don't cover.

Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ♻️ Scope the shadow devtools to the dev build

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-03 15:48:25 +02:00
Andrey Antukh
3858993a57 Merge remote-tracking branch 'origin/staging' into develop 2026-05-27 13:37:02 +02:00
Andrey Antukh
676e7f8646 🔧 Update devenv postgresql config 2026-05-27 11:35:21 +02:00
Dominik Jain
c7a4532838 Add Serena update mechanism for agentic devenv
Add SERENA_UPDATE_VERSION env var (in devenv docker-compose.yml)
to dynamically update Serena on agentic devenv without requiring
an image rebuild.
Apply for update to v1.5.0 (also changing initial installation
in Dockerfile to this version).
2026-05-21 14:20:10 +02:00
Marina López
d26412740a
♻️ Rename control center to admin console (#9705) 2026-05-18 14:33:24 +02:00
Michael Panchenko
c2a1d5c6f7 Add run-devenv-agentic command, starting the Serena MCP server in the container
Serena provides useful tools for the agentic workflow for penpot.
The following additional extensions are added:

1. uv and Serena installation, including a suitable serena_config.yml, are added to the devenv docker image
2. Serena configuration options are set via env vars and flags in manage.sh
3. run-devenv can now take -e flags which it forwards to docker exec

GitHub #9315
2026-05-12 12:49:47 +02:00
Dominik Jain
66d518f15d 🎉 Add MCP tool for ClojureScript expression evaluation
New tool to evaluate ClojureScript expressions by connecting to the
nREPL service already provided in devenv.

Add dependency 'nrepl-client' and a corresponding client class
as well as types to support this.

Add a new environment variable for 'devenv mode', which enables
the new tool (PENPOT_MCP_DEVENV).
2026-05-12 12:36:44 +02:00
Dominik Jain
b7b31f6ee3 Start MCP server in devenv if PENPOT_FLAGS contains 'enable-mcp' 2026-05-12 12:35:38 +02:00
Andrey Antukh
cd882f9ebd 🔧 Add minor changes to devenv config 2026-05-11 12:39:03 +02:00
Francis Santiago
e9588f3939
🐳 Reuse shared Nginx security headers (#9473)
Signed-off-by: Francis Santiago <francis.santiago@kaleidos.net>
2026-05-08 14:11:09 +02:00
Andrey Antukh
c0989d4261 Merge remote-tracking branch 'origin/main' into main-staging 2026-04-28 10:08:51 +02:00
Andrey Antukh
4e1968bbab 📎 Add updated version of github cli to devenv 2026-04-28 00:01:46 +02:00
Andrey Antukh
3a39676969 Backport MCP from staging (part 1) 2026-04-20 19:37:02 +02:00
Andrey Antukh
6db3c6cf89
🐛 Fix regression on subpath support (#8793) 2026-03-26 15:43:30 +01:00
Andrey Antukh
920fbe34ad 🐛 Fix invalid deps passed to http management routes service 2026-02-10 13:46:29 +01:00
Andrey Antukh
5ec345162a Add mcp plugin into the frontend bundle 2026-02-09 17:38:40 +01:00
Andrey Antukh
1656fefdc9 Merge remote-tracking branch 'origin/staging-render' into develop 2026-02-04 16:23:46 +01:00
Andrey Antukh
c14ccc18b8 Import mcp from develop 2026-02-04 16:21:19 +01:00
Andrey Antukh
61d7dd3167 Update devenv with mcp required dependencies 2026-02-04 12:22:36 +01:00
Andrey Antukh
e8fd4698c9 🔧 Update caddy configuration 2026-01-27 15:10:53 +01:00
Pablo Alba
d5abc52dac
🎉 Add first integration with nitrate (#7803)
* 🐛 Display missing selected tokens set info (#8098)

* 🐛 Display missing selected tokens set info

*  Add integration tests to verify current active set

* 🎉 Integration with nitrate platform

* 🐛 Fix nitrate get-teams returns deleted teams

*  Add nitrate to tmux devenv

*  Add retry and validation to nitrate module

*  Add photoUrl to profile on nitrate authenticate

*  Move nitrate url to an env variable

* ♻️ Change Nitrate organization-id schema to text

* ♻️ Cleanup unused imports

* 🔧 Add control-center to nginx

*  Add create org link

* 🔧 Fix nginx entrypoint

* 🐛 Fix control-center proxy pass

* 🎉 Add nitrate licence check

* Revert " Add nitrate to tmux devenv"

This reverts commit dc6f6c458995dac55cab7be365ced0972760a058.

*  Add feature flag check

* 🐛 Rename licences for licenses

*  MR changes

*  MR changes 2

* 📎 Add the ability to have local config on start backend

* 📎 Add FIXME comment

---------

Co-authored-by: Xaviju <xavier.julian@kaleidos.net>
Co-authored-by: Juanfran <juanfran.ag@gmail.com>
Co-authored-by: Yamila Moreno <yamila.moreno@kaleidos.net>
Co-authored-by: Marina López <marina.lopez.yap@gmail.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-01-27 10:04:53 +01:00
Andrey Antukh
6470db8d5f Remove mention of yarn on several files 2026-01-22 13:55:41 +01:00
Andrey Antukh
3ecf509f3b 🔧 Add missing gif files handling on defaut nginx config 2026-01-19 13:32:01 +01:00
Andrey Antukh
b7cd315872 🐛 Fix wasm-playground on devenv 2026-01-08 13:48:09 +01:00
Andrey Antukh
36732a4bd3
Make the devenv runtine initialization yarn independent (#8023) 2026-01-07 11:21:58 +01:00
Andrey Antukh
52b8560b70 Merge branch 'staging-render' into develop 2025-12-30 15:30:56 +01:00
Andrey Antukh
824ca1bbca 🔧 Make devenv init yarn indpendent 2025-12-30 15:28:19 +01:00
Pablo Alba
35fb376a78
Add proxypass to caddyfile on devenv (#7985) 2025-12-22 09:21:22 +01:00
Andrey Antukh
eb1eeb4750 Merge remote-tracking branch 'origin/staging-render' into niwinz-develop-merge 2025-12-10 13:53:15 +01:00
Andrey Antukh
f111cbb2a4 Add better cache config on devenv nginx 2025-12-09 19:38:30 +01:00
Andrey Antukh
d04fdb5fbd
Make the dist bundle use consistent and cache-aware uris (#7911) 2025-12-09 08:05:28 +01:00
Andrey Antukh
eeaf28bb25 📎 Disable caddy logging 2025-12-02 13:27:09 +01:00
Andrey Antukh
6b8091bb90 Make devenv https and http2 capable (#7871)
Making it more similar on how it runs on production
environments and improves large amount of files loading
thanks to http2.
2025-12-02 10:49:37 +01:00
Andrey Antukh
7fe20b65dc 🔧 Add more cache efficient configuration for devenv nginx 2025-11-27 17:59:12 +01:00
Andrey Antukh
3ec001de44
🔧 Add nitrate url to devenv nginx (#7800) 2025-11-21 12:30:49 +01:00
Andrey Antukh
683db071d6 Merge remote-tracking branch 'origin/staging' into develop 2025-10-06 12:09:57 +02:00
Andrey Antukh
b72704e54b Set explicit no-buffering for sse responses 2025-10-06 09:19:48 +02:00
Andrey Antukh
e8434c3370 📎 Update devenv tmux script to start exporter using yarn 2025-10-01 10:59:41 +02:00
Belén Albeza
cb8aae4d5f 🔧 Drop the -R in chown (dockerfile mac) 2025-07-08 15:45:34 +02:00
Andrey Antukh
2d4fc3e05f ♻️ Refactor devenv build mechanism
This introduces multistage build process for devenv making
different dependencies build depend on its own (per example, when
jvm version is changed, only the jvm stage is rebuild)

This commit also introduces imagemagick 7.x custom build
in the same way as we have on public docker images, so on
devenv we use the same version.
2025-06-18 09:46:15 +02:00
Andrey Antukh
74430aad21 📎 Add missing emsd loading to devenv bashrc 2025-06-09 15:57:31 +02:00
Andrey Antukh
9584e1b02d 📎 Add missing playwright chromium install to devenv 2025-06-06 15:10:52 +02:00
Andrey Antukh
1909189ce0 Use different approach for setup cargo home 2025-06-02 22:29:39 +02:00
Andrey Antukh
46f330fef3 Move several logic from init to entrypoint on devenv
For make commands consistent independently if they are executed
inside devenv or from manage.sh
2025-06-02 19:10:48 +02:00
Andrey Antukh
f067c86b02 🔥 Remove unnecesary env vars from bashrc (devenv) 2025-06-02 19:10:20 +02:00
Andrey Antukh
d8913ab18b
Add minor changes to devenv for avoid repeated deps download (#6600)
*  Add minor changes to devenv for avoid repeated dependency download

*  Add minor changes to devenv for integrate payments service

*  Remove playwright deps install from circleci config

*  Move cargo_home to userspace on devenv start

*  Improve cache management on CI

*  Improve cargo installation

*  Add missing playwright install cmd on CI

*  Install cargo-watch on devenv

---------

Co-authored-by: David Barragán Merino <david.barragan@kaleidos.net>
2025-06-01 09:16:28 +02:00
Alejandro Alonso
2aaa2f3033
🐛 Fix template import (#6299) 2025-04-15 10:39:22 +02:00
Andrey Antukh
982118c942 🐳 Update devenv corepack setup 2025-04-01 10:47:49 +02:00
Andrey Antukh
d15f0f1fd0 Improve build process
Remove hardcoded yarn version on devenv
2025-01-31 13:39:49 +01:00
alonso.torres
33e8c8b3c9 🔧 Improved storybook configuration 2025-01-23 12:06:02 +01:00