179 Commits

Author SHA1 Message Date
Michael Panchenko
16dc83616a
Add the ability to launch parallel devenv instances (#9906)
* 🐳 Split devenv compose for parallel workspaces

Move shared services into an infra compose file and keep the main devenv container plus Valkey in a separate compose file driven by defaults.env. Parameterize host-side ports, container names, source path, and runtime env while keeping container-internal ports fixed for same-origin proxying.

Make tmux startup idempotent, add attach-devenv for the live instance, move shared MinIO user setup to infra startup, and let exporter scripts load backend _env.local overrides.

Co-authored-by: Codex <codex@openai.com>

* 🐳 Run parallel devenv instances against shared infra

Add support for running N parallel devenv instances under separate compose
projects sharing Postgres, MinIO, mailer, and LDAP. Each instance has its
own main container, Valkey, source checkout, tmux session, and host port
range offset by 10000 (3449 -> 13449 -> 23449, etc.).

./manage.sh run-devenv-agentic --n-instances N reconciles the running set
to exactly {ws0..ws(N-1)}: missing instances are created (workspace sync
from the live repo via git ls-files + per-instance env-file generation
under docker/devenv/instances/ + detached tmux startup), surplus instances
are stopped highest-first via compose down (never -v), already-running
instances are left untouched. ws0 binds the live repo at PWD; ws1+ are
scratch clones under ~/.penpot/penpot_workspaces/.

Backend workers (enable-backend-worker) are gated on PENPOT_BACKEND_WORKER
in backend/scripts/_env; ws1+ overlays disable them so async-task
notifications stay bound to a single Valkey Pub/Sub instance.

Compose helpers wrap docker compose with env -i so per-instance overlay
--env-file actually overrides defaults.env -- without the strip, the shell
env from sourcing defaults.env at startup would shadow the overlay (Compose
gives shell precedence over --env-file).

Other:
- Drop network aliases (- main, - redis); use container_name for
  cross-container DNS so multiple instances on the shared network don't
  fight over the same DNS name.
- Pin volume names via name: (PENPOT_*_VOLUME) so volumes survive project
  renames; ws0 keeps the pre-existing physical names (penpotdev_*).
- Remove cross-project depends_on from main.yml (postgres/minio-setup now
  live in penpotdev-infra); manage.sh ensure-infra-up docker-waits on the
  minio-setup one-shot.
- Strict arg parsing in run-devenv / run-devenv-agentic; --n-instances 0
  rejected.
- Remove unused Host-matched server block from the Caddyfile.

Memory mem:devenv/core and developer docs updated.

Co-authored-by: Codex <codex@openai.com>

*  Document and stabilise the parallel-workspace CLI; wire AI agents

Improve parallel-workspaces developer CLI,
and add an opt-in layer that lets four AI
coding agents (Claude Code, opencode, VS Code Copilot, OpenAI Codex CLI)
drive a specific workspace through a single launcher command.

Parallel-workspace semantics
----------------------------

each run-devenv-agentic call brings up one wsN;
--ws N (integer; default 0) targets a specific workspace and auto-starts
ws0 first when N>=1 so the worker invariant holds. --sync is forbidden on
ws0 and re-seeds the workspace from the live repo for ws1+. Stop semantics
mirror the start invariant -- ws0 is the last to stop, shared infra stops
with it, --all walks every instance highest-first. The worker policy
section explains why workers run only on ws0 (Postgres FOR UPDATE
SKIP LOCKED is safe across many workers but the cron dedup primitive is
best-effort, and :telemetry / :audit-log-archive are not idempotent).
Per-instance Valkey Pub/Sub isolation, msgbus topology, and the
"async task notifications miss ws1+ tabs" caveat are stated explicitly.

The mem:prod-infra/core memory captures the same external-services and
task-queue / Pub-Sub topology in agent-readable form, and
mem:backend/core and mem:critical-info now cross-link it so backend work
surfaces the horizontal-scaling constraints from the start.

AI coding agent integration
---------------------------

New top-level .devenv/ directory holds committed templates
(templates/{claude-code,opencode,vscode}.json and templates/codex.toml,
each with \${PENPOT_MCP_PORT} and \${SERENA_MCP_PORT} placeholders) plus
committed shared entries (matching shared/* files for Playwright, the
only workspace-independent server we ship today).

./manage.sh start-coding-agent <claude|opencode|vscode|codex> [--ws N]
launches the chosen client against one workspace. It cd's into the
target's directory (the live repo for ws0; workspace-path "wsN" for ws1+)
and refuses to launch unless (a) the binary is on PATH, (b) the
workspace directory exists for ws1+, and (c) the instance is up
(devenv-main-running) -- the MCP servers only exist while the devenv is
running. The agentic-devenv guide is restructured around this Quick
start path, with a per-client table and a Manual configuration fallback
for clients we don't cover.

Co-Authored-By: Codex <codex@openai.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ♻️ Scope the shadow devtools to the dev build

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-03 15:48:25 +02:00
Andrey Antukh
4a8fb5af53 Merge remote-tracking branch 'origin/staging' into develop 2026-06-01 13:15:57 +02:00
Yamila Moreno
ddba2ffa75
📎 Update Kaleidos Copyright (#9929) 2026-05-29 11:24:58 +02:00
Marina López
d26412740a
♻️ Rename control center to admin console (#9705) 2026-05-18 14:33:24 +02:00
Andrey Antukh
cbd5f7795b Add minor compatibility adjustments for audit archive task (#8491) 2026-04-27 16:15:35 +02:00
Yamila Moreno
7031052c4e 🐛 Prevent invitations to blacklisted domains 2026-04-24 16:48:59 +02:00
Andrey Antukh
de27ea904d
Add minor adjustments to the auth events (#9027) 2026-04-16 09:59:45 +02:00
Andrey Antukh
f5996a7235 ♻️ Make several improvements to management API authentication 2026-01-27 15:14:32 +01:00
Pablo Alba
d5abc52dac
🎉 Add first integration with nitrate (#7803)
* 🐛 Display missing selected tokens set info (#8098)

* 🐛 Display missing selected tokens set info

*  Add integration tests to verify current active set

* 🎉 Integration with nitrate platform

* 🐛 Fix nitrate get-teams returns deleted teams

*  Add nitrate to tmux devenv

*  Add retry and validation to nitrate module

*  Add photoUrl to profile on nitrate authenticate

*  Move nitrate url to an env variable

* ♻️ Change Nitrate organization-id schema to text

* ♻️ Cleanup unused imports

* 🔧 Add control-center to nginx

*  Add create org link

* 🔧 Fix nginx entrypoint

* 🐛 Fix control-center proxy pass

* 🎉 Add nitrate licence check

* Revert " Add nitrate to tmux devenv"

This reverts commit dc6f6c458995dac55cab7be365ced0972760a058.

*  Add feature flag check

* 🐛 Rename licences for licenses

*  MR changes

*  MR changes 2

* 📎 Add the ability to have local config on start backend

* 📎 Add FIXME comment

---------

Co-authored-by: Xaviju <xavier.julian@kaleidos.net>
Co-authored-by: Juanfran <juanfran.ag@gmail.com>
Co-authored-by: Yamila Moreno <yamila.moreno@kaleidos.net>
Co-authored-by: Marina López <marina.lopez.yap@gmail.com>
Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2026-01-27 10:04:53 +01:00
Andrey Antukh
fe72d0af82
Add self-signed cert to caddy (#7872) 2025-12-02 10:45:26 +01:00
Andrey Antukh
4fddf3d986 ♻️ Make management key derivable from secret key
Still preserves the ability to set management
2025-11-20 12:20:13 +01:00
Andrey Antukh
363b4e3778
♻️ Make the SSO code more modular (#7575)
* 📎 Disable by default social auth on devenv

* 🎉 Add the ability to import profile picture from SSO provider

* 📎 Add srepl helper for insert custom sso config

* 🎉 Add custom SSO auth flow
2025-11-12 12:49:10 +01:00
Andrey Antukh
42c416e3cb 📎 Add user feedback defaults to backend scripts/_env 2025-10-17 09:39:58 +02:00
Alejandro Alonso
fad148e6a6 📎 Reorder jvm opts on _env 2025-10-15 10:15:06 +02:00
Andrey Antukh
c4cd665594 📎 Enable redis-cache flag on devenv start scripts 2025-10-13 12:32:29 +02:00
Andrey Antukh
5717708b56 ♻️ Refactor file storage
Make it more scallable and make it easily extensible
2025-10-13 12:24:05 +02:00
Andrey Antukh
0aadc3b6b3 Add management shared key authentication 2025-10-13 11:49:58 +02:00
Andrey Antukh
de25a24a6d 🐛 Fix backend repl start issue with jdk 24 2025-09-29 13:35:48 +02:00
Andrey Antukh
9d907071aa
⬆️ Update dependencies (#7330)
* ⬆️ Update to JDK25 on the devenv

* ⬆️ Update dependencies

* 🔥 Remove unused flag from devenv backend startup scripts

*  Enable shenandoah gc on backend scripts/repl
2025-09-26 13:43:43 +02:00
Andrey Antukh
fd62141c04 Disable pointer-map feature (temporary)
Because the upcoming refactor changes several aspects
of that feature and it not make sense to continue have
this active for now, until refactor is merged.
2025-07-30 12:06:41 +02:00
Elena Torro
cf8006ce9c 🔧 Add option to skip tutorial/walkthrough when creating profiles for dev purposes 2025-06-18 17:00:46 +02:00
Andrey Antukh
2d4fc3e05f ♻️ Refactor devenv build mechanism
This introduces multistage build process for devenv making
different dependencies build depend on its own (per example, when
jvm version is changed, only the jvm stage is rebuild)

This commit also introduces imagemagick 7.x custom build
in the same way as we have on public docker images, so on
devenv we use the same version.
2025-06-18 09:46:15 +02:00
Andrey Antukh
6dd0f4f164 🔥 Remove unused jvm options from backend start-dev script 2025-06-18 09:46:15 +02:00
Marina López
e5bc369e56
Visual indicators subscription for teams and project settings (#6546)
*  Visual indicators subscription for teams and project settings

* 📎 Fixes PR feedback

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2025-05-26 12:56:40 +02:00
Andrey Antukh
2df6f2b8b1 ♻️ Refactor prepl interface
Make prepl to be json message based protocol
instead of clojure expression. This facilitates
implementing internal RPC over socket server.
2025-04-28 10:23:02 +02:00
Marina López
f2977cf938
Visual indicators for unlimited tier users (#6270)
*  Visual indicators for unlimited tier users

* ♻️ Refactor to organize properly subscription

* ♻️ Refactor with PR feedback

* 💄 Add minor cosmetic changes

---------

Co-authored-by: Andrey Antukh <niwi@niwi.nz>
2025-04-15 12:47:37 +02:00
Andrey Antukh
a4701866a4 ⬆️ Upgrade devenv (ubuntu, jvm, node) 2025-04-07 16:26:04 +02:00
Andrey Antukh
b913c75c41 🔥 Remove unused parameters from backend run template script 2025-02-04 15:36:22 +01:00
Andrey Antukh
f871f88f30
♻️ Refactor file data migrations subsystem (#5692)
* ♻️ Refactor file data migrations subsystem

* 📎 Add backend scripts/run helper script
2025-01-31 13:37:41 +01:00
Andrey Antukh
754ba304a7
⬆️ Update dependencies (#5694)
* ⬆️ Update system dependencies on devenv

* ⬆️ Update partially frontend dependencies

* ⬆️ Update application dependencies

* ⬆️ Update storybook dependency
2025-01-29 11:21:38 +01:00
Andrey Antukh
e92ddee33a 🐳 Move devenv secret key env asignation to scripts
from the docker compose
2025-01-16 09:59:19 +01:00
Andrey Antukh
88fb5e7ab5 ♻️ Update integrant to latest version
This upgrade also includes complete elimination of use spec
from the backend codebase, completing the long running migration
to fully use malli for validation and decoding.
2024-11-13 19:09:19 +01:00
Andrey Antukh
65f182001b 📎 Update backend scripts/repl with a default config 2024-10-29 11:17:01 +01:00
Andrey Antukh
cc6e071f48 ♻️ Remove all usage of graalvm js runtime
And replace it with a commandline call to nodejs
for execute a custom svgo based command line script.
2024-10-22 23:30:56 +02:00
Andrey Antukh
a1f5bcae80 ♻️ Add better ergonomics for the internal quotes API 2024-10-08 14:51:14 +02:00
Andrey Antukh
9da891e9b0 📎 Enable auto-file-snapshot feature scripts/repl 2024-09-04 12:18:31 +02:00
Andrey Antukh
0e92bcc0de 🎉 Add file-data offload mechanism 2024-08-09 14:28:18 +02:00
Andrey Antukh
5cf54c6384 Improve file snapshoting mechanism 2024-07-29 10:19:34 +02:00
Andrey Antukh
fa00fed694 🐛 Fix issue with v2 manual migration script 2024-04-11 13:29:33 +02:00
Andrey Antukh
7d36cf1b5e Add missing jvm parameter on backend run.sh template 2024-04-10 15:31:49 +02:00
Andrey Antukh
34534c924f Set smaller default deletion delay for devenv
And make the deletion delay configurable
2024-04-10 15:31:49 +02:00
Andrey Antukh
7b7820952c Update docker related files 2024-04-10 15:31:49 +02:00
Andrey Antukh
5924f3bc41 Simplify v2 migration helpers on srepl ns 2024-04-10 15:31:49 +02:00
Andrey Antukh
fd5b1c0341 Enable by default components v2 feature 2024-04-08 11:05:16 +02:00
Andrey Antukh
542b27a779 📎 Add minor changes to compv2 related scripts 2024-04-07 14:07:40 +02:00
Andrey Antukh
67cdaa397c Add minor improvements to devenv initial flags 2024-03-19 11:21:16 +01:00
Andrey Antukh
5573f467b7 📎 Increase devenv limits for multipart requests 2024-02-19 14:28:39 +01:00
Andrey Antukh
a89f16e594 Add better logging config for devenv 2024-02-19 09:20:47 +01:00
Andrey Antukh
02d8208553 📎 Add temporal repl and log4j config 2024-01-25 16:03:29 +01:00
Andrey Antukh
fede8c9975 Setup better media max file-size on devenv 2024-01-10 14:00:31 +01:00