mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
35f6a98cd9
* feat(dev): add a two-Gateway multi-instance harness script There was no way to run the deployment.multi_instance topology locally: scripts/serve.sh pins one Gateway to port 8001, the local nginx config has a single upstream, and getting two processes past the multi-instance startup gate needs Postgres, Redis, a shared DEER_FLOW_HOME, shared auth secrets and a per-process DeerMem retrieval index by hand. scripts/dev_multi_instance.sh (make dev-multi / dev-multi-check / dev-multi-down) starts throwaway Postgres and Redis containers on loopback, renders a config from the developer's config.yaml (or config.example.yaml), and runs Gateway A on 8001 and Gateway B on 8011 with one shared home, one copy of extensions_config.json and generated AUTH_JWT_SECRET, DEER_FLOW_INTERNAL_AUTH_TOKEN and DEER_FLOW_CREDENTIALS_KEY values, plus an optional round-robin nginx on 2027. B starts after A answers /health: Alembic migrations are advisory-locked, but the LangGraph checkpointer/store setup() that follows is not. scripts/dev_multi_instance.py holds the testable parts: the config overlay (neutralizing and reporting settings the gate refuses), the secrets file, the nginx config, and `check`, which automates the scriptable cross-instance acceptance checks: readiness with stream_bridge ok, A-issued sessions and the internal token accepted by B, thread/upload/artifact visibility, a skill toggle seen by B, nginx balancing, and SSE join on B with Last-Event-ID resume on A. Tests pin that the rendered config passes the real _enforce_postgres_for_multi_worker gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): link the two-Gateway harness entries to #6613 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(dev): keep the multi-instance harness inside its own state Address review findings on the two-Gateway harness (#6613): - Containers carry a deerflow.harness.state-dir label; up/down remove only containers of their own state dir and refuse those of another checkout or DEERFLOW_MI_STATE_DIR instead of force-removing them. - Containers are removed with `docker rm -f -v`, so the anonymous volumes the postgres and redis images declare no longer leak on every up/down. - `up` installs an EXIT trap once it starts containers, so any later failure (including B's port being taken after preflight) tears down A and both containers; start_gateway returns failures instead of exiting the script, and the Gateway is disowned so stopping it prints no job notice. - The overlay removes any checkpointer section (a postgres one would write checkpoints and the Store into the developer's database), points an explicit sandbox.ownership and a Redis checkpoint_cache at the harness Redis (keeping ownership timing), drops DeerMem storage_path (both spellings) and the blob_storage root so data lands in the harness home, gives a local AIO sandbox a state-dir-specific container_prefix, and reports a non-DeerMem memory backend it keeps sharing. - nginx paths are rendered as quoted string literals ('$' and control characters are refused), so a state dir with spaces works. - `check` skips the run-based SSE check when model discovery did not succeed, unless --with-llm, and enforces the SSE time budget per raw line so heartbeat-only traffic cannot hold it open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(scripts): keep the harness guidance under the AGENTS.md soft limit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>