* fix(sandbox): leave SIGTERM/SIGINT teardown to the Gateway lifespan
AIO and E2B providers built on the event-loop thread installed SIGTERM/SIGINT
handlers that force-destroyed every sandbox synchronously, before uvicorn
started the run drain. The Gateway lifespan now holds sandbox shutdown
ownership from its first step, so providers skip those handlers and the
existing hook closes the provider after run drain and MCP teardown. SIGHUP
keeps the provider fallback. AIO acquire now refuses once shutdown began.
* docs(sandbox): document Gateway-owned sandbox shutdown and AIO admission fence
* test(gateway): inspect the lifespan body now that lifespan wraps it
* docs(changelog): reference #7560 for the sandbox SIGTERM drain-order fix
* ci: retrigger sandbox image smoke after a registry pull timeout
* fix(gateway): rebase scheduler shutdown bound onto latest main
* style(gateway): format scheduled task shutdown
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(deploy): wire the multi-replica prerequisites into the Helm chart and compose
The chart shipped gateway.replicas: 1 with a README citing the long-closed
issue #3948, while its rendered config lacked run_ownership heartbeats and
db run events: raising replicas made every starting Pod write its peers'
lease-less runs off as orphans, and the Gateway's startup gate could not
notice because its worker count only sees one Pod.
Enable heartbeats and db run events in the default config, export
DEER_FLOW_MULTI_INSTANCE from the replica count for the startup gate,
generate and inject a shared AUTH_JWT_SECRET (per-Pod .jwt_secret files
logged users out across replicas), declare a surge-then-drain rollout
strategy, render a PodDisruptionBudget when replicas > 1, and bound
uvicorn's graceful shutdown inside a 90s termination grace period.
docker-compose gets the same uvicorn bound and a 60s stop_grace_period so
the 30s memory flush is no longer SIGKILLed. Docs and comments now list
what actually remains single-instance: IM channels, WeChat QR login and
browser tools.
* fix(deploy): honor existingAppSecret, declare multi-instance explicitly, cover the dev stack's shutdown
Follow-up to the multi-replica prerequisites wiring, addressing review:
- `deer-flow.appSecret` returns `existingAppSecret` when it is set. The
gateway and frontend kept referencing the generated `<release>-app`
Secret, which the chart stopped rendering, so every Pod sat in
CreateContainerConfigError. The value is now declared in values.yaml.
- `gateway.multiInstance` (boolean, default null) overrides the
replica-derived DEER_FLOW_MULTI_INSTANCE, PodDisruptionBudget and
AUTH_JWT_SECRET requirement for deployments scaled with kubectl or an
HPA, which change the replica count without a Helm upgrade.
- AUTH_JWT_SECRET is required, not optional, whenever the gateway is
multi-instance: concurrently booting Pods otherwise race to write
different `.jwt_secret` files on the home volume and sign sessions with
different keys.
- `gateway.uvicornGracefulShutdownSeconds: 0` is honored instead of being
swallowed by `default 10`.
- docker-compose-dev.yaml and dev-entrypoint.sh get the same uvicorn
graceful-shutdown bound and `stop_grace_period: 60s`.
- The chart README documents the one-time sign-out when upgrading a
release that predates AUTH_JWT_SECRET and how to seed the previous key
into the Secret first.
- Tests read the shutdown budget from the Gateway through a shared helper
instead of hard-coding 5/5/30, the raw-template string matches give way
to `helm template` renders, and new renders cover multiInstance,
existingAppSecret, a disabled PDB and a zero graceful-shutdown bound.
* docs(changelog): link the multi-replica deploy entry to #6347
* fix(deploy): budget every bounded teardown hook and flag the multiInstance PDB window
Review follow-ups on #6347:
- The shutdown budget modeled only channel stop + run drain + memory flush
(40s). The lifespan also runs the startup trash sweep, the notification
delivery worker stop, browser-session close and the MCP session-pool
close, each bounded at _SHUTDOWN_HOOK_TIMEOUT_SECONDS, plus a 1s
retrieval-warm wait: about 61s worst case. _gateway_shutdown_budget.py
now reads all of them from the Gateway, a drift guard pins the hook
count against app.py, and both compose files move stop_grace_period
from 60s to 90s (Helm's 90s already had room: 5 + 10 + 61). Comments
and the chart README cite the full arithmetic.
- gateway.multiInstance: true renders the PodDisruptionBudget immediately,
which blocks drains of the single Pod until the scale-up happens; the
README, values comment and both deployment guides say so and name
podDisruptionBudget.enabled: false as the interim escape hatch.
