15 Commits

Author SHA1 Message Date
阿泽
9c7cd4cad3
feat(sandbox): add thread data mount override for upload sync (#4536) 2026-07-28 23:41:14 +08:00
Zhipeng Zheng
838037188e
feat(channels): share inbound webhook dedupe across pods via Postgres (#4210)
* feat(channels): share inbound webhook dedupe across pods via Postgres (#4120)

* ci: run cross-pod inbound dedupe integration tests in CI

Expose the job Postgres service via DEDUPE_TEST_POSTGRES_URL so the integration tests (issue #4120) actually execute instead of silently skipping. Normalize the URL for asyncpg (postgresql:// -> +asyncpg, drop libpq-only sslmode) and await the now-async _is_duplicate_inbound in test_github_dispatcher.
2026-07-27 23:07:40 +08:00
Matt Van Horn
3949340610
fix: harden Postgres async engine with pool_recycle and command_timeout to stop stale-connection 504s (#4230)
* fix: harden Postgres async engine with pool_recycle and command_timeout to stop stale-connection 504s

* fix(persistence): make postgres command timeout a configurable database setting

Default the app-ORM command timeout to 30s (below nginx's 60s proxy deadline), expose it as database.command_timeout with null to disable, and add regression coverage. Addresses P1 review feedback.

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

* fix(persistence): decouple DB command timeout from nonexistent proxy deadline

The command timeout bounds stalled ORM queries independently; drop the incorrect
coupling to a 60s nginx deadline (actual proxy timeout is 600s).

* feat(config): make pool_recycle configurable

Expose pool_recycle alongside command_timeout and pool_size in config,
example config, and the helm chart, keeping the 300s default, per
review.

---------

Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-21 08:52:31 +08:00
Aari
09e25b8a32
fix(auth): let deployments close local self-registration (#4311)
* fix(auth): let deployments close local self-registration

The OIDC provisioning policy (allowed_email_domains, require_verified_email,
auto_create_users) is enforced only in the SSO callback via
get_or_provision_oidc_user. POST /api/v1/auth/register creates a local account
without consulting any of it, and nothing can turn that path off, so a
deployment declaring an email-domain allowlist can still be joined by any
address through local registration.

Add auth.local.allow_registration (default true, so existing deployments are
unchanged) and gate /register on it before the account is created. Report the
flag from /setup-status so the login page stops offering a signup entry the
Gateway will reject.

/initialize is deliberately not gated: it is the bootstrap path, guarded by
admin_count == 0, and closing it would leave a fresh install unable to create
its first admin.

An unreadable config.yaml falls back to the pre-gate default (open) rather than
making these two endpoints a hard dependency on the file.

* docs(auth): align registration-gate fallback wording with the FileNotFoundError catch
2026-07-20 23:33:09 +08:00
lllyfff
de024646a7
feat(memory): memory template externalization (#4286)
* feat(memory): template externalization — externalized prompts + signal patterns

Externalize two categories of hardcoded content in the DeerMem memory
backend so operators can customise them without touching Python code.

Prompt externalization (plugin 06):
- 4 memory extraction prompts moved from Python string constants to YAML
  files under core/prompts/ (consolidation / fact_extraction /
  memory_update.chat / staleness_review).
- load_prompt(name) and load_prompt_messages(name, variables) loaders.
- memory_update uses the chat format (system / user message split) for
  prompt-caching-friendly system prefixes; other prompts stay text.
- The extraction callback + per-agent prompt directories + Jinja2
  dependency are intentionally not included (minimal surface).
- Compatible with the upstream prompt additions from #4143
  (expected_valid_days, staleFactsToExtend, KEEP/REMOVE/EXTEND in
  staleness review).

Signal-pattern externalization (plugin 07, regex only):
- Correction and reinforcement detection patterns externalised to
  core/message_patterns/{correction,reinforcement}.yaml.
- load_patterns(name, patterns_dir) loader with bundled defaults.
- detect_correction / detect_reinforcement accept a keyword-only
  patterns= parameter; signatures are backward-compatible.
- patterns_dir config field added to DeerMemConfig.
- Importance scorer (importance.py, build_importance_scorer,
  _prepare_update changes) is NOT included — deferred to a later PR.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): fail loudly on invalid yaml in prompt/pattern loaders

Replace silent error handling in load_prompt / load_prompt_messages /
load_patterns so malformed yaml or missing required keys raise ValueError
with the file path rather than a raw YAMLError traceback or silent
empty-string / empty-list return.

Changes:
- load_prompt: YAMLError -> ValueError(path); missing/empty 'template'
  key -> ValueError
- load_prompt_messages: YAMLError -> ValueError(path); missing/empty
  'messages' key -> ValueError; KeyError from .format (placeholder
  mismatch) -> ValueError
- load_patterns: YAMLError -> ValueError (was silent warn+return []).
  OSError still degrades (permission, not format). Not-a-list yaml
  -> ValueError (was silent warn+return []).

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): wire prompts_dir through updater, fail-loud patterns_dir, bump config_version

Addresses PR feedback on the template-externalization PR:

P1 — Wire prompts_dir into the DeerMem update path:
- Add prompts_dir field to DeerMemConfig (default None = bundled defaults).
- Thread it through DeerMem -> MemoryUpdater.__init__(prompts_dir=).
- _build_staleness_section / _build_consolidation_section accept
  prompts_dir= keyword and call load_prompt() instead of relying on
  module-level shim constants.
- _prepare_update_prompt passes prompts_dir to load_prompt_messages
  and to both section builders.
- Add _PROMPT_CACHE to load_prompt so repeated lookups (once per
  memory-update cycle) do not re-read yaml from disk.

P2 — Explicit patterns_dir must find its files:
- When patterns_dir is explicitly set and a YAML file is missing,
  load_patterns() raises FileNotFoundError instead of silently caching
  an empty list. Bundled defaults (patterns_dir=None) still log a
  WARNING and return [] for a missing bundled file (packaging bug).

P3 — Bump config_version and sync Helm:
- config.example.yaml: 26 -> 27
- deploy/helm/deer-flow/values.yaml: 26 -> 27
- deploy/helm/deer-flow/README.md: 26 -> 27
- scripts/check_config_version.sh confirms parity.

Tests: 224 passed (218 memory + 6 config_version).
Lint: ruff check + ruff format --check pass.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): cache load_prompt_messages, validate format fields, fail-loud load_patterns

Review feedback from willem-bd:

Cache for load_prompt_messages:
  Add _CHAT_TEMPLATE_CACHE + _render_messages() helper so the parsed chat
  templates are cached per (name, agent, prompts_dir). On cache hit only
  .format() rendering runs; the yaml file is read once per key.

Validate format field in both loaders:
  load_prompt rejects format='chat' (redirects to load_prompt_messages);
  load_prompt_messages rejects format='text' (redirects to load_prompt).
  Prevents operators from loading a chat yaml via the text loader (or vice
  versa) without a clear error.

Load_patterns fail-loud for explicit directories:
  - OSError (permission, etc.) now raises for explicit patterns_dir
    instead of silently disabling detection.
  - Malformed entries (missing/empty pattern key, wrong type) are skipped
    with a WARNING instead of silent skip.
  - Unknown flag names are warned instead of silently dropped.
  - re.error from compile() raises ValueError with file path and entry
    index instead of a bare re.error traceback.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): thread agent_name through section builders, validate explicit prompts at construction, bump config_version to 28

P1 — Thread agent_name through staleness/consolidation section builders:
  _build_staleness_section and _build_consolidation_section now accept
  agent_name= and pass it to load_prompt(), so per-agent prompt overrides
  work for section templates too (not just memory_update). Previously
  only prompts_dir was threaded; agent_name was ignored for section
  builders.

P1 — Validate explicit prompts at DeerMem construction:
  When DeerMemConfig.prompts_dir is explicitly set, DeerMem.__init__ now
  pre-loads all four prompt templates (staleness_review, consolidation,
  fact_extraction, memory_update with dummy variables) at construction
  time. A missing file, malformed YAML, or invalid placeholder raises
  immediately at startup instead of being caught by the updater's
  generic error handler and silently dropped as a failed update.

P2 — Advance config_version to 28:
  Upstream main already consumed the previous 26→27 bump with a
  different schema change. Bump config.example.yaml, Helm values.yaml,
  and Helm README.md to 28 to version this PR's patterns_dir and
  prompts_dir additions.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): render text templates at construction, propagate PromptConfigurationError

Keep prompt-configuration failures out of the recoverable update catch:

- Define PromptConfigurationError(ValueError) in prompt.py. Raised by
  load_prompt, load_prompt_messages, and _render_messages for bad yaml,
  missing keys, and invalid placeholders.

- _do_update_memory_sync_impl, update_memory's executor path, and
  _process_queue all re-raise (PromptConfigurationError, FileNotFoundError,
  OSError) before the generic except Exception, so a bad explicit prompt
  is never silently returned as False.

- DeerMem.__init__ prompt validation now renders text templates with
  dummy variables (.format(stale_facts="") etc.) so an unknown
  placeholder in staleness_review.yaml, consolidation.yaml, or
  fact_extraction.yaml is caught at construction.

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(memory): drop re-raise guards, soften validation comment, exclude dormant fact_extraction

- Remove (PromptConfigurationError, FileNotFoundError, OSError) re-raise
  guards from _do_update_memory_sync_impl, update_memory executor path,
  and _process_queue. The existing except Exception: logger.exception(...)
  already logs prompt-config errors at ERROR with full traceback; the
  re-raise was killing the entire batch and only surfacing via stderr.
  Per-agent override errors now log at ERROR per-item without aborting
  co-tenant updates.

- Soften the construction validation comment to clarify it only covers
  global templates. Per-agent overrides are validated lazily at first use
  and logged at ERROR by the updater's exception handler.

- Drop fact_extraction from construction validation (dormant prompt with
  no runtime caller; the shim + yaml remain for backward compat).

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-20 07:24:34 +08:00
Hanchen Qiu
d2f8f61e3a
fix(skills): add security_fail_closed option for moderation model outages (#4297)
* fix(skills): add security_fail_closed option for moderation model outages

When the skill security moderation model call fails, scan_skill_content
previously blocked ALL content (executable and non-executable), which
turns a moderation-model outage into a denial of service for skill writes.

Add a skill_evolution.security_fail_closed option (default True, preserving
current behavior). When set to False, non-executable content is allowed with
a warn decision during an outage while executable content is still blocked.

Closes #3021

* fix(config): bump config_version to 27 and format skill_evolution config

Address review feedback on #4297:
- Bump config_version 26 -> 27 so existing installs are flagged outdated
  and pick up skill_evolution.security_fail_closed via make config-upgrade.
- Apply ruff format to skill_evolution_config.py to satisfy the backend
  formatting gate.
- Add config-version/upgrade regression tests covering the v26 outdated
  warning and merging security_fail_closed without changing user values.

* fix(helm): bump chart config_version to 27 to match config.example.yaml

Keeps deploy/helm/deer-flow/values.yaml and its README example in sync
with the config schema bump, satisfying scripts/check_config_version.sh
(validate-chart CI).

* fix(skills): surface fail-open security scan in logs

Address @willem-bd review feedback on #4297:
- Log an operator-visible warning when the moderation model is
  unavailable and fail-open lets non-executable skill content through
  as a warn, so a skipped scan is no longer silent.
- Reword the model-call-failed log so it stays accurate under both
  fail-closed and fail-open policy instead of always claiming a
  "conservative fallback".
- Add a regression test asserting the fail-open warn path emits the
  warning log.
2026-07-19 18:56:03 +08:00
Daoyuan Li
7757e38b7f
fix(nginx): allow long chat prompts through /api/langgraph/ without a raw 500 (#4277)
nginx's default client_max_body_size (1m) and proxy_request_buffering (on)
were never overridden for the chat/LangGraph route, only for the upload
route. A long pasted prompt either exceeded the default size limit (413) or
got spooled to a temp file before reaching Gateway, which can fail with a
raw nginx 500 on a non-root local run if that temp directory isn't
writable -- reproduced independently (real nginx 1.28.3, byte-identical
error page and matching error-log Permission denied line) while confirming
fancyboi999's diagnosis in issue #3952.

Mirrors the upload route's existing client_max_body_size / proxy_request_
buffering fix onto /api/langgraph/ in all three places this nginx config is
maintained (Docker prod, local make dev, and the Kubernetes/Helm ConfigMap),
sized smaller (20M) since this route only ever carries JSON chat text, never
binary file uploads.
2026-07-18 17:48:42 +08:00
Zheng Feng
d57f695769
fix(helm): default sandbox Services to ClusterIP (#3929) (#4190)
* fix(helm): default sandbox Services to ClusterIP (#3929)

The K8s sandbox provisioner supports both NodePort and ClusterIP via
SANDBOX_SERVICE_TYPE (added in #4016), but the Helm chart never set it,
so real-cluster installs inherited the NodePort default. That bound the
code-execution sandbox on every node's interfaces - including externally
reachable ones on GKE/EKS/AKS - and pinned every sandbox URL to one node
IP (SPOF on node reboot/drain/ephemeral-IP).

Default the chart to ClusterIP: the provisioner returns a cluster-DNS URL
(http://sandbox-<id>-svc.<ns>.svc.cluster.local:8080) so the gateway->
sandbox hop stays inside the cluster network - no node IP, no 30xxx port,
no external exposure. The chart always runs the gateway in-cluster, so
ClusterIP is always correct there.

NodePort remains an opt-in (provisioner.sandboxServiceType: NodePort +
nodeHost) for the Docker-Compose/hybrid path where the gateway is not in
K8s and cannot resolve .svc.cluster.local; the provisioner code default
stays NodePort for that path.

- values.yaml: add provisioner.sandboxServiceType ("ClusterIP")
- provisioner-deployment.yaml: emit SANDBOX_SERVICE_TYPE; gate the
  NODE_HOST block on NodePort mode (default "ClusterIP" for upgrade safety)
- NOTES.txt + README.md: document ClusterIP default + NodePort opt-in

No change to docker/provisioner/app.py (already mode-aware since #4016)
or RBAC (services verbs already cover ClusterIP).

* test(helm): assert sandbox Service-type gating + CHANGELOG the default flip (#3929)

Address review on #4190:

- Add scripts/check_chart_sandbox_service.sh: renders the chart for the
  default (ClusterIP, no NODE_HOST), the NodePort opt-in (both emitted),
  and NodePort+nodeHost (literal value, not downward API). Locks in the
  #3929 gating so a regression (e.g. re-adding an unconditional NODE_HOST,
  or dropping the `default "ClusterIP"` upgrade-safety fallback) fails CI.
  Wired into .github/workflows/chart.yaml validate-chart job. (#2)
- CHANGELOG [Unreleased] -> Changed: note the NodePort->ClusterIP default
  flip on upgrade + the `sandboxServiceType: NodePort` opt-back-in. (#4)

No chart template changes (the gating itself landed in the first commit).

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-16 14:30:05 +08:00
Tu Naichao
959bf13406
fix(memory): flush memory queue on graceful shutdown to prevent loss (#4181)
* fix(memory): bounded shutdown flush via MemoryManager.shutdown_flush

Re-applies the memory-queue shutdown drain on top of the pluggable
MemoryManager abstraction (#4122): the old top-level MemoryUpdateQueue
singleton is gone, so the drain is now a backend contract instead of
host code reaching into the queue.

- MemoryManager ABC: shutdown_flush(timeout) -> bool. Every backend
  implements a bounded graceful-shutdown drain.
- DeerMem: queue.flush_sync (daemon-thread + Event.wait hard timeout
  for the uninterruptible sync LLM call; joins an in-flight worker
  first so contexts a debounce Timer already pulled out are not lost on
  exit; skips inter-item sleep on the drain path; per-item
  succeeded/failed count), exposed via shutdown_flush.
- noop: shutdown_flush is a clean no-op success.
- Gateway lifespan: call get_memory_manager().shutdown_flush(timeout)
  after channels/scheduler stop, via asyncio.to_thread, try/except
  bounded. No host-level pending/processing guard -- the backend
  short-circuits on an idle buffer, so the host cannot "forget" the
  in-flight case (structurally eliminates the guard race flagged on the
  prior revision).
- shutdown_flush_timeout_seconds added to the shared MemoryConfig
  (host-owned lifecycle budget, default 30, 1-300) + exposed on
  MemoryConfigResponse and the embedded client; config_version 25 -> 26.

Tests: queue flush_sync (7), lifespan drain incl. False-branch caplog
assertion + disabled gate (3), ABC contract noop/deermem (3).

* fix(chart): gateway grace period so memory drain is not SIGKILLed

K8s defaults terminationGracePeriodSeconds to 30s, shorter than the
Gateway's graceful-shutdown work (channel stop ~5s + memory queue drain
default 30s). Without an explicit grace period, K8s SIGKILLs the memory
drain mid-flight and silently re-introduces the loss shutdown_flush is
fixing (flagged on the prior revision).

- gateway pod: terminationGracePeriodSeconds (default 45, configurable).
- gateway container: preStop sleep (default 5, 0 disables) so the
  Service/ingress deregisters the pod before SIGTERM begins the drain.
- values.yaml + README: both configurable; README documents that the
  grace period must track memory.shutdown_flush_timeout_seconds.

* docs(memory): document shutdown_flush_timeout_seconds + lifespan drain

Add the host-shared field to the memory config list and Config Schema
summary in backend/AGENTS.md, noting the lifespan drain and the K8s
grace-period relationship.

* fix(chart): bump embedded config_version to 26

The chart's embedded `config:` block (values.yaml + README example) still
had config_version: 25 after commit f3ca8e9f raised config.example.yaml to
26, failing the validate-chart config_version drift check. Bump both to 26.
2026-07-15 20:25:41 +08:00
hataa
1300c6d36b
feat(authz): add pluggable AuthorizationProvider protocol and config scaffolding (Phase 0, #4063) (#4127)
Phase 0 of the RFC in #4063: scaffolding only, zero behavior change.

New deerflow/authz/ package (sibling to deerflow/guardrails/):
- AuthorizationProvider Protocol: authorize() + aauthorize() + filter_resources()
- Principal/AuthzRequest/AuthzDecision/AuthzReason dataclasses
- GuardrailAuthorizationAdapter: bridges AuthorizationProvider → GuardrailProvider
  so existing GuardrailMiddleware can enforce authz decisions without a new middleware

New authorization config section (default enabled: false):
- AuthorizationConfig wired into AppConfig alongside guardrails
- Singleton load/reset mirrors GuardrailsConfig pattern
- config.example.yaml documents the RBAC provider schema

29 tests covering protocol conformance, dataclass construction, adapter
request/decision mapping, and config singleton behavior.

Per RFC #4063 Phase 0 (foundations). Layer 1/2 wiring and Principal builder
in services.py deferred to Phase 1.
2026-07-15 10:03:33 +08:00
Zheng Feng
f3e4e5f8f0
ci(helm): publish chart to charts/ namespace prefix on GHCR (#4175)
* ci(helm): publish chart to charts/ namespace prefix on GHCR

Push the Helm chart to ghcr.io/<owner>/charts/deer-flow (via a `charts/`
prefix on the `helm push` target) instead of the bare `deer-flow`
package. This namespaces the chart apart from the image packages
(deer-flow-{backend,frontend,provisioner}) without renaming the chart:
Chart.yaml `name` stays `deer-flow`, so dir = chart name = in-cluster
resource/selector names, and no `nameOverride` hack is needed.

The chart is new in 2.1.0 (chart infra landed in #3987, after v2.0.0),
so 2.1.0 is the first chart release. Early nightly builds remain at the
legacy non-prefixed ghcr.io/<owner>/deer-flow.

Also refresh chart release docs:
- Replace the removed scripts/build-and-push.sh in the chart README with
  raw `docker build`/`push` commands (contexts/args match container.yaml).
- Point NOTES.txt's empty-registry warning at the README section instead
  of the removed script.
- Retarget RELEASING.md version examples from 2.2.0 to 2.1.0.
- Bump the chart README's helm prerequisite to 3+.

* docs(helm): require helm 3.8+ and note legacy chart package cleanup

Address PR #4175 review:
- bump documented minimum to helm 3.8 (OCI registry support stabilized
  there; earlier 3.x needs HELM_EXPERIMENTAL_OCI=1)
- add a post-release note to delete/revoke the legacy bare
  ghcr.io/<owner>/deer-flow chart package after 2.1.0
2026-07-15 08:54:36 +08:00
DanielWalnut
4e209827f3
feat(agent): Add subagent total delegation cap (#4115)
* fix subagent total delegation cap

* fix embedded subagent run cap context

* fix subagent cap config consistency

* fix resumed subagent run cap boundary

* fix legacy resume subagent boundary

* address subagent cap review feedback

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-13 19:26:07 +08:00
Vanzeren
c2002d9fac
feat(memory): add memory tool sets (#4023)
* feat: add memory-as-tool mode alongside existing middleware mode

- Add memory.mode config field (middleware|tool, default middleware)
- Add search_memory_facts() for case-insensitive fact lookup
- Add 4 memory tools: memory_search, memory_add, memory_update, memory_delete
- Wire mode gating in factory.py and lead_agent/agent.py
- 256 memory tests passing, zero regressions

* fix: harden tool-mode memory scoping and docs

* fix: address memory tool mode review feedback

* fix(memory): address tool mode review feedback

* fix: update config_version to 22 in values.yaml
2026-07-11 15:28:16 +08:00
Vanzeren
dc38d2d003
perf(boxlite): add benchmark-driven warm-pool reclaim tuning (#4001)
* bench: add provider-agnostic sandbox benchmark with BoxLite warm pool results

- scripts/bench_sandbox_provider.py: CLI for measuring acquire/run/release
  across providers, scenarios, workloads, and concurrency levels
- scripts/summarize_bench.py: JSONL aggregation with p50/p95/p99 tables
- bench_results.jsonl: 110 turns across 7 scenarios on real BoxLite 0.9.7

Key findings:
  cold acquire:  ~860ms
  warm reclaim:   ~14ms  (60x speedup)
  release:         ~0ms
  warm_hit_rate:  95%   (warm_same_thread)

* perf(boxlite): skip health check for recently-released warm pool boxes

Boxes released within health_check_skip_seconds (default 5.0s)
are promoted directly without the ~14ms echo-ok round-trip.
A VM alive seconds ago is overwhelmingly likely to still be alive.

Add sandbox.health_check_skip_seconds config option.
Set to 0 to always health-check (old behaviour).

Benchmark (warm_same_thread, noop, 20 iters):
  acquire p50: 14.9ms → 0.0ms
  total p50:   29.9ms → 14.0ms

* chore: move benchmark scripts into backend/scripts/benchmark/

* fix: address BoxLite benchmark review findings

* fix(boxlite): only skip warm reclaim checks for released boxes

* fix(benchmark): keep BoxLite shim workaround off the event loop

* fix(boxlite): invalidate dead boxes from command path

* test(boxlite): cover skip window and invalidation edge cases

* fix(boxlite): treat sandbox-has-been-closed as terminal in _exec

* fix(boxlite): harden warm-pool reclaim and benchmark accounting

* fix(boxlite): validate warm-pool reclaims by default

* fix(config): expose boxlite health-check skip setting

* fix(boxlite): tighten failure classification and benchmark workaround

* Update config_version to 21 in values.yaml

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-07-09 17:21:25 +08:00
Zheng Feng
bc9ee9645c
feat(deploy): first-class Helm chart for Kubernetes deployment (#3987)
* feat(helm): add production-ready Helm chart for Kubernetes deployment

Adds deploy/helm/deer-flow, a native-Kubernetes translation of the
production docker-compose stack, plus CI to publish its images and chart.

* ci(release): gate releases on version-source consistency

Add a reusable verify-versions workflow invoked by both chart.yaml and
container.yaml on v* tags. It runs scripts/verify_versions.sh against the
tag and fails the release — skipping all image and chart publishing — when
Chart.yaml (version + appVersion), backend/pyproject.toml, or
frontend/package.json don't all match the tag.

Add scripts/verify_versions.sh (the check, also runnable locally) and
scripts/bump_version.sh (bumps all four sources in lockstep, then
self-verifies). Document the release flow in RELEASING.md and link it from
AGENTS.md.

* fix(deploy): address Helm chart review feedback (#3987)

Three review items from willem-bd:

1. nginx IPv6 listen strip never matched. The sed pattern required a `;`
   immediately after `2026`, but the rendered config emits
   `listen [::]:2026 default_server;` (space + `default_server` before the
   `;`), so the line was never deleted and nginx crash-looped on pods
   without IPv6 (`socket() :::2026 failed (97: Address family not
   supported)`). Drop the trailing `;` from the pattern so it matches.
   Same latent bug fixed in docker-compose-dev.yaml.

2. Passwords were spliced into DSNs verbatim, so a password containing
   URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and
   a confusing parse error. Add a `deer-flow.urlEscape` helper
   (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral
   treats the replacement as a regex template so `[`/`]`/`?` break it) and
   apply it to the password in the postgres and redis DSNs. The raw
   `postgres-password` / `redis-password` keys stay unencoded - they back
   POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment.

3. NODE_HOST defaulted to "gateway", which can never route: the gateway
   Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a
   user who skips the caveat gets unreachable sandboxes with no error at
   install time. Default NODE_HOST to the provisioner pod's node IP via
   the downward API (status.hostIP) - a NodePort is exposed on every node,
   so <node-IP>:<NodePort> routes from the gateway on most clusters.
   `provisioner.nodeHost` remains an override for CNIs/policies that block
   pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart
   README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL
   removes NODE_HOST and the NodePort exposure entirely.)

Validated with helm lint, helm template (incl. a special-char password
rendering the encoded DSNs), and a sed pattern-match check.

* fix(deploy): address round-2 Helm chart review feedback (#3987)

Three "Medium" items from willem-bd:

1. No helm lint / helm template gate before publish. A template regression
   ships as an immutable OCI artifact (GHCR won't overwrite --version), so
   gate packaging on `helm lint` + `helm template --include-crds` in
   chart.yaml before `helm package`. (ct lint / helm-unittest deferred.)

2. Action pinning inconsistent + PR body overstates it. SHA-pin
   actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance
   (v2.4.0, e8998f94) across the publishing workflows (chart.yaml,
   container.yaml, verify-versions.yml), matching the existing docker/*
   SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the
   "SHA-pinned actions" claim accurate. Other pre-existing workflows left
   untouched (out of scope for this PR).

3. Provisioner RBAC broader than needed. Dropped the unused update/patch
   verbs and the pods/exec + events rules from the provisioner Role -
   audited against docker/provisioner/app.py, which only calls
   get/create/delete on pods and get/list/create/delete on services. Fixed
   NOTES.txt to accurately describe the grant instead of understating it as
   "create Pods and Services". The remaining scope concern - verbs apply to
   all Pods in the namespace, not just sandbox Pods - is still deferred
   (RBAC can't scope by label; needs a dedicated namespace or admission
   control), now noted in NOTES.txt and README.

Validated with helm lint + helm template (narrowed Role renders with
exactly get/list/watch/create/delete).

* feat(helm): enable sandbox+web tools out of the box

The chart's default config loaded zero agent tools (config.tools empty ->
"Total tools loaded: 0"), so a fresh install gave an agent that could do
nothing useful. Add tool_groups + tools to the default config block:

- web: web_search (ddg), web_fetch (jina), image_search - no API key
- file:read: ls, read_file, glob, grep
- file:write: write_file, str_replace
- bash

The file/bash tools run inside the AIO sandbox the chart already
configures; the web tools need outbound internet from the gateway pod
(swap backends or drop entries for air-gapped clusters - see
config.example.yaml).

Also bump config_version 15 -> 19 to match config.example.yaml (the chart
had drifted behind). NOTES.txt and the README example updated to match.

* ci(helm): add chart validation + config_version drift check on PR

Extend the chart workflow with a PR-triggered validate-chart job that runs
helm lint, helm template --include-crds, and a config_version drift check:
it parses config_version from both config.example.yaml and the chart's
values.yaml and fails the build (with a ::error:: naming the files to bump)
if the chart is behind the example. This catches the kind of drift this
PR is fixing - the chart sat at v15 while the example moved to v19 - before
it can merge again.

verify-versions and publish-chart stay tag-only; publish-chart now
needs: [verify-versions, validate-chart]. validate-chart runs on both
PRs and tag pushes: the tag arm is required because a job that `needs`
a skipped job is itself skipped under the default success() check, so
validate-chart must actually run on tag pushes or publish-chart would
never fire.

* Bump config version to 20
2026-07-09 15:40:53 +08:00