fix(config): upgrade the config.yaml the Gateway loads (#5991)

* fix(config): upgrade the config.yaml the Gateway loads

scripts/config-upgrade.sh resolved DEER_FLOW_CONFIG_PATH, then
backend/config.yaml, then <checkout>/config.yaml. make dev exports
DEER_FLOW_PROJECT_ROOT=<checkout>, so the Gateway reads the checkout copy
first: with both files present the script upgraded a file nothing loads
and left the live config outdated while reporting success. It also
ignored DEER_FLOW_PROJECT_ROOT and a DEER_FLOW_CONFIG_PATH set in .env,
and fell back to another file when DEER_FLOW_CONFIG_PATH named a
missing one.

Ask the harness resolver (AppConfig.resolve_config_path) instead of
copying its order, defaulting DEER_FLOW_PROJECT_ROOT to the checkout as
serve.sh does. A missing DEER_FLOW_CONFIG_PATH or invalid project root
now fails with the Gateway's error; no config anywhere still creates
<checkout>/config.yaml from the example.

* docs(changelog): reference #5991

* test(config): pin config-upgrade seeding a missing config from the example

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

CHANGELOG.md

@@ -327,6 +327,18 @@ This release closes that milestone with **181 merged pull requests**.
 
 ### Fixed
 
+- **config:** `make config-upgrade` (also run by `make dev` / `make start`)
+  upgrades the `config.yaml` the Gateway loads. With both
+  `<checkout>/config.yaml` and `backend/config.yaml` present, the script
+  upgraded the `backend/` copy while the Gateway read the checkout copy, so the
+  file in use stayed outdated even though the upgrade reported success. It also
+  ignored `DEER_FLOW_PROJECT_ROOT` and a `DEER_FLOW_CONFIG_PATH` set in `.env`,
+  and fell back to another file when `DEER_FLOW_CONFIG_PATH` named a missing
+  one. The script now asks the harness resolver
+  (`AppConfig.resolve_config_path`) for the file, and a missing
+  `DEER_FLOW_CONFIG_PATH` or invalid `DEER_FLOW_PROJECT_ROOT` fails with the
+  Gateway's error instead of upgrading a fallback. ([#5991])
+
 - **sandbox:** Unwrap `Overwrite`-wrapped sandbox state in
   `ToolOutputBudgetMiddleware` and `ReadBeforeWriteMiddleware`. In delta
   checkpoint mode, forked or restored threads deliver the `sandbox` channel
@@ -6233,5 +6245,6 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
 [#5981]: https://github.com/bytedance/deer-flow/pull/5981
 [#5982]: https://github.com/bytedance/deer-flow/pull/5982
 [#5987]: https://github.com/bytedance/deer-flow/pull/5987
+[#5991]: https://github.com/bytedance/deer-flow/pull/5991
 [#6015]: https://github.com/bytedance/deer-flow/pull/6015
 

CHANGELOG_zh.md

@@ -332,6 +332,17 @@
 
 ### 修复
 
+- **配置：** `make config-upgrade`（`make dev` / `make start` 也会执行）现在升级的是
+  Gateway 实际加载的 `config.yaml`。当 `<checkout>/config.yaml` 与
+  `backend/config.yaml` 同时存在时，脚本升级的是 `backend/` 下的副本，而 Gateway
+  读取的是 checkout 根目录的副本，因此实际使用的文件仍停留在旧版本，升级却显示
+  成功。脚本还会忽略 `DEER_FLOW_PROJECT_ROOT` 以及 `.env` 中设置的
+  `DEER_FLOW_CONFIG_PATH`，并在 `DEER_FLOW_CONFIG_PATH` 指向不存在的文件时退回到
+  其他文件。现在脚本通过 harness 的解析器
+  （`AppConfig.resolve_config_path`）确定文件；`DEER_FLOW_CONFIG_PATH` 不存在或
+  `DEER_FLOW_PROJECT_ROOT` 无效时，会以 Gateway 相同的错误失败，而不是升级
+  回退文件。([#5991])
+
 - **沙箱：** 在中间件（`ToolOutputBudgetMiddleware` 与 `ReadBeforeWriteMiddleware`）
   中解包 `Overwrite` 包装的沙箱状态。在 delta checkpoint 模式下，分叉或回滚
   的对话交付的 `sandbox` 通道状态会被 LangGraph 的 `Overwrite` 包装。此前直接判断
@@ -5287,4 +5298,5 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写，核心包含子
 [#5981]: https://github.com/bytedance/deer-flow/pull/5981
 [#5982]: https://github.com/bytedance/deer-flow/pull/5982
 [#5987]: https://github.com/bytedance/deer-flow/pull/5987
+[#5991]: https://github.com/bytedance/deer-flow/pull/5991
 [#6015]: https://github.com/bytedance/deer-flow/pull/6015

backend/docs/CONFIGURATION.md

@@ -109,7 +109,7 @@ Run `make config-upgrade` to merge new fields into your config.
 ```
 
 - **Missing `config_version`** in your config is treated as version 0.
-- Run `make config-upgrade` to auto-merge missing fields (your existing values are preserved, a `.bak` backup is created).
+- Run `make config-upgrade` to auto-merge missing fields (your existing values are preserved, a `.bak` backup is created). It upgrades the file the Gateway loads, resolved as in [Configuration Priority](#configuration-priority).
 - When changing the config schema, bump `config_version` in `config.example.yaml`.
 
 ## Configuration Sections

backend/packages/harness/deerflow/config/AGENTS.md

@@ -57,6 +57,8 @@ Configuration priority:
 3. `config.yaml` under `DEER_FLOW_PROJECT_ROOT`, or the current directory when it is unset
 4. Legacy `backend/config.yaml`, then repository-root `config.yaml` (project root is the **recommended location**)
 
+`scripts/config-upgrade.sh` calls `AppConfig.resolve_config_path` rather than copying this order.
+The legacy locations are anchored to the installed harness source, not to the caller's checkout.
 `scripts/doctor.py` calls `AppConfig.resolve_config_path` rather than copying this order.
 
 Config values starting with `$` are resolved as environment variables (e.g., `$OPENAI_API_KEY`).

backend/tests/test_config_version.py

@@ -11,7 +11,7 @@ import pytest
 import yaml
 from support.shell import find_script_bash
 
-from deerflow.config.app_config import AppConfig
+from deerflow.config.app_config import AppConfig, _legacy_config_candidates
 from deerflow.config.knowledge_base_config import KnowledgeBaseConfig
 from deerflow.tools.tools import get_available_tools
 
@@ -625,3 +625,154 @@ def test_version_46_pii_disabled_config_upgrade_skips_token_secret(tmp_path):
     assert upgraded["config_version"] == expected_version
     assert "token_secret" not in upgraded["pii_redaction"]
     AppConfig.model_validate(upgraded)
+
+
+def _run_config_upgrade_in_checkout(checkout: Path, **env_overrides: str):
+    """Run a copy of scripts/config-upgrade.sh from a throwaway checkout.
+
+    The script resolves the checkout from its own location, so the copy sees
+    ``checkout`` as the repository root while ``UV_PROJECT`` keeps its
+    ``uv run`` on the real backend environment. Resolution overrides are
+    cleared so the developer's shell cannot pick the file under test.
+
+    The harness's legacy fallback is anchored to its install location, which
+    here is the real repository, so a run may only reach it when no legacy
+    candidate exists: otherwise the script would upgrade the developer's own
+    ``config.yaml``.
+    """
+    import shutil
+    import subprocess
+
+    project_root = Path(env_overrides.get("DEER_FLOW_PROJECT_ROOT", checkout))
+    if "DEER_FLOW_CONFIG_PATH" not in env_overrides and project_root.is_dir() and not (project_root / "config.yaml").is_file():
+        assert not any(path.exists() for path in _legacy_config_candidates()), "would fall back to the real repository's config.yaml"
+
+    repo_root = Path(__file__).resolve().parents[2]
+    (checkout / "scripts").mkdir(parents=True, exist_ok=True)
+    (checkout / "backend").mkdir(exist_ok=True)
+    shutil.copy2(repo_root / "scripts" / "config-upgrade.sh", checkout / "scripts" / "config-upgrade.sh")
+    shutil.copy2(repo_root / "config.example.yaml", checkout / "config.example.yaml")
+
+    env = {key: value for key, value in os.environ.items() if key not in {"DEER_FLOW_CONFIG_PATH", "DEER_FLOW_PROJECT_ROOT"}}
+    env["UV_PROJECT"] = str(repo_root / "backend")
+    env.update(env_overrides)
+    return subprocess.run(
+        [SCRIPT_BASH, str(checkout / "scripts" / "config-upgrade.sh")],
+        env=env,
+        capture_output=True,
+        text=True,
+        timeout=120,
+    )
+
+
+def _write_outdated_config(path: Path) -> str:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    text = yaml.dump({"config_version": 1, "sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}})
+    path.write_text(text, encoding="utf-8")
+    return text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_targets_checkout_config_over_legacy_backend_copy(tmp_path):
+    """With both copies present, upgrade the checkout config.yaml that `make dev` loads."""
+    checkout = tmp_path / "checkout"
+    _write_outdated_config(checkout / "config.yaml")
+    legacy_text = _write_outdated_config(checkout / "backend" / "config.yaml")
+
+    result = _run_config_upgrade_in_checkout(checkout)
+    assert result.returncode == 0, result.stdout + result.stderr
+
+    expected_version = yaml.safe_load((checkout / "config.example.yaml").read_text(encoding="utf-8"))["config_version"]
+    upgraded = yaml.safe_load((checkout / "config.yaml").read_text(encoding="utf-8"))
+    assert upgraded["config_version"] == expected_version
+    assert (checkout / "backend" / "config.yaml").read_text(encoding="utf-8") == legacy_text
+    assert not (checkout / "backend" / "config.yaml.bak").exists()
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_honors_deer_flow_project_root(tmp_path):
+    """An exported DEER_FLOW_PROJECT_ROOT decides the file, as it does for the Gateway."""
+    checkout = tmp_path / "checkout"
+    project_root = tmp_path / "project"
+    checkout_text = _write_outdated_config(checkout / "config.yaml")
+    _write_outdated_config(project_root / "config.yaml")
+
+    result = _run_config_upgrade_in_checkout(checkout, DEER_FLOW_PROJECT_ROOT=str(project_root))
+    assert result.returncode == 0, result.stdout + result.stderr
+
+    upgraded = yaml.safe_load((project_root / "config.yaml").read_text(encoding="utf-8"))
+    assert upgraded["config_version"] > 1
+    assert (checkout / "config.yaml").read_text(encoding="utf-8") == checkout_text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_fails_on_missing_deer_flow_config_path(tmp_path):
+    """A missing explicit config stops the Gateway, so no fallback file is upgraded instead."""
+    checkout = tmp_path / "checkout"
+    checkout_text = _write_outdated_config(checkout / "config.yaml")
+    legacy_text = _write_outdated_config(checkout / "backend" / "config.yaml")
+    missing = tmp_path / "missing" / "config.yaml"
+
+    result = _run_config_upgrade_in_checkout(checkout, DEER_FLOW_CONFIG_PATH=str(missing))
+    assert result.returncode != 0
+    assert "DEER_FLOW_CONFIG_PATH" in result.stderr
+    assert (checkout / "config.yaml").read_text(encoding="utf-8") == checkout_text
+    assert (checkout / "backend" / "config.yaml").read_text(encoding="utf-8") == legacy_text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_reports_invalid_deer_flow_project_root(tmp_path):
+    """A project root the Gateway rejects fails with its message, not a traceback."""
+    checkout = tmp_path / "checkout"
+    checkout_text = _write_outdated_config(checkout / "config.yaml")
+
+    result = _run_config_upgrade_in_checkout(checkout, DEER_FLOW_PROJECT_ROOT=str(tmp_path / "missing"))
+    assert result.returncode != 0
+    assert "DEER_FLOW_PROJECT_ROOT is set to" in result.stderr
+    assert "Traceback" not in result.stderr
+    assert (checkout / "config.yaml").read_text(encoding="utf-8") == checkout_text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_honors_deer_flow_config_path_from_dotenv(tmp_path):
+    """`make config-upgrade` does not source .env, but the Gateway loads it; follow the Gateway."""
+    checkout = tmp_path / "checkout"
+    live = tmp_path / "live" / "config.yaml"
+    checkout_text = _write_outdated_config(checkout / "config.yaml")
+    _write_outdated_config(live)
+    (checkout / ".env").write_text(f"DEER_FLOW_CONFIG_PATH={live}\n", encoding="utf-8")
+
+    result = _run_config_upgrade_in_checkout(checkout)
+    assert result.returncode == 0, result.stdout + result.stderr
+
+    assert yaml.safe_load(live.read_text(encoding="utf-8"))["config_version"] > 1
+    assert (checkout / "config.yaml").read_text(encoding="utf-8") == checkout_text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_resolves_relative_deer_flow_config_path_from_backend(tmp_path):
+    """The Gateway runs from backend/, so a relative DEER_FLOW_CONFIG_PATH is relative to it."""
+    checkout = tmp_path / "checkout"
+    checkout_text = _write_outdated_config(checkout / "config.yaml")
+    _write_outdated_config(checkout / "backend" / "custom.yaml")
+
+    result = _run_config_upgrade_in_checkout(checkout, DEER_FLOW_CONFIG_PATH="custom.yaml")
+    assert result.returncode == 0, result.stdout + result.stderr
+
+    upgraded = yaml.safe_load((checkout / "backend" / "custom.yaml").read_text(encoding="utf-8"))
+    assert upgraded["config_version"] > 1
+    assert (checkout / "config.yaml").read_text(encoding="utf-8") == checkout_text
+
+
+@pytest.mark.skipif(SCRIPT_BASH is None, reason="repo shell-script tests need Git Bash on Windows")
+def test_config_upgrade_creates_checkout_config_when_none_exists(tmp_path):
+    """With no config anywhere, the resolver miss is not an error: the example seeds <checkout>/config.yaml."""
+    if any(path.exists() for path in _legacy_config_candidates()):
+        pytest.skip("the resolver's legacy fallback would find this repository's own config.yaml")
+    checkout = tmp_path / "checkout"
+
+    result = _run_config_upgrade_in_checkout(checkout)
+    assert result.returncode == 0, result.stdout + result.stderr
+
+    assert "creating from example" in result.stdout
+    assert (checkout / "config.yaml").read_bytes() == (checkout / "config.example.yaml").read_bytes()

scripts/AGENTS.md

@@ -57,6 +57,14 @@ Pinned by `backend/tests/test_doctor.py::TestMainConfigResolution`.
 Root `make install` runs pre-commit through uv, so uv's tool bin directory
 need not be on `PATH`.
 
+`config-upgrade.sh` upgrades the file the Gateway loads by asking the harness
+(`AppConfig.resolve_config_path`) rather than copying its lookup order. It
+defaults `DEER_FLOW_PROJECT_ROOT` to the checkout, as `serve.sh` does, so
+`<checkout>/config.yaml` wins over a legacy `backend/config.yaml`. A missing
+`DEER_FLOW_CONFIG_PATH` or invalid project root is an error, never a fallback.
+Only "no config anywhere" creates `<checkout>/config.yaml` from the example.
+`backend/tests/test_config_version.py::test_config_upgrade_*` pins this.
+
 ## Shell Script Invocation Contract
 
 Root Makefile recipes must invoke repository `.sh` files through

scripts/config-upgrade.sh

@@ -11,17 +11,35 @@ set -e
 REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 EXAMPLE="$REPO_ROOT/config.example.yaml"
 
-# Resolve config.yaml location: env var > backend/ > repo root
-if [ -n "$DEER_FLOW_CONFIG_PATH" ] && [ -f "$DEER_FLOW_CONFIG_PATH" ]; then
-    CONFIG="$DEER_FLOW_CONFIG_PATH"
-elif [ -f "$REPO_ROOT/backend/config.yaml" ]; then
-    CONFIG="$REPO_ROOT/backend/config.yaml"
-elif [ -f "$REPO_ROOT/config.yaml" ]; then
-    CONFIG="$REPO_ROOT/config.yaml"
+if command -v cygpath >/dev/null 2>&1; then
+    REPO_ROOT_WIN="$(cygpath -w "$REPO_ROOT")"
 else
-    CONFIG=""
+    REPO_ROOT_WIN="$REPO_ROOT"
 fi
 
+# Upgrade the config.yaml the Gateway loads. Ask the harness resolver rather
+# than copying its order: with both <checkout>/config.yaml and
+# backend/config.yaml present, `make dev` reads the checkout copy. The import
+# loads .env as the Gateway does; DEER_FLOW_PROJECT_ROOT then defaults to the
+# checkout, as in serve.sh. Prints nothing when no config exists yet.
+CONFIG="$(cd "$REPO_ROOT/backend" && REPO_ROOT_WIN_PATH="$REPO_ROOT_WIN" uv run python -c "
+import os
+import sys
+
+from deerflow.config.app_config import AppConfig
+
+os.environ.setdefault('DEER_FLOW_PROJECT_ROOT', os.environ['REPO_ROOT_WIN_PATH'])
+try:
+    sys.stdout.write(str(AppConfig.resolve_config_path()))
+except FileNotFoundError as exc:
+    # An explicit DEER_FLOW_CONFIG_PATH that does not exist stops the Gateway
+    # too; never upgrade a fallback file in its place.
+    if os.environ.get('DEER_FLOW_CONFIG_PATH'):
+        sys.exit(f'ERROR {exc}')
+except ValueError as exc:
+    sys.exit(f'ERROR {exc}')
+")"
+
 if [ ! -f "$EXAMPLE" ]; then
     echo "✗ config.example.yaml not found at $EXAMPLE"
     exit 1