From c09c33454458f2d6b7dc1c1352a440ba49746072 Mon Sep 17 00:00:00 2001 From: Nan Gao Date: Fri, 1 May 2026 16:19:50 +0200 Subject: [PATCH] fix(harness): resolve runtime paths from project root (#2642) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(harness): resolve runtime paths from project root * docs(config): update * fix(config): address runtime path review feedback * test(config): fix skills path e2e root * test(config): cover legacy config fallback when project root lacks config files Verifies that when DEER_FLOW_PROJECT_ROOT is unset and cwd has no config.yaml/extensions_config.json, AppConfig and ExtensionsConfig fall back to the legacy backend/repo-root candidates — the backward-compat path requested in PR #2642 review. --------- Co-authored-by: Willem Jiang --- README.md | 2 +- README_zh.md | 2 +- backend/docs/CONFIGURATION.md | 16 +- backend/docs/SETUP.md | 22 ++- .../harness/deerflow/config/app_config.py | 16 +- .../deerflow/config/extensions_config.py | 13 +- .../packages/harness/deerflow/config/paths.py | 9 +- .../harness/deerflow/config/runtime_paths.py | 41 +++++ .../harness/deerflow/config/skills_config.py | 22 +-- .../skills/storage/local_skill_storage.py | 7 +- backend/tests/test_client_e2e.py | 7 + backend/tests/test_runtime_paths.py | 145 ++++++++++++++++++ backend/tests/test_skills_loader.py | 21 ++- config.example.yaml | 10 +- docker/docker-compose-dev.yaml | 1 + docker/docker-compose.yaml | 5 +- 16 files changed, 284 insertions(+), 55 deletions(-) create mode 100644 backend/packages/harness/deerflow/config/runtime_paths.py create mode 100644 backend/tests/test_runtime_paths.py diff --git a/README.md b/README.md index c67fdc005..0fc8f173e 100644 --- a/README.md +++ b/README.md @@ -251,7 +251,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed Docker development guide. If you prefer running services locally: -Prerequisite: complete the "Configuration" steps above first (`make setup`). `make dev` requires a valid `config.yaml` in the project root (can be overridden via `DEER_FLOW_CONFIG_PATH`). Run `make doctor` to verify your setup before starting. +Prerequisite: complete the "Configuration" steps above first (`make setup`). `make dev` requires a valid `config.yaml` in the project root. Set `DEER_FLOW_PROJECT_ROOT` to define that root explicitly, or `DEER_FLOW_CONFIG_PATH` to point at a specific config file. Runtime state defaults to `.deer-flow` under the project root and can be moved with `DEER_FLOW_HOME`; skills default to `skills/` under the project root and can be moved with `DEER_FLOW_SKILLS_PATH`. Run `make doctor` to verify your setup before starting. On Windows, run the local development flow from Git Bash. Native `cmd.exe` and PowerShell shells are not supported for the bash-based service scripts, and WSL is not guaranteed because some scripts rely on Git for Windows utilities such as `cygpath`. 1. **Check prerequisites**: diff --git a/README_zh.md b/README_zh.md index 6e4a618c7..d5317082e 100644 --- a/README_zh.md +++ b/README_zh.md @@ -194,7 +194,7 @@ make down # 停止并移除容器 如果你更希望直接在本地启动各个服务： -前提：先完成上面的“配置”步骤（`make config` 和模型 API key 配置）。`make dev` 需要有效配置文件，默认读取项目根目录下的 `config.yaml`，也可以通过 `DEER_FLOW_CONFIG_PATH` 覆盖。 +前提：先完成上面的“配置”步骤（`make config` 和模型 API key 配置）。`make dev` 需要有效配置文件，默认读取项目根目录下的 `config.yaml`。可以用 `DEER_FLOW_PROJECT_ROOT` 显式指定项目根目录，也可以用 `DEER_FLOW_CONFIG_PATH` 指向某个具体配置文件。运行期状态默认写到项目根目录下的 `.deer-flow`，可用 `DEER_FLOW_HOME` 覆盖；skills 默认读取项目根目录下的 `skills/`，可用 `DEER_FLOW_SKILLS_PATH` 覆盖。 在 Windows 上，请使用 Git Bash 运行本地开发流程。基于 bash 的服务脚本不支持直接在原生 `cmd.exe` 或 PowerShell 中执行，且 WSL 也不保证可用，因为部分脚本依赖 Git for Windows 的 `cygpath` 等工具。 1. **检查依赖环境**： diff --git a/backend/docs/CONFIGURATION.md b/backend/docs/CONFIGURATION.md index f87fdd236..26137951f 100644 --- a/backend/docs/CONFIGURATION.md +++ b/backend/docs/CONFIGURATION.md @@ -321,12 +321,16 @@ models: - `DEEPSEEK_API_KEY` - DeepSeek API key - `NOVITA_API_KEY` - Novita API key (OpenAI-compatible endpoint) - `TAVILY_API_KEY` - Tavily search API key +- `DEER_FLOW_PROJECT_ROOT` - Project root for relative runtime paths - `DEER_FLOW_CONFIG_PATH` - Custom config file path +- `DEER_FLOW_EXTENSIONS_CONFIG_PATH` - Custom extensions config file path +- `DEER_FLOW_HOME` - Runtime state directory (defaults to `.deer-flow` under the project root) +- `DEER_FLOW_SKILLS_PATH` - Skills directory when `skills.path` is omitted - `GATEWAY_ENABLE_DOCS` - Set to `false` to disable Swagger UI (`/docs`), ReDoc (`/redoc`), and OpenAPI schema (`/openapi.json`) endpoints (default: `true`) ## Configuration Location -The configuration file should be placed in the **project root directory** (`deer-flow/config.yaml`), not in the backend directory. +The configuration file should be placed in the **project root directory** (`deer-flow/config.yaml`). Set `DEER_FLOW_PROJECT_ROOT` when the process may start from another working directory, or set `DEER_FLOW_CONFIG_PATH` to point at a specific file. ## Configuration Priority @@ -334,12 +338,12 @@ DeerFlow searches for configuration in this order: 1. Path specified in code via `config_path` argument 2. Path from `DEER_FLOW_CONFIG_PATH` environment variable -3. `config.yaml` in current working directory (typically `backend/` when running) -4. `config.yaml` in parent directory (project root: `deer-flow/`) +3. `config.yaml` under `DEER_FLOW_PROJECT_ROOT`, or under the current working directory when `DEER_FLOW_PROJECT_ROOT` is unset +4. Legacy backend/repository-root locations for monorepo compatibility ## Best Practices -1. **Place `config.yaml` in project root** - Not in `backend/` directory +1. **Place `config.yaml` in project root** - Set `DEER_FLOW_PROJECT_ROOT` if the runtime starts elsewhere 2. **Never commit `config.yaml`** - It's already in `.gitignore` 3. **Use environment variables for secrets** - Don't hardcode API keys 4. **Keep `config.example.yaml` updated** - Document all new options @@ -350,7 +354,7 @@ DeerFlow searches for configuration in this order: ### "Config file not found" - Ensure `config.yaml` exists in the **project root** directory (`deer-flow/config.yaml`) -- The backend searches parent directory by default, so root location is preferred +- If the runtime starts outside the project root, set `DEER_FLOW_PROJECT_ROOT` - Alternatively, set `DEER_FLOW_CONFIG_PATH` environment variable to custom location ### "Invalid API key" @@ -360,7 +364,7 @@ DeerFlow searches for configuration in this order: ### "Skills not loading" - Check that `deer-flow/skills/` directory exists - Verify skills have valid `SKILL.md` files -- Check `skills.path` configuration if using custom path +- Check `skills.path` or `DEER_FLOW_SKILLS_PATH` if using a custom path ### "Docker sandbox fails to start" - Ensure Docker is running diff --git a/backend/docs/SETUP.md b/backend/docs/SETUP.md index 50885eb3f..aff0e287f 100644 --- a/backend/docs/SETUP.md +++ b/backend/docs/SETUP.md @@ -23,6 +23,9 @@ DeerFlow uses a YAML configuration file that should be placed in the **project r # Option A: Set environment variables (recommended) export OPENAI_API_KEY="your-key-here" + # Optional: pin the project root when running from another directory + export DEER_FLOW_PROJECT_ROOT="/path/to/deer-flow" + # Option B: Edit config.yaml directly vim config.yaml # or your preferred editor ``` @@ -35,17 +38,20 @@ DeerFlow uses a YAML configuration file that should be placed in the **project r ## Important Notes -- **Location**: `config.yaml` should be in `deer-flow/` (project root), not `deer-flow/backend/` +- **Location**: `config.yaml` should be in `deer-flow/` (project root) - **Git**: `config.yaml` is automatically ignored by git (contains secrets) -- **Priority**: If both `backend/config.yaml` and `../config.yaml` exist, backend version takes precedence +- **Runtime root**: Set `DEER_FLOW_PROJECT_ROOT` if DeerFlow may start from outside the project root +- **Runtime data**: State defaults to `.deer-flow` under the project root; set `DEER_FLOW_HOME` to move it +- **Skills**: Skills default to `skills/` under the project root; set `DEER_FLOW_SKILLS_PATH` or `skills.path` to move them ## Configuration File Locations The backend searches for `config.yaml` in this order: -1. `DEER_FLOW_CONFIG_PATH` environment variable (if set) -2. `backend/config.yaml` (current directory when running from backend/) -3. `deer-flow/config.yaml` (parent directory - **recommended location**) +1. Explicit `config_path` argument from code +2. `DEER_FLOW_CONFIG_PATH` environment variable (if set) +3. `config.yaml` under `DEER_FLOW_PROJECT_ROOT`, or the current working directory when `DEER_FLOW_PROJECT_ROOT` is unset +4. Legacy backend/repository-root locations for monorepo compatibility **Recommended**: Place `config.yaml` in project root (`deer-flow/config.yaml`). @@ -77,8 +83,8 @@ python -c "from deerflow.config.app_config import AppConfig; print(AppConfig.res If it can't find the config: 1. Ensure you've copied `config.example.yaml` to `config.yaml` -2. Verify you're in the correct directory -3. Check the file exists: `ls -la ../config.yaml` +2. Verify you're in the project root, or set `DEER_FLOW_PROJECT_ROOT` +3. Check the file exists: `ls -la config.yaml` ### Permission denied @@ -89,4 +95,4 @@ chmod 600 ../config.yaml # Protect sensitive configuration ## See Also - [Configuration Guide](CONFIGURATION.md) - Detailed configuration options -- [Architecture Overview](../CLAUDE.md) - System architecture \ No newline at end of file +- [Architecture Overview](../CLAUDE.md) - System architecture diff --git a/backend/packages/harness/deerflow/config/app_config.py b/backend/packages/harness/deerflow/config/app_config.py index b31d396a5..a41108372 100644 --- a/backend/packages/harness/deerflow/config/app_config.py +++ b/backend/packages/harness/deerflow/config/app_config.py @@ -17,6 +17,7 @@ from deerflow.config.guardrails_config import GuardrailsConfig, load_guardrails_ from deerflow.config.memory_config import MemoryConfig, load_memory_config_from_dict from deerflow.config.model_config import ModelConfig from deerflow.config.run_events_config import RunEventsConfig +from deerflow.config.runtime_paths import existing_project_file from deerflow.config.sandbox_config import SandboxConfig from deerflow.config.skill_evolution_config import SkillEvolutionConfig from deerflow.config.skills_config import SkillsConfig @@ -46,8 +47,8 @@ class CircuitBreakerConfig(BaseModel): recovery_timeout_sec: int = Field(default=60, description="Time in seconds before attempting to recover the circuit") -def _default_config_candidates() -> tuple[Path, ...]: - """Return deterministic config.yaml locations without relying on cwd.""" +def _legacy_config_candidates() -> tuple[Path, ...]: + """Return source-tree config.yaml locations for monorepo compatibility.""" backend_dir = Path(__file__).resolve().parents[4] repo_root = backend_dir.parent return (backend_dir / "config.yaml", repo_root / "config.yaml") @@ -110,7 +111,8 @@ class AppConfig(BaseModel): Priority: 1. If provided `config_path` argument, use it. 2. If provided `DEER_FLOW_CONFIG_PATH` environment variable, use it. - 3. Otherwise, search deterministic backend/repository-root defaults from `_default_config_candidates()`. + 3. Otherwise, search the caller project root. + 4. Finally, search legacy backend/repository-root defaults for monorepo compatibility. """ if config_path: path = Path(config_path) @@ -123,10 +125,14 @@ class AppConfig(BaseModel): raise FileNotFoundError(f"Config file specified by environment variable `DEER_FLOW_CONFIG_PATH` not found at {path}") return path else: - for path in _default_config_candidates(): + project_config = existing_project_file(("config.yaml",)) + if project_config is not None: + return project_config + + for path in _legacy_config_candidates(): if path.exists(): return path - raise FileNotFoundError("`config.yaml` file not found at the default backend or repository root locations") + raise FileNotFoundError("`config.yaml` file not found in the project root or legacy backend/repository root locations") @classmethod def from_file(cls, config_path: str | None = None) -> Self: diff --git a/backend/packages/harness/deerflow/config/extensions_config.py b/backend/packages/harness/deerflow/config/extensions_config.py index e7a48d166..a2daa71f4 100644 --- a/backend/packages/harness/deerflow/config/extensions_config.py +++ b/backend/packages/harness/deerflow/config/extensions_config.py @@ -7,6 +7,8 @@ from typing import Any, Literal from pydantic import BaseModel, ConfigDict, Field +from deerflow.config.runtime_paths import existing_project_file + class McpOAuthConfig(BaseModel): """OAuth configuration for an MCP server (HTTP/SSE transports).""" @@ -73,8 +75,8 @@ class ExtensionsConfig(BaseModel): Priority: 1. If provided `config_path` argument, use it. 2. If provided `DEER_FLOW_EXTENSIONS_CONFIG_PATH` environment variable, use it. - 3. Otherwise, check for `extensions_config.json` in the current directory, then in the parent directory. - 4. For backward compatibility, also check for `mcp_config.json` if `extensions_config.json` is not found. + 3. Otherwise, search the caller project root for `extensions_config.json`, then `mcp_config.json`. + 4. For backward compatibility, also search legacy backend/repository-root defaults. 5. If not found, return None (extensions are optional). Args: @@ -83,8 +85,9 @@ class ExtensionsConfig(BaseModel): Resolution order: 1. If provided `config_path` argument, use it. 2. If provided `DEER_FLOW_EXTENSIONS_CONFIG_PATH` environment variable, use it. - 3. Otherwise, search backend/repository-root defaults for + 3. Otherwise, search the caller project root for `extensions_config.json`, then legacy `mcp_config.json`. + 4. Finally, search backend/repository-root defaults for monorepo compatibility. Returns: Path to the extensions config file if found, otherwise None. @@ -100,6 +103,10 @@ class ExtensionsConfig(BaseModel): raise FileNotFoundError(f"Extensions config file specified by environment variable `DEER_FLOW_EXTENSIONS_CONFIG_PATH` not found at {path}") return path else: + project_config = existing_project_file(("extensions_config.json", "mcp_config.json")) + if project_config is not None: + return project_config + backend_dir = Path(__file__).resolve().parents[4] repo_root = backend_dir.parent for path in ( diff --git a/backend/packages/harness/deerflow/config/paths.py b/backend/packages/harness/deerflow/config/paths.py index f1ce7eae1..9fa633f54 100644 --- a/backend/packages/harness/deerflow/config/paths.py +++ b/backend/packages/harness/deerflow/config/paths.py @@ -3,6 +3,8 @@ import re import shutil from pathlib import Path, PureWindowsPath +from deerflow.config.runtime_paths import runtime_home + # Virtual path prefix seen by agents inside the sandbox VIRTUAL_PATH_PREFIX = "/mnt/user-data" @@ -11,9 +13,8 @@ _SAFE_USER_ID_RE = re.compile(r"^[A-Za-z0-9_\-]+$") def _default_local_base_dir() -> Path: - """Return the repo-local DeerFlow state directory without relying on cwd.""" - backend_dir = Path(__file__).resolve().parents[4] - return backend_dir / ".deer-flow" + """Return the caller project's writable DeerFlow state directory.""" + return runtime_home() def _validate_thread_id(thread_id: str) -> str: @@ -81,7 +82,7 @@ class Paths: BaseDir resolution (in priority order): 1. Constructor argument `base_dir` 2. DEER_FLOW_HOME environment variable - 3. Repo-local fallback derived from this module path: `{backend_dir}/.deer-flow` + 3. Caller project fallback: `{project_root}/.deer-flow` """ def __init__(self, base_dir: str | Path | None = None) -> None: diff --git a/backend/packages/harness/deerflow/config/runtime_paths.py b/backend/packages/harness/deerflow/config/runtime_paths.py new file mode 100644 index 000000000..25157106f --- /dev/null +++ b/backend/packages/harness/deerflow/config/runtime_paths.py @@ -0,0 +1,41 @@ +"""Runtime path resolution for standalone harness usage.""" + +import os +from pathlib import Path + + +def project_root() -> Path: + """Return the caller project root for runtime-owned files.""" + if env_root := os.getenv("DEER_FLOW_PROJECT_ROOT"): + root = Path(env_root).resolve() + if not root.exists(): + raise ValueError(f"DEER_FLOW_PROJECT_ROOT is set to '{env_root}', but the resolved path '{root}' does not exist.") + if not root.is_dir(): + raise ValueError(f"DEER_FLOW_PROJECT_ROOT is set to '{env_root}', but the resolved path '{root}' is not a directory.") + return root + return Path.cwd().resolve() + + +def runtime_home() -> Path: + """Return the writable DeerFlow state directory.""" + if env_home := os.getenv("DEER_FLOW_HOME"): + return Path(env_home).resolve() + return project_root() / ".deer-flow" + + +def resolve_path(value: str | os.PathLike[str], *, base: Path | None = None) -> Path: + """Resolve absolute paths as-is and relative paths against the project root.""" + path = Path(value) + if not path.is_absolute(): + path = (base or project_root()) / path + return path.resolve() + + +def existing_project_file(names: tuple[str, ...]) -> Path | None: + """Return the first existing named file under the project root.""" + root = project_root() + for name in names: + candidate = root / name + if candidate.is_file(): + return candidate + return None diff --git a/backend/packages/harness/deerflow/config/skills_config.py b/backend/packages/harness/deerflow/config/skills_config.py index 266a98b91..671b48fde 100644 --- a/backend/packages/harness/deerflow/config/skills_config.py +++ b/backend/packages/harness/deerflow/config/skills_config.py @@ -1,11 +1,9 @@ +import os from pathlib import Path from pydantic import BaseModel, Field - -def _default_repo_root() -> Path: - """Resolve the repo root without relying on the current working directory.""" - return Path(__file__).resolve().parents[5] +from deerflow.config.runtime_paths import project_root, resolve_path class SkillsConfig(BaseModel): @@ -17,7 +15,7 @@ class SkillsConfig(BaseModel): ) path: str | None = Field( default=None, - description="Path to skills directory. If not specified, defaults to ../skills relative to backend directory", + description="Path to skills directory. If not specified, defaults to skills under the caller project root.", ) container_path: str = Field( default="/mnt/skills", @@ -32,15 +30,11 @@ class SkillsConfig(BaseModel): Path to the skills directory """ if self.path: - # Use configured path (can be absolute or relative) - path = Path(self.path) - if not path.is_absolute(): - # If relative, resolve from the repo root for deterministic behavior. - path = _default_repo_root() / path - return path.resolve() - else: - # Default: /skills - return _default_repo_root() / "skills" + # Use configured path (can be absolute or relative to project root) + return resolve_path(self.path) + if env_path := os.getenv("DEER_FLOW_SKILLS_PATH"): + return resolve_path(env_path) + return project_root() / "skills" def get_skill_container_path(self, skill_name: str, category: str = "public") -> str: """ diff --git a/backend/packages/harness/deerflow/skills/storage/local_skill_storage.py b/backend/packages/harness/deerflow/skills/storage/local_skill_storage.py index 047cd6163..4b7dffde4 100644 --- a/backend/packages/harness/deerflow/skills/storage/local_skill_storage.py +++ b/backend/packages/harness/deerflow/skills/storage/local_skill_storage.py @@ -12,7 +12,7 @@ from collections.abc import Iterable from datetime import UTC, datetime from pathlib import Path -from deerflow.config.skills_config import _default_repo_root +from deerflow.config.runtime_paths import resolve_path from deerflow.skills.storage.skill_storage import SKILL_MD_FILE, SkillStorage from deerflow.skills.types import SkillCategory @@ -44,10 +44,7 @@ class LocalSkillStorage(SkillStorage): config = app_config or get_app_config() self._host_root: Path = config.skills.get_skills_path() else: - path = Path(host_path) - if not path.is_absolute(): - path = _default_repo_root() / path - self._host_root = path.resolve() + self._host_root = resolve_path(host_path) # ------------------------------------------------------------------ # Abstract operation implementations diff --git a/backend/tests/test_client_e2e.py b/backend/tests/test_client_e2e.py index 0c3872e41..4b6a62ea9 100644 --- a/backend/tests/test_client_e2e.py +++ b/backend/tests/test_client_e2e.py @@ -17,6 +17,7 @@ import json import os import uuid import zipfile +from pathlib import Path import pytest from dotenv import load_dotenv @@ -94,12 +95,18 @@ def e2e_env(tmp_path, monkeypatch): """Isolated filesystem environment for E2E tests. - DEER_FLOW_HOME → tmp_path (all thread data lands in a temp dir) + - DEER_FLOW_PROJECT_ROOT → repository root (shared skills/config assets + still resolve correctly when tests run from backend/) - Singletons reset so they pick up the new env - Title/memory/summarization disabled to avoid extra LLM calls - AppConfig built programmatically (avoids config.yaml param-name issues) """ # 1. Filesystem isolation monkeypatch.setenv("DEER_FLOW_HOME", str(tmp_path)) + monkeypatch.setenv( + "DEER_FLOW_PROJECT_ROOT", + str(Path(__file__).resolve().parents[2]), + ) monkeypatch.setattr("deerflow.config.paths._paths", None) monkeypatch.setattr("deerflow.sandbox.sandbox_provider._default_sandbox_provider", None) diff --git a/backend/tests/test_runtime_paths.py b/backend/tests/test_runtime_paths.py new file mode 100644 index 000000000..aa9e94641 --- /dev/null +++ b/backend/tests/test_runtime_paths.py @@ -0,0 +1,145 @@ +"""Runtime path policy tests for standalone harness usage.""" + +from pathlib import Path + +import pytest +import yaml + +from deerflow.config import app_config as app_config_module +from deerflow.config import extensions_config as extensions_config_module +from deerflow.config.app_config import AppConfig +from deerflow.config.extensions_config import ExtensionsConfig +from deerflow.config.paths import Paths +from deerflow.config.runtime_paths import project_root +from deerflow.config.skills_config import SkillsConfig +from deerflow.skills.storage import get_or_new_skill_storage + + +def _clear_path_env(monkeypatch): + for name in ( + "DEER_FLOW_CONFIG_PATH", + "DEER_FLOW_EXTENSIONS_CONFIG_PATH", + "DEER_FLOW_HOME", + "DEER_FLOW_PROJECT_ROOT", + "DEER_FLOW_SKILLS_PATH", + ): + monkeypatch.delenv(name, raising=False) + + +def test_default_runtime_paths_resolve_from_current_project(tmp_path: Path, monkeypatch): + _clear_path_env(monkeypatch) + monkeypatch.chdir(tmp_path) + + (tmp_path / "config.yaml").write_text( + yaml.safe_dump({"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}}), + encoding="utf-8", + ) + (tmp_path / "extensions_config.json").write_text('{"mcpServers": {}, "skills": {}}', encoding="utf-8") + + assert AppConfig.resolve_config_path() == tmp_path / "config.yaml" + assert ExtensionsConfig.resolve_config_path() == tmp_path / "extensions_config.json" + assert Paths().base_dir == tmp_path / ".deer-flow" + assert SkillsConfig().get_skills_path() == tmp_path / "skills" + assert get_or_new_skill_storage(skills_path=SkillsConfig().get_skills_path()).get_skills_root_path() == tmp_path / "skills" + + +def test_deer_flow_project_root_overrides_current_directory(tmp_path: Path, monkeypatch): + _clear_path_env(monkeypatch) + project_root = tmp_path / "project" + other_cwd = tmp_path / "other" + project_root.mkdir() + other_cwd.mkdir() + monkeypatch.chdir(other_cwd) + monkeypatch.setenv("DEER_FLOW_PROJECT_ROOT", str(project_root)) + + (project_root / "config.yaml").write_text( + yaml.safe_dump({"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}}), + encoding="utf-8", + ) + (project_root / "mcp_config.json").write_text('{"mcpServers": {}, "skills": {}}', encoding="utf-8") + + assert AppConfig.resolve_config_path() == project_root / "config.yaml" + assert ExtensionsConfig.resolve_config_path() == project_root / "mcp_config.json" + assert Paths().base_dir == project_root / ".deer-flow" + assert SkillsConfig(path="custom-skills").get_skills_path() == project_root / "custom-skills" + + +def test_deer_flow_skills_path_overrides_project_default(tmp_path: Path, monkeypatch): + _clear_path_env(monkeypatch) + monkeypatch.chdir(tmp_path) + monkeypatch.setenv("DEER_FLOW_SKILLS_PATH", "team-skills") + + assert SkillsConfig().get_skills_path() == tmp_path / "team-skills" + assert get_or_new_skill_storage(skills_path=SkillsConfig().get_skills_path()).get_skills_root_path() == tmp_path / "team-skills" + + +def test_deer_flow_project_root_must_exist(tmp_path: Path, monkeypatch): + _clear_path_env(monkeypatch) + missing_root = tmp_path / "missing" + monkeypatch.setenv("DEER_FLOW_PROJECT_ROOT", str(missing_root)) + + with pytest.raises(ValueError, match="does not exist"): + project_root() + + +def test_deer_flow_project_root_must_be_directory(tmp_path: Path, monkeypatch): + _clear_path_env(monkeypatch) + project_root_file = tmp_path / "project-root" + project_root_file.write_text("", encoding="utf-8") + monkeypatch.setenv("DEER_FLOW_PROJECT_ROOT", str(project_root_file)) + + with pytest.raises(ValueError, match="not a directory"): + project_root() + + +def test_app_config_falls_back_to_legacy_when_project_root_lacks_config(tmp_path: Path, monkeypatch): + """When DEER_FLOW_PROJECT_ROOT is unset and cwd has no config.yaml, the + legacy backend/repo-root candidates must be used for monorepo compatibility.""" + _clear_path_env(monkeypatch) + cwd = tmp_path / "cwd" + cwd.mkdir() + monkeypatch.chdir(cwd) + + legacy_backend = tmp_path / "legacy-backend" + legacy_repo = tmp_path / "legacy-repo" + legacy_backend.mkdir() + legacy_repo.mkdir() + legacy_backend_config = legacy_backend / "config.yaml" + legacy_backend_config.write_text( + yaml.safe_dump({"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"}}), + encoding="utf-8", + ) + repo_root_config = legacy_repo / "config.yaml" + repo_root_config.write_text("", encoding="utf-8") + + monkeypatch.setattr( + app_config_module, + "_legacy_config_candidates", + lambda: (legacy_backend_config, repo_root_config), + ) + + assert AppConfig.resolve_config_path() == legacy_backend_config + + +def test_extensions_config_falls_back_to_legacy_when_project_root_lacks_file(tmp_path: Path, monkeypatch): + """ExtensionsConfig should hit the legacy backend/repo-root locations when + the caller project root has no extensions_config.json/mcp_config.json.""" + _clear_path_env(monkeypatch) + cwd = tmp_path / "cwd" + cwd.mkdir() + monkeypatch.chdir(cwd) + + fake_backend = tmp_path / "fake-backend" + fake_repo = tmp_path / "fake-repo" + fake_backend.mkdir() + fake_repo.mkdir() + legacy_extensions = fake_backend / "extensions_config.json" + legacy_extensions.write_text('{"mcpServers": {}, "skills": {}}', encoding="utf-8") + + fake_paths_module_file = fake_backend / "packages" / "harness" / "deerflow" / "config" / "extensions_config.py" + fake_paths_module_file.parent.mkdir(parents=True) + fake_paths_module_file.write_text("", encoding="utf-8") + + monkeypatch.setattr(extensions_config_module, "__file__", str(fake_paths_module_file)) + + assert ExtensionsConfig.resolve_config_path() == legacy_extensions diff --git a/backend/tests/test_skills_loader.py b/backend/tests/test_skills_loader.py index 5a03532c6..886090f71 100644 --- a/backend/tests/test_skills_loader.py +++ b/backend/tests/test_skills_loader.py @@ -14,12 +14,25 @@ def _write_skill(skill_dir: Path, name: str, description: str) -> None: (skill_dir / "SKILL.md").write_text(content, encoding="utf-8") -def test_get_skills_root_path_points_to_project_root_skills(): - """get_skills_root_path() should point to deer-flow/skills (sibling of backend/), not backend/packages/skills.""" +def test_get_skills_root_path_points_to_current_project_skills(tmp_path: Path, monkeypatch): + """get_skills_root_path() should point to the caller project skills directory.""" + monkeypatch.delenv("DEER_FLOW_SKILLS_PATH", raising=False) + monkeypatch.delenv("DEER_FLOW_PROJECT_ROOT", raising=False) + monkeypatch.chdir(tmp_path) + app_config = SimpleNamespace(skills=SkillsConfig()) path = get_or_new_skill_storage(app_config=app_config).get_skills_root_path() - assert path.name == "skills", f"Expected 'skills', got '{path.name}'" - assert (path.parent / "backend").is_dir(), f"Expected skills path's parent to be project root containing 'backend/', but got {path}" + assert path == tmp_path / "skills" + + +def test_get_skills_root_path_honors_env_override(tmp_path: Path, monkeypatch): + """DEER_FLOW_SKILLS_PATH should override the caller project skills directory.""" + skills_root = tmp_path / "team-skills" + monkeypatch.setenv("DEER_FLOW_SKILLS_PATH", str(skills_root)) + + app_config = SimpleNamespace(skills=SkillsConfig()) + path = get_or_new_skill_storage(app_config=app_config).get_skills_root_path() + assert path == skills_root def test_load_skills_discovers_nested_skills_and_sets_container_paths(tmp_path: Path): diff --git a/config.example.yaml b/config.example.yaml index 04ccd0b12..b16b4a6bb 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -2,8 +2,11 @@ # # Guidelines: # - Copy this file to `config.yaml` and customize it for your environment -# - The default path of this configuration file is `config.yaml` in the current working directory. -# However you can change it using the `DEER_FLOW_CONFIG_PATH` environment variable. +# - The default path of this configuration file is `config.yaml` in the project root. +# You can set `DEER_FLOW_PROJECT_ROOT` to define that root explicitly, or use +# `DEER_FLOW_CONFIG_PATH` to point at a specific config file. +# - Runtime state defaults to `.deer-flow` under the project root. Override it +# with `DEER_FLOW_HOME` when you need a different writable data directory. # - Environment variables are available for all field values. Example: `api_key: $OPENAI_API_KEY` # - The `use` path is a string that looks like "package_name.sub_package_name.module_name:class_name/variable_name". @@ -678,7 +681,8 @@ sandbox: skills: # Path to skills directory on the host (relative to project root or absolute) - # Default: ../skills (relative to backend directory) + # Default: skills under the project root + # Override with DEER_FLOW_SKILLS_PATH when this field is omitted. # Uncomment to customize: # path: /absolute/path/to/custom/skills diff --git a/docker/docker-compose-dev.yaml b/docker/docker-compose-dev.yaml index 8fb95124d..6d00d71ff 100644 --- a/docker/docker-compose-dev.yaml +++ b/docker/docker-compose-dev.yaml @@ -157,6 +157,7 @@ services: working_dir: /app environment: - CI=true + - DEER_FLOW_PROJECT_ROOT=/app - DEER_FLOW_HOME=/app/backend/.deer-flow - DEER_FLOW_CHANNELS_LANGGRAPH_URL=${DEER_FLOW_CHANNELS_LANGGRAPH_URL:-http://gateway:8001/api} - DEER_FLOW_CHANNELS_GATEWAY_URL=${DEER_FLOW_CHANNELS_GATEWAY_URL:-http://gateway:8001} diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml index 82cb62425..8d82980d3 100644 --- a/docker/docker-compose.yaml +++ b/docker/docker-compose.yaml @@ -8,9 +8,11 @@ # - provisioner: (optional) Sandbox provisioner for Kubernetes mode # # Key environment variables (set via environment/.env or scripts/deploy.sh): -# DEER_FLOW_HOME — runtime data dir, default $REPO_ROOT/backend/.deer-flow +# DEER_FLOW_PROJECT_ROOT — project root for relative runtime paths +# DEER_FLOW_HOME — runtime data dir, default .deer-flow under $DEER_FLOW_PROJECT_ROOT (or cwd) # DEER_FLOW_CONFIG_PATH — path to config.yaml # DEER_FLOW_EXTENSIONS_CONFIG_PATH — path to extensions_config.json +# DEER_FLOW_SKILLS_PATH — skills dir, default $DEER_FLOW_PROJECT_ROOT/skills # DEER_FLOW_DOCKER_SOCKET — Docker socket path, default /var/run/docker.sock # DEER_FLOW_REPO_ROOT — repo root (used for skills host path in DooD) # BETTER_AUTH_SECRET — required for frontend auth/session security @@ -93,6 +95,7 @@ services: working_dir: /app environment: - CI=true + - DEER_FLOW_PROJECT_ROOT=/app - DEER_FLOW_HOME=/app/backend/.deer-flow - DEER_FLOW_CONFIG_PATH=/app/backend/config.yaml - DEER_FLOW_EXTENSIONS_CONFIG_PATH=/app/backend/extensions_config.json