mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-25 14:06:18 +00:00
fix(mcp): honor configured stdio working directories (#5643)
* fix(mcp): honor configured stdio working directories * fix(mcp): preserve defaults for empty working directories Treat empty stdio cwd values as omitted, including unresolved environment references. Add real subprocess regressions for discovery and pooled-call defaults, plus direct connection-dictionary coverage.
This commit is contained in:
parent
4bc4241531
commit
0ff2e9ddcf
@ -1130,6 +1130,11 @@ Public-skill CI waivers are exact, expiring exceptions in `.github/skill-review-
|
||||
|
||||
Tools follow the same philosophy. DeerFlow comes with a core toolset — web search, web fetch, rendered web capture, file operations, bash execution — and supports custom tools via MCP servers and Python functions. The bundled DDG, Brave, Tavily, and SearXNG search providers accept an optional `time_range` of `day`, `week`, `month`, or `year`; omitting it preserves existing search behavior. For DDG recency searches, DeerFlow excludes DDGS backends that ignore time limits. Swap anything. Add anything.
|
||||
|
||||
Stdio MCP servers can set `cwd` in `extensions_config.json` when their entrypoint
|
||||
or data files depend on a specific working directory. The setting applies to
|
||||
both discovery and tool calls; see [MCP configuration](backend/docs/MCP_SERVER.md#stdio-working-directory).
|
||||
Omitted, `null`, or empty values keep the default working directories.
|
||||
|
||||
Tavily `web_search` also accepts optional `include_domains` and `exclude_domains`
|
||||
lists in its `config.yaml` tool entry to control search sources. Non-empty
|
||||
`include_domains` uses Tavily's `filter` mode to restrict results to those domains.
|
||||
|
||||
@ -14,6 +14,34 @@ DeerFlow supports configurable MCP servers and skills to extend its capabilities
|
||||
3. Configure each server’s command, arguments, and environment variables as needed.
|
||||
4. Restart the application to load and register MCP tools.
|
||||
|
||||
## Stdio Working Directory
|
||||
|
||||
Set `cwd` when a stdio server needs to resolve its entrypoint or data files
|
||||
relative to a specific directory:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"local": {
|
||||
"type": "stdio",
|
||||
"command": "python",
|
||||
"args": ["server.py"],
|
||||
"cwd": "/absolute/path/to/server"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The directory must exist on the Gateway host (inside the container for Docker).
|
||||
Use an absolute path for consistent behavior across launch locations; a
|
||||
whole-string environment reference such as `"$MCP_SERVER_CWD"` is also supported.
|
||||
The configured directory applies to discovery and subsequent tool calls.
|
||||
When `cwd` is omitted, `null`, or an empty string (including an unset environment
|
||||
reference), discovery inherits the Gateway's working directory and pooled calls
|
||||
use the thread workspace. HTTP/SSE servers ignore it.
|
||||
Files created outside the thread's user-data tree are not exposed through the
|
||||
sandbox/artifact API.
|
||||
|
||||
## OpenViking MCP Tools
|
||||
|
||||
OpenViking's official server exposes a Streamable HTTP MCP endpoint at `/mcp`.
|
||||
|
||||
@ -210,6 +210,7 @@ class McpServerConfig(BaseModel):
|
||||
type: str = Field(default="stdio", description="Transport type: 'stdio', 'sse', or 'http'")
|
||||
command: str | None = Field(default=None, description="Command to execute to start the MCP server (for stdio type)")
|
||||
args: list[str] = Field(default_factory=list, description="Arguments to pass to the command (for stdio type)")
|
||||
cwd: str | None = Field(default=None, description="Working directory for the MCP server process (for stdio type)")
|
||||
env: dict[str, str] = Field(default_factory=dict, description="Environment variables for the MCP server")
|
||||
url: str | None = Field(default=None, description="URL of the MCP server (for sse or http type)")
|
||||
headers: dict[str, str] = Field(default_factory=dict, description="HTTP headers to send (for sse or http type)")
|
||||
|
||||
@ -37,6 +37,7 @@
|
||||
references `tool_search` when a hinted MCP tool is currently deferred; do not
|
||||
add a parallel routing middleware for PR1-style preference hints.
|
||||
- **Stdio file outputs**: Persistent stdio sessions are scoped by `user_id:thread_id`. For stdio transports only, DeerFlow pins the subprocess default `cwd` to the thread workspace and `TMPDIR`/`TMP`/`TEMP` to `workspace/.mcp/tmp/`, unless the operator explicitly configured `cwd` or temp env values. `.mcp` is a DeerFlow-owned internal namespace: its temporary/debug files remain addressable when returned by a tool but are excluded from run workspace-change summaries — by directory name at any depth, consistent with the other reserved names in `EXCLUDED_DIR_NAMES` (`.git`, `node_modules`, …) and robust if a server ever creates a relative `.mcp` from a different cwd. Both launch paths pin it at the workspace root today. SSE/HTTP transports skip this filesystem prep entirely.
|
||||
- **Configured stdio working directory**: A non-empty `McpServerConfig.cwd` is forwarded by `build_server_params` for discovery, pooled Agent calls, and durable task calls. Omit the connection key when the setting is `None` or empty (including unresolved environment references), so the runtime's thread-workspace default still applies; HTTP/SSE connections must not receive it. `tests/test_mcp_cwd.py` exercises the JSON-config-to-subprocess path with relative entrypoints, relative tool inputs, and discovery/pool defaults for omitted, null, empty, and unresolved values.
|
||||
- **Stdio disconnect recovery**: Ordinary Agent tool calls and durable task submit/status/cancel calls that receive the MCP SDK's explicit `Connection closed` error or an AnyIO closed-stream error evict only that `(server_name, user_id:thread_id)` session when the registered entry is still the same `ClientSession` that failed. A late error from an old concurrent call cannot evict its replacement or a new in-flight creation. The failing call still surfaces its original error and is never replayed automatically; a later retry creates a fresh subprocess/session. Protocol timeouts, normal `isError=true` tool results, and interceptor failures do not evict a healthy stateful session.
|
||||
- **Stdio path translation**: MCP-returned local file references are not copied. If a `ResourceLink` or conservative free-text path resolves to an existing file inside the thread's mounted user-data tree, it is translated deterministically to `/mnt/user-data/...`; paths outside that tree remain unchanged.
|
||||
- **Runtime updates**: Gateway API saves to extensions_config.json; the Gateway-embedded runtime detects changes via the resolved-path + content-signature check above, so multi-worker / stale-mtime deployments still pick up an added/removed MCP server without a restart (`PUT /api/mcp/config` keeps whole-payload validation, while `PATCH /api/mcp/config` changes only one server's `enabled` field, normalizes the same `type`/MCP-spec `transport` alias as the runtime config model, and validates the target only when enabling it; either endpoint's reset clears the cache only in its own worker). MCP, skill, and embedded-client writers hold the process-local `extensions_config_write_lock` plus the sidecar advisory `extensions_config_file_lock` for the complete read-modify-write/reload cycle, then share `atomic_write_extensions_config()`, which writes and fsyncs a same-directory temporary file before `os.replace()` and preserves an existing file's mode and symlink target; failed serialization or replacement leaves the prior config intact and cleans up the temporary file.
|
||||
|
||||
@ -27,6 +27,8 @@ def build_server_params(server_name: str, config: McpServerConfig) -> dict[str,
|
||||
raise ValueError(f"MCP server '{server_name}' with stdio transport requires 'command' field")
|
||||
params["command"] = config.command
|
||||
params["args"] = config.args
|
||||
if config.cwd:
|
||||
params["cwd"] = config.cwd
|
||||
# Add environment variables if present
|
||||
if config.env:
|
||||
params["env"] = config.env
|
||||
|
||||
@ -26,6 +26,17 @@ def test_build_server_params_stdio_success():
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cwd", [None, ""], ids=["null", "empty"])
|
||||
def test_build_server_params_omits_empty_stdio_cwd(cwd: str | None):
|
||||
config = McpServerConfig(command="python", args=["server.py"], cwd=cwd)
|
||||
|
||||
assert build_server_params("local", config) == {
|
||||
"transport": "stdio",
|
||||
"command": "python",
|
||||
"args": ["server.py"],
|
||||
}
|
||||
|
||||
|
||||
def test_extensions_config_resolves_env_variables_inside_nested_collections(monkeypatch):
|
||||
monkeypatch.setenv("MCP_TOKEN", "secret")
|
||||
monkeypatch.delenv("MISSING_TOKEN", raising=False)
|
||||
@ -70,6 +81,16 @@ def test_build_server_params_http_like_success(transport: str):
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("transport", ["sse", "http"])
|
||||
def test_build_server_params_does_not_forward_stdio_cwd_to_remote_transports(transport: str):
|
||||
config = McpServerConfig(type=transport, url="https://example.com/mcp", cwd="/local/server")
|
||||
|
||||
assert build_server_params("remote-server", config) == {
|
||||
"transport": transport,
|
||||
"url": "https://example.com/mcp",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("transport", ["sse", "http"])
|
||||
def test_build_server_params_http_like_requires_url(transport: str):
|
||||
config = McpServerConfig(type=transport, url=None)
|
||||
|
||||
114
backend/tests/test_mcp_cwd.py
Normal file
114
backend/tests/test_mcp_cwd.py
Normal file
@ -0,0 +1,114 @@
|
||||
"""Configured stdio working directories must reach discovery and pooled calls."""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from deerflow.config.paths import Paths
|
||||
from deerflow.mcp import tools as mcp_tools
|
||||
from deerflow.mcp.session_pool import MCPSessionPool
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("relative_script", [True, False], ids=["relative-entrypoint", "relative-tool-input"])
|
||||
async def test_stdio_cwd_from_config_reaches_discovery_and_tool_calls(tmp_path, monkeypatch, relative_script):
|
||||
server_dir = tmp_path / "mcp server"
|
||||
server_dir.mkdir()
|
||||
(server_dir / "marker.txt").write_text("configured-directory", encoding="utf-8")
|
||||
server_path = server_dir / "server.py"
|
||||
server_path.write_text(
|
||||
"""
|
||||
from pathlib import Path
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
|
||||
mcp = FastMCP("cwd-test")
|
||||
|
||||
@mcp.tool()
|
||||
def read_marker() -> str:
|
||||
return Path("marker.txt").read_text(encoding="utf-8")
|
||||
|
||||
mcp.run(transport="stdio")
|
||||
""",
|
||||
encoding="utf-8",
|
||||
)
|
||||
config_path = tmp_path / "extensions_config.json"
|
||||
config_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"mcpServers": {
|
||||
"local": {
|
||||
"command": sys.executable,
|
||||
"args": ["server.py" if relative_script else str(server_path)],
|
||||
"cwd": "$TEST_MCP_CWD",
|
||||
}
|
||||
}
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(config_path))
|
||||
monkeypatch.setenv("TEST_MCP_CWD", str(server_dir))
|
||||
monkeypatch.setattr(mcp_tools, "get_paths", lambda: Paths(tmp_path / "deerflow"))
|
||||
pool = MCPSessionPool()
|
||||
monkeypatch.setattr(mcp_tools, "get_session_pool", lambda: pool)
|
||||
runtime = SimpleNamespace(context={"thread_id": "thread", "user_id": "user"}, config={})
|
||||
|
||||
try:
|
||||
tools = await mcp_tools.get_mcp_tools()
|
||||
assert [tool.name for tool in tools] == ["local_read_marker"]
|
||||
content, _artifact = await tools[0].coroutine(runtime=runtime)
|
||||
assert content[0]["text"] == "configured-directory"
|
||||
finally:
|
||||
await pool.close_all()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"cwd_config",
|
||||
[{}, {"cwd": None}, {"cwd": ""}, {"cwd": "$TEST_UNSET_MCP_CWD"}],
|
||||
ids=["omitted", "null", "empty", "unset-env"],
|
||||
)
|
||||
async def test_empty_stdio_cwd_preserves_default_working_directories(tmp_path, monkeypatch, cwd_config):
|
||||
launch_dir = tmp_path / "gateway"
|
||||
launch_dir.mkdir()
|
||||
monkeypatch.chdir(launch_dir)
|
||||
monkeypatch.delenv("TEST_UNSET_MCP_CWD", raising=False)
|
||||
server_path = tmp_path / "cwd_server.py"
|
||||
server_path.write_text(
|
||||
"""
|
||||
from pathlib import Path
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
|
||||
mcp = FastMCP("cwd-defaults-test")
|
||||
|
||||
@mcp.tool(description=str(Path.cwd()))
|
||||
def read_cwd() -> str:
|
||||
return str(Path.cwd())
|
||||
|
||||
mcp.run(transport="stdio")
|
||||
""",
|
||||
encoding="utf-8",
|
||||
)
|
||||
config_path = tmp_path / "extensions_config.json"
|
||||
config_path.write_text(
|
||||
json.dumps({"mcpServers": {"local": {"command": sys.executable, "args": [str(server_path)], **cwd_config}}}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(config_path))
|
||||
paths = Paths(tmp_path / "deerflow")
|
||||
monkeypatch.setattr(mcp_tools, "get_paths", lambda: paths)
|
||||
pool = MCPSessionPool()
|
||||
monkeypatch.setattr(mcp_tools, "get_session_pool", lambda: pool)
|
||||
runtime = SimpleNamespace(context={"thread_id": "thread", "user_id": "user"}, config={})
|
||||
|
||||
try:
|
||||
tools = await mcp_tools.get_mcp_tools()
|
||||
assert [tool.name for tool in tools] == ["local_read_cwd"]
|
||||
assert Path(tools[0].description) == launch_dir.resolve()
|
||||
content, _artifact = await tools[0].coroutine(runtime=runtime)
|
||||
assert Path(content[0]["text"]) == paths.sandbox_work_dir("thread", user_id="user").resolve()
|
||||
finally:
|
||||
await pool.close_all()
|
||||
Loading…
x
Reference in New Issue
Block a user