deer-flow/backend/packages/harness/deerflow/mcp/AGENTS.md

MCP System (packages/harness/deerflow/mcp/)

• Uses langchain-mcp-adapters MultiServerMCPClient for multi-server management

• Long-running task foundation: mcp/tasks/ defines the protocol-neutral McpTaskDriver contract and normalized TaskSnapshot states (submitted, working, input_required, completed, failed, cancelled). A driver-supplied poll_after_seconds must be a finite positive number, validated at the TaskSnapshot boundary so every driver is held to the same invariant rather than each one guarding the consumer that turns the interval into a timedelta. persistence/mcp_tasks/ owns the durable remote-handle mapping, poll schedule, notification state, lease owner, and a consecutive poll-error counter (incremented on failed polls, reset on any applied snapshot — the total poll_attempt_count grows on every claim and cannot distinguish failure streaks) for a later driver-layer backoff/terminal-failure policy; app/mcp_tasks/McpTaskService performs status calls outside the Agent/LLM loop. A status result is applied only when the worker still owns an unexpired lease, so a stale result cannot be written after expiry even before another worker reclaims the row. Poll timestamps and retry schedules are based on the remote call's completion time rather than the scan start. If submission succeeds but persistence fails, the service best-effort cancels the remote task and preserves the original persistence error if that compensation also fails. The exact uq_mcp_tasks_user_server_remote conflict is different: an existing durable row already owns the remote handle, so the conflict surfaces without cancelling that tracked task. Unexpected per-task poll failures are isolated from sibling claims and remain recoverable through lease expiry; Gateway shutdown cancels the poller so a hung external status call cannot block process exit. input_required and terminal states stop polling and become notification_status=pending for later Agent/UI delivery. Durable recovery requires a SQL database backend (sqlite or postgres); the in-memory backend leaves the repository/service unavailable. The runtime is startup-configured by mcp_tasks and disabled by default until a concrete driver is registered; this foundation does not alter ordinary MCP tool behavior on its own.

• Long-running ordinary task driver: extensions_config.json -> mcpServers.<server>.task_toolsets binds exact raw submit/status/cancel names; one raw tool may occupy only one role across that server's groups. mcp/tools.py hides status/cancel and replaces submit with a wrapper that returns only the local task ID after persistence. ordinary.py reads only MCP structuredContent, maps remote running to working, and treats error_code=task_not_found or malformed structured output as permanent failure. A status call with isError=true is a retryable call failure: the first text content block is retained as a bounded diagnostic, while a permanent remote-task outcome must arrive in a normal result with structured status=failed. task_tool_caller.py restores the same (server_name, user_id:thread_id) stdio session scope; HTTP/SSE calls remain ephemeral, apply session_init_timeout to initialization and tool_call_timeout to task calls, and support server-level OAuth refresh outside an Agent run. McpTaskService exponentially backs off transient errors without a maximum attempt count, derives API tracking_degraded from the consecutive-error threshold, keeps input_required on a slower poll, and caps finite positive remote poll hints at 24 hours. Task-enabled server runtime/binding configuration and mcpInterceptors are frozen to the Gateway startup snapshot; hot drift fails clearly before tool discovery can diverge from background calls, while presentation-only fields and non-task servers remain reloadable. Configured task toolsets fail startup when the runtime is disabled or persistence is memory. This stage intentionally has no completion wake-up, natural-language cancellation, ThreadState projection, or frontend panel.

• Durable task payload bounds: persisted task errors are capped at 4,000 characters. input_required and result_artifact must each serialize as valid JSON within 64 KiB; an invalid or oversized payload becomes a permanent protocol failure rather than being truncated and changing its semantics. Remote task IDs/task names are limited to 255 characters and task-enabled server names to 128, matching the SQL schema; an oversized submitted remote ID is rejected only after the Service has the handle so compensation cancellation still runs. Oversized results retain the existing bounded preview/truncation/artifact behavior.

• Lazy initialization: Tools loaded on first use via get_cached_mcp_tools()

• Cache invalidation: Detects extensions-config changes by comparing the resolved config path and a (mtime, size, sha256) content signature against the values recorded at initialization, not a strict mtime > comparison. This catches same-second edits, mtime that stays put or moves backward (git checkout, cp -p / backup restore, tar / rsync, object-store / network mounts), and a switch to a different config file with an equal-or-older mtime. The signature helper (config/file_signature.py::get_config_signature) is shared with config/app_config.py::get_app_config() for the sibling runtime-editable config file, rather than each maintaining its own copy. ExtensionsConfig.resolve_config_path() raises FileNotFoundError for an explicit config_path/DEER_FLOW_EXTENSIONS_CONFIG_PATH that points at a missing file — an operator-asserted path going missing is a real misconfiguration, so this is intentionally loud for callers that load the config for actual use (e.g. from_file() via get_mcp_tools()); only the fallback search mode returns None. The MCP cache's own path resolution (mcp/cache.py::_resolve_config_path) is narrower: it catches that specific FileNotFoundError locally and treats it the same as "unconfigured", so this staleness check degrades to "not stale" instead of propagating an exception when a previously-valid explicit/env-var config disappears mid-run

• Transports: stdio (command-based), SSE, HTTP

• Per-server tool-name prefixing: mcpServers.<server>.tool_name_prefix defaults to true, preserving the collision-safe <server_name>_ prefix. Servers whose tools already carry a stable namespace may set it to false; discovery then calls langchain_mcp_adapters.tools.load_mcp_tools with that server's flag. Source routing and stdio session-pool wrapping are based on the producing server and transport, never on whether the visible tool name starts with the server prefix.

• OAuth (HTTP/SSE): Supports token endpoint flows (client_credentials, refresh_token) with automatic token refresh + Authorization header injection

• Routing hints: extensions_config.json -> mcpServers.<server>.routing and tools.<original_tool_name>.routing are soft preference metadata. The effective routing is resolved while mcp/tools.py::get_mcp_tools() still has both source_name and the original MCP tool name, then stored on tool.metadata under deerflow_mcp_routing. Prompt rendering uses tools/builtins/tool_search.py::get_mcp_routing_hints_prompt_section, which references tool_search when a hinted MCP tool is currently deferred; do not add a parallel routing middleware for PR1-style preference hints.

• Stdio file outputs: Persistent stdio sessions are scoped by user_id:thread_id. For stdio transports only, DeerFlow pins the subprocess default cwd to the thread workspace and TMPDIR/TMP/TEMP to workspace/.mcp/tmp/, unless the operator explicitly configured cwd or temp env values. SSE/HTTP transports skip this filesystem prep entirely.

• Stdio path translation: MCP-returned local file references are not copied. If a ResourceLink or conservative free-text path resolves to an existing file inside the thread's mounted user-data tree, it is translated deterministically to /mnt/user-data/...; paths outside that tree remain unchanged.

• Runtime updates: Gateway API saves to extensions_config.json; the Gateway-embedded runtime detects changes via the resolved-path + content-signature check above, so multi-worker / stale-mtime deployments still pick up an added/removed MCP server without a restart (PUT /api/mcp/config keeps whole-payload validation, while PATCH /api/mcp/config changes only one server's enabled field, normalizes the same type/MCP-spec transport alias as the runtime config model, and validates the target only when enabling it; either endpoint's reset clears the cache only in its own worker). MCP, skill, and embedded-client writers share atomic_write_extensions_config(), which writes and fsyncs a same-directory temporary file before os.replace() and preserves an existing file's mode and symlink target; failed serialization or replacement leaves the prior config intact and cleans up the temporary file.

• Stdio launch policy at the HTTP boundary (routers/mcp.py::_validate_mcp_update_request, shared by PUT and the enable branch of PATCH): a config file may express anything, but the API is untrusted input, so an API-registered stdio server must (a) name a bare executable from the allowlist — _DEFAULT_MCP_STDIO_COMMAND_ALLOWLIST = {npx, uvx}, extended by DEER_FLOW_MCP_STDIO_COMMAND_ALLOWLIST, with path separators, whitespace, and shell metacharacters rejected in command; (b) carry no args flag in _ARBITRARY_EXEC_ARGS; and (c) set no env name in _CODE_INJECTING_ENV_VARS. Checks (b) and (c) exist because the command check alone names a binary without constraining what that binary runs. The env denylist applies to every allowlisted command, and both denylists match --flag=value as well as --flag value. The args denylist's scope depends on the command, because where a launcher stops parsing its own flags is what decides whether a token is an exec flag at all:

  • For a package launcher in _PACKAGE_LAUNCHERS ({npx, uvx}) only the launcher's own option region is screened. npx/uvx stop parsing their flags at the package name and hand every later token to the spawned server's argv, where -c is routinely "config" and -e "env" — screening those rejected ordinary third-party servers while covering nothing. A bare -- ends the region too: only the first token after it is the package name. Finding that boundary needs each launcher's option arity, since a value is not a positional — npx -p <pkg> -c '<command>' runs the command (-p is npm exec's --package, so <pkg> is its value and npm keeps parsing), so ending the region at the first non-flag token would walk straight past it. _NPX_BOOLEAN_ARGS is generated from @npmcli/config's definitions (npm 10.9.4) minus the -p exec override; _UVX_VALUE_ARGS comes from uvx --help (uv 0.11.1). Regenerate these against a newer launcher rather than hand-editing. The unknown-option default is deliberately opposite per launcher, following the exec set rather than symmetry: npx owns real exec flags (-c/--call), so an unknown option consumes a value and keeps the region open (npm errors on options it does not define, so this cannot reject a working invocation); uvx owns no string-eval flag at all, so its screen is a tripwire, an unknown option consumes nothing, and uv's large boolean surface cannot over-block. uvx's exec set also drops the short spellings, because -c is uv's --constraints and -p its --python.
  • Every other command is screened whole, with two extra rules, because it is an interpreter rather than a package runner: -p counts as an exec flag there (node's --print), and single-dash short-option clusters are decomposed letter by letter so node -pe cannot pass a check that only splits on =.

  Verdicts are pinned against the real launchers: for npx, every argument vector the validator rejects is one npx actually executes, and every vector it allows is one npx passes through to the server. env screening covers names that execute code unconditionally at process startup, e.g. PYTHONPATH/PYTHONHOME, which run a caller-controlled sitecustomize.py at interpreter startup under plain uvx. Caller-controlled search paths are a weaker, conditional class and are an accepted residual: LD_LIBRARY_PATH/DYLD_LIBRARY_PATH (conditional on the process loading a shadowable library, and legitimately set by native-dependency servers) and NODE_PATH (searched after the local node_modules chain, so it cannot shadow an installed dependency, and ignored entirely by ESM import — it can only supply a CJS module that would otherwise fail to resolve). Do not move a search path into the set: it would make the "unconditional" rule untrue, which is how a defense-in-depth list starts being mistaken for a boundary. Remote transports skip all three — they spawn nothing. This is defense in depth, not a trust boundary. npx/uvx exist to fetch and execute remote packages, so an admin can still point one at a package they published; the boundary is admin authentication plus network reachability. Do not add a check here on the assumption that it makes MCP registration safe for untrusted admins — it does not, and the fix for that is not a bigger denylist.