fix(mcp): skip task servers whose discovery failed instead of dropping all tools (#6481)

* fix(mcp): skip task servers whose discovery failed instead of dropping all tools

Per-server discovery replaced a failed or timed-out server with an empty
tool list. For a server with task_toolsets, _configure_task_tools_for_server
then reported submit/status/cancel as missing and raised
McpTaskConfigurationError, which get_mcp_tools re-raises, so the tools of
every healthy server were discarded and the cache was never published:
each agent build rediscovered every server.

load_server_tools now returns None for a failed discovery, distinct from a
server that answered with zero tools, and the wrapping loop skips such
servers before task-toolset validation. A server that answers without its
configured raw tools still raises as a configuration error.

* docs(changelog): reference #6481

* docs(mcp): name the cache-reset endpoint as a task-server recovery path

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

CHANGELOG.md

@@ -880,6 +880,15 @@ This release closes that milestone with **439 merged pull requests**.
   goal." and `get_goal`/`set_goal`/`clear_goal` raised. The goal helpers now
   fall back to the synchronous methods for those savers. The web UI was not
   affected. ([#6448])
+- **mcp:** An MCP server with `task_toolsets` that is unreachable or times out
+  during tool discovery no longer removes every MCP tool. Discovery skipped the
+  failed server with an empty tool list, the task-toolset check then reported
+  its submit, status and cancel tools as missing, and the resulting error
+  discarded the tools of every healthy server. Because the cache was never
+  published, each agent build repeated discovery for all servers, respawning
+  stdio servers and re-requesting OAuth tokens. A server whose discovery fails
+  is now skipped like any other failed server; a server that answers without
+  its configured tools still fails as a configuration error. ([#6481])
 - **frontend:** A failed side-chat send no longer clears the composer. The side
   chat's submit handler showed the error toast and then resolved, which the
   composer treats as success, so the typed text and attachments were lost when
@@ -9192,3 +9201,4 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
 [#6447]: https://github.com/bytedance/deer-flow/pull/6447
 [#6448]: https://github.com/bytedance/deer-flow/pull/6448
 [#6450]: https://github.com/bytedance/deer-flow/pull/6450
+[#6481]: https://github.com/bytedance/deer-flow/pull/6481

CHANGELOG_zh.md

@@ -721,6 +721,10 @@
   同步检查点保存器定义了异步方法，但调用时会抛出 `NotImplementedError`，而目标读写只要异步方法存在就会调用它，因此
   `/goal` 只会显示 "Could not set goal."，`get_goal`/`set_goal`/`clear_goal` 则直接抛出异常。现在目标读写在这类保存器上
   会改用同步方法。Web UI 不受影响。([#6448])
+- **mcp：** 配置了 `task_toolsets` 的 MCP 服务器在工具发现时无法连接或超时，不再导致所有 MCP 工具丢失。此前发现阶段以空
+  工具列表跳过失败的服务器，随后任务工具集检查把它的 submit、status、cancel 工具报告为缺失，抛出的错误连同所有正常服务器
+  的工具一起丢弃；由于缓存从未发布，每次构建智能体都会重新发现全部服务器，重启 stdio 服务器并重新请求 OAuth 令牌。现在发现
+  失败的服务器与其他失败服务器一样被跳过；已应答但缺少所配置工具的服务器仍按配置错误处理。([#6481])
 - **前端：** 侧边对话发送失败时不再清空输入框。侧边对话的提交处理在弹出错误提示后仍以成功返回，输入框据此视为成功，
   因此在创建侧边对话或上传附件失败时，已输入的文字和附件都会丢失；发往新侧边对话的第一条消息也会在排队时（实际发送前）
   就被清空。现在提交处理会在提示后抛出错误，排队的首条发送也以其自身结果完成提交，因此草稿会保留以便重试，只有消息
@@ -7523,3 +7527,4 @@ DeerFlow 2.0 是围绕"超级智能体"框架的彻底重写，核心包含子
 [#6441]: https://github.com/bytedance/deer-flow/pull/6441
 [#6448]: https://github.com/bytedance/deer-flow/pull/6448
 [#6450]: https://github.com/bytedance/deer-flow/pull/6450
+[#6481]: https://github.com/bytedance/deer-flow/pull/6481

backend/docs/MCP_SERVER.md

@@ -396,6 +396,15 @@ the task alive and recognize its ID after DeerFlow reconnects. A stdio server
 must therefore persist its own tasks; multi-instance deployments should
 normally use an independently running HTTP/SSE service.
 
+A task-enabled server that fails or times out during tool discovery is skipped
+like any other MCP server: its submit tool is absent, other servers keep their
+tools, and the server returns on the next MCP tool reload. After the server
+recovers, an administrator can force that reload with the cache-reset endpoint
+(`POST /api/mcp/cache/reset`, described above) instead of editing the config or
+restarting the Gateway; a config change or restart also reloads it. A server
+that answers discovery without one of its configured raw tools is a
+configuration error instead: no MCP tools load until the names match.
+
 For deployment-level HTTP/SSE servers with `task_toolsets`, discovery, ordinary
 tool calls, and background submit/status/cancel calls share the cached access
 token, rotated refresh token, and refresh lock for one Gateway process lifetime.

backend/packages/harness/deerflow/mcp/AGENTS.md

@@ -5,7 +5,7 @@
 - **Long-running task foundation**: `mcp/tasks/` defines the protocol-neutral `McpTaskDriver` contract and normalized `TaskSnapshot` states (`submitted`, `working`, `input_required`, `completed`, `failed`, `cancelled`). A driver-supplied `poll_after_seconds` must be a finite positive number, validated at the `TaskSnapshot` boundary so every driver is held to the same invariant rather than each one guarding the consumer that turns the interval into a `timedelta`. `persistence/mcp_tasks/` owns the durable remote-handle mapping, poll schedule, notification state, lease owner, and separate consecutive poll/delivery error counters; `app/mcp_tasks/McpTaskService` performs status, cancellation, and notification work outside the Agent/LLM loop. Notification retries keep their idempotency attempt separate from the delivery-failure count, use capped exponential backoff, and stop after five failures; strict existing-thread admission dead-letters a deleted/mismatched target immediately. A status result is applied only when the worker still owns an unexpired lease, so a stale result cannot be written after expiry even before another worker reclaims the row. Every claim carries a fresh per-claim token, and every poll/cancel/notification release or snapshot apply is fenced by the owner plus that token, so a release that completes after the same service reclaims the row cannot clear the newer lease. Poll, cancel, and notification claims also have separate phase-level single-flight owners that protect only an ambiguous claim outcome: after an uncancelled timeout or caller cancellation, later scans do not start another claim while the outcome is unknown. Once the claim resolves, the phase owner is released immediately, and any returned rows are released through bounded, service-owned background work; per-claim token fencing prevents a late release from mutating a newer claim generation, so a stuck release does not stall the whole phase. While the claim outcome remains unresolved, later scans skip the phase and emit a warning rather than overlapping an ambiguous database claim. Once the claim resolves, the handoff may continue releasing returned rows in the background without blocking later scans. Lease expiry remains the cross-process crash fallback rather than the normal recovery for a live service's late claim. Poll timestamps and retry schedules are based on the remote call's completion time rather than the scan start. If submission succeeds but persistence fails or the caller is cancelled while persistence is in flight, the service best-effort cancels the remote task and preserves the original error or cancellation if that compensation also fails. The exact `uq_mcp_tasks_user_server_remote` conflict is different: an existing durable row already owns the remote handle, so the conflict surfaces without cancelling that tracked task. Unexpected per-task poll failures are isolated from sibling claims and remain recoverable through lease expiry; Gateway shutdown cancels the poller so a hung external status call cannot block process exit. Routine cancellation releases clear only the lease/token and preserve any pre-existing `last_cancel_error` or `notification_error`; cancellation itself is not persisted as a task failure diagnostic. Cancelling an in-flight poll releases only the owner- and per-claim-token-fenced lease and preserves its preclaim schedule and poll-failure state; real poll failures retain exponential backoff and tracking-degradation behavior. `input_required` and terminal states stop polling and become `notification_status=pending` for later Agent/UI delivery. Durable recovery requires a SQL database backend (`sqlite` or `postgres`); the in-memory backend leaves the repository/service unavailable. The runtime is startup-configured by `mcp_tasks` and disabled by default until a concrete driver is registered; this foundation does not alter ordinary MCP tool behavior on its own.
 - **Runtime availability boundary**: the installed process-local submitter is the source of truth for durable task-management tool exposure. `mcp_tasks` is startup-only; changing it on disk does not alter the live toolset until the Gateway restarts.
 - **Task OAuth lifetime**: `tasks/runtime.py` owns shared OAuth state only for enabled deployment servers with `task_toolsets`, from `set_mcp_task_config_snapshot(config)` until its `None` teardown. Use `_task_enabled_servers` for both snapshot and OAuth selection. Validate the frozen snapshot before reusing that state for discovery, ordinary tools, or durable calls. Cache resets must not discard it. Keep tokens and the cross-loop refresh lock together; rotate a private config copy, never the parsed snapshot or config file. Non-task servers remain reloadable and personal callers use independent managers, even on a name collision. This is not cross-process or cross-restart credential storage.
-- **Long-running ordinary task driver**: `extensions_config.json -> mcpServers.<server>.task_toolsets` binds exact raw submit/status/cancel names; one raw tool may occupy only one role across that server's groups. `mcp/tools.py` hides status/cancel and replaces submit with a wrapper that returns only the local task ID after persistence. `ordinary.py` reads only MCP `structuredContent`, maps remote `running` to `working`, and treats `error_code=task_not_found` or malformed structured output as permanent failure. A status call with `isError=true` is a retryable call failure: the first text content block is retained as a bounded diagnostic, while a permanent remote-task outcome must arrive in a normal result with structured `status=failed`. `task_tool_caller.py` restores the same canonical `(server_name, versioned user/thread/incarnation scope)` stdio session as ordinary calls inside the persisted deployment/personal ownership domain; non-NULL incarnations use an unambiguous versioned encoding, while an explicitly captured legacy NULL retains the pre-activation `user_id:thread_id` scope. A runtime with a missing/invalid incarnation key fails closed. HTTP/SSE calls remain ephemeral, apply `session_init_timeout` to initialization and `tool_call_timeout` to task calls, and support server-level OAuth refresh outside an Agent run. `McpTaskService` exponentially backs off transient status/cancel errors without a maximum attempt count, derives API `tracking_degraded` from the consecutive-error threshold, keeps `input_required` on a slower poll, and caps finite positive remote poll hints at 24 hours. Task-enabled server runtime/binding configuration and `mcpInterceptors` are frozen to the Gateway startup snapshot; hot drift fails clearly before tool discovery can diverge from background calls, while presentation-only fields and non-task servers remain reloadable. Configured task toolsets fail startup when the runtime is disabled or persistence is memory. Users still cannot submit an answer back to an `input_required` remote task.
+- **Long-running ordinary task driver**: `extensions_config.json -> mcpServers.<server>.task_toolsets` binds exact raw submit/status/cancel names; one raw tool may occupy only one role across that server's groups. `mcp/tools.py` hides status/cancel and replaces submit with a wrapper that returns only the local task ID after persistence. `ordinary.py` reads only MCP `structuredContent`, maps remote `running` to `working`, and treats `error_code=task_not_found` or malformed structured output as permanent failure. A status call with `isError=true` is a retryable call failure: the first text content block is retained as a bounded diagnostic, while a permanent remote-task outcome must arrive in a normal result with structured `status=failed`. `task_tool_caller.py` restores the same canonical `(server_name, versioned user/thread/incarnation scope)` stdio session as ordinary calls inside the persisted deployment/personal ownership domain; non-NULL incarnations use an unambiguous versioned encoding, while an explicitly captured legacy NULL retains the pre-activation `user_id:thread_id` scope. A runtime with a missing/invalid incarnation key fails closed. HTTP/SSE calls remain ephemeral, apply `session_init_timeout` to initialization and `tool_call_timeout` to task calls, and support server-level OAuth refresh outside an Agent run. `McpTaskService` exponentially backs off transient status/cancel errors without a maximum attempt count, derives API `tracking_degraded` from the consecutive-error threshold, keeps `input_required` on a slower poll, and caps finite positive remote poll hints at 24 hours. Task-enabled server runtime/binding configuration and `mcpInterceptors` are frozen to the Gateway startup snapshot; hot drift fails clearly before tool discovery can diverge from background calls, while presentation-only fields and non-task servers remain reloadable. Configured task toolsets fail startup when the runtime is disabled or persistence is memory. A task server whose discovery fails or times out is skipped like any other server (`load_server_tools` returns `None`, not `[]`); only a server that answered without its configured raw tools raises `McpTaskConfigurationError`. Users still cannot submit an answer back to an `input_required` remote task.
 - **Durable task payload bounds**: persisted task errors are capped at 4,000 characters. `input_required` and `result_artifact` must each serialize as valid JSON within 64 KiB; an invalid or oversized payload becomes a permanent protocol failure rather than being truncated and changing its semantics. Remote task IDs/task names are limited to 255 characters and task-enabled server names to 128, matching the SQL schema; an oversized submitted remote ID is rejected only after the Service has the handle so compensation cancellation still runs. Oversized results retain the existing bounded preview/truncation/artifact behavior.
 - **Remote task bring-up**: One `session_init_timeout` deadline covers transport context entry and MCP initialization. Report the server and configured bound on expiry; disable it before calling the tool. Enter and exit the adapter's AnyIO context managers in the same task; tests include real SSE endpoint/initialization stalls and connection cleanup.
 - **Background task identity**: `McpTaskToolCaller` binds an ID-only `_TaskOwner` in the existing user ContextVar only for background HTTP/SSE calls with enabled `user_auth`, restoring the previous value in `finally`. The `CurrentUser` contract guarantees only `.id`, not profile fields or roles. This lets `user_auth` select configured credentials without an active Agent run, including recovery from SQL; concurrent owners remain task-local. Submit, stdio, and calls without enabled `user_auth` keep their original context. Do not persist request secrets or enable `headers_from_context` for background status/cancel calls.

backend/packages/harness/deerflow/mcp/tools.py

@@ -986,7 +986,10 @@ async def get_mcp_tools(
             tool_name_prefix=True,
         )
 
-        async def load_server_tools(server_name: str) -> list[BaseTool]:
+        async def load_server_tools(server_name: str) -> list[BaseTool] | None:
+            # ``None`` marks a server whose discovery failed, which is distinct
+            # from a server that answered with zero tools: only the latter is
+            # evidence that configured task_toolsets name missing tools.
             try:
                 server_cfg = extensions_config.mcp_servers.get(server_name)
                 tool_name_prefix = server_cfg.tool_name_prefix if server_cfg is not None else True
@@ -1032,19 +1035,19 @@ async def get_mcp_tools(
                             server_name,
                             session_init_timeout,
                         )
-                        return []
+                        return None
                 return await discovery
             except Exception as e:
                 logger.warning(
                     f"Skipping MCP server '{server_name}' after tool discovery failed: {e}",
                     exc_info=True,
                 )
-                return []
+                return None
 
         # Get tools from each server independently so one broken MCP server does
         # not prevent healthy servers from contributing their tools.
         tools_by_server = await asyncio.gather(*(load_server_tools(name) for name in servers_config))
-        tools = [tool for server_tools in tools_by_server for tool in server_tools]
+        tools = [tool for server_tools in tools_by_server if server_tools is not None for tool in server_tools]
         logger.info(f"Successfully loaded {len(tools)} tool(s) from MCP servers")
 
         # Wrap each tool with persistent-session logic.
@@ -1059,6 +1062,11 @@ async def get_mcp_tools(
         # "web_") matches "web" first), which pools the tool under the wrong server. Using the
         # source grouping makes routing exact even when a server opts out of name prefixing.
         for source_name, server_tools in zip(servers_config.keys(), tools_by_server, strict=True):
+            if server_tools is None:
+                # Discovery already logged the skip. Validating task_toolsets
+                # against a server that never answered would report every
+                # raw tool as missing and discard the healthy servers' tools.
+                continue
             transport = servers_config[source_name].get("transport", "stdio")
             server_cfg = extensions_config.mcp_servers.get(source_name)
             tool_name_prefix = server_cfg.tool_name_prefix if server_cfg is not None else True

backend/tests/test_mcp_cache.py

@@ -1504,3 +1504,41 @@ class TestLazyInitializationFailure:
 
         assert len(calls) == 1
         assert cache_module._cache_initialized is True
+
+
+def test_failed_task_server_discovery_publishes_healthy_tools_once(cache_globals, monkeypatch, tmp_path):
+    """An unreachable task-enabled server must not leave the cache uninitialized."""
+    from langchain_core.tools import StructuredTool
+
+    cfg = tmp_path / "extensions_config.json"
+    toolset = {"name": "reports", "submit_tool": "submit_report", "status_tool": "get_report_status", "cancel_tool": "cancel_report"}
+    _write_extensions_config(
+        cfg,
+        {
+            "healthy": {"type": "http", "url": "http://healthy.invalid/mcp"},
+            "reports": {"type": "http", "url": "http://reports.invalid/mcp", "task_toolsets": [toolset]},
+        },
+    )
+    monkeypatch.setenv("DEER_FLOW_EXTENSIONS_CONFIG_PATH", str(cfg))
+    discovered: list[str] = []
+
+    class FakeClient:
+        def __init__(self, connections, *, callbacks=None, tool_interceptors=None, tool_name_prefix=False):
+            self.callbacks = callbacks
+            self.tool_interceptors = tool_interceptors or []
+
+        async def get_tools(self, *, server_name=None):
+            discovered.append(server_name)
+            if server_name == "reports":
+                raise ConnectionError("reports is down")
+            return [StructuredTool.from_function(lambda: "ok", name="healthy_search", description="search")]
+
+    monkeypatch.setattr("langchain_mcp_adapters.client.MultiServerMCPClient", FakeClient)
+
+    first = cache_module.get_cached_mcp_tools()
+    second = cache_module.get_cached_mcp_tools()
+
+    assert [tool.name for tool in first] == ["healthy_search"]
+    assert second == first
+    assert cache_module._cache_initialized is True
+    assert sorted(discovered) == ["healthy", "reports"]

backend/tests/test_mcp_task_tool_wrapping.py

@@ -1,3 +1,4 @@
+import asyncio
 from types import SimpleNamespace
 from unittest.mock import patch
 
@@ -204,3 +205,68 @@ async def test_tool_reload_rejects_task_server_runtime_config_drift() -> None:
             await get_mcp_tools()
     finally:
         set_mcp_task_config_snapshot(None)
+
+
+def _discovery_config(*, reports_init_timeout: float | None = None) -> ExtensionsConfig:
+    reports = {"type": "http", "url": "http://reports.invalid/mcp", "task_toolsets": _server_config().model_dump()["task_toolsets"]}
+    if reports_init_timeout is not None:
+        reports["session_init_timeout"] = reports_init_timeout
+    return ExtensionsConfig.model_validate(
+        {
+            "mcpServers": {
+                "healthy": {"type": "http", "url": "http://healthy.invalid/mcp"},
+                "reports": reports,
+            }
+        }
+    )
+
+
+def _discovery_client(reports_discovery):
+    class FakeClient:
+        def __init__(self, connections, *, callbacks=None, tool_interceptors=None, tool_name_prefix=False) -> None:
+            self.callbacks = callbacks
+            self.tool_interceptors = tool_interceptors or []
+
+        async def get_tools(self, *, server_name=None):
+            if server_name == "reports":
+                return await reports_discovery()
+            return [_tool("healthy_search")]
+
+    return FakeClient
+
+
+async def _reports_unreachable():
+    raise ConnectionError("reports is down")
+
+
+async def _reports_hangs():
+    await asyncio.sleep(60)
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+    ("reports_discovery", "reports_init_timeout"),
+    [(_reports_unreachable, None), (_reports_hangs, 0.05)],
+    ids=["discovery-error", "discovery-timeout"],
+)
+async def test_failed_task_server_discovery_keeps_healthy_server_tools(reports_discovery, reports_init_timeout) -> None:
+    # A task server that never answered has not shown its task_toolsets to be
+    # wrong; it is skipped like any other failed server instead of raising.
+    with patch("langchain_mcp_adapters.client.MultiServerMCPClient", _discovery_client(reports_discovery)):
+        tools = await asyncio.wait_for(get_mcp_tools(_discovery_config(reports_init_timeout=reports_init_timeout)), timeout=5)
+
+    assert [tool.name for tool in tools] == ["healthy_search"]
+
+
+@pytest.mark.asyncio
+async def test_task_server_answering_with_no_tools_still_fails() -> None:
+    # An empty answer is a successful discovery, so the missing raw tools are a
+    # configuration error rather than an outage.
+    async def reports_without_tools():
+        return []
+
+    with (
+        patch("langchain_mcp_adapters.client.MultiServerMCPClient", _discovery_client(reports_without_tools)),
+        pytest.raises(McpTaskConfigurationError, match="cancel_report, get_report_status, submit_report"),
+    ):
+        await get_mcp_tools(_discovery_config())

backend/tests/test_personal_mcp.py

@@ -580,3 +580,43 @@ async def test_gateway_registers_driver_for_personal_only_task_toolsets(personal
             submitted = app.state.mcp_task_service.submit.await_args.kwargs
             assert submitted["driver_name"] == ORDINARY_MCP_TASK_DRIVER
             assert submitted["request"].driver_data["connection_scope"] == "personal"
+
+
+@pytest.mark.asyncio
+async def test_unreachable_personal_task_server_keeps_other_personal_tools(personal_client):
+    from langchain_core.tools import StructuredTool
+
+    from deerflow.config.extensions_config import atomic_write_extensions_config
+    from deerflow.mcp.user_tools import _load
+
+    toolset = {"name": "reports", "submit_tool": "submit_report", "status_tool": "status_report", "cancel_tool": "cancel_report"}
+    atomic_write_extensions_config(
+        user_mcp_config_path("alice"),
+        {
+            "mcpServers": {
+                "notes": {"type": "http", "url": "https://notes.example.com/mcp"},
+                "reports": {"type": "http", "url": "https://reports.example.com/mcp", "task_toolsets": [toolset]},
+            }
+        },
+    )
+
+    class FakeClient:
+        def __init__(self, servers, *, tool_interceptors, **_kwargs):
+            self.servers = servers
+            self.tool_interceptors = tool_interceptors
+            self.callbacks = None
+
+        async def get_tools(self, *, server_name):
+            if "reports" in self.servers[server_name]["url"]:
+                raise ConnectionError("reports is down")
+            return [StructuredTool.from_function(lambda: "ok", name=f"{server_name}_search", description="search")]
+
+    with (
+        patch("deerflow.mcp.tasks.runtime.is_mcp_task_runtime_available", return_value=True),
+        patch("langchain_mcp_adapters.client.MultiServerMCPClient", FakeClient),
+    ):
+        personal = load_user_mcp_config("alice")
+        tools = await _load("alice", personal)
+
+    notes_name = next(name for name, server in personal.mcp_servers.items() if server.url == "https://notes.example.com/mcp")
+    assert [tool.name for tool in tools] == [f"{notes_name}_search"]