fix(groundroute): contain malformed web search and fetch payloads (#6603)

* fix(groundroute): contain malformed web search and fetch payloads

web_search and web_fetch assumed the GroundRoute 200 body was an object with
a `results` list of objects, and read it outside the request try/except. A
body that arrived as a bare list, as an object with a non-list `results`, or
with non-object entries raised AttributeError / KeyError out of the tool
boundary, so the tool call failed with a stack trace instead of the provider's
format error.

Validate the payload before normalizing it, following the containment rule
already applied to brave (#6550), serper, serply, sofya, tencent_wsa and
unbrowse: a non-object payload, a non-list `results` container, or a non-empty
list holding no objects returns the structured format error, non-object entries
in an otherwise valid list are skipped in order, and missing, null or empty
results keep returning "No results found".

* fix(groundroute): coerce non-string fetch fields

web_fetch sliced the fetched page's `content` (falling back to `snippet`)
to the snippet limit without checking its type, so a 200 body whose result
object held a number, boolean, or object there raised TypeError / KeyError
out of the tool boundary -- the same escape the previous commit closed for
the container shape. A null title also rendered as "# None".

Coerce non-string content to text before truncating, and render a missing
or non-string title as empty, matching the coercion sofya and unbrowse
already apply to their fetch fields.

* docs(groundroute): scope the falsy-title note to actual behavior

The guidance said a non-string title renders as empty, but the code only
empties falsy titles; a truthy non-string title still renders as its text
form via the f-string. Say "falsy" instead.

README.md

@@ -174,6 +174,8 @@ streams simulate chunks from a non-streaming response, while no-tool streams sta
 
    Brave web search preserves valid entries in mixed result lists. Malformed response containers or lists containing no usable entries return a structured format error; missing, null, or empty results keep the existing "No results found" response. Format errors also log the malformed container's path and type, or the absence of usable result objects, without including search queries, credentials, or payload values.
 
+   GroundRoute web search and fetch contain malformed HTTP-success payloads the same way: a non-object payload, a `results` container that is not a list, or a non-empty list with no object entries returns the provider's format error, while non-object entries in an otherwise valid list are skipped in order. Missing, null, or empty results keep the existing "No results found" response.
+
    Jina, Browserless, and InfoQuest web fetches resolve relative links and image sources using the requested page URL (or a usable HTML base URL), so returned Markdown includes complete destinations. Link resolution preserves the surrounding HTML source, including malformed-page formatting.
 
    Jina fetches support opt-in bounded retries via `max_retries` (default `0`) and `retry_budget_seconds` (default `30`) in the tool configuration. Valid `Retry-After` hints set a minimum wait for HTTP 429/503; 429 without a valid hint stays terminal. Hints that cannot fit the remaining budget stop retries. Local backoff remains randomized. Retries may increase upstream requests and cost; see [Jina fetch retries](backend/docs/CONFIGURATION.md#jina-fetch-retries).

backend/packages/harness/deerflow/community/groundroute/AGENTS.md

@@ -0,0 +1,22 @@
+# GroundRoute
+
+Validate an HTTP-success payload before normalizing it. A top level that is not an
+object, a `results` container that is not a list, or a non-empty list whose entries
+hold no objects is a format error, not an empty search. Skip non-object entries while
+preserving valid objects in order. Missing or null `results` and an empty list keep
+"No results found".
+
+`web_fetch` coerces non-string `content` (or its `snippet` fallback) to text before
+truncating to the snippet limit, and renders a missing or falsy title as empty, so a
+well-formed result object whose leaf fields are not strings still returns text rather
+than raising.
+
+`web_search` returns the structured `{"error": ..., "query": ...}` response; `web_fetch`
+returns the `Error: ...` string. Log the malformed container's type, or that no usable
+result objects remain, before returning — never the query, URL, credentials, or payload
+values.
+
+`tests/test_groundroute_response_shapes.py` exercises the real tools through an offline
+HTTPX transport with synthetic JSON bodies. Patch the provider's `httpx` reference with a
+local client factory and `HTTPStatusError`; never replace the shared `httpx.Client`.
+Cover both the unchanged global client and HTTP error handling.

backend/packages/harness/deerflow/community/groundroute/tools.py

@@ -71,6 +71,34 @@ def _missing_key_error(tool_name: str, **context: str) -> str:
     return json.dumps({"error": "GROUNDROUTE_API_KEY is not configured", **context}, ensure_ascii=False)
 
 
+def _unexpected_format_error(query: str) -> str:
+    return json.dumps({"error": "GroundRoute returned an unexpected response format", "query": query}, ensure_ascii=False)
+
+
+def _response_results(data: object) -> list[dict] | None:
+    """Return the result objects of a GroundRoute response, or None if malformed.
+
+    A missing or null ``results`` is an empty search. A non-list container, or a
+    non-empty list holding no objects, is malformed: the caller reports a format
+    error rather than "No results found", which would tell the agent the query
+    legitimately matched nothing.
+    """
+    if not isinstance(data, dict):
+        logger.error("GroundRoute returned unexpected payload type: %s", type(data).__name__)
+        return None
+    results = data.get("results")
+    if results is None:
+        return []
+    if not isinstance(results, list):
+        logger.error("GroundRoute returned unexpected 'results' payload type: %s", type(results).__name__)
+        return None
+    usable = [item for item in results if isinstance(item, dict)]
+    if results and not usable:
+        logger.error("GroundRoute returned 'results' with no usable result objects")
+        return None
+    return usable
+
+
 def _post_search(api_key: str, body: dict) -> dict:
     with httpx.Client(timeout=_TIMEOUT_S) as client:
         response = client.post(
@@ -116,7 +144,9 @@ def web_search_tool(query: str, max_results: int | None = None) -> str:
         logger.error("GroundRoute search failed: %s: %s", type(e).__name__, e)
         return json.dumps({"error": str(e), "query": query}, ensure_ascii=False)
 
-    results = data.get("results") or []
+    results = _response_results(data)
+    if results is None:
+        return _unexpected_format_error(query)
     if not results:
         return json.dumps({"error": "No results found", "query": query}, ensure_ascii=False)
 
@@ -156,11 +186,14 @@ def web_fetch_tool(url: str) -> str:
         logger.error("GroundRoute fetch failed: %s: %s", type(e).__name__, e)
         return f"Error: {e}"
 
-    results = data.get("results") or []
+    results = _response_results(data)
+    if results is None:
+        return "Error: GroundRoute returned an unexpected response format"
     if not results:
         return "Error: No results found"
 
     result = results[0]
     content = result.get("content") or result.get("snippet") or ""
-    title = result.get("title", "")
+    content = content if isinstance(content, str) else str(content)
+    title = result.get("title") or ""
     return f"# {title}\n\n{content[:_FETCH_SNIPPET_LIMIT]}"

backend/tests/test_agent_guidance_check.py

@@ -21,6 +21,7 @@ EXPECTED_GUIDANCE_PATHS = {
     "backend/packages/harness/deerflow/agents/memory/AGENTS.md",
     "backend/packages/harness/deerflow/agents/task_continuity/AGENTS.md",
     "backend/packages/harness/deerflow/community/brave/AGENTS.md",
+    "backend/packages/harness/deerflow/community/groundroute/AGENTS.md",
     "backend/packages/harness/deerflow/community/jina_ai/AGENTS.md",
     "backend/packages/harness/deerflow/community/ragflow/AGENTS.md",
     "backend/packages/harness/deerflow/community/serper/AGENTS.md",

backend/tests/test_groundroute_response_shapes.py

@@ -0,0 +1,178 @@
+"""GroundRoute web_search/web_fetch must contain malformed HTTP-success payloads."""
+
+import json
+import logging
+from types import SimpleNamespace
+
+import httpx
+import pytest
+
+_LOGGER_NAME = "deerflow.community.groundroute.tools"
+_FORMAT_ERROR = "GroundRoute returned an unexpected response format"
+_SEARCH_QUERY = "文档 search"
+_FETCH_URL = "https://example.com/文档"
+
+
+@pytest.fixture(autouse=True)
+def reset_api_key_warned():
+    import deerflow.community.groundroute.tools as gr_mod
+
+    gr_mod._api_key_warned = set()
+    yield
+    gr_mod._api_key_warned = set()
+
+
+def _invoker(monkeypatch, tool_name):
+    """Patch the provider's httpx reference with a local MockTransport client factory."""
+    from deerflow.community.groundroute import tools
+
+    config = SimpleNamespace(get_tool_config=lambda _: SimpleNamespace(model_extra={"api_key": "synthetic-test-key"}))
+    monkeypatch.setattr(tools, "get_app_config", lambda: config)
+    real_client = httpx.Client
+
+    def invoke(payload, *, status_code=200):
+        requests = []
+
+        def respond(request):
+            requests.append(request)
+            return httpx.Response(status_code, content=json.dumps(payload).encode("utf-8"), headers={"Content-Type": "application/json"})
+
+        transport = httpx.MockTransport(respond)
+        monkeypatch.setattr(
+            tools,
+            "httpx",
+            SimpleNamespace(Client=lambda **kwargs: real_client(transport=transport, **kwargs), HTTPStatusError=httpx.HTTPStatusError),
+        )
+        if tool_name == "web_search":
+            result = tools.web_search_tool.invoke({"query": _SEARCH_QUERY})
+        else:
+            result = tools.web_fetch_tool.invoke({"url": _FETCH_URL})
+        assert len(requests) == 1
+        return result
+
+    return invoke
+
+
+@pytest.fixture
+def invoke_search(monkeypatch):
+    return _invoker(monkeypatch, "web_search")
+
+
+@pytest.fixture
+def invoke_fetch(monkeypatch):
+    return _invoker(monkeypatch, "web_fetch")
+
+
+def _search(invoke_search, payload, *, status_code=200):
+    return json.loads(invoke_search(payload, status_code=status_code))
+
+
+def _error_logs(caplog):
+    return [record.getMessage() for record in caplog.records if record.name == _LOGGER_NAME and record.levelno == logging.ERROR]
+
+
+def _assert_no_payload_values_leaked(caplog):
+    assert "文档 search" not in caplog.text
+    assert "example.com" not in caplog.text
+    assert "synthetic-test-key" not in caplog.text
+    assert "provider-private-payload" not in caplog.text
+
+
+@pytest.mark.parametrize("payload", [None, False, 0, "invalid", [], ["provider-private-payload"]])
+def test_malformed_top_level_payload_returns_format_error(invoke_search, payload, caplog):
+    assert _search(invoke_search, payload) == {"error": _FORMAT_ERROR, "query": "文档 search"}
+    assert _error_logs(caplog) == [f"GroundRoute returned unexpected payload type: {type(payload).__name__}"]
+    _assert_no_payload_values_leaked(caplog)
+
+
+@pytest.mark.parametrize("results", [False, 0, "", "provider-private-payload", {}, {"title": "provider-private-payload"}])
+def test_malformed_results_container_returns_format_error(invoke_search, results, caplog):
+    assert _search(invoke_search, {"results": results}) == {"error": _FORMAT_ERROR, "query": "文档 search"}
+    assert _error_logs(caplog) == [f"GroundRoute returned unexpected 'results' payload type: {type(results).__name__}"]
+    _assert_no_payload_values_leaked(caplog)
+
+
+@pytest.mark.parametrize("results", [[None], [False, 0, "provider-private-payload", []]])
+def test_all_malformed_entries_return_format_error(invoke_search, results, caplog):
+    assert _search(invoke_search, {"results": results}) == {"error": _FORMAT_ERROR, "query": "文档 search"}
+    assert _error_logs(caplog) == ["GroundRoute returned 'results' with no usable result objects"]
+    _assert_no_payload_values_leaked(caplog)
+
+
+def test_mixed_entries_preserve_valid_results_in_order(invoke_search, caplog):
+    payload = {"results": [None, {"title": "文档", "url": "https://other.example/one", "snippet": "概述", "source_engine": "serper"}, "invalid", {}, [], {"title": "Second"}]}
+    assert _search(invoke_search, payload) == [
+        {"title": "文档", "url": "https://other.example/one", "snippet": "概述", "source_engine": "serper"},
+        {"title": "", "url": "", "snippet": "", "source_engine": ""},
+        {"title": "Second", "url": "", "snippet": "", "source_engine": ""},
+    ]
+    assert _error_logs(caplog) == []
+
+
+@pytest.mark.parametrize("payload", [{}, {"results": None}, {"request_id": "r1"}, {"results": []}])
+def test_missing_or_empty_results_keep_no_results_response(invoke_search, payload, caplog):
+    assert _search(invoke_search, payload) == {"error": "No results found", "query": "文档 search"}
+    assert _error_logs(caplog) == []
+
+
+def test_valid_results_keep_existing_normalization(invoke_search, caplog):
+    payload = {"results": [{"title": "T", "url": "https://other.example", "snippet": "s", "source_engine": "exa"}, {}]}
+    assert _search(invoke_search, payload) == [
+        {"title": "T", "url": "https://other.example", "snippet": "s", "source_engine": "exa"},
+        {"title": "", "url": "", "snippet": "", "source_engine": ""},
+    ]
+    assert _error_logs(caplog) == []
+
+
+@pytest.mark.parametrize("payload", [None, False, [], "provider-private-payload", {"results": {"0": "provider-private-payload"}}, {"results": [None]}])
+def test_fetch_malformed_payload_returns_format_error(invoke_fetch, payload, caplog):
+    assert invoke_fetch(payload) == "Error: GroundRoute returned an unexpected response format"
+    assert len(_error_logs(caplog)) == 1
+    _assert_no_payload_values_leaked(caplog)
+
+
+def test_fetch_missing_or_empty_results_keep_no_results_error(invoke_fetch, caplog):
+    assert invoke_fetch({"results": []}) == "Error: No results found"
+    assert invoke_fetch({}) == "Error: No results found"
+    assert _error_logs(caplog) == []
+
+
+def test_fetch_skips_malformed_entries_and_returns_the_first_usable_result(invoke_fetch):
+    assert invoke_fetch({"results": [None, {"title": "Page", "content": "Body text", "url": "https://other.example"}]}) == "# Page\n\nBody text"
+
+
+def test_valid_fetch_keeps_existing_response(invoke_fetch, caplog):
+    assert invoke_fetch({"results": [{"title": "Page", "content": "Body text"}]}) == "# Page\n\nBody text"
+    assert _error_logs(caplog) == []
+
+
+@pytest.mark.parametrize(
+    ("payload", "expected"),
+    [
+        ({"results": [{"title": "Page", "content": 12345}]}, "# Page\n\n12345"),
+        ({"results": [{"title": "Page", "content": True}]}, "# Page\n\nTrue"),
+        ({"results": [{"title": "Page", "content": {"a": 1}}]}, "# Page\n\n{'a': 1}"),
+        ({"results": [{"title": "Page", "snippet": 6789}]}, "# Page\n\n6789"),
+        ({"results": [{"title": None, "content": "Body text"}]}, "# \n\nBody text"),
+    ],
+)
+def test_fetch_coerces_non_string_fields(invoke_fetch, payload, expected, caplog):
+    assert invoke_fetch(payload) == expected
+    assert _error_logs(caplog) == []
+
+
+def test_http_transport_patch_does_not_replace_global_client(invoke_search):
+    real_client = httpx.Client
+    invoke_search({"results": []})
+    assert httpx.Client is real_client
+    with httpx.Client(transport=httpx.MockTransport(lambda request: httpx.Response(204))) as client:
+        assert client.get("https://unrelated.invalid/probe").status_code == 204
+
+
+@pytest.mark.parametrize("status_code", [402, 503])
+def test_search_http_error_keeps_existing_structured_response(invoke_search, status_code):
+    assert _search(invoke_search, {"error": "synthetic-denial"}, status_code=status_code) == {"error": f"GroundRoute API error: HTTP {status_code}", "query": "文档 search"}
+
+
+def test_fetch_http_error_keeps_existing_error_string(invoke_fetch):
+    assert invoke_fetch({"error": "synthetic-denial"}, status_code=402) == "Error: GroundRoute API error: HTTP 402"