fix(search): surface DuckDuckGo search failures instead of reporting an empty result (#6648)

* fix(search): surface DuckDuckGo failures instead of reporting an empty result

DDGS execution failures (rate limits, timeouts, blocked HTML/vqd extraction,
SDK errors, and the missing ddgs dependency) were caught in _search_text and
converted into an empty list, so web_search reported "No results found" for
every failure. The agent then read an outage as a query that matched nothing
and rewrote the query instead of trying an alternative.

_search_text now raises DDGSearchError carrying the exception type and message
for those failures, and web_search_tool answers them with a structured
"DuckDuckGo search failed: ..." error. A genuinely empty result set (the SDK's
exact DDGSException("No results found.") sentinel) still returns the existing
"No results found" response, and tool_result_meta classifies the two cases
differently (transient/config vs no_results).

Tests: tests/test_ddg_search_tools.py (three new failure tests red on main),
tests/test_agent_guidance_check.py, tests/test_tool_result_meta.py; ruff check
and ruff format --check clean.

* fix(search): keep DDGS client construction inside the error boundary

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>

README.md

@@ -178,6 +178,8 @@ streams simulate chunks from a non-streaming response, while no-tool streams sta
 
    GroundRoute web search and fetch contain malformed HTTP-success payloads the same way: a non-object payload, a `results` container that is not a list, or a non-empty list with no object entries returns the provider's format error, while non-object entries in an otherwise valid list are skipped in order. Missing, null, or empty results keep the existing "No results found" response.
 
+   DuckDuckGo web search distinguishes execution failures from empty results: rate limits, timeouts, blocked requests, client-setup failures, SDK errors, and a missing `ddgs` dependency return a structured `DuckDuckGo search failed: ...` error instead of "No results found", while a genuinely empty result set keeps the existing "No results found" response. The structured error text lets the harness classify an outage as transient or a configuration problem instead of a query that matched nothing.
+   
    Tavily web search and fetch contain malformed HTTP-success payloads the same way: a non-object payload, or a `results`/`failed_results` container that is not a list, returns the provider's format error, and non-object entries in an otherwise valid list are skipped in order. Missing, null, or empty results keep the existing response (`[]` for search, `Error: No results found` for fetch); missing or null search fields (`title`, `url`, `content`) normalize to empty text, and a non-string fetch `raw_content` is coerced to text before the truncation limit.
 
    Sofya web search and fetch contain malformed HTTP-success payloads the same way: a non-object payload, a `results` container that is not a list, or a non-empty list with no object entries returns the provider's format error, while non-object entries in an otherwise valid list are skipped in order. Missing, null, or empty results keep the existing "No results found" response.

backend/packages/harness/deerflow/community/ddg_search/AGENTS.md

@@ -0,0 +1,21 @@
+# DuckDuckGo Search
+
+`web_search` distinguishes a search that matched nothing from one that never
+ran. `_search_text` returns `[]` only for a genuinely empty result set, which
+`ddgs` reports as `DDGSException("No results found.")`, and raises
+`DDGSearchError` for every execution failure: rate limits, timeouts, blocked
+HTML (vqd extraction), SDK errors, and the missing `ddgs` dependency. Building
+the client is part of the boundary too: keep `DDGS(timeout=30)` inside the
+`try` so proxy or client-setup failures surface as the same structured error
+instead of escaping raw.
+`web_search_tool` answers a failure with
+`{"error": "DuckDuckGo search failed: <exception type>: <message>", "query": ...}`
+and keeps `{"error": "No results found", "query": ...}` for the empty result.
+Keep the exception type and message in the error text: `tool_result_meta`
+classifies on it, and an outage downgraded to "No results found" reads as
+`no_results` (rewrite the query) instead of transient or config recovery.
+
+`tests/test_ddg_search_tools.py` exercises the real tool by patching
+`sys.modules["ddgs"]` with a fake `DDGS` whose `__init__` or `text()` raises -
+never the normalizer or the tool's final output. Keep the SDK's "No results
+found." sentinel mapped back to the empty-search response.

backend/packages/harness/deerflow/community/ddg_search/tools.py

@@ -13,6 +13,18 @@ from deerflow.config import get_app_config
 
 logger = logging.getLogger(__name__)
 
+
+class DDGSearchError(RuntimeError):
+    """Raised when a DuckDuckGo search cannot be executed.
+
+    ``_search_text`` returns an empty list when the search ran and matched
+    nothing. Execution failures (client initialization, rate limits,
+    timeouts, blocked requests, SDK errors, a missing ddgs dependency) raise
+    this error instead, so ``web_search_tool`` can report the failure rather
+    than "No results found".
+    """
+
+
 DEFAULT_BACKEND = "auto"
 DEFAULT_REGION = "wt-wt"
 DEFAULT_SAFESEARCH = "moderate"
@@ -125,16 +137,22 @@ def _search_text(
 
     Returns:
         List of search results
+
+    Raises:
+        DDGSearchError: If the search could not be executed, including a
+            ``DDGS`` client that fails to initialize. A genuinely empty
+            result set is not an error and still returns an empty list.
     """
     try:
         from ddgs import DDGS
-    except ImportError:
-        logger.error("ddgs library not installed. Run: pip install ddgs")
-        return []
-
-    ddgs = DDGS(timeout=30)
+    except ImportError as e:
+        raise DDGSearchError("ddgs library not installed. Run: pip install ddgs") from e
 
     try:
+        # Constructing the client can already fail (proxy or client setup), so
+        # it stays inside the boundary that maps failures to DDGSearchError.
+        ddgs = DDGS(timeout=30)
+
         backend = _resolve_time_range_backend(backend) if time_range is not None else _normalize_backend(backend)
         safesearch = _normalize_setting(safesearch, DEFAULT_SAFESEARCH)
         effective_region = _resolve_ddgs_region(query, region, backend)
@@ -150,8 +168,13 @@ def _search_text(
         return list(results) if results else []
 
     except Exception as e:
-        logger.error(f"Failed to search web: {e}")
-        return []
+        # ddgs signals a genuinely empty result set with the exact message
+        # "No results found."; every other failure (rate limit, timeout,
+        # blocked HTML, SDK error) must surface as an error instead, or the
+        # agent reads an outage as "no matches" and rewrites the query.
+        if str(e).strip().rstrip(".!").lower() == "no results found":
+            return []
+        raise DDGSearchError(f"{type(e).__name__}: {e}") from e
 
 
 @tool("web_search", parse_docstring=True)
@@ -179,14 +202,18 @@ def web_search_tool(
         safesearch = config.model_extra.get("safesearch", safesearch)
         backend = config.model_extra.get("backend", backend)
 
-    results = _search_text(
-        query=query,
-        max_results=coerce_max_results(max_results, provider="DDG Search", logger=logger),
-        region=region,
-        safesearch=safesearch,
-        backend=backend,
-        time_range=time_range,
-    )
+    try:
+        results = _search_text(
+            query=query,
+            max_results=coerce_max_results(max_results, provider="DDG Search", logger=logger),
+            region=region,
+            safesearch=safesearch,
+            backend=backend,
+            time_range=time_range,
+        )
+    except DDGSearchError as e:
+        logger.error("DuckDuckGo search failed: %s", e)
+        return json.dumps({"error": f"DuckDuckGo search failed: {e}", "query": query}, ensure_ascii=False)
 
     if not results:
         return json.dumps({"error": "No results found", "query": query}, ensure_ascii=False)

backend/tests/test_agent_guidance_check.py

@@ -27,6 +27,7 @@ EXPECTED_GUIDANCE_PATHS = {
     "backend/packages/harness/deerflow/community/ragflow/AGENTS.md",
     "backend/packages/harness/deerflow/community/serper/AGENTS.md",
     "backend/packages/harness/deerflow/community/tavily/AGENTS.md",
+    "backend/packages/harness/deerflow/community/ddg_search/AGENTS.md",
     "backend/packages/harness/deerflow/community/e2b_sandbox/AGENTS.md",
     "backend/packages/harness/deerflow/community/aio_sandbox/AGENTS.md",
     "backend/packages/harness/deerflow/community/opensandbox/AGENTS.md",

backend/tests/test_ddg_search_tools.py

@@ -213,3 +213,116 @@ def test_web_search_tool_reads_ddgs_options_from_config() -> None:
         backend="auto",
         time_range="week",
     )
+
+
+def test_web_search_tool_reports_search_failure_instead_of_no_results(monkeypatch, caplog) -> None:
+    """A failed search must not masquerade as a genuinely empty result set."""
+
+    class FailingDDGS:
+        def __init__(self, timeout: int) -> None:
+            self.timeout = timeout
+
+        def text(self, query: str, **kwargs):
+            msg = "HTTP 429 Too Many Requests"
+            raise RuntimeError(msg)
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=FailingDDGS))
+    monkeypatch.setattr(tools, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: None))
+    caplog.set_level(logging.ERROR, logger=tools.__name__)
+
+    parsed = json.loads(tools.web_search_tool.invoke({"query": "latest news"}))
+
+    assert parsed["query"] == "latest news"
+    assert "No results found" not in parsed["error"]
+    assert "429" in parsed["error"]
+    assert any(record.getMessage().startswith("DuckDuckGo search failed") for record in caplog.records)
+
+
+def test_web_search_tool_reports_missing_ddgs_dependency(monkeypatch) -> None:
+    monkeypatch.setitem(sys.modules, "ddgs", None)
+    monkeypatch.setattr(tools, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: None))
+
+    parsed = json.loads(tools.web_search_tool.invoke({"query": "latest news"}))
+
+    assert "not installed" in parsed["error"]
+
+
+def test_web_search_tool_reports_client_initialization_failure(monkeypatch, caplog) -> None:
+    """A DDGS client that fails to construct must not escape as a raw exception."""
+
+    class BrokenInitDDGS:
+        def __init__(self, timeout: int) -> None:
+            msg = "client setup failed: invalid proxy configuration"
+            raise RuntimeError(msg)
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=BrokenInitDDGS))
+    monkeypatch.setattr(tools, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: None))
+    caplog.set_level(logging.ERROR, logger=tools.__name__)
+
+    parsed = json.loads(tools.web_search_tool.invoke({"query": "latest news"}))
+
+    assert parsed["query"] == "latest news"
+    assert parsed["error"] == "DuckDuckGo search failed: RuntimeError: client setup failed: invalid proxy configuration"
+    assert any(record.getMessage().startswith("DuckDuckGo search failed") for record in caplog.records)
+
+
+def test_search_text_raises_on_execution_failure(monkeypatch) -> None:
+    from ddgs.exceptions import DDGSException
+
+    class FailingDDGS:
+        def __init__(self, timeout: int) -> None:
+            self.timeout = timeout
+
+        def text(self, query: str, **kwargs):
+            raise DDGSException("TimeoutException: search timed out")
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=FailingDDGS))
+
+    with pytest.raises(tools.DDGSearchError, match="timed out"):
+        tools._search_text("latest news")
+
+
+def test_search_text_reports_client_initialization_failure(monkeypatch) -> None:
+    """Constructing DDGS can already fail; it must raise DDGSearchError, not leak."""
+
+    class BrokenInitDDGS:
+        def __init__(self, timeout: int) -> None:
+            msg = "client setup failed: invalid proxy configuration"
+            raise RuntimeError(msg)
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=BrokenInitDDGS))
+
+    with pytest.raises(tools.DDGSearchError, match="client setup failed: invalid proxy configuration"):
+        tools._search_text("latest news")
+
+
+def test_search_text_keeps_sdk_no_results_sentinel_empty(monkeypatch) -> None:
+    from ddgs.exceptions import DDGSException
+
+    class EmptyDDGS:
+        def __init__(self, timeout: int) -> None:
+            self.timeout = timeout
+
+        def text(self, query: str, **kwargs):
+            raise DDGSException("No results found.")
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=EmptyDDGS))
+
+    assert tools._search_text("latest news") == []
+
+
+def test_search_text_treats_only_the_exact_sdk_sentinel_as_empty(monkeypatch) -> None:
+    """A failure message that merely embeds the sentinel phrase stays an error."""
+    from ddgs.exceptions import DDGSException
+
+    class AmbiguousDDGS:
+        def __init__(self, timeout: int) -> None:
+            self.timeout = timeout
+
+        def text(self, query: str, **kwargs):
+            raise DDGSException("No results found for 'latest news': HTTP 429")
+
+    monkeypatch.setitem(sys.modules, "ddgs", SimpleNamespace(DDGS=AmbiguousDDGS))
+
+    with pytest.raises(tools.DDGSearchError, match="429"):
+        tools._search_text("latest news")