mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 21:26:39 +00:00
73f6749068
* fix(searxng): contain malformed search payloads instead of crashing on them
`_search_page` indexed the decoded body directly:
data = resp.json()
return data.get("results") or []
so a self-hosted instance that answers HTTP 200 with something other than a
result page -- an HTML or plain-text error page from a reverse proxy or captive
portal -- reached the model as a raw Python error, or as an empty search:
{"results": "maintenance window"} -> {"error": "'str' object has no attribute 'get'"}
{"results": {"a": 1}} -> {"error": "'str' object has no attribute 'get'"}
{"results": [1, 2, 3]} -> {"error": "'int' object has no attribute 'get'"}
{"results": [None]} -> {"error": "'NoneType' object has no attribute 'get'"}
Worse, a single malformed entry in an otherwise valid page crashed the whole
call and discarded every good row:
{"results": [{"title": "ok", ...}, 7]} -> {"error": "'int' object has no attribute 'get'"}
This is the containment rule the sibling providers already follow (brave #6550,
groundroute #6603, tavily #6645, sofya #6719, serper #6779, serply #6796,
tencent-wsa #6799): a payload that is not a result page is a format error, not an
empty result, because "No results found" tells the agent the query legitimately
matched nothing and ends a research loop as "no hits".
The client now raises `SearxngResponseError` for a payload that is not an object,
a body that is not JSON, a `results` container that is not a list, and a
non-empty list holding no object entries, while still skipping non-object entries
in an otherwise valid page so the valid rows survive. `web_search_tool` reports
the provider's format error instead of leaking the exception text. Missing, null,
and empty `results` keep the existing "No results found" behavior.
Diagnostics log the container field name and its type, or the absence of usable
objects -- never the query, credentials, or payload values.
* test(searxng): pin the mid-walk format-error trade-off and fix the docstring
The paginating client aborts the walk and discards already-collected rows when
a later page comes back malformed, which is a different outcome from the
single-request siblings. Pin that trade-off with a test so the behaviour is a
deliberate decision rather than an accident, and reword the
_unexpected_format_error docstring: it promises the query is not echoed, but
the sibling providers all echo it back on purpose and only withhold the
payload.
* test(searxng): log skipped entries and pin the dedicated format-error handler
Two follow-ups from review, both about making the family contract observable
and pinned:
- Skipped non-object entries now log their index ("Skipping malformed SearXNG
result entry at results[1]"), matching the merged InfoQuest sibling. SearXNG
is the provider where drift is most expected -- a self-hosted instance serves
whatever sits behind its reverse proxy -- and a search spans up to
_MAX_PAGES requests, so rows could otherwise be shed with no trace at all.
- The dedicated `except SearxngResponseError` branch had no test that could
fail: the exception is a ValueError whose message is already the format-error
string, so the generic handler returns byte-identical JSON. The new test
asserts the generic branch's log line is absent, which is what actually
distinguishes the two paths.
* docs(searxng): name the empty result it actually returns, and pin the dedicated except
Two low findings from the follow-up review:
- The README said missing/null/empty `results` keep the existing "No results
found" response. That response does not exist in this provider: all three
produce a JSON empty list, and the string appears only in comments
explaining what must NOT be reported. The family convention distinguishes
(Tavily: "`[]` for search, Error: No results found for fetch"; InfoQuest:
"keep the existing empty result"), so this now reads "keep the existing
empty result (`[]`)".
- The client-level `except SearxngResponseError: raise` was unpinned: deleting
it left the whole file green, because the tool-level test mocks the client
and the payload tests only assert the raise type. The observable regression
is the generic branch logging "An unexpected error occurred" for every
contained page, so the existing payload test now asserts that line is absent
from caplog.
Mutation check: removing the clause now fails 8 parameterised cases (was 33
green); with the clause in place the suite is 33 passed and 91 across the two
provider files, ruff check/format clean.