mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-11 14:38:38 +00:00
23bd76046a
* feat(community): add Sofya web search provider Add a community provider backed by Sofya (https://sofya.co). Its search endpoint returns the content of the result pages, not only their snippets, and its fetch endpoint returns a page as markdown. Both are plain JSON over HTTP, so this needs no extra Python package (uses httpx, already a dependency). Changes: - backend/packages/harness/deerflow/community/sofya/__init__.py - backend/packages/harness/deerflow/community/sofya/tools.py Implements web_search_tool and web_fetch_tool using httpx. API key is read from the config.yaml `api_key` field or the SOFYA_API_KEY env var. Follows the same interface and output shape as the existing ddg_search and serper providers, including the max_results parameter with config override and the structured "No results found" error. - backend/tests/test_sofya_tools.py Unit tests covering API key resolution, config overrides, result mapping, time range, HTTP errors, empty results, and fetch failures. - config.example.yaml: add commented-out Sofya web_search and web_fetch examples alongside the other providers - .env.example: add SOFYA_API_KEY placeholder - backend/docs/CONFIGURATION.md: list Sofya under web_search, web_fetch and the environment variables * fix(sofya): honor caller max_results, validate search_depth, join time_range contract test - Caller-supplied max_results now wins; config is used only when the argument is omitted, matching GroundRoute. - search_depth is clamped to basic/snippets; an unsupported value logs a warning and falls back to basic. - Sofya added to the shared time_range schema contract test. * fix(sofya): cap per-result content so a search stays inline An unbounded search payload (up to 20 read pages) crossed the tool output budget middleware's externalize_min_chars threshold, which replaces the result list with a file reference. Cap each result's content at contents_max_characters (default 2000, 0 disables), matching Exa's config key. Five capped results stay under the 12000 char threshold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * fix(sofya): list Sofya in the recency contract, coerce non-string content _clip subscripted its input, so a non-string content or description from the API raised TypeError instead of degrading. Coerce to text first, the way _sofya_post and _response_results guard the shapes around it. Also add Sofya to the Web Search Recency section in backend/AGENTS.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * fix(sofya): coerce web_fetch content, list sofya in the tools guide, add changelog web_fetch sliced its content the same way web_search did before the last push: a truthy non-string from the API passed the falsiness guard and then raised TypeError. Reuse _clip, keeping the `or ""` so empty content still reports "No content found". Also add sofya to the community provider inventory in packages/harness/deerflow/tools/AGENTS.md and an [Unreleased] changelog entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 * docs(zh): add the missing InfoQuest and Firecrawl web_fetch tabs The ZH web_fetch tab list named five providers where EN names seven. Both tabs mirror their EN counterparts, so the two locales list the same web_fetch providers again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016TZhyPNCX2GYyBPkvTgJV5 --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
28 lines
1.3 KiB
Python
28 lines
1.3 KiB
Python
"""Shared contract tests for provider-native web-search recency filtering."""
|
|
|
|
import pytest
|
|
from langchain_core.utils.function_calling import convert_to_openai_tool
|
|
|
|
from deerflow.community.brave.tools import web_search_tool as brave_web_search
|
|
from deerflow.community.ddg_search.tools import web_search_tool as ddg_web_search
|
|
from deerflow.community.searxng.tools import web_search_tool as searxng_web_search
|
|
from deerflow.community.sofya.tools import web_search_tool as sofya_web_search
|
|
from deerflow.community.tavily.tools import web_search_tool as tavily_web_search
|
|
|
|
EXPECTED_TIME_RANGES = {"day", "week", "month", "year"}
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"tool_obj",
|
|
[ddg_web_search, brave_web_search, tavily_web_search, searxng_web_search, sofya_web_search],
|
|
ids=["ddg", "brave", "tavily", "searxng", "sofya"],
|
|
)
|
|
def test_web_search_time_range_schema_is_consistent(tool_obj) -> None:
|
|
parameters = convert_to_openai_tool(tool_obj)["function"]["parameters"]
|
|
time_range_schema = parameters["properties"]["time_range"]
|
|
branches = time_range_schema.get("anyOf", [time_range_schema])
|
|
enum_values = next(branch["enum"] for branch in branches if "enum" in branch)
|
|
|
|
assert set(enum_values) == EXPECTED_TIME_RANGES
|
|
assert "time_range" not in parameters.get("required", [])
|