feat(search): add DeepSeek web search with configurable thinking (#8299)

* feat(search): add DeepSeek web search with configurable thinking

* fix(search): drain DeepSeek teardown and retain citation spans

Reuse the cancellation-safe context helper so repeated cancellation cannot interrupt HTTP transport cleanup. Preserve validated Unicode citation ranges on retained public sources so the agent can associate claims with their supporting pages. Add regressions for cancellation, timeout, normal completion, invalid ranges, URL screening, and result limits.

* fix(search): preserve DeepSeek citation excerpts through guardrails

* fix(search): share DeepSeek citation excerpts across sources

Group each validated answer range once and associate it with response-local source IDs after URL filtering and result limits. Preserve full-answer PII redaction and existing source-card fields, document the model-facing citation mapping, and repair live-test configuration for both thinking modes. Add HTTP transport, guardrail/budget, attribution, and browser reload regressions.

---------

Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
yong 2026-11-18 21:19:41 +08:00 committed by GitHub
parent 39fa6447e8
commit 8547666a31
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
12 changed files with 1267 additions and 7 deletions

View File

@ -109,6 +109,7 @@ Each guide is the full reference for its topic. Open the one you need.
| Topic | Guide |
| --- | --- |
| Models, providers, `config.yaml`, environment variables | [Configuration](docs/guide/configuration.md) |
| DeepSeek web search, thinking mode and citation excerpts | [DeepSeek Web Search](backend/docs/CONFIGURATION.md#deepseek-web-search) |
| Docker, local development, upgrades, production deployment | [Running the Application](docs/guide/deployment.md) |
| Sandbox modes, file system, uploads and outputs | [Sandbox](docs/guide/sandbox.md) |
| Skills, built-in tools, script workflows, Claude Code integration | [Skills & Tools](docs/guide/skills-and-tools.md) |

View File

@ -1315,8 +1315,82 @@ operator handling has not been verified here. Upstream operators are best-effort
offline mocked tests verify request composition and local filtering, not provider
retrieval behavior. No paid API calls are needed for the regression suite.
#### DeepSeek Web Search
DeepSeek's `web_search` provider calls the official Chat Completions endpoint
with the server-side `deepseek_web_search` tool. Set `DEEPSEEK_API_KEY` in the
Gateway environment and replace the active search provider in `config.yaml`:
```yaml
tools:
- name: web_search
group: web
use: deerflow.community.deepseek.tools:web_search_tool
api_key: $DEEPSEEK_API_KEY
model: deepseek-flash
thinking: false
max_results: 5
timeout_seconds: 180
max_tokens: 8192
```
Set `thinking: true` to enable DeepSeek's multi-step thinking search. This
controls the model inside the search tool independently of the main DeerFlow
model. Boolean strings such as `"false"` are accepted for environment-variable
configuration. The default model is `deepseek-flash`; use a model that supports
DeepSeek's server-side web tools. See the official
[web tools documentation](https://api-docs.deepseek.com/zh-cn/guides/web_tools/).
The tool accepts `query` and `max_results` and returns JSON with `query`,
`total_results`, `results` (`title`, `url`, `snippet`), `answer`, and `truncated`.
Source URLs use the shared public-URL policy and are deduplicated. Sources cited
by the generated answer take priority, with search snippets preserved when
available. When retained sources have valid citation ranges, each result also
has a response-local `source_id`, and a top-level `citations` list pairs each
`excerpt` with its supporting `source_ids`. Resolve those IDs to the returned
source titles and URLs when citing a claim. For example:
```json
{"excerpt": "A claim supported by two pages.", "source_ids": [1, 2]}
```
Each distinct original range is represented once, in answer order, even when
several sources cite it. Identical text at different ranges remains separate.
Excerpts stay paired with their sources through tool-result sanitization and
optional PII redaction. If either boundary falls inside a redacted value, the
excerpt includes the whole placeholder. Responses without valid retained ranges
omit `citations` and `source_id`, keeping their usual fields. Long answers or
distinct long overlapping excerpts still use the normal tool-output budget and
externalization policy. `max_results` defaults to 5 and is capped at 50.
A configured value takes precedence over the tool argument. It limits returned
source links, while DeepSeek controls how many searches it executes and what
answer it generates.
`timeout_seconds` bounds the request, including server-side search and reasoning.
`max_tokens` bounds generated output, including thinking. Both accept positive
numeric values or environment-variable strings. Boolean limits are rejected.
The API key can also be set directly through the tool's `api_key` field.
Configured keys take precedence over `DEEPSEEK_API_KEY`.
DeepSeek decides whether to search. A completed response with no usable public
sources returns `No results found`. Provider failures, malformed payloads,
timeouts and output-budget exhaustion return structured errors. A partial
response with usable sources sets `truncated: true` when it reaches its output
limit. Cancellation waits for HTTP transport cleanup to complete, including when
the caller is cancelled repeatedly during client teardown. Requests have no
automatic retries. DeepSeek bills model inference and server-side search/fetch
calls.
Offline regression tests use HTTPX mock transports. To explicitly run the paid
API checks for both thinking modes, export `DEEPSEEK_API_KEY` and run from
`backend/`:
```bash
DEER_FLOW_RUN_LIVE_TESTS=1 uv run pytest tests/test_deepseek_search_live.py -q
```
**Built-in Tools**:
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Serper, Serply, Exa, InfoQuest, Tencent Cloud WSA, Firecrawl, fastCRW, GroundRoute, Sofya, SpecStream, You.com)
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Serper, Serply, Exa, InfoQuest, Tencent Cloud WSA, DeepSeek, Firecrawl, fastCRW, GroundRoute, Sofya, SpecStream, You.com)
- `web_fetch` - Fetch web pages (Jina AI, Crawl4AI, Exa, InfoQuest, Firecrawl, fastCRW, GroundRoute, Browserless, Sofya, SpecStream, Unbrowse)
- `web_capture` - Capture rendered webpage screenshots as artifacts (Browserless)
- `image_search` - Search for reference images (DuckDuckGo, InfoQuest, Serper, Brave)

View File

@ -59,6 +59,7 @@ import hashlib
import hmac
import logging
import re
from bisect import bisect_right
from collections.abc import Awaitable, Callable, Iterator, Sequence
from copy import deepcopy
from dataclasses import dataclass
@ -266,6 +267,15 @@ def redact_texts(texts: Sequence[str], config: PiiRedactionConfig | None) -> lis
return [redactor.redact(text) for text in texts]
def redact_text_excerpts(text: str, ranges: Sequence[tuple[int, int]], config: PiiRedactionConfig) -> list[str]:
"""Extract validated ranges after redacting the full text.
When an endpoint falls inside a redacted value, include the entire
placeholder. Redacting a substring in isolation could miss a clipped value.
"""
return _make_redactor(config).excerpts(text, ranges)
def _make_redactor(config: PiiRedactionConfig | None) -> _Redactor:
"""Build the scan redactor: active detectors + the configured token key."""
if config is None:
@ -329,6 +339,40 @@ class _Redactor:
text = detector.sub(self._replacer(detector), text)
return text
def excerpts(self, text: str, ranges: Sequence[tuple[int, int]]) -> list[str]:
for detector in self._detectors:
text, ranges = self._redact_excerpt_ranges(text, ranges, detector)
return [text[start:end] for start, end in ranges]
def _redact_excerpt_ranges(self, text: str, ranges: Sequence[tuple[int, int]], detector: _Detector) -> tuple[str, Sequence[tuple[int, int]]]:
replace = self._replacer(detector)
edits: list[tuple[int, int, int]] = []
deltas = [0]
def record_edit(match: re.Match[str]) -> str:
replacement = replace(match)
if replacement != match.group(0):
start, end = match.span()
edits.append((start, end, len(replacement)))
deltas.append(deltas[-1] + len(replacement) - (end - start))
return replacement
redacted = detector.sub(record_edit, text)
if not edits:
return redacted, ranges
def remap(boundary: int, *, is_end: bool) -> int:
# Completed edits shift the boundary by their cumulative length change.
# A boundary inside the next edit moves to the edge of its placeholder.
preceding = bisect_right(edits, boundary, key=lambda edit: edit[1])
if preceding < len(edits):
start, _, size = edits[preceding]
if start < boundary:
boundary = start + (size if is_end else 0)
return boundary + deltas[preceding]
return redacted, [(remap(start, is_end=False), remap(end, is_end=True)) for start, end in ranges]
def _replacer(self, detector: _Detector) -> Callable[[re.Match[str]], str]:
def replace(match: re.Match[str]) -> str:
value = match.group(0)

View File

@ -25,6 +25,9 @@ make test-blocking-io
still enforces the address policy at connection time. Normalize the Unicode dot
separators of IDNA before the check, and return the original URL for an
allowed result.
- **Pair cited DeepSeek sources with excerpts.** Validate provider offsets
against the full answer and apply configured PII redaction before slicing
so clipped identifiers and later tool-result rewrites preserve attribution.
- **A diagnostic names the malformed container and its type.** Queries, URLs,
credentials and payload values stay out of logs.
- **Resolve API keys through `provider_credentials.resolve_api_key`.**

View File

@ -0,0 +1 @@
"""DeepSeek server-side web search provider."""

View File

@ -0,0 +1,280 @@
"""DeepSeek server-side web search exposed as a DeerFlow search provider."""
import asyncio
import json
import logging
import os
from typing import Any
import httpx
from langchain_core.tools import tool
from pydantic import BaseModel, ConfigDict, Field, ValidationError, field_validator
from deerflow.agents.middlewares.pii_redaction_middleware import redact_text_excerpts
from deerflow.community.provider_credentials import resolve_api_key
from deerflow.community.search_max_results import coerce_max_results
from deerflow.community.url_safety import safe_public_result_url
from deerflow.config import get_app_config
from deerflow.config.pii_redaction_config import PiiRedactionConfig
from deerflow.runtime.cancellation import drained_async_context
logger = logging.getLogger(__name__)
_ENDPOINT = "https://api.deepseek.com/chat/completions"
_DEFAULT_MAX_RESULTS = 5
_MAX_RESULTS = 50
class _Options(BaseModel):
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
model: str = Field(default="deepseek-flash", min_length=1)
thinking: bool = False
timeout_seconds: float = Field(default=180, gt=0, allow_inf_nan=False)
max_tokens: int = Field(default=8192, gt=0)
@field_validator("timeout_seconds", "max_tokens", mode="before")
@classmethod
def reject_boolean_limits(cls, value: object) -> object:
if isinstance(value, bool):
raise ValueError("Search limits must be numeric")
return value
def _objects(value: object, path: str) -> list[dict[str, Any]]:
if value is None:
return []
if not isinstance(value, list):
logger.error(
"DeepSeek returned malformed %s container of type %s",
path,
type(value).__name__,
)
raise ValueError
objects = [item for item in value if isinstance(item, dict)]
if value and not objects:
logger.error(
"DeepSeek returned %s with no object entries (type %s)",
path,
type(value[0]).__name__,
)
raise ValueError
return objects
def _text(value: object) -> str:
return value if isinstance(value, str) else ""
def _normalize(data: object, query: str, max_results: int, pii_config: PiiRedactionConfig) -> dict[str, Any]:
if not isinstance(data, dict):
logger.error(
"DeepSeek returned malformed response container of type %s",
type(data).__name__,
)
raise ValueError
# DeepSeek 可能在 HTTP 200 响应体中报告生成失败。
if data.get("error") is not None:
return {"error": "DeepSeek search generation failed", "query": query}
choices = _objects(data.get("choices"), "choices")
message = choices[0].get("message") if choices else None
if not isinstance(message, dict):
logger.error(
"DeepSeek returned malformed choices[0].message container of type %s",
type(message).__name__,
)
raise ValueError
finish_reason = choices[0].get("finish_reason")
if finish_reason not in {"stop", "length"}:
return {"error": "DeepSeek search did not complete", "query": query}
sources: list[dict[str, Any]] = []
search_failed = False
for step in _objects(message.get("server_tool_calls"), "server_tool_calls"):
for call in _objects(step.get("calls"), "server_tool_calls[].calls"):
rows = _objects(call.get("sources"), "server_tool_calls[].calls[].sources")
page = call.get("page")
if page is not None and not isinstance(page, dict):
logger.error("DeepSeek returned malformed server_tool_calls[].calls[].page container of type %s", type(page).__name__)
raise ValueError
if call.get("status") == "completed":
sources.extend(rows)
if page is not None:
sources.append(page)
elif call.get("status") == "failed":
search_failed = True
# 引用可能指向深度搜索中打开、但未出现在搜索列表中的网页。
answer = _text(message.get("content"))
cited_urls: set[str] = set()
spans_by_url: dict[str, dict[tuple[int, int], None]] = {}
for annotation in _objects(message.get("annotations"), "annotations"):
if annotation.get("type") != "url_citation":
continue
citation = annotation.get("url_citation")
if not isinstance(citation, dict):
logger.error(
"DeepSeek returned malformed annotations[].url_citation container of type %s",
type(citation).__name__,
)
raise ValueError
sources.append(citation)
url = safe_public_result_url(citation.get("url"))
cited_urls.add(url)
start, end = citation.get("start_index"), citation.get("end_index")
if not url or type(start) is not int or type(end) is not int:
continue
if not 0 <= start < end <= len(answer):
continue
spans_by_url.setdefault(url, {})[start, end] = None
results_by_url: dict[str, dict[str, Any]] = {}
for source in sources:
url = safe_public_result_url(source.get("url"))
if not url:
continue
result = results_by_url.setdefault(url, {"title": "", "url": url, "snippet": ""})
for field in ("title", "snippet"):
if not result[field]:
result[field] = _text(source.get(field))
results = list(results_by_url.values())
if not results:
if finish_reason == "length":
return {"error": "DeepSeek search reached max_tokens before returning sources", "query": query}
return {
"error": "DeepSeek web search failed" if search_failed else "No results found",
"query": query,
}
# 优先保留回答实际引用的来源,同时保留搜索结果提供的摘要。
results.sort(key=lambda result: result["url"] not in cited_urls)
results = results[:max_results]
output = {
"query": query,
"total_results": len(results),
"results": results,
"answer": answer,
"truncated": finish_reason == "length",
}
retained_spans = sorted({span for result in results for span in spans_by_url.get(result["url"], {})})
if retained_spans:
source_ids_by_span: dict[tuple[int, int], list[int]] = {span: [] for span in retained_spans}
for source_id, result in enumerate(results, start=1):
result["source_id"] = source_id
for span in spans_by_url.get(result["url"], {}):
source_ids_by_span[span].append(source_id)
excerpts = redact_text_excerpts(answer, retained_spans, pii_config)
output["citations"] = [{"excerpt": excerpt, "source_ids": source_ids_by_span[span]} for span, excerpt in zip(retained_spans, excerpts, strict=True)]
return output
def _prepare() -> tuple[dict[str, Any], httpx.AsyncClient | None, str | None, PiiRedactionConfig]:
app_config = get_app_config()
config = app_config.get_tool_config("web_search")
extras = dict(config.model_extra or {}) if config is not None else {}
api_key = resolve_api_key(extras.get("api_key")) or resolve_api_key(os.getenv("DEEPSEEK_API_KEY"))
if not api_key:
return extras, None, "DEEPSEEK_API_KEY is not configured", app_config.pii_redaction
# 配置读取与 TLS 初始化统一放到工作线程。
client = httpx.AsyncClient(headers={"Authorization": f"Bearer {api_key}"})
return extras, client, None, app_config.pii_redaction
async def _close_prepared_client(preparation: asyncio.Task) -> None:
try:
_, client, _, _ = await preparation
if client is not None:
await client.aclose()
except Exception as exc:
logger.error("DeepSeek search initialization cleanup failed (%s)", type(exc).__name__)
@tool("web_search", parse_docstring=True)
async def web_search_tool(query: str, max_results: int = _DEFAULT_MAX_RESULTS) -> str:
"""Search the web using DeepSeek and return an answer with source links.
Returns JSON with an answer and results containing title, url, and snippet.
When citations are available, results also have source_id values, and each
citations entry pairs an excerpt with source_ids referring to those results.
Resolve source_ids to result titles and URLs when writing Markdown citations.
Args:
query: Search keywords or a research question describing what to find.
max_results: Maximum number of source links to return. Default is 5.
"""
query = query.strip()
if not query:
return json.dumps(
{"error": "Search query must not be empty", "query": query},
ensure_ascii=False,
)
preparation = asyncio.create_task(asyncio.to_thread(_prepare))
try:
extras, client, error, pii_config = await asyncio.shield(preparation)
except asyncio.CancelledError:
# 工作线程仍会创建客户端。独立清理任务持有资源,重复取消时也先等它关闭。
cleanup = asyncio.create_task(_close_prepared_client(preparation))
while not cleanup.done():
try:
await asyncio.shield(cleanup)
except asyncio.CancelledError:
continue
raise
except Exception as exc:
logger.error("DeepSeek search initialization failed (%s)", type(exc).__name__)
return json.dumps({"error": "DeepSeek search initialization failed", "query": query}, ensure_ascii=False)
if error:
return json.dumps({"error": error, "query": query}, ensure_ascii=False)
assert client is not None
async with drained_async_context(client):
try:
options = _Options.model_validate(extras)
except ValidationError:
logger.error("Invalid DeepSeek search tool configuration")
return json.dumps(
{"error": "Invalid DeepSeek search tool configuration", "query": query},
ensure_ascii=False,
)
max_results = min(
coerce_max_results(
extras.get("max_results", max_results),
provider="DeepSeek",
logger=logger,
),
_MAX_RESULTS,
)
payload = {
"model": options.model,
"thinking": {"type": "enabled" if options.thinking else "disabled"},
"tools": [{"type": "deepseek_web_search"}],
"messages": [
{
"role": "system",
"content": "Search the web for the user's query and answer using retrieved sources. Cite your sources.",
},
{"role": "user", "content": query},
],
"max_tokens": options.max_tokens,
"stream": False,
}
try:
async with asyncio.timeout(options.timeout_seconds):
response = await client.post(_ENDPOINT, json=payload, timeout=options.timeout_seconds)
response.raise_for_status()
result = _normalize(response.json(), query, max_results, pii_config)
except httpx.HTTPStatusError as exc:
logger.error("DeepSeek search returned HTTP %s", exc.response.status_code)
result = {
"error": f"DeepSeek search API error: HTTP {exc.response.status_code}",
"query": query,
}
except (httpx.TimeoutException, TimeoutError):
result = {"error": "DeepSeek search timed out", "query": query}
except httpx.HTTPError as exc:
logger.error("DeepSeek search request failed (%s)", type(exc).__name__)
result = {"error": "DeepSeek search request failed", "query": query}
except (ValueError, TypeError):
logger.error("DeepSeek search returned an invalid response format")
result = {
"error": "DeepSeek returned an unexpected response format",
"query": query,
}
return json.dumps(result, ensure_ascii=False)

View File

@ -0,0 +1,46 @@
"""Keep DeepSeek provider config reads and HTTPX TLS setup off the loop."""
import importlib
import json
import threading
from pathlib import Path
from types import SimpleNamespace
import httpx
import pytest
from deerflow.config.pii_redaction_config import PiiRedactionConfig
@pytest.fixture
def provider(monkeypatch):
module = importlib.import_module("deerflow.community.deepseek.tools")
threads = []
def config():
threads.append(threading.get_ident())
Path(__file__).read_text(encoding="utf-8")
return SimpleNamespace(get_tool_config=lambda _name: SimpleNamespace(model_extra={"api_key": "test-key"}), pii_redaction=PiiRedactionConfig())
def respond(request):
return httpx.Response(200, json={"choices": [{"finish_reason": "stop", "message": {"content": "", "server_tool_calls": []}}]})
def client(**kwargs):
threads.append(threading.get_ident())
Path(__file__).read_text(encoding="utf-8")
return httpx.AsyncClient(transport=httpx.MockTransport(respond), **kwargs)
private_httpx = SimpleNamespace(**vars(httpx))
private_httpx.AsyncClient = client
monkeypatch.setattr(module, "get_app_config", config)
monkeypatch.setattr(module, "httpx", private_httpx)
return module.web_search_tool, threads
@pytest.mark.asyncio
async def test_deepseek_search_initializes_off_loop(provider):
tool, threads = provider
result = json.loads(await tool.ainvoke({"query": "test"}))
assert result["error"] == "No results found"
assert len(threads) == 2
assert all(thread != threading.get_ident() for thread in threads)

View File

@ -0,0 +1,34 @@
"""Opt-in API checks. Set DEER_FLOW_RUN_LIVE_TESTS=1 and DEEPSEEK_API_KEY."""
import json
import os
from types import SimpleNamespace
import pytest
from deerflow.community.deepseek import tools as deepseek
from deerflow.config.pii_redaction_config import PiiRedactionConfig
pytestmark = [
pytest.mark.live,
pytest.mark.skipif(os.getenv("DEER_FLOW_RUN_LIVE_TESTS") != "1", reason="Live DeepSeek requests are opt-in"),
pytest.mark.skipif(not os.getenv("DEEPSEEK_API_KEY"), reason="DEEPSEEK_API_KEY is required"),
]
@pytest.mark.asyncio
@pytest.mark.parametrize("thinking", [False, True])
async def test_deepseek_search_api_returns_public_sources(monkeypatch, thinking):
extras = {"thinking": thinking, "max_results": 2}
monkeypatch.setattr(deepseek, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: SimpleNamespace(model_extra=extras), pii_redaction=PiiRedactionConfig()))
result = json.loads(
await deepseek.web_search_tool.ainvoke(
{
"query": "Search the official Python website for the Python 3.13.0 release date and provide its release page URL.",
}
)
)
assert "error" not in result
assert 0 < result["total_results"] <= 2
assert result["answer"]
assert all(row["url"].startswith(("https://", "http://")) for row in result["results"])

View File

@ -0,0 +1,741 @@
"""Offline HTTP boundary tests for the DeepSeek community search provider."""
import asyncio
import importlib
import json
import threading
from types import SimpleNamespace
import httpx
import pytest
from langchain_core.messages import AIMessage
from langchain_core.tools import BaseTool
from langgraph.graph import START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode
from pydantic import SecretStr
from deerflow.agents.middlewares.pii_redaction_middleware import PiiRedactionMiddleware, redact_text
from deerflow.agents.middlewares.tool_output_budget_middleware import ToolOutputBudgetMiddleware
from deerflow.agents.middlewares.tool_result_sanitization_middleware import ToolResultSanitizationMiddleware
from deerflow.config.app_config import AppConfig
from deerflow.config.pii_redaction_config import PiiRedactionConfig
from deerflow.config.tool_output_config import ToolOutputConfig
from deerflow.extensions.registry import ExtensionRegistry
from deerflow.reflection import resolve_variable
from deerflow.tools import get_available_tools
def _completion(*, sources=None, **message):
return {
"choices": [
{
"finish_reason": "stop",
"message": {
"content": "带来源的回答",
"server_tool_calls": [
{
"calls": [
{
"status": "completed",
"sources": sources
if sources is not None
else [
{"url": "https://example.com/article", "title": "标题", "snippet": "摘要"},
],
}
]
}
],
**message,
},
}
],
}
@pytest.fixture
def provider(monkeypatch):
module = importlib.import_module("deerflow.community.deepseek.tools")
state = SimpleNamespace(extras={"api_key": SecretStr(" test-key\n")}, requests=[], clients=[], threads=[], response=_completion(), pii_redaction=PiiRedactionConfig())
monkeypatch.delenv("DEEPSEEK_API_KEY", raising=False)
def config():
state.threads.append(threading.get_ident())
return SimpleNamespace(get_tool_config=lambda _name: SimpleNamespace(model_extra=state.extras), pii_redaction=state.pii_redaction)
async def respond(request):
state.requests.append(request)
if isinstance(state.response, Exception):
raise state.response
if callable(state.response):
return await state.response(request)
return httpx.Response(200, json=state.response)
def client(**kwargs):
state.threads.append(threading.get_ident())
result = httpx.AsyncClient(transport=httpx.MockTransport(respond), **kwargs)
state.clients.append(result)
return result
private_httpx = SimpleNamespace(**vars(httpx))
private_httpx.AsyncClient = client
monkeypatch.setattr(module, "httpx", private_httpx)
monkeypatch.setattr(module, "get_app_config", config)
state.tool = resolve_variable("deerflow.community.deepseek.tools:web_search_tool", BaseTool)
return state
def test_provider_entry_preserves_the_web_search_interface(provider):
assert provider.tool.name == "web_search"
assert set(provider.tool.args) == {"query", "max_results"}
def test_configured_provider_is_assembled_and_invoked_by_the_host(provider):
config = AppConfig.model_validate(
{
"tools": [{"name": "web_search", "group": "web", "use": "deerflow.community.deepseek.tools:web_search_tool"}],
"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
}
)
tools = get_available_tools(
app_config=config,
groups=["web"],
include_mcp=False,
subagent_enabled=False,
include_upload_tool=False,
extensions=ExtensionRegistry().build(),
)
(search,) = [tool for tool in tools if tool.name == "web_search"]
result = json.loads(search.invoke({"query": "test"}))
assert result["total_results"] == 1
assert len(provider.requests) == 1
@pytest.mark.asyncio
@pytest.mark.parametrize(("thinking", "wire_value"), [(None, "disabled"), (False, "disabled"), (True, "enabled"), ("false", "disabled"), ("true", "enabled")])
async def test_thinking_payload_and_normalized_results(provider, thinking, wire_value):
if thinking is not None:
provider.extras["thinking"] = thinking
result = json.loads(await provider.tool.ainvoke({"query": " 最新研究 "}))
(request,) = provider.requests
assert str(request.url) == "https://api.deepseek.com/chat/completions"
assert request.headers["Authorization"] == "Bearer test-key"
assert json.loads(request.content) == {
"model": "deepseek-flash",
"thinking": {"type": wire_value},
"tools": [{"type": "deepseek_web_search"}],
"messages": [
{"role": "system", "content": "Search the web for the user's query and answer using retrieved sources. Cite your sources."},
{"role": "user", "content": "最新研究"},
],
"max_tokens": 8192,
"stream": False,
}
assert result == {
"query": "最新研究",
"total_results": 1,
"results": [{"title": "标题", "url": "https://example.com/article", "snippet": "摘要"}],
"answer": "带来源的回答",
"truncated": False,
}
assert provider.clients[0].is_closed
assert all(thread != threading.get_ident() for thread in provider.threads)
@pytest.mark.asyncio
async def test_sources_and_citations_are_screened_deduplicated_and_limited(provider):
provider.extras.update(max_results=2, model="deployment-model", max_tokens=4096)
provider.response = _completion(
sources=[
"bad-entry",
{"url": "http://127。0。0。1/private"},
{"url": "javascript:alert(1)"},
{"url": "https://example.com/one", "title": None, "snippet": 123},
{"url": "https://example.com/one", "title": "duplicate"},
],
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/two", "title": "打开的网页"}}],
)
provider.response["choices"][0]["finish_reason"] = "length"
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
assert result["results"] == [
{"url": "https://example.com/two", "title": "打开的网页", "snippet": ""},
{"url": "https://example.com/one", "title": "duplicate", "snippet": ""},
]
assert result["total_results"] == 2
assert result["truncated"] is True
payload = json.loads(provider.requests[0].content)
assert payload["model"] == "deployment-model"
assert payload["max_tokens"] == 4096
@pytest.mark.asyncio
async def test_cited_source_beyond_result_limit_keeps_its_search_snippet(provider):
provider.response = _completion(
sources=[{"url": f"https://example.com/{index}", "snippet": f"summary-{index}"} for index in range(6)],
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/5", "title": "引用"}}],
)
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
assert result["results"] == [{"url": "https://example.com/5", "title": "引用", "snippet": "summary-5"}]
@pytest.mark.asyncio
async def test_multistep_search_preserves_sources_and_citations_from_all_steps(provider):
provider.response = _completion(
server_tool_calls=[
{"reasoning_content": "first step", "calls": [{"status": "completed", "kind": "deep_search", "sources": [{"url": "https://example.com/one", "snippet": "first"}]}]},
{
"reasoning_content": "second step",
"calls": [
{"status": "completed", "action": {"type": "open_page"}, "page": {"url": "https://example.com/page", "title": "opened"}},
{"status": "completed", "sources": [None, {"url": "https://example.com/two", "title": "second"}]},
{"status": "failed", "error": {"code": "unavailable"}},
],
},
],
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/page", "title": "cited page"}}],
)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["results"] == [
{"url": "https://example.com/page", "title": "opened", "snippet": ""},
{"url": "https://example.com/one", "title": "", "snippet": "first"},
{"url": "https://example.com/two", "title": "second", "snippet": ""},
]
assert "reasoning_content" not in result
assert "first step" not in json.dumps(result)
@pytest.mark.asyncio
async def test_opened_page_is_a_source_without_annotations(provider):
provider.response = _completion(
server_tool_calls=[
{
"calls": [
{"status": "completed", "page": {"url": "https://example.com/page", "title": "opened"}},
{"status": "completed", "page": {"url": "http://127.0.0.1/private", "title": "private"}},
]
}
]
)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["results"] == [{"url": "https://example.com/page", "title": "opened", "snippet": ""}]
@pytest.mark.asyncio
@pytest.mark.parametrize(
"payload",
[
[],
{"choices": "bad"},
{"choices": ["bad"]},
{"choices": [{"message": []}]},
_completion(server_tool_calls="bad"),
_completion(server_tool_calls=[{"calls": 12}]),
_completion(server_tool_calls=[{"calls": [{"status": "completed", "page": "bad"}]}]),
_completion(sources="bad"),
_completion(sources=["bad"]),
_completion(annotations=[{"type": "url_citation", "url_citation": []}]),
],
)
async def test_malformed_containers_are_errors(provider, payload, caplog):
provider.response = payload
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
assert "unexpected response format" in result["error"]
assert "private-query" not in caplog.text
assert "test-key" not in caplog.text
assert provider.clients[0].is_closed
@pytest.mark.asyncio
@pytest.mark.parametrize("rows", [[], None])
async def test_empty_search_is_reported(provider, rows):
provider.response = _completion(server_tool_calls=[{"calls": [{"status": "completed", "sources": rows}]}])
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "No results found"
@pytest.mark.asyncio
async def test_missing_key_and_empty_query_do_not_create_clients(provider):
provider.extras.clear()
assert "not configured" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
assert "empty" in json.loads(await provider.tool.ainvoke({"query": " "}))["error"]
assert provider.clients == []
@pytest.mark.asyncio
async def test_environment_key_fallback(provider, monkeypatch):
provider.extras["api_key"] = "your-deepseek-api-key"
monkeypatch.setenv("DEEPSEEK_API_KEY", " env-key\n")
await provider.tool.ainvoke({"query": "test"})
assert provider.requests[0].headers["Authorization"] == "Bearer env-key"
@pytest.mark.asyncio
async def test_configured_key_precedes_environment(provider, monkeypatch):
monkeypatch.setenv("DEEPSEEK_API_KEY", "environment-key")
await provider.tool.ainvoke({"query": "test"})
assert provider.requests[0].headers["Authorization"] == "Bearer test-key"
@pytest.mark.asyncio
async def test_environment_key_works_without_tool_extras(provider, monkeypatch):
module = importlib.import_module("deerflow.community.deepseek.tools")
monkeypatch.setattr(module, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: None, pii_redaction=provider.pii_redaction))
monkeypatch.setenv("DEEPSEEK_API_KEY", "environment-key")
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["total_results"] == 1
assert provider.requests[0].headers["Authorization"] == "Bearer environment-key"
@pytest.mark.asyncio
@pytest.mark.parametrize("options", [{"thinking": "invalid"}, {"timeout_seconds": 0}, {"timeout_seconds": True}, {"timeout_seconds": float("inf")}, {"max_tokens": True}, {"max_tokens": 1.5}, {"max_tokens": 0}, {"model": " "}])
async def test_invalid_options_return_configuration_error_and_close_client(provider, options):
provider.extras.update(options)
assert "configuration" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
assert provider.requests == []
assert provider.clients[0].is_closed
@pytest.mark.asyncio
async def test_numeric_settings_accept_environment_strings(provider):
provider.extras.update(max_tokens="4096", timeout_seconds="120")
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["total_results"] == 1
assert json.loads(provider.requests[0].content)["max_tokens"] == 4096
assert provider.requests[0].extensions["timeout"]["read"] == 120.0
@pytest.mark.asyncio
async def test_http_and_embedded_errors_do_not_leak_provider_details(provider, caplog):
async def unauthorized(request):
return httpx.Response(401, text="secret test-key private-query", request=request)
provider.response = unauthorized
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
assert result["error"] == "DeepSeek search API error: HTTP 401"
provider.response = {"error": {"message": "secret test-key"}, "usage": {}}
assert "generation failed" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
assert "secret" not in caplog.text
assert "test-key" not in caplog.text
assert "private-query" not in caplog.text
@pytest.mark.asyncio
async def test_transport_timeout_and_invalid_json(provider):
provider.response = httpx.ReadTimeout("secret test-key")
assert "timed out" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
provider.response = httpx.ConnectError("secret test-key")
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "DeepSeek search request failed"
async def html(request):
return httpx.Response(200, text="<html>Error</html>")
provider.response = html
assert "unexpected response format" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
assert all(client.is_closed for client in provider.clients)
@pytest.mark.asyncio
async def test_request_deadline_cancels_slow_transport(provider):
provider.extras["timeout_seconds"] = 0.01
async def wait(request):
await asyncio.Event().wait()
provider.response = wait
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["error"] == "DeepSeek search timed out"
assert provider.clients[0].is_closed
@pytest.mark.asyncio
async def test_failed_search_is_distinct_from_empty_results(provider):
provider.response = _completion(server_tool_calls=[{"calls": [{"status": "failed", "error": {"code": "timeout"}}]}])
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "DeepSeek web search failed"
@pytest.mark.asyncio
async def test_exhausted_generation_budget_is_distinct_from_empty_results(provider):
provider.response = _completion(server_tool_calls=[])
provider.response["choices"][0]["finish_reason"] = "length"
assert "max_tokens" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
@pytest.mark.asyncio
async def test_content_filter_does_not_return_a_fabricated_answer(provider):
provider.response = _completion()
provider.response["choices"][0]["finish_reason"] = "content_filter"
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["error"] == "DeepSeek search did not complete"
assert "answer" not in result
@pytest.mark.asyncio
async def test_client_initialization_failure_contains_sensitive_exception(provider, monkeypatch, caplog):
module = importlib.import_module("deerflow.community.deepseek.tools")
def fail(**kwargs):
raise OSError("test-key private-query")
monkeypatch.setattr(module.httpx, "AsyncClient", fail)
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
assert result["error"] == "DeepSeek search initialization failed"
assert "test-key" not in caplog.text
assert "private-query" not in caplog.text
assert provider.requests == []
@pytest.mark.asyncio
@pytest.mark.parametrize("value, count", [(1000, 50), ("3", 3), (False, 5), (0, 5)])
async def test_configured_result_count_is_bounded(provider, value, count):
provider.extras["max_results"] = value
provider.response = _completion(sources=[{"url": f"https://example.com/{index}"} for index in range(60)])
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["total_results"] == count
@pytest.mark.asyncio
@pytest.mark.parametrize("cancel_count", [1, 2])
async def test_cancellation_during_client_creation_closes_late_client(provider, monkeypatch, cancel_count):
module = importlib.import_module("deerflow.community.deepseek.tools")
original = module.httpx.AsyncClient
started = asyncio.Event()
created = asyncio.Event()
release = threading.Event()
loop = asyncio.get_running_loop()
def delayed_client(**kwargs):
loop.call_soon_threadsafe(started.set)
release.wait()
client = original(**kwargs)
loop.call_soon_threadsafe(created.set)
return client
monkeypatch.setattr(module.httpx, "AsyncClient", delayed_client)
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
try:
await asyncio.wait_for(started.wait(), 5)
task.cancel()
if cancel_count == 2:
checkpoint = asyncio.Event()
loop.call_soon(checkpoint.set)
await checkpoint.wait()
task.cancel()
finally:
release.set()
if not task.cancelling():
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
await asyncio.wait_for(created.wait(), 5)
assert provider.clients[0].is_closed
assert provider.requests == []
@pytest.mark.asyncio
async def test_cancellation_propagates_and_closes_client(provider):
started = asyncio.Event()
async def wait(request):
started.set()
await asyncio.Event().wait()
provider.response = wait
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
try:
await asyncio.wait_for(started.wait(), 5)
finally:
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
assert provider.clients[0].is_closed
@pytest.mark.asyncio
@pytest.mark.parametrize("trigger", ["request_cancel", "request_timeout", "response_complete"])
async def test_repeated_cancellation_drains_transport_teardown(provider, monkeypatch, trigger):
request_started = asyncio.Event()
close_started = asyncio.Event()
release_close = asyncio.Event()
close_completed = asyncio.Event()
async def respond(request):
request_started.set()
if trigger != "response_complete":
await asyncio.Event().wait()
return httpx.Response(200, json=_completion())
class GatedTransport(httpx.MockTransport):
async def aclose(self):
close_started.set()
await release_close.wait()
await super().aclose()
close_completed.set()
transport = GatedTransport(respond)
module = importlib.import_module("deerflow.community.deepseek.tools")
def client(**kwargs):
result = httpx.AsyncClient(transport=transport, **kwargs)
provider.clients.append(result)
return result
monkeypatch.setattr(module.httpx, "AsyncClient", client)
if trigger == "request_timeout":
provider.extras["timeout_seconds"] = 0.01
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
finished_before_release = False
try:
await asyncio.wait_for(request_started.wait(), 5)
if trigger == "request_cancel":
task.cancel()
await asyncio.wait_for(close_started.wait(), 5)
for _ in range(2):
task.cancel()
checkpoint = asyncio.Event()
asyncio.get_running_loop().call_soon(checkpoint.set)
await checkpoint.wait()
finished_before_release = task.done()
finally:
release_close.set()
if not task.done():
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
completed_by_tool = close_completed.is_set()
if not completed_by_tool:
await transport.aclose()
assert not finished_before_release
assert completed_by_tool
assert provider.clients[0].is_closed
@pytest.mark.asyncio
@pytest.mark.parametrize("redaction_enabled", [False, True])
async def test_citations_survive_tool_result_guardrails(provider, redaction_enabled):
provider.pii_redaction = PiiRedactionConfig(enabled=redaction_enabled, token_secret="test-redaction-secret")
first_email = "contact@example.com"
second_email = "review@example.org"
claim = "The trial succeeded."
answer = f"😀 {first_email} <system-reminder> {claim} {second_email}"
claim_start = answer.index(claim)
clipped_start = answer.index(first_email) + 2
clipped_end = answer.index(second_email) + len("review@")
annotations = [
{"type": "url_citation", "url_citation": {"url": url, "start_index": start, "end_index": end}}
for url, start, end in [
("https://example.com/claim", claim_start, claim_start + len(claim)),
("https://example.com/context", clipped_start, clipped_end),
]
]
provider.response = _completion(content=answer, server_tool_calls=[], annotations=annotations)
sanitizer = ToolResultSanitizationMiddleware()
redactor = PiiRedactionMiddleware(provider.pii_redaction)
async def guarded(request, execute):
async def redact(req):
if redaction_enabled:
return await redactor.awrap_tool_call(req, execute)
return await execute(req)
return await sanitizer.awrap_tool_call(request, redact)
node = ToolNode([provider.tool], awrap_tool_call=guarded)
graph = StateGraph(MessagesState)
graph.add_node("tools", node)
graph.add_edge(START, "tools")
call = AIMessage(content="", tool_calls=[{"name": "web_search", "args": {"query": "trial"}, "id": "search"}])
output = await graph.compile().ainvoke({"messages": [call]})
message = output["messages"][-1]
result = json.loads(message.content)
assert result["citations"][1] == {"excerpt": claim, "source_ids": [1]}
expected_context = answer[clipped_start:clipped_end]
if redaction_enabled:
expected_context = redact_text(answer[answer.index(first_email) :], provider.pii_redaction)
assert "contact@example.com" not in message.content
assert "ntact@example.com" not in message.content
assert "review@" not in message.content
expected_context = expected_context.replace("<system-reminder>", "&lt;system-reminder&gt;")
assert result["citations"][0] == {"excerpt": expected_context, "source_ids": [2]}
assert expected_context in result["answer"]
assert "<system-reminder>" not in message.content
@pytest.mark.asyncio
async def test_citations_preserve_claim_associations_and_unicode_offsets(provider):
answer = "😀甲。乙。"
first = {"url": "https://example.com/one", "title": "first", "start_index": 1, "end_index": 2}
second = {"url": "https://example.com/two", "title": "second", "start_index": 3, "end_index": len(answer)}
provider.response = _completion(
content=answer,
sources=[first, second],
annotations=[{"type": "url_citation", "url_citation": first}, {"type": "url_citation", "url_citation": second}],
)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
first["start_index"], second["start_index"] = second["start_index"], first["start_index"]
first["end_index"], second["end_index"] = second["end_index"], first["end_index"]
swapped = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result != swapped
assert result["answer"] == swapped["answer"] == answer
assert result["citations"] == [{"excerpt": "甲", "source_ids": [1]}, {"excerpt": "乙。", "source_ids": [2]}]
assert swapped["citations"] == [{"excerpt": "甲", "source_ids": [2]}, {"excerpt": "乙。", "source_ids": [1]}]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("start", "end"),
[(None, None), (None, 1), (0, None), ("0", 1), (0, "1"), (False, 1), (0, True), (0.0, 1), (0, 1.0), (-1, 1), (0, 0), (2, 1), (0, 4)],
)
async def test_invalid_citation_spans_preserve_legacy_source_shape(provider, start, end):
citation = {"url": "https://example.com/cited", "title": "source", "start_index": start, "end_index": end}
provider.response = _completion(content="甲😀乙", server_tool_calls=[], annotations=[{"type": "url_citation", "url_citation": citation}])
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result == {
"query": "test",
"total_results": 1,
"results": [{"title": "source", "url": citation["url"], "snippet": ""}],
"answer": "甲😀乙",
"truncated": False,
}
@pytest.mark.asyncio
async def test_citations_preserve_whitespace_multiple_sources_and_distinct_ranges(provider):
answer = " 😀甲乙\n"
annotations = [
{"type": "url_citation", "url_citation": {"url": url, "start_index": start, "end_index": end}}
for url, start, end in [
("https://example.com/one", 2, 4),
("https://example.com/one", 2, 4),
("https://example.com/one", 3, len(answer)),
("https://example.com/two", 2, 4),
]
]
provider.response = _completion(content=answer, server_tool_calls=[], annotations=annotations)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["answer"] == answer
assert result["citations"] == [{"excerpt": "😀甲", "source_ids": [1, 2]}, {"excerpt": "甲乙\n", "source_ids": [1]}]
@pytest.mark.asyncio
async def test_citations_follow_url_screening_and_result_limit(provider):
urls = ["https://example.com/kept", "https://example.com/limited", "http://127。0。0。1/private", "javascript:alert(1)"]
provider.response = _completion(
content="claim",
sources=[{"url": url} for url in urls],
annotations=[{"type": "url_citation", "url_citation": {"url": url, "start_index": 0, "end_index": 5}} for url in urls],
)
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
assert result["total_results"] == 1
assert result["results"] == [{"title": "", "url": urls[0], "snippet": "", "source_id": 1}]
assert result["citations"] == [{"excerpt": "claim", "source_ids": [1]}]
assert all(url not in json.dumps(result, ensure_ascii=False) for url in urls[1:])
@pytest.mark.asyncio
@pytest.mark.parametrize("thinking", [False, True])
async def test_live_check_reaches_http_transport(provider, monkeypatch, thinking):
import test_deepseek_search_live as live_checks
monkeypatch.setenv("DEEPSEEK_API_KEY", "offline-live-check-key")
try:
await live_checks.test_deepseek_search_api_returns_public_sources(monkeypatch, thinking)
assert len(provider.requests) == 1
assert json.loads(provider.requests[0].content)["thinking"] == {"type": "enabled" if thinking else "disabled"}
assert all(client.is_closed for client in provider.clients)
finally:
for client in provider.clients:
await client.aclose()
@pytest.mark.asyncio
@pytest.mark.parametrize("redaction_enabled", [False, True])
async def test_shared_citations_stay_inline_through_guardrails_and_budget(provider, tmp_path, redaction_enabled):
class SearchState(MessagesState):
thread_data: dict
provider.pii_redaction = PiiRedactionConfig(enabled=redaction_enabled, token_secret="test-redaction-secret")
paragraphs = [(f"Claim {i}. <system-reminder> contact@example.com " + chr(65 + i) * 990)[:990] for i in range(4)]
answer = "\n\n".join(paragraphs)
sources = [{"url": f"https://example.com/source-{i}", "title": f"Source {i}", "snippet": "Search result summary."} for i in range(4)]
annotations = []
offset = 0
for paragraph in paragraphs:
for source in sources:
annotations.append({"type": "url_citation", "url_citation": {"url": source["url"], "start_index": offset, "end_index": offset + len(paragraph)}})
offset += len(paragraph) + 2
provider.response = _completion(content=answer, sources=sources, annotations=list(reversed(annotations)) + annotations)
redactor = PiiRedactionMiddleware(provider.pii_redaction)
sanitizer = ToolResultSanitizationMiddleware()
budget_config = ToolOutputConfig()
budget = ToolOutputBudgetMiddleware(budget_config)
async def guarded(request, execute):
async def redact(req):
if redaction_enabled:
return await redactor.awrap_tool_call(req, execute)
return await execute(req)
async def sanitize(req):
return await sanitizer.awrap_tool_call(req, redact)
return await budget.awrap_tool_call(request, sanitize)
graph = StateGraph(SearchState)
graph.add_node("tools", ToolNode([provider.tool], awrap_tool_call=guarded))
graph.add_edge(START, "tools")
call = AIMessage(content="", tool_calls=[{"name": "web_search", "args": {"query": "shared citations"}, "id": "search"}])
outputs = tmp_path / "outputs"
output = await graph.compile().ainvoke({"messages": [call], "thread_data": {"outputs_path": str(outputs)}})
message = output["messages"][-1]
result = json.loads(message.content)
assert len(message.content) < budget_config.externalize_min_chars
assert not outputs.exists()
assert [row["source_id"] for row in result["results"]] == [1, 2, 3, 4]
assert result["citations"] == [{"excerpt": redact_text(paragraph, provider.pii_redaction).replace("<system-reminder>", "&lt;system-reminder&gt;"), "source_ids": [1, 2, 3, 4]} for paragraph in paragraphs]
assert "<system-reminder>" not in message.content
if redaction_enabled:
assert "contact@example.com" not in message.content
@pytest.mark.asyncio
async def test_equal_excerpt_text_at_distinct_ranges_keeps_separate_attribution(provider):
answer = "同一句。同一句。"
provider.response = _completion(
content=answer,
sources=[{"url": "https://example.com/one"}, {"url": "https://example.com/two"}],
annotations=[
{"type": "url_citation", "url_citation": {"url": "https://example.com/two", "start_index": 4, "end_index": 8}},
{"type": "url_citation", "url_citation": {"url": "https://example.com/one", "start_index": 0, "end_index": 4}},
],
)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["citations"] == [{"excerpt": "同一句。", "source_ids": [1]}, {"excerpt": "同一句。", "source_ids": [2]}]
@pytest.mark.asyncio
@pytest.mark.parametrize("max_results", [1, 2, 3])
async def test_citation_ids_follow_retained_source_order(provider, max_results):
provider.response = _completion(
content="AB",
sources=[{"url": "https://example.com/uncited"}, {"url": "https://example.com/b"}, {"url": "https://example.com/c"}],
annotations=[
{"type": "url_citation", "url_citation": {"url": "https://example.com/c", "start_index": 0, "end_index": 1}},
{"type": "url_citation", "url_citation": {"url": "https://example.com/b", "start_index": 1, "end_index": 2}},
],
)
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": max_results}))
assert [(row["source_id"], row["url"]) for row in result["results"]] == [(1, "https://example.com/b"), (2, "https://example.com/c"), (3, "https://example.com/uncited")][:max_results]
assert result["citations"] == ([{"excerpt": "B", "source_ids": [1]}] if max_results == 1 else [{"excerpt": "A", "source_ids": [2]}, {"excerpt": "B", "source_ids": [1]}])
@pytest.mark.asyncio
async def test_distinct_long_overlapping_ranges_preserve_full_excerpts(provider):
answer = "abcdef" * 660
provider.response = _completion(
content=answer,
sources=[{"url": f"https://example.com/{i}"} for i in range(4)],
annotations=[{"type": "url_citation", "url_citation": {"url": f"https://example.com/{i}", "start_index": i, "end_index": len(answer)}} for i in range(4)],
)
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
assert result["answer"] == answer
assert result["citations"] == [{"excerpt": answer[i:], "source_ids": [i + 1]} for i in range(4)]
assert len(json.dumps(result)) > ToolOutputConfig().externalize_min_chars

View File

@ -1010,6 +1010,18 @@ tools:
# # mode: 0 # Optional: 0=web, 1=VR, 2=mixed
# # api_key: $TENCENTCLOUD_WSA_APIKEY # Optional explicit env-var reference
# Web search tool (DeepSeek server-side search, requires DEEPSEEK_API_KEY)
# Replace the active web_search entry with this provider.
# - name: web_search
# group: web
# use: deerflow.community.deepseek.tools:web_search_tool
# api_key: $DEEPSEEK_API_KEY
# model: deepseek-flash
# thinking: false # true enables multi-step thinking search
# max_results: 5 # Returned sources, capped at 50
# timeout_seconds: 180 # Includes server-side search and reasoning
# max_tokens: 8192
# Web search tool (uses Exa, requires EXA_API_KEY)
# - name: web_search
# group: web

View File

@ -119,6 +119,8 @@ fences, including unclosed fences, in complete and streamed replies.
The wizard also lets you configure an optional web search provider, or skip it for now.
[DeepSeek web search](../../backend/docs/CONFIGURATION.md#deepseek-web-search) uses `deerflow.community.deepseek.tools:web_search_tool`. Configure `DEEPSEEK_API_KEY` in the Gateway environment and replace the active `web_search` entry with the example in `config.example.yaml`. Set `thinking: true` for multi-step thinking search, independently of the main DeerFlow model. The default is `false`. Results include a generated answer and screened, deduplicated source links.
Brave web search preserves valid entries in mixed result lists. Malformed response containers or lists containing no usable entries return a structured format error; missing, null, or empty results keep the existing "No results found" response. Format errors also log the malformed container's path and type, or the absence of usable result objects, without including search queries, credentials, or payload values.
Several providers, including You.com, GroundRoute, Webz, InfoQuest and Jsonify, initialize HTTPX clients in a worker thread so synchronous TLS trust loading does not block the agent event loop; requests and session cleanup remain asynchronous.

View File

@ -2,7 +2,12 @@ import { expect, test } from "@playwright/test";
import { mockLangGraphAPI } from "./utils/mock-api";
for (const shape of ["array", "webz", "webz-malformed"] as const) {
for (const shape of [
"array",
"webz",
"webz-malformed",
"deepseek-citations",
] as const) {
test(`search source links from ${shape} results survive reload`, async ({
page,
}) => {
@ -23,14 +28,31 @@ for (const shape of ["array", "webz", "webz-malformed"] as const) {
});
results.push({ title: "Invalid URL", url: 42 });
}
if (shape === "deepseek-citations") {
results[0] = {
source_id: 1,
title: "Renewable energy news",
url: "https://example.com/news",
snippet: "Matching passage",
};
}
const payload =
shape === "array"
? results
: {
query: "renewable energy",
returned_results: results.length,
results,
};
: shape === "deepseek-citations"
? {
query: "renewable energy",
total_results: results.length,
results,
answer: "Matching passage",
citations: [{ excerpt: "Matching passage", source_ids: [1] }],
truncated: false,
}
: {
query: "renewable energy",
returned_results: results.length,
results,
};
mockLangGraphAPI(page, {
threads: [
{