mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-11-20 03:10:34 +00:00
feat(search): add DeepSeek web search with configurable thinking (#8299)
* feat(search): add DeepSeek web search with configurable thinking * fix(search): drain DeepSeek teardown and retain citation spans Reuse the cancellation-safe context helper so repeated cancellation cannot interrupt HTTP transport cleanup. Preserve validated Unicode citation ranges on retained public sources so the agent can associate claims with their supporting pages. Add regressions for cancellation, timeout, normal completion, invalid ranges, URL screening, and result limits. * fix(search): preserve DeepSeek citation excerpts through guardrails * fix(search): share DeepSeek citation excerpts across sources Group each validated answer range once and associate it with response-local source IDs after URL filtering and result limits. Preserve full-answer PII redaction and existing source-card fields, document the model-facing citation mapping, and repair live-test configuration for both thinking modes. Add HTTP transport, guardrail/budget, attribution, and browser reload regressions. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
39fa6447e8
commit
8547666a31
@ -109,6 +109,7 @@ Each guide is the full reference for its topic. Open the one you need.
|
||||
| Topic | Guide |
|
||||
| --- | --- |
|
||||
| Models, providers, `config.yaml`, environment variables | [Configuration](docs/guide/configuration.md) |
|
||||
| DeepSeek web search, thinking mode and citation excerpts | [DeepSeek Web Search](backend/docs/CONFIGURATION.md#deepseek-web-search) |
|
||||
| Docker, local development, upgrades, production deployment | [Running the Application](docs/guide/deployment.md) |
|
||||
| Sandbox modes, file system, uploads and outputs | [Sandbox](docs/guide/sandbox.md) |
|
||||
| Skills, built-in tools, script workflows, Claude Code integration | [Skills & Tools](docs/guide/skills-and-tools.md) |
|
||||
|
||||
@ -1315,8 +1315,82 @@ operator handling has not been verified here. Upstream operators are best-effort
|
||||
offline mocked tests verify request composition and local filtering, not provider
|
||||
retrieval behavior. No paid API calls are needed for the regression suite.
|
||||
|
||||
#### DeepSeek Web Search
|
||||
|
||||
DeepSeek's `web_search` provider calls the official Chat Completions endpoint
|
||||
with the server-side `deepseek_web_search` tool. Set `DEEPSEEK_API_KEY` in the
|
||||
Gateway environment and replace the active search provider in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
- name: web_search
|
||||
group: web
|
||||
use: deerflow.community.deepseek.tools:web_search_tool
|
||||
api_key: $DEEPSEEK_API_KEY
|
||||
model: deepseek-flash
|
||||
thinking: false
|
||||
max_results: 5
|
||||
timeout_seconds: 180
|
||||
max_tokens: 8192
|
||||
```
|
||||
|
||||
Set `thinking: true` to enable DeepSeek's multi-step thinking search. This
|
||||
controls the model inside the search tool independently of the main DeerFlow
|
||||
model. Boolean strings such as `"false"` are accepted for environment-variable
|
||||
configuration. The default model is `deepseek-flash`; use a model that supports
|
||||
DeepSeek's server-side web tools. See the official
|
||||
[web tools documentation](https://api-docs.deepseek.com/zh-cn/guides/web_tools/).
|
||||
|
||||
The tool accepts `query` and `max_results` and returns JSON with `query`,
|
||||
`total_results`, `results` (`title`, `url`, `snippet`), `answer`, and `truncated`.
|
||||
Source URLs use the shared public-URL policy and are deduplicated. Sources cited
|
||||
by the generated answer take priority, with search snippets preserved when
|
||||
available. When retained sources have valid citation ranges, each result also
|
||||
has a response-local `source_id`, and a top-level `citations` list pairs each
|
||||
`excerpt` with its supporting `source_ids`. Resolve those IDs to the returned
|
||||
source titles and URLs when citing a claim. For example:
|
||||
|
||||
```json
|
||||
{"excerpt": "A claim supported by two pages.", "source_ids": [1, 2]}
|
||||
```
|
||||
|
||||
Each distinct original range is represented once, in answer order, even when
|
||||
several sources cite it. Identical text at different ranges remains separate.
|
||||
Excerpts stay paired with their sources through tool-result sanitization and
|
||||
optional PII redaction. If either boundary falls inside a redacted value, the
|
||||
excerpt includes the whole placeholder. Responses without valid retained ranges
|
||||
omit `citations` and `source_id`, keeping their usual fields. Long answers or
|
||||
distinct long overlapping excerpts still use the normal tool-output budget and
|
||||
externalization policy. `max_results` defaults to 5 and is capped at 50.
|
||||
A configured value takes precedence over the tool argument. It limits returned
|
||||
source links, while DeepSeek controls how many searches it executes and what
|
||||
answer it generates.
|
||||
|
||||
`timeout_seconds` bounds the request, including server-side search and reasoning.
|
||||
`max_tokens` bounds generated output, including thinking. Both accept positive
|
||||
numeric values or environment-variable strings. Boolean limits are rejected.
|
||||
The API key can also be set directly through the tool's `api_key` field.
|
||||
Configured keys take precedence over `DEEPSEEK_API_KEY`.
|
||||
|
||||
DeepSeek decides whether to search. A completed response with no usable public
|
||||
sources returns `No results found`. Provider failures, malformed payloads,
|
||||
timeouts and output-budget exhaustion return structured errors. A partial
|
||||
response with usable sources sets `truncated: true` when it reaches its output
|
||||
limit. Cancellation waits for HTTP transport cleanup to complete, including when
|
||||
the caller is cancelled repeatedly during client teardown. Requests have no
|
||||
automatic retries. DeepSeek bills model inference and server-side search/fetch
|
||||
calls.
|
||||
|
||||
Offline regression tests use HTTPX mock transports. To explicitly run the paid
|
||||
API checks for both thinking modes, export `DEEPSEEK_API_KEY` and run from
|
||||
`backend/`:
|
||||
|
||||
```bash
|
||||
DEER_FLOW_RUN_LIVE_TESTS=1 uv run pytest tests/test_deepseek_search_live.py -q
|
||||
```
|
||||
|
||||
**Built-in Tools**:
|
||||
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Serper, Serply, Exa, InfoQuest, Tencent Cloud WSA, Firecrawl, fastCRW, GroundRoute, Sofya, SpecStream, You.com)
|
||||
- `web_search` - Search the web (DuckDuckGo, Tavily, Brave, Serper, Serply, Exa, InfoQuest, Tencent Cloud WSA, DeepSeek, Firecrawl, fastCRW, GroundRoute, Sofya, SpecStream, You.com)
|
||||
- `web_fetch` - Fetch web pages (Jina AI, Crawl4AI, Exa, InfoQuest, Firecrawl, fastCRW, GroundRoute, Browserless, Sofya, SpecStream, Unbrowse)
|
||||
- `web_capture` - Capture rendered webpage screenshots as artifacts (Browserless)
|
||||
- `image_search` - Search for reference images (DuckDuckGo, InfoQuest, Serper, Brave)
|
||||
|
||||
@ -59,6 +59,7 @@ import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import re
|
||||
from bisect import bisect_right
|
||||
from collections.abc import Awaitable, Callable, Iterator, Sequence
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass
|
||||
@ -266,6 +267,15 @@ def redact_texts(texts: Sequence[str], config: PiiRedactionConfig | None) -> lis
|
||||
return [redactor.redact(text) for text in texts]
|
||||
|
||||
|
||||
def redact_text_excerpts(text: str, ranges: Sequence[tuple[int, int]], config: PiiRedactionConfig) -> list[str]:
|
||||
"""Extract validated ranges after redacting the full text.
|
||||
|
||||
When an endpoint falls inside a redacted value, include the entire
|
||||
placeholder. Redacting a substring in isolation could miss a clipped value.
|
||||
"""
|
||||
return _make_redactor(config).excerpts(text, ranges)
|
||||
|
||||
|
||||
def _make_redactor(config: PiiRedactionConfig | None) -> _Redactor:
|
||||
"""Build the scan redactor: active detectors + the configured token key."""
|
||||
if config is None:
|
||||
@ -329,6 +339,40 @@ class _Redactor:
|
||||
text = detector.sub(self._replacer(detector), text)
|
||||
return text
|
||||
|
||||
def excerpts(self, text: str, ranges: Sequence[tuple[int, int]]) -> list[str]:
|
||||
for detector in self._detectors:
|
||||
text, ranges = self._redact_excerpt_ranges(text, ranges, detector)
|
||||
return [text[start:end] for start, end in ranges]
|
||||
|
||||
def _redact_excerpt_ranges(self, text: str, ranges: Sequence[tuple[int, int]], detector: _Detector) -> tuple[str, Sequence[tuple[int, int]]]:
|
||||
replace = self._replacer(detector)
|
||||
edits: list[tuple[int, int, int]] = []
|
||||
deltas = [0]
|
||||
|
||||
def record_edit(match: re.Match[str]) -> str:
|
||||
replacement = replace(match)
|
||||
if replacement != match.group(0):
|
||||
start, end = match.span()
|
||||
edits.append((start, end, len(replacement)))
|
||||
deltas.append(deltas[-1] + len(replacement) - (end - start))
|
||||
return replacement
|
||||
|
||||
redacted = detector.sub(record_edit, text)
|
||||
if not edits:
|
||||
return redacted, ranges
|
||||
|
||||
def remap(boundary: int, *, is_end: bool) -> int:
|
||||
# Completed edits shift the boundary by their cumulative length change.
|
||||
# A boundary inside the next edit moves to the edge of its placeholder.
|
||||
preceding = bisect_right(edits, boundary, key=lambda edit: edit[1])
|
||||
if preceding < len(edits):
|
||||
start, _, size = edits[preceding]
|
||||
if start < boundary:
|
||||
boundary = start + (size if is_end else 0)
|
||||
return boundary + deltas[preceding]
|
||||
|
||||
return redacted, [(remap(start, is_end=False), remap(end, is_end=True)) for start, end in ranges]
|
||||
|
||||
def _replacer(self, detector: _Detector) -> Callable[[re.Match[str]], str]:
|
||||
def replace(match: re.Match[str]) -> str:
|
||||
value = match.group(0)
|
||||
|
||||
@ -25,6 +25,9 @@ make test-blocking-io
|
||||
still enforces the address policy at connection time. Normalize the Unicode dot
|
||||
separators of IDNA before the check, and return the original URL for an
|
||||
allowed result.
|
||||
- **Pair cited DeepSeek sources with excerpts.** Validate provider offsets
|
||||
against the full answer and apply configured PII redaction before slicing
|
||||
so clipped identifiers and later tool-result rewrites preserve attribution.
|
||||
- **A diagnostic names the malformed container and its type.** Queries, URLs,
|
||||
credentials and payload values stay out of logs.
|
||||
- **Resolve API keys through `provider_credentials.resolve_api_key`.**
|
||||
|
||||
@ -0,0 +1 @@
|
||||
"""DeepSeek server-side web search provider."""
|
||||
280
backend/packages/harness/deerflow/community/deepseek/tools.py
Normal file
280
backend/packages/harness/deerflow/community/deepseek/tools.py
Normal file
@ -0,0 +1,280 @@
|
||||
"""DeepSeek server-side web search exposed as a DeerFlow search provider."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from langchain_core.tools import tool
|
||||
from pydantic import BaseModel, ConfigDict, Field, ValidationError, field_validator
|
||||
|
||||
from deerflow.agents.middlewares.pii_redaction_middleware import redact_text_excerpts
|
||||
from deerflow.community.provider_credentials import resolve_api_key
|
||||
from deerflow.community.search_max_results import coerce_max_results
|
||||
from deerflow.community.url_safety import safe_public_result_url
|
||||
from deerflow.config import get_app_config
|
||||
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
||||
from deerflow.runtime.cancellation import drained_async_context
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
_ENDPOINT = "https://api.deepseek.com/chat/completions"
|
||||
_DEFAULT_MAX_RESULTS = 5
|
||||
_MAX_RESULTS = 50
|
||||
|
||||
|
||||
class _Options(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
|
||||
|
||||
model: str = Field(default="deepseek-flash", min_length=1)
|
||||
thinking: bool = False
|
||||
timeout_seconds: float = Field(default=180, gt=0, allow_inf_nan=False)
|
||||
max_tokens: int = Field(default=8192, gt=0)
|
||||
|
||||
@field_validator("timeout_seconds", "max_tokens", mode="before")
|
||||
@classmethod
|
||||
def reject_boolean_limits(cls, value: object) -> object:
|
||||
if isinstance(value, bool):
|
||||
raise ValueError("Search limits must be numeric")
|
||||
return value
|
||||
|
||||
|
||||
def _objects(value: object, path: str) -> list[dict[str, Any]]:
|
||||
if value is None:
|
||||
return []
|
||||
if not isinstance(value, list):
|
||||
logger.error(
|
||||
"DeepSeek returned malformed %s container of type %s",
|
||||
path,
|
||||
type(value).__name__,
|
||||
)
|
||||
raise ValueError
|
||||
objects = [item for item in value if isinstance(item, dict)]
|
||||
if value and not objects:
|
||||
logger.error(
|
||||
"DeepSeek returned %s with no object entries (type %s)",
|
||||
path,
|
||||
type(value[0]).__name__,
|
||||
)
|
||||
raise ValueError
|
||||
return objects
|
||||
|
||||
|
||||
def _text(value: object) -> str:
|
||||
return value if isinstance(value, str) else ""
|
||||
|
||||
|
||||
def _normalize(data: object, query: str, max_results: int, pii_config: PiiRedactionConfig) -> dict[str, Any]:
|
||||
if not isinstance(data, dict):
|
||||
logger.error(
|
||||
"DeepSeek returned malformed response container of type %s",
|
||||
type(data).__name__,
|
||||
)
|
||||
raise ValueError
|
||||
# DeepSeek 可能在 HTTP 200 响应体中报告生成失败。
|
||||
if data.get("error") is not None:
|
||||
return {"error": "DeepSeek search generation failed", "query": query}
|
||||
choices = _objects(data.get("choices"), "choices")
|
||||
message = choices[0].get("message") if choices else None
|
||||
if not isinstance(message, dict):
|
||||
logger.error(
|
||||
"DeepSeek returned malformed choices[0].message container of type %s",
|
||||
type(message).__name__,
|
||||
)
|
||||
raise ValueError
|
||||
finish_reason = choices[0].get("finish_reason")
|
||||
if finish_reason not in {"stop", "length"}:
|
||||
return {"error": "DeepSeek search did not complete", "query": query}
|
||||
|
||||
sources: list[dict[str, Any]] = []
|
||||
search_failed = False
|
||||
for step in _objects(message.get("server_tool_calls"), "server_tool_calls"):
|
||||
for call in _objects(step.get("calls"), "server_tool_calls[].calls"):
|
||||
rows = _objects(call.get("sources"), "server_tool_calls[].calls[].sources")
|
||||
page = call.get("page")
|
||||
if page is not None and not isinstance(page, dict):
|
||||
logger.error("DeepSeek returned malformed server_tool_calls[].calls[].page container of type %s", type(page).__name__)
|
||||
raise ValueError
|
||||
if call.get("status") == "completed":
|
||||
sources.extend(rows)
|
||||
if page is not None:
|
||||
sources.append(page)
|
||||
elif call.get("status") == "failed":
|
||||
search_failed = True
|
||||
|
||||
# 引用可能指向深度搜索中打开、但未出现在搜索列表中的网页。
|
||||
answer = _text(message.get("content"))
|
||||
cited_urls: set[str] = set()
|
||||
spans_by_url: dict[str, dict[tuple[int, int], None]] = {}
|
||||
for annotation in _objects(message.get("annotations"), "annotations"):
|
||||
if annotation.get("type") != "url_citation":
|
||||
continue
|
||||
citation = annotation.get("url_citation")
|
||||
if not isinstance(citation, dict):
|
||||
logger.error(
|
||||
"DeepSeek returned malformed annotations[].url_citation container of type %s",
|
||||
type(citation).__name__,
|
||||
)
|
||||
raise ValueError
|
||||
sources.append(citation)
|
||||
url = safe_public_result_url(citation.get("url"))
|
||||
cited_urls.add(url)
|
||||
start, end = citation.get("start_index"), citation.get("end_index")
|
||||
if not url or type(start) is not int or type(end) is not int:
|
||||
continue
|
||||
if not 0 <= start < end <= len(answer):
|
||||
continue
|
||||
spans_by_url.setdefault(url, {})[start, end] = None
|
||||
|
||||
results_by_url: dict[str, dict[str, Any]] = {}
|
||||
for source in sources:
|
||||
url = safe_public_result_url(source.get("url"))
|
||||
if not url:
|
||||
continue
|
||||
result = results_by_url.setdefault(url, {"title": "", "url": url, "snippet": ""})
|
||||
for field in ("title", "snippet"):
|
||||
if not result[field]:
|
||||
result[field] = _text(source.get(field))
|
||||
results = list(results_by_url.values())
|
||||
if not results:
|
||||
if finish_reason == "length":
|
||||
return {"error": "DeepSeek search reached max_tokens before returning sources", "query": query}
|
||||
return {
|
||||
"error": "DeepSeek web search failed" if search_failed else "No results found",
|
||||
"query": query,
|
||||
}
|
||||
# 优先保留回答实际引用的来源,同时保留搜索结果提供的摘要。
|
||||
results.sort(key=lambda result: result["url"] not in cited_urls)
|
||||
results = results[:max_results]
|
||||
output = {
|
||||
"query": query,
|
||||
"total_results": len(results),
|
||||
"results": results,
|
||||
"answer": answer,
|
||||
"truncated": finish_reason == "length",
|
||||
}
|
||||
retained_spans = sorted({span for result in results for span in spans_by_url.get(result["url"], {})})
|
||||
if retained_spans:
|
||||
source_ids_by_span: dict[tuple[int, int], list[int]] = {span: [] for span in retained_spans}
|
||||
for source_id, result in enumerate(results, start=1):
|
||||
result["source_id"] = source_id
|
||||
for span in spans_by_url.get(result["url"], {}):
|
||||
source_ids_by_span[span].append(source_id)
|
||||
excerpts = redact_text_excerpts(answer, retained_spans, pii_config)
|
||||
output["citations"] = [{"excerpt": excerpt, "source_ids": source_ids_by_span[span]} for span, excerpt in zip(retained_spans, excerpts, strict=True)]
|
||||
return output
|
||||
|
||||
|
||||
def _prepare() -> tuple[dict[str, Any], httpx.AsyncClient | None, str | None, PiiRedactionConfig]:
|
||||
app_config = get_app_config()
|
||||
config = app_config.get_tool_config("web_search")
|
||||
extras = dict(config.model_extra or {}) if config is not None else {}
|
||||
api_key = resolve_api_key(extras.get("api_key")) or resolve_api_key(os.getenv("DEEPSEEK_API_KEY"))
|
||||
if not api_key:
|
||||
return extras, None, "DEEPSEEK_API_KEY is not configured", app_config.pii_redaction
|
||||
# 配置读取与 TLS 初始化统一放到工作线程。
|
||||
client = httpx.AsyncClient(headers={"Authorization": f"Bearer {api_key}"})
|
||||
return extras, client, None, app_config.pii_redaction
|
||||
|
||||
|
||||
async def _close_prepared_client(preparation: asyncio.Task) -> None:
|
||||
try:
|
||||
_, client, _, _ = await preparation
|
||||
if client is not None:
|
||||
await client.aclose()
|
||||
except Exception as exc:
|
||||
logger.error("DeepSeek search initialization cleanup failed (%s)", type(exc).__name__)
|
||||
|
||||
|
||||
@tool("web_search", parse_docstring=True)
|
||||
async def web_search_tool(query: str, max_results: int = _DEFAULT_MAX_RESULTS) -> str:
|
||||
"""Search the web using DeepSeek and return an answer with source links.
|
||||
|
||||
Returns JSON with an answer and results containing title, url, and snippet.
|
||||
When citations are available, results also have source_id values, and each
|
||||
citations entry pairs an excerpt with source_ids referring to those results.
|
||||
Resolve source_ids to result titles and URLs when writing Markdown citations.
|
||||
|
||||
Args:
|
||||
query: Search keywords or a research question describing what to find.
|
||||
max_results: Maximum number of source links to return. Default is 5.
|
||||
"""
|
||||
query = query.strip()
|
||||
if not query:
|
||||
return json.dumps(
|
||||
{"error": "Search query must not be empty", "query": query},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
preparation = asyncio.create_task(asyncio.to_thread(_prepare))
|
||||
try:
|
||||
extras, client, error, pii_config = await asyncio.shield(preparation)
|
||||
except asyncio.CancelledError:
|
||||
# 工作线程仍会创建客户端。独立清理任务持有资源,重复取消时也先等它关闭。
|
||||
cleanup = asyncio.create_task(_close_prepared_client(preparation))
|
||||
while not cleanup.done():
|
||||
try:
|
||||
await asyncio.shield(cleanup)
|
||||
except asyncio.CancelledError:
|
||||
continue
|
||||
raise
|
||||
except Exception as exc:
|
||||
logger.error("DeepSeek search initialization failed (%s)", type(exc).__name__)
|
||||
return json.dumps({"error": "DeepSeek search initialization failed", "query": query}, ensure_ascii=False)
|
||||
if error:
|
||||
return json.dumps({"error": error, "query": query}, ensure_ascii=False)
|
||||
assert client is not None
|
||||
async with drained_async_context(client):
|
||||
try:
|
||||
options = _Options.model_validate(extras)
|
||||
except ValidationError:
|
||||
logger.error("Invalid DeepSeek search tool configuration")
|
||||
return json.dumps(
|
||||
{"error": "Invalid DeepSeek search tool configuration", "query": query},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
max_results = min(
|
||||
coerce_max_results(
|
||||
extras.get("max_results", max_results),
|
||||
provider="DeepSeek",
|
||||
logger=logger,
|
||||
),
|
||||
_MAX_RESULTS,
|
||||
)
|
||||
payload = {
|
||||
"model": options.model,
|
||||
"thinking": {"type": "enabled" if options.thinking else "disabled"},
|
||||
"tools": [{"type": "deepseek_web_search"}],
|
||||
"messages": [
|
||||
{
|
||||
"role": "system",
|
||||
"content": "Search the web for the user's query and answer using retrieved sources. Cite your sources.",
|
||||
},
|
||||
{"role": "user", "content": query},
|
||||
],
|
||||
"max_tokens": options.max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
try:
|
||||
async with asyncio.timeout(options.timeout_seconds):
|
||||
response = await client.post(_ENDPOINT, json=payload, timeout=options.timeout_seconds)
|
||||
response.raise_for_status()
|
||||
result = _normalize(response.json(), query, max_results, pii_config)
|
||||
except httpx.HTTPStatusError as exc:
|
||||
logger.error("DeepSeek search returned HTTP %s", exc.response.status_code)
|
||||
result = {
|
||||
"error": f"DeepSeek search API error: HTTP {exc.response.status_code}",
|
||||
"query": query,
|
||||
}
|
||||
except (httpx.TimeoutException, TimeoutError):
|
||||
result = {"error": "DeepSeek search timed out", "query": query}
|
||||
except httpx.HTTPError as exc:
|
||||
logger.error("DeepSeek search request failed (%s)", type(exc).__name__)
|
||||
result = {"error": "DeepSeek search request failed", "query": query}
|
||||
except (ValueError, TypeError):
|
||||
logger.error("DeepSeek search returned an invalid response format")
|
||||
result = {
|
||||
"error": "DeepSeek returned an unexpected response format",
|
||||
"query": query,
|
||||
}
|
||||
return json.dumps(result, ensure_ascii=False)
|
||||
46
backend/tests/blocking_io/test_deepseek_search.py
Normal file
46
backend/tests/blocking_io/test_deepseek_search.py
Normal file
@ -0,0 +1,46 @@
|
||||
"""Keep DeepSeek provider config reads and HTTPX TLS setup off the loop."""
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider(monkeypatch):
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
threads = []
|
||||
|
||||
def config():
|
||||
threads.append(threading.get_ident())
|
||||
Path(__file__).read_text(encoding="utf-8")
|
||||
return SimpleNamespace(get_tool_config=lambda _name: SimpleNamespace(model_extra={"api_key": "test-key"}), pii_redaction=PiiRedactionConfig())
|
||||
|
||||
def respond(request):
|
||||
return httpx.Response(200, json={"choices": [{"finish_reason": "stop", "message": {"content": "", "server_tool_calls": []}}]})
|
||||
|
||||
def client(**kwargs):
|
||||
threads.append(threading.get_ident())
|
||||
Path(__file__).read_text(encoding="utf-8")
|
||||
return httpx.AsyncClient(transport=httpx.MockTransport(respond), **kwargs)
|
||||
|
||||
private_httpx = SimpleNamespace(**vars(httpx))
|
||||
private_httpx.AsyncClient = client
|
||||
monkeypatch.setattr(module, "get_app_config", config)
|
||||
monkeypatch.setattr(module, "httpx", private_httpx)
|
||||
return module.web_search_tool, threads
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_deepseek_search_initializes_off_loop(provider):
|
||||
tool, threads = provider
|
||||
result = json.loads(await tool.ainvoke({"query": "test"}))
|
||||
assert result["error"] == "No results found"
|
||||
assert len(threads) == 2
|
||||
assert all(thread != threading.get_ident() for thread in threads)
|
||||
34
backend/tests/test_deepseek_search_live.py
Normal file
34
backend/tests/test_deepseek_search_live.py
Normal file
@ -0,0 +1,34 @@
|
||||
"""Opt-in API checks. Set DEER_FLOW_RUN_LIVE_TESTS=1 and DEEPSEEK_API_KEY."""
|
||||
|
||||
import json
|
||||
import os
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from deerflow.community.deepseek import tools as deepseek
|
||||
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
||||
|
||||
pytestmark = [
|
||||
pytest.mark.live,
|
||||
pytest.mark.skipif(os.getenv("DEER_FLOW_RUN_LIVE_TESTS") != "1", reason="Live DeepSeek requests are opt-in"),
|
||||
pytest.mark.skipif(not os.getenv("DEEPSEEK_API_KEY"), reason="DEEPSEEK_API_KEY is required"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("thinking", [False, True])
|
||||
async def test_deepseek_search_api_returns_public_sources(monkeypatch, thinking):
|
||||
extras = {"thinking": thinking, "max_results": 2}
|
||||
monkeypatch.setattr(deepseek, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: SimpleNamespace(model_extra=extras), pii_redaction=PiiRedactionConfig()))
|
||||
result = json.loads(
|
||||
await deepseek.web_search_tool.ainvoke(
|
||||
{
|
||||
"query": "Search the official Python website for the Python 3.13.0 release date and provide its release page URL.",
|
||||
}
|
||||
)
|
||||
)
|
||||
assert "error" not in result
|
||||
assert 0 < result["total_results"] <= 2
|
||||
assert result["answer"]
|
||||
assert all(row["url"].startswith(("https://", "http://")) for row in result["results"])
|
||||
741
backend/tests/test_deepseek_tools.py
Normal file
741
backend/tests/test_deepseek_tools.py
Normal file
@ -0,0 +1,741 @@
|
||||
"""Offline HTTP boundary tests for the DeepSeek community search provider."""
|
||||
|
||||
import asyncio
|
||||
import importlib
|
||||
import json
|
||||
import threading
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from langchain_core.messages import AIMessage
|
||||
from langchain_core.tools import BaseTool
|
||||
from langgraph.graph import START, MessagesState, StateGraph
|
||||
from langgraph.prebuilt import ToolNode
|
||||
from pydantic import SecretStr
|
||||
|
||||
from deerflow.agents.middlewares.pii_redaction_middleware import PiiRedactionMiddleware, redact_text
|
||||
from deerflow.agents.middlewares.tool_output_budget_middleware import ToolOutputBudgetMiddleware
|
||||
from deerflow.agents.middlewares.tool_result_sanitization_middleware import ToolResultSanitizationMiddleware
|
||||
from deerflow.config.app_config import AppConfig
|
||||
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
||||
from deerflow.config.tool_output_config import ToolOutputConfig
|
||||
from deerflow.extensions.registry import ExtensionRegistry
|
||||
from deerflow.reflection import resolve_variable
|
||||
from deerflow.tools import get_available_tools
|
||||
|
||||
|
||||
def _completion(*, sources=None, **message):
|
||||
return {
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"message": {
|
||||
"content": "带来源的回答",
|
||||
"server_tool_calls": [
|
||||
{
|
||||
"calls": [
|
||||
{
|
||||
"status": "completed",
|
||||
"sources": sources
|
||||
if sources is not None
|
||||
else [
|
||||
{"url": "https://example.com/article", "title": "标题", "snippet": "摘要"},
|
||||
],
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
**message,
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider(monkeypatch):
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
state = SimpleNamespace(extras={"api_key": SecretStr(" test-key\n")}, requests=[], clients=[], threads=[], response=_completion(), pii_redaction=PiiRedactionConfig())
|
||||
monkeypatch.delenv("DEEPSEEK_API_KEY", raising=False)
|
||||
|
||||
def config():
|
||||
state.threads.append(threading.get_ident())
|
||||
return SimpleNamespace(get_tool_config=lambda _name: SimpleNamespace(model_extra=state.extras), pii_redaction=state.pii_redaction)
|
||||
|
||||
async def respond(request):
|
||||
state.requests.append(request)
|
||||
if isinstance(state.response, Exception):
|
||||
raise state.response
|
||||
if callable(state.response):
|
||||
return await state.response(request)
|
||||
return httpx.Response(200, json=state.response)
|
||||
|
||||
def client(**kwargs):
|
||||
state.threads.append(threading.get_ident())
|
||||
result = httpx.AsyncClient(transport=httpx.MockTransport(respond), **kwargs)
|
||||
state.clients.append(result)
|
||||
return result
|
||||
|
||||
private_httpx = SimpleNamespace(**vars(httpx))
|
||||
private_httpx.AsyncClient = client
|
||||
monkeypatch.setattr(module, "httpx", private_httpx)
|
||||
monkeypatch.setattr(module, "get_app_config", config)
|
||||
state.tool = resolve_variable("deerflow.community.deepseek.tools:web_search_tool", BaseTool)
|
||||
return state
|
||||
|
||||
|
||||
def test_provider_entry_preserves_the_web_search_interface(provider):
|
||||
assert provider.tool.name == "web_search"
|
||||
assert set(provider.tool.args) == {"query", "max_results"}
|
||||
|
||||
|
||||
def test_configured_provider_is_assembled_and_invoked_by_the_host(provider):
|
||||
config = AppConfig.model_validate(
|
||||
{
|
||||
"tools": [{"name": "web_search", "group": "web", "use": "deerflow.community.deepseek.tools:web_search_tool"}],
|
||||
"sandbox": {"use": "deerflow.sandbox.local:LocalSandboxProvider"},
|
||||
}
|
||||
)
|
||||
tools = get_available_tools(
|
||||
app_config=config,
|
||||
groups=["web"],
|
||||
include_mcp=False,
|
||||
subagent_enabled=False,
|
||||
include_upload_tool=False,
|
||||
extensions=ExtensionRegistry().build(),
|
||||
)
|
||||
(search,) = [tool for tool in tools if tool.name == "web_search"]
|
||||
result = json.loads(search.invoke({"query": "test"}))
|
||||
assert result["total_results"] == 1
|
||||
assert len(provider.requests) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(("thinking", "wire_value"), [(None, "disabled"), (False, "disabled"), (True, "enabled"), ("false", "disabled"), ("true", "enabled")])
|
||||
async def test_thinking_payload_and_normalized_results(provider, thinking, wire_value):
|
||||
if thinking is not None:
|
||||
provider.extras["thinking"] = thinking
|
||||
result = json.loads(await provider.tool.ainvoke({"query": " 最新研究 "}))
|
||||
(request,) = provider.requests
|
||||
assert str(request.url) == "https://api.deepseek.com/chat/completions"
|
||||
assert request.headers["Authorization"] == "Bearer test-key"
|
||||
assert json.loads(request.content) == {
|
||||
"model": "deepseek-flash",
|
||||
"thinking": {"type": wire_value},
|
||||
"tools": [{"type": "deepseek_web_search"}],
|
||||
"messages": [
|
||||
{"role": "system", "content": "Search the web for the user's query and answer using retrieved sources. Cite your sources."},
|
||||
{"role": "user", "content": "最新研究"},
|
||||
],
|
||||
"max_tokens": 8192,
|
||||
"stream": False,
|
||||
}
|
||||
assert result == {
|
||||
"query": "最新研究",
|
||||
"total_results": 1,
|
||||
"results": [{"title": "标题", "url": "https://example.com/article", "snippet": "摘要"}],
|
||||
"answer": "带来源的回答",
|
||||
"truncated": False,
|
||||
}
|
||||
assert provider.clients[0].is_closed
|
||||
assert all(thread != threading.get_ident() for thread in provider.threads)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sources_and_citations_are_screened_deduplicated_and_limited(provider):
|
||||
provider.extras.update(max_results=2, model="deployment-model", max_tokens=4096)
|
||||
provider.response = _completion(
|
||||
sources=[
|
||||
"bad-entry",
|
||||
{"url": "http://127。0。0。1/private"},
|
||||
{"url": "javascript:alert(1)"},
|
||||
{"url": "https://example.com/one", "title": None, "snippet": 123},
|
||||
{"url": "https://example.com/one", "title": "duplicate"},
|
||||
],
|
||||
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/two", "title": "打开的网页"}}],
|
||||
)
|
||||
provider.response["choices"][0]["finish_reason"] = "length"
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
|
||||
assert result["results"] == [
|
||||
{"url": "https://example.com/two", "title": "打开的网页", "snippet": ""},
|
||||
{"url": "https://example.com/one", "title": "duplicate", "snippet": ""},
|
||||
]
|
||||
assert result["total_results"] == 2
|
||||
assert result["truncated"] is True
|
||||
payload = json.loads(provider.requests[0].content)
|
||||
assert payload["model"] == "deployment-model"
|
||||
assert payload["max_tokens"] == 4096
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cited_source_beyond_result_limit_keeps_its_search_snippet(provider):
|
||||
provider.response = _completion(
|
||||
sources=[{"url": f"https://example.com/{index}", "snippet": f"summary-{index}"} for index in range(6)],
|
||||
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/5", "title": "引用"}}],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
|
||||
assert result["results"] == [{"url": "https://example.com/5", "title": "引用", "snippet": "summary-5"}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multistep_search_preserves_sources_and_citations_from_all_steps(provider):
|
||||
provider.response = _completion(
|
||||
server_tool_calls=[
|
||||
{"reasoning_content": "first step", "calls": [{"status": "completed", "kind": "deep_search", "sources": [{"url": "https://example.com/one", "snippet": "first"}]}]},
|
||||
{
|
||||
"reasoning_content": "second step",
|
||||
"calls": [
|
||||
{"status": "completed", "action": {"type": "open_page"}, "page": {"url": "https://example.com/page", "title": "opened"}},
|
||||
{"status": "completed", "sources": [None, {"url": "https://example.com/two", "title": "second"}]},
|
||||
{"status": "failed", "error": {"code": "unavailable"}},
|
||||
],
|
||||
},
|
||||
],
|
||||
annotations=[{"type": "url_citation", "url_citation": {"url": "https://example.com/page", "title": "cited page"}}],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["results"] == [
|
||||
{"url": "https://example.com/page", "title": "opened", "snippet": ""},
|
||||
{"url": "https://example.com/one", "title": "", "snippet": "first"},
|
||||
{"url": "https://example.com/two", "title": "second", "snippet": ""},
|
||||
]
|
||||
assert "reasoning_content" not in result
|
||||
assert "first step" not in json.dumps(result)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_opened_page_is_a_source_without_annotations(provider):
|
||||
provider.response = _completion(
|
||||
server_tool_calls=[
|
||||
{
|
||||
"calls": [
|
||||
{"status": "completed", "page": {"url": "https://example.com/page", "title": "opened"}},
|
||||
{"status": "completed", "page": {"url": "http://127.0.0.1/private", "title": "private"}},
|
||||
]
|
||||
}
|
||||
]
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["results"] == [{"url": "https://example.com/page", "title": "opened", "snippet": ""}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
"payload",
|
||||
[
|
||||
[],
|
||||
{"choices": "bad"},
|
||||
{"choices": ["bad"]},
|
||||
{"choices": [{"message": []}]},
|
||||
_completion(server_tool_calls="bad"),
|
||||
_completion(server_tool_calls=[{"calls": 12}]),
|
||||
_completion(server_tool_calls=[{"calls": [{"status": "completed", "page": "bad"}]}]),
|
||||
_completion(sources="bad"),
|
||||
_completion(sources=["bad"]),
|
||||
_completion(annotations=[{"type": "url_citation", "url_citation": []}]),
|
||||
],
|
||||
)
|
||||
async def test_malformed_containers_are_errors(provider, payload, caplog):
|
||||
provider.response = payload
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
|
||||
assert "unexpected response format" in result["error"]
|
||||
assert "private-query" not in caplog.text
|
||||
assert "test-key" not in caplog.text
|
||||
assert provider.clients[0].is_closed
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("rows", [[], None])
|
||||
async def test_empty_search_is_reported(provider, rows):
|
||||
provider.response = _completion(server_tool_calls=[{"calls": [{"status": "completed", "sources": rows}]}])
|
||||
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "No results found"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_key_and_empty_query_do_not_create_clients(provider):
|
||||
provider.extras.clear()
|
||||
assert "not configured" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
assert "empty" in json.loads(await provider.tool.ainvoke({"query": " "}))["error"]
|
||||
assert provider.clients == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_environment_key_fallback(provider, monkeypatch):
|
||||
provider.extras["api_key"] = "your-deepseek-api-key"
|
||||
monkeypatch.setenv("DEEPSEEK_API_KEY", " env-key\n")
|
||||
await provider.tool.ainvoke({"query": "test"})
|
||||
assert provider.requests[0].headers["Authorization"] == "Bearer env-key"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_configured_key_precedes_environment(provider, monkeypatch):
|
||||
monkeypatch.setenv("DEEPSEEK_API_KEY", "environment-key")
|
||||
await provider.tool.ainvoke({"query": "test"})
|
||||
assert provider.requests[0].headers["Authorization"] == "Bearer test-key"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_environment_key_works_without_tool_extras(provider, monkeypatch):
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
monkeypatch.setattr(module, "get_app_config", lambda: SimpleNamespace(get_tool_config=lambda name: None, pii_redaction=provider.pii_redaction))
|
||||
monkeypatch.setenv("DEEPSEEK_API_KEY", "environment-key")
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["total_results"] == 1
|
||||
assert provider.requests[0].headers["Authorization"] == "Bearer environment-key"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("options", [{"thinking": "invalid"}, {"timeout_seconds": 0}, {"timeout_seconds": True}, {"timeout_seconds": float("inf")}, {"max_tokens": True}, {"max_tokens": 1.5}, {"max_tokens": 0}, {"model": " "}])
|
||||
async def test_invalid_options_return_configuration_error_and_close_client(provider, options):
|
||||
provider.extras.update(options)
|
||||
assert "configuration" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
assert provider.requests == []
|
||||
assert provider.clients[0].is_closed
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_numeric_settings_accept_environment_strings(provider):
|
||||
provider.extras.update(max_tokens="4096", timeout_seconds="120")
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["total_results"] == 1
|
||||
assert json.loads(provider.requests[0].content)["max_tokens"] == 4096
|
||||
assert provider.requests[0].extensions["timeout"]["read"] == 120.0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_http_and_embedded_errors_do_not_leak_provider_details(provider, caplog):
|
||||
async def unauthorized(request):
|
||||
return httpx.Response(401, text="secret test-key private-query", request=request)
|
||||
|
||||
provider.response = unauthorized
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
|
||||
assert result["error"] == "DeepSeek search API error: HTTP 401"
|
||||
provider.response = {"error": {"message": "secret test-key"}, "usage": {}}
|
||||
assert "generation failed" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
assert "secret" not in caplog.text
|
||||
assert "test-key" not in caplog.text
|
||||
assert "private-query" not in caplog.text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transport_timeout_and_invalid_json(provider):
|
||||
provider.response = httpx.ReadTimeout("secret test-key")
|
||||
assert "timed out" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
provider.response = httpx.ConnectError("secret test-key")
|
||||
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "DeepSeek search request failed"
|
||||
|
||||
async def html(request):
|
||||
return httpx.Response(200, text="<html>Error</html>")
|
||||
|
||||
provider.response = html
|
||||
assert "unexpected response format" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
assert all(client.is_closed for client in provider.clients)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_deadline_cancels_slow_transport(provider):
|
||||
provider.extras["timeout_seconds"] = 0.01
|
||||
|
||||
async def wait(request):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
provider.response = wait
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["error"] == "DeepSeek search timed out"
|
||||
assert provider.clients[0].is_closed
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_failed_search_is_distinct_from_empty_results(provider):
|
||||
provider.response = _completion(server_tool_calls=[{"calls": [{"status": "failed", "error": {"code": "timeout"}}]}])
|
||||
assert json.loads(await provider.tool.ainvoke({"query": "test"}))["error"] == "DeepSeek web search failed"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_exhausted_generation_budget_is_distinct_from_empty_results(provider):
|
||||
provider.response = _completion(server_tool_calls=[])
|
||||
provider.response["choices"][0]["finish_reason"] = "length"
|
||||
assert "max_tokens" in json.loads(await provider.tool.ainvoke({"query": "test"}))["error"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_content_filter_does_not_return_a_fabricated_answer(provider):
|
||||
provider.response = _completion()
|
||||
provider.response["choices"][0]["finish_reason"] = "content_filter"
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["error"] == "DeepSeek search did not complete"
|
||||
assert "answer" not in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_client_initialization_failure_contains_sensitive_exception(provider, monkeypatch, caplog):
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
|
||||
def fail(**kwargs):
|
||||
raise OSError("test-key private-query")
|
||||
|
||||
monkeypatch.setattr(module.httpx, "AsyncClient", fail)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "private-query"}))
|
||||
assert result["error"] == "DeepSeek search initialization failed"
|
||||
assert "test-key" not in caplog.text
|
||||
assert "private-query" not in caplog.text
|
||||
assert provider.requests == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("value, count", [(1000, 50), ("3", 3), (False, 5), (0, 5)])
|
||||
async def test_configured_result_count_is_bounded(provider, value, count):
|
||||
provider.extras["max_results"] = value
|
||||
provider.response = _completion(sources=[{"url": f"https://example.com/{index}"} for index in range(60)])
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["total_results"] == count
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("cancel_count", [1, 2])
|
||||
async def test_cancellation_during_client_creation_closes_late_client(provider, monkeypatch, cancel_count):
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
original = module.httpx.AsyncClient
|
||||
started = asyncio.Event()
|
||||
created = asyncio.Event()
|
||||
release = threading.Event()
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
def delayed_client(**kwargs):
|
||||
loop.call_soon_threadsafe(started.set)
|
||||
release.wait()
|
||||
client = original(**kwargs)
|
||||
loop.call_soon_threadsafe(created.set)
|
||||
return client
|
||||
|
||||
monkeypatch.setattr(module.httpx, "AsyncClient", delayed_client)
|
||||
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
|
||||
try:
|
||||
await asyncio.wait_for(started.wait(), 5)
|
||||
task.cancel()
|
||||
if cancel_count == 2:
|
||||
checkpoint = asyncio.Event()
|
||||
loop.call_soon(checkpoint.set)
|
||||
await checkpoint.wait()
|
||||
task.cancel()
|
||||
finally:
|
||||
release.set()
|
||||
if not task.cancelling():
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
await asyncio.wait_for(created.wait(), 5)
|
||||
assert provider.clients[0].is_closed
|
||||
assert provider.requests == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancellation_propagates_and_closes_client(provider):
|
||||
started = asyncio.Event()
|
||||
|
||||
async def wait(request):
|
||||
started.set()
|
||||
await asyncio.Event().wait()
|
||||
|
||||
provider.response = wait
|
||||
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
|
||||
try:
|
||||
await asyncio.wait_for(started.wait(), 5)
|
||||
finally:
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
assert provider.clients[0].is_closed
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("trigger", ["request_cancel", "request_timeout", "response_complete"])
|
||||
async def test_repeated_cancellation_drains_transport_teardown(provider, monkeypatch, trigger):
|
||||
request_started = asyncio.Event()
|
||||
close_started = asyncio.Event()
|
||||
release_close = asyncio.Event()
|
||||
close_completed = asyncio.Event()
|
||||
|
||||
async def respond(request):
|
||||
request_started.set()
|
||||
if trigger != "response_complete":
|
||||
await asyncio.Event().wait()
|
||||
return httpx.Response(200, json=_completion())
|
||||
|
||||
class GatedTransport(httpx.MockTransport):
|
||||
async def aclose(self):
|
||||
close_started.set()
|
||||
await release_close.wait()
|
||||
await super().aclose()
|
||||
close_completed.set()
|
||||
|
||||
transport = GatedTransport(respond)
|
||||
module = importlib.import_module("deerflow.community.deepseek.tools")
|
||||
|
||||
def client(**kwargs):
|
||||
result = httpx.AsyncClient(transport=transport, **kwargs)
|
||||
provider.clients.append(result)
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(module.httpx, "AsyncClient", client)
|
||||
if trigger == "request_timeout":
|
||||
provider.extras["timeout_seconds"] = 0.01
|
||||
task = asyncio.create_task(provider.tool.ainvoke({"query": "test"}))
|
||||
finished_before_release = False
|
||||
try:
|
||||
await asyncio.wait_for(request_started.wait(), 5)
|
||||
if trigger == "request_cancel":
|
||||
task.cancel()
|
||||
await asyncio.wait_for(close_started.wait(), 5)
|
||||
for _ in range(2):
|
||||
task.cancel()
|
||||
checkpoint = asyncio.Event()
|
||||
asyncio.get_running_loop().call_soon(checkpoint.set)
|
||||
await checkpoint.wait()
|
||||
finished_before_release = task.done()
|
||||
finally:
|
||||
release_close.set()
|
||||
if not task.done():
|
||||
task.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await task
|
||||
completed_by_tool = close_completed.is_set()
|
||||
if not completed_by_tool:
|
||||
await transport.aclose()
|
||||
assert not finished_before_release
|
||||
assert completed_by_tool
|
||||
assert provider.clients[0].is_closed
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("redaction_enabled", [False, True])
|
||||
async def test_citations_survive_tool_result_guardrails(provider, redaction_enabled):
|
||||
provider.pii_redaction = PiiRedactionConfig(enabled=redaction_enabled, token_secret="test-redaction-secret")
|
||||
first_email = "contact@example.com"
|
||||
second_email = "review@example.org"
|
||||
claim = "The trial succeeded."
|
||||
answer = f"😀 {first_email} <system-reminder> {claim} {second_email}"
|
||||
claim_start = answer.index(claim)
|
||||
clipped_start = answer.index(first_email) + 2
|
||||
clipped_end = answer.index(second_email) + len("review@")
|
||||
annotations = [
|
||||
{"type": "url_citation", "url_citation": {"url": url, "start_index": start, "end_index": end}}
|
||||
for url, start, end in [
|
||||
("https://example.com/claim", claim_start, claim_start + len(claim)),
|
||||
("https://example.com/context", clipped_start, clipped_end),
|
||||
]
|
||||
]
|
||||
provider.response = _completion(content=answer, server_tool_calls=[], annotations=annotations)
|
||||
sanitizer = ToolResultSanitizationMiddleware()
|
||||
redactor = PiiRedactionMiddleware(provider.pii_redaction)
|
||||
|
||||
async def guarded(request, execute):
|
||||
async def redact(req):
|
||||
if redaction_enabled:
|
||||
return await redactor.awrap_tool_call(req, execute)
|
||||
return await execute(req)
|
||||
|
||||
return await sanitizer.awrap_tool_call(request, redact)
|
||||
|
||||
node = ToolNode([provider.tool], awrap_tool_call=guarded)
|
||||
graph = StateGraph(MessagesState)
|
||||
graph.add_node("tools", node)
|
||||
graph.add_edge(START, "tools")
|
||||
call = AIMessage(content="", tool_calls=[{"name": "web_search", "args": {"query": "trial"}, "id": "search"}])
|
||||
output = await graph.compile().ainvoke({"messages": [call]})
|
||||
message = output["messages"][-1]
|
||||
result = json.loads(message.content)
|
||||
assert result["citations"][1] == {"excerpt": claim, "source_ids": [1]}
|
||||
expected_context = answer[clipped_start:clipped_end]
|
||||
if redaction_enabled:
|
||||
expected_context = redact_text(answer[answer.index(first_email) :], provider.pii_redaction)
|
||||
assert "contact@example.com" not in message.content
|
||||
assert "ntact@example.com" not in message.content
|
||||
assert "review@" not in message.content
|
||||
expected_context = expected_context.replace("<system-reminder>", "<system-reminder>")
|
||||
assert result["citations"][0] == {"excerpt": expected_context, "source_ids": [2]}
|
||||
assert expected_context in result["answer"]
|
||||
assert "<system-reminder>" not in message.content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_citations_preserve_claim_associations_and_unicode_offsets(provider):
|
||||
answer = "😀甲。乙。"
|
||||
first = {"url": "https://example.com/one", "title": "first", "start_index": 1, "end_index": 2}
|
||||
second = {"url": "https://example.com/two", "title": "second", "start_index": 3, "end_index": len(answer)}
|
||||
provider.response = _completion(
|
||||
content=answer,
|
||||
sources=[first, second],
|
||||
annotations=[{"type": "url_citation", "url_citation": first}, {"type": "url_citation", "url_citation": second}],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
first["start_index"], second["start_index"] = second["start_index"], first["start_index"]
|
||||
first["end_index"], second["end_index"] = second["end_index"], first["end_index"]
|
||||
swapped = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result != swapped
|
||||
assert result["answer"] == swapped["answer"] == answer
|
||||
assert result["citations"] == [{"excerpt": "甲", "source_ids": [1]}, {"excerpt": "乙。", "source_ids": [2]}]
|
||||
assert swapped["citations"] == [{"excerpt": "甲", "source_ids": [2]}, {"excerpt": "乙。", "source_ids": [1]}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
("start", "end"),
|
||||
[(None, None), (None, 1), (0, None), ("0", 1), (0, "1"), (False, 1), (0, True), (0.0, 1), (0, 1.0), (-1, 1), (0, 0), (2, 1), (0, 4)],
|
||||
)
|
||||
async def test_invalid_citation_spans_preserve_legacy_source_shape(provider, start, end):
|
||||
citation = {"url": "https://example.com/cited", "title": "source", "start_index": start, "end_index": end}
|
||||
provider.response = _completion(content="甲😀乙", server_tool_calls=[], annotations=[{"type": "url_citation", "url_citation": citation}])
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result == {
|
||||
"query": "test",
|
||||
"total_results": 1,
|
||||
"results": [{"title": "source", "url": citation["url"], "snippet": ""}],
|
||||
"answer": "甲😀乙",
|
||||
"truncated": False,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_citations_preserve_whitespace_multiple_sources_and_distinct_ranges(provider):
|
||||
answer = " 😀甲乙\n"
|
||||
annotations = [
|
||||
{"type": "url_citation", "url_citation": {"url": url, "start_index": start, "end_index": end}}
|
||||
for url, start, end in [
|
||||
("https://example.com/one", 2, 4),
|
||||
("https://example.com/one", 2, 4),
|
||||
("https://example.com/one", 3, len(answer)),
|
||||
("https://example.com/two", 2, 4),
|
||||
]
|
||||
]
|
||||
provider.response = _completion(content=answer, server_tool_calls=[], annotations=annotations)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["answer"] == answer
|
||||
assert result["citations"] == [{"excerpt": "😀甲", "source_ids": [1, 2]}, {"excerpt": "甲乙\n", "source_ids": [1]}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_citations_follow_url_screening_and_result_limit(provider):
|
||||
urls = ["https://example.com/kept", "https://example.com/limited", "http://127。0。0。1/private", "javascript:alert(1)"]
|
||||
provider.response = _completion(
|
||||
content="claim",
|
||||
sources=[{"url": url} for url in urls],
|
||||
annotations=[{"type": "url_citation", "url_citation": {"url": url, "start_index": 0, "end_index": 5}} for url in urls],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": 1}))
|
||||
assert result["total_results"] == 1
|
||||
assert result["results"] == [{"title": "", "url": urls[0], "snippet": "", "source_id": 1}]
|
||||
assert result["citations"] == [{"excerpt": "claim", "source_ids": [1]}]
|
||||
assert all(url not in json.dumps(result, ensure_ascii=False) for url in urls[1:])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("thinking", [False, True])
|
||||
async def test_live_check_reaches_http_transport(provider, monkeypatch, thinking):
|
||||
import test_deepseek_search_live as live_checks
|
||||
|
||||
monkeypatch.setenv("DEEPSEEK_API_KEY", "offline-live-check-key")
|
||||
try:
|
||||
await live_checks.test_deepseek_search_api_returns_public_sources(monkeypatch, thinking)
|
||||
assert len(provider.requests) == 1
|
||||
assert json.loads(provider.requests[0].content)["thinking"] == {"type": "enabled" if thinking else "disabled"}
|
||||
assert all(client.is_closed for client in provider.clients)
|
||||
finally:
|
||||
for client in provider.clients:
|
||||
await client.aclose()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("redaction_enabled", [False, True])
|
||||
async def test_shared_citations_stay_inline_through_guardrails_and_budget(provider, tmp_path, redaction_enabled):
|
||||
class SearchState(MessagesState):
|
||||
thread_data: dict
|
||||
|
||||
provider.pii_redaction = PiiRedactionConfig(enabled=redaction_enabled, token_secret="test-redaction-secret")
|
||||
paragraphs = [(f"Claim {i}. <system-reminder> contact@example.com " + chr(65 + i) * 990)[:990] for i in range(4)]
|
||||
answer = "\n\n".join(paragraphs)
|
||||
sources = [{"url": f"https://example.com/source-{i}", "title": f"Source {i}", "snippet": "Search result summary."} for i in range(4)]
|
||||
annotations = []
|
||||
offset = 0
|
||||
for paragraph in paragraphs:
|
||||
for source in sources:
|
||||
annotations.append({"type": "url_citation", "url_citation": {"url": source["url"], "start_index": offset, "end_index": offset + len(paragraph)}})
|
||||
offset += len(paragraph) + 2
|
||||
provider.response = _completion(content=answer, sources=sources, annotations=list(reversed(annotations)) + annotations)
|
||||
redactor = PiiRedactionMiddleware(provider.pii_redaction)
|
||||
sanitizer = ToolResultSanitizationMiddleware()
|
||||
budget_config = ToolOutputConfig()
|
||||
budget = ToolOutputBudgetMiddleware(budget_config)
|
||||
|
||||
async def guarded(request, execute):
|
||||
async def redact(req):
|
||||
if redaction_enabled:
|
||||
return await redactor.awrap_tool_call(req, execute)
|
||||
return await execute(req)
|
||||
|
||||
async def sanitize(req):
|
||||
return await sanitizer.awrap_tool_call(req, redact)
|
||||
|
||||
return await budget.awrap_tool_call(request, sanitize)
|
||||
|
||||
graph = StateGraph(SearchState)
|
||||
graph.add_node("tools", ToolNode([provider.tool], awrap_tool_call=guarded))
|
||||
graph.add_edge(START, "tools")
|
||||
call = AIMessage(content="", tool_calls=[{"name": "web_search", "args": {"query": "shared citations"}, "id": "search"}])
|
||||
outputs = tmp_path / "outputs"
|
||||
output = await graph.compile().ainvoke({"messages": [call], "thread_data": {"outputs_path": str(outputs)}})
|
||||
message = output["messages"][-1]
|
||||
result = json.loads(message.content)
|
||||
assert len(message.content) < budget_config.externalize_min_chars
|
||||
assert not outputs.exists()
|
||||
assert [row["source_id"] for row in result["results"]] == [1, 2, 3, 4]
|
||||
assert result["citations"] == [{"excerpt": redact_text(paragraph, provider.pii_redaction).replace("<system-reminder>", "<system-reminder>"), "source_ids": [1, 2, 3, 4]} for paragraph in paragraphs]
|
||||
assert "<system-reminder>" not in message.content
|
||||
if redaction_enabled:
|
||||
assert "contact@example.com" not in message.content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_equal_excerpt_text_at_distinct_ranges_keeps_separate_attribution(provider):
|
||||
answer = "同一句。同一句。"
|
||||
provider.response = _completion(
|
||||
content=answer,
|
||||
sources=[{"url": "https://example.com/one"}, {"url": "https://example.com/two"}],
|
||||
annotations=[
|
||||
{"type": "url_citation", "url_citation": {"url": "https://example.com/two", "start_index": 4, "end_index": 8}},
|
||||
{"type": "url_citation", "url_citation": {"url": "https://example.com/one", "start_index": 0, "end_index": 4}},
|
||||
],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["citations"] == [{"excerpt": "同一句。", "source_ids": [1]}, {"excerpt": "同一句。", "source_ids": [2]}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("max_results", [1, 2, 3])
|
||||
async def test_citation_ids_follow_retained_source_order(provider, max_results):
|
||||
provider.response = _completion(
|
||||
content="AB",
|
||||
sources=[{"url": "https://example.com/uncited"}, {"url": "https://example.com/b"}, {"url": "https://example.com/c"}],
|
||||
annotations=[
|
||||
{"type": "url_citation", "url_citation": {"url": "https://example.com/c", "start_index": 0, "end_index": 1}},
|
||||
{"type": "url_citation", "url_citation": {"url": "https://example.com/b", "start_index": 1, "end_index": 2}},
|
||||
],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test", "max_results": max_results}))
|
||||
assert [(row["source_id"], row["url"]) for row in result["results"]] == [(1, "https://example.com/b"), (2, "https://example.com/c"), (3, "https://example.com/uncited")][:max_results]
|
||||
assert result["citations"] == ([{"excerpt": "B", "source_ids": [1]}] if max_results == 1 else [{"excerpt": "A", "source_ids": [2]}, {"excerpt": "B", "source_ids": [1]}])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_distinct_long_overlapping_ranges_preserve_full_excerpts(provider):
|
||||
answer = "abcdef" * 660
|
||||
provider.response = _completion(
|
||||
content=answer,
|
||||
sources=[{"url": f"https://example.com/{i}"} for i in range(4)],
|
||||
annotations=[{"type": "url_citation", "url_citation": {"url": f"https://example.com/{i}", "start_index": i, "end_index": len(answer)}} for i in range(4)],
|
||||
)
|
||||
result = json.loads(await provider.tool.ainvoke({"query": "test"}))
|
||||
assert result["answer"] == answer
|
||||
assert result["citations"] == [{"excerpt": answer[i:], "source_ids": [i + 1]} for i in range(4)]
|
||||
assert len(json.dumps(result)) > ToolOutputConfig().externalize_min_chars
|
||||
@ -1010,6 +1010,18 @@ tools:
|
||||
# # mode: 0 # Optional: 0=web, 1=VR, 2=mixed
|
||||
# # api_key: $TENCENTCLOUD_WSA_APIKEY # Optional explicit env-var reference
|
||||
|
||||
# Web search tool (DeepSeek server-side search, requires DEEPSEEK_API_KEY)
|
||||
# Replace the active web_search entry with this provider.
|
||||
# - name: web_search
|
||||
# group: web
|
||||
# use: deerflow.community.deepseek.tools:web_search_tool
|
||||
# api_key: $DEEPSEEK_API_KEY
|
||||
# model: deepseek-flash
|
||||
# thinking: false # true enables multi-step thinking search
|
||||
# max_results: 5 # Returned sources, capped at 50
|
||||
# timeout_seconds: 180 # Includes server-side search and reasoning
|
||||
# max_tokens: 8192
|
||||
|
||||
# Web search tool (uses Exa, requires EXA_API_KEY)
|
||||
# - name: web_search
|
||||
# group: web
|
||||
|
||||
@ -119,6 +119,8 @@ fences, including unclosed fences, in complete and streamed replies.
|
||||
|
||||
The wizard also lets you configure an optional web search provider, or skip it for now.
|
||||
|
||||
[DeepSeek web search](../../backend/docs/CONFIGURATION.md#deepseek-web-search) uses `deerflow.community.deepseek.tools:web_search_tool`. Configure `DEEPSEEK_API_KEY` in the Gateway environment and replace the active `web_search` entry with the example in `config.example.yaml`. Set `thinking: true` for multi-step thinking search, independently of the main DeerFlow model. The default is `false`. Results include a generated answer and screened, deduplicated source links.
|
||||
|
||||
Brave web search preserves valid entries in mixed result lists. Malformed response containers or lists containing no usable entries return a structured format error; missing, null, or empty results keep the existing "No results found" response. Format errors also log the malformed container's path and type, or the absence of usable result objects, without including search queries, credentials, or payload values.
|
||||
|
||||
Several providers, including You.com, GroundRoute, Webz, InfoQuest and Jsonify, initialize HTTPX clients in a worker thread so synchronous TLS trust loading does not block the agent event loop; requests and session cleanup remain asynchronous.
|
||||
|
||||
@ -2,7 +2,12 @@ import { expect, test } from "@playwright/test";
|
||||
|
||||
import { mockLangGraphAPI } from "./utils/mock-api";
|
||||
|
||||
for (const shape of ["array", "webz", "webz-malformed"] as const) {
|
||||
for (const shape of [
|
||||
"array",
|
||||
"webz",
|
||||
"webz-malformed",
|
||||
"deepseek-citations",
|
||||
] as const) {
|
||||
test(`search source links from ${shape} results survive reload`, async ({
|
||||
page,
|
||||
}) => {
|
||||
@ -23,14 +28,31 @@ for (const shape of ["array", "webz", "webz-malformed"] as const) {
|
||||
});
|
||||
results.push({ title: "Invalid URL", url: 42 });
|
||||
}
|
||||
if (shape === "deepseek-citations") {
|
||||
results[0] = {
|
||||
source_id: 1,
|
||||
title: "Renewable energy news",
|
||||
url: "https://example.com/news",
|
||||
snippet: "Matching passage",
|
||||
};
|
||||
}
|
||||
const payload =
|
||||
shape === "array"
|
||||
? results
|
||||
: {
|
||||
query: "renewable energy",
|
||||
returned_results: results.length,
|
||||
results,
|
||||
};
|
||||
: shape === "deepseek-citations"
|
||||
? {
|
||||
query: "renewable energy",
|
||||
total_results: results.length,
|
||||
results,
|
||||
answer: "Matching passage",
|
||||
citations: [{ excerpt: "Matching passage", source_ids: [1] }],
|
||||
truncated: false,
|
||||
}
|
||||
: {
|
||||
query: "renewable energy",
|
||||
returned_results: results.length,
|
||||
results,
|
||||
};
|
||||
mockLangGraphAPI(page, {
|
||||
threads: [
|
||||
{
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user