github/deer-flow
1
0
Fork 0
mirror of https://github.com/bytedance/deer-flow.git synced 2026-10-26 21:26:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
deer-flow/scripts/wizard/providers.py
Value Add 4beb6c4ea4
feat(community): add You.com web search provider (#6515) 
* feat(community): add You.com web search provider

Adds You.com as an opt-in community `web_search` engine, mirroring the
existing community engines (tavily, brave, exa, firecrawl, groundroute).

You.com runs its own web index and returns extracted page snippets per
hit rather than a single meta description, so a research loop gets usable
context without a follow-up fetch.

The provider picks its endpoint by whether a key is configured, so it
works out of the box and scales up when the operator adds one:

- no key  -> /v1/agents/search, keyless free tier
- YDC_API_KEY -> /v1/search with X-API-Key, full Search API

Endpoint and headers are always chosen together; the keyless endpoint
rejects an auth header, so a stale key can never be sent there. Both
return the same body shape, so one normalizer covers both.

httpx only, no new dependency and no pyproject extra, so uv.lock is
untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(community/youcom): review follow-ups — drop stray .pr-body.md, guard OverflowError, add 429 hint

* fix(doctor): make the You.com free-provider detail true with or without YDC_API_KEY

The doctor detail returned before any key check, so operators who had
set YDC_API_KEY still saw 'set YDC_API_KEY for the full Search API'.
Reworded to match the sibling free-provider entries and stay accurate in
both states.

* fix(search): sweep youcom max_results onto the shared owner

The single-owner gate that landed with #6563 flags this branch's private
_coerce_max_results as a newly hand-rolled copy, so backend-unit-tests
now fails after merging main. Follow the coordination note from review:
delegate to deerflow.community.search_max_results.coerce_max_results,
which owns the rejection profile (booleans, non-integral floats,
OverflowError), and keep only the 1-100 mirror of You.com's server-side
count clamp on top — clamping policy stays per-provider.

The .inf regression test now goes through the public tool path and also
pins the bool and non-integral-float shapes of the shared bar.

* docs(tools): use the bare you.com platform link in config and docs

Match every other provider entry: no other provider in config.example.yaml
or the en/zh tools.mdx uses campaign-tagged URLs, so drop the utm params
from the three you.com/platform links. Integration attribution stays in
the identifying User-Agent sent to You.com hosts.

* test(youcom): cover shared max_results ownership and bounds

Register You.com among the shared-owner providers and assert default handling and the 100-result cap through outbound request parameters.

---------

Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: valueadd-lgtm <339269770+valueadd-lgtm@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-11 11:56:32 +08:00

792 lines
28 KiB
Python
Raw Blame History

"""LLM and search provider definitions for the Setup Wizard."""
from __future__ import annotations
from dataclasses import dataclass, field, replace
@dataclass
class LLMProvider:
name: str
display_name: str
description: str
use: str
models: list[str]
default_model: str
env_var: str | None
package: str | None
# Optional: some providers use a different field name for the API key in YAML
api_key_field: str = "api_key"
# Extra config fields beyond the common ones (merged into YAML)
extra_config: dict = field(default_factory=dict)
# Per-model supports_vision overrides for providers whose models differ in
# capability (e.g. MiniMax M3 supports vision but M2.7 is text-only). The
# provider-level extra_config holds the default (default_model) capability.
model_vision_overrides: dict[str, bool] = field(default_factory=dict)
auth_hint: str | None = None
base_url_prompt: str | None = None
model_prompt: str | None = None
# For generic OpenAI-compatible gateways the wizard cannot infer whether the
# user-supplied model supports thinking/reasoning, so prompt for it explicitly.
ask_thinking_support: bool = False
def extra_config_for(self, model_name: str) -> dict:
"""Return extra_config for a selected model, applying per-model overrides.
Does not mutate the shared provider-level ``extra_config``.
"""
config = dict(self.extra_config)
if model_name in self.model_vision_overrides:
config["supports_vision"] = self.model_vision_overrides[model_name]
return config
@dataclass
class WebProvider:
name: str
display_name: str
description: str
use: str
env_var: str | None # None = no API key required
tool_name: str
extra_config: dict = field(default_factory=dict)
@dataclass
class SearchProvider:
name: str
display_name: str
description: str
use: str
env_var: str | None # None = no API key required
tool_name: str = "web_search"
extra_config: dict = field(default_factory=dict)
OPENAI_COMPAT_THINKING_CONFIG = {
"supports_thinking": True,
"when_thinking_enabled": {
"extra_body": {
"thinking": {
"type": "enabled",
}
}
},
"when_thinking_disabled": {
"extra_body": {
"thinking": {
"type": "disabled",
}
}
},
}
ANTHROPIC_THINKING_CONFIG = {
"supports_thinking": True,
"when_thinking_enabled": {
"thinking": {
"type": "enabled",
"budget_tokens": 4096,
}
},
"when_thinking_disabled": {
"thinking": {
"type": "disabled",
}
},
}
def with_thinking_support(provider: LLMProvider, supports_thinking: bool) -> LLMProvider:
"""Return a copy of *provider* with thinking-capability flags applied.
For generic OpenAI-compatible gateways the wizard cannot infer whether the
user-supplied model supports thinking/reasoning. When the user confirms
support we also wire the common OpenAI-compatible enable/disable toggles so
the runtime can switch thinking on and off; otherwise we record the
capability as unsupported. The shared provider definition is never mutated.
"""
if supports_thinking:
extra_config = {**provider.extra_config, **OPENAI_COMPAT_THINKING_CONFIG}
else:
extra_config = {**provider.extra_config, "supports_thinking": False}
return replace(provider, extra_config=extra_config)
LLM_PROVIDERS: list[LLMProvider] = [
LLMProvider(
name="volcengine",
display_name="Volcengine Doubao",
description="Doubao Seed with thinking support",
use="deerflow.models.patched_deepseek:PatchedChatDeepSeek",
models=["doubao-seed-1-8-251228"],
default_model="doubao-seed-1-8-251228",
env_var="VOLCENGINE_API_KEY",
package="langchain-deepseek",
extra_config={
"api_base": "https://ark.cn-beijing.volces.com/api/v3",
"timeout": 600.0,
"max_retries": 2,
"supports_vision": True,
"supports_reasoning_effort": True,
**OPENAI_COMPAT_THINKING_CONFIG,
},
),
LLMProvider(
name="volcengine_codingplan",
display_name="Volcengine Coding Plan",
description="One key, multi-vendor models (Doubao/GLM/DeepSeek/Kimi/MiniMax)",
use="deerflow.models.patched_deepseek:PatchedChatDeepSeek",
models=[
"doubao-seed-2.0-code",
"doubao-seed-2.0-pro",
"doubao-seed-2.0-lite",
"doubao-seed-code",
"minimax-m2.7",
"minimax-m3",
"glm-5.2",
"deepseek-v4-flash",
"deepseek-v4-pro",
"kimi-k2.6",
"kimi-k2.7-code",
],
default_model="glm-5.2",
env_var="VOLCENGINE_API_KEY",
package="langchain-deepseek",
extra_config={
"api_base": "https://ark.cn-beijing.volces.com/api/coding/v3",
"timeout": 600.0,
"max_retries": 2,
"supports_vision": True,
"supports_reasoning_effort": True,
**OPENAI_COMPAT_THINKING_CONFIG,
},
model_vision_overrides={
"doubao-seed-2.0-code": True,
"doubao-seed-2.0-pro": True,
"doubao-seed-2.0-lite": True,
"doubao-seed-code": True,
"minimax-m2.7": False,
"minimax-m3": True,
"glm-5.2": False,
"deepseek-v4-flash": False,
"deepseek-v4-pro": False,
"kimi-k2.6": False,
"kimi-k2.7-code": False,
},
),
LLMProvider(
name="zai",
display_name="Z.AI GLM-5.3-Flash",
description="GLM-5.3-Flash with required thinking, low/high/max effort, and native vision",
use="deerflow.models.patched_deepseek:PatchedChatDeepSeek",
models=["glm-5.3-flash"],
default_model="glm-5.3-flash",
env_var="ZAI_API_KEY",
package="langchain-deepseek",
extra_config={
"api_base": "https://api.z.ai/api/paas/v4",
"timeout": 600.0,
"max_retries": 2,
"temperature": 1.0,
"top_p": 0.95,
"max_tokens": 131072,
"context_window": 1000000,
"supports_vision": True,
# Declarative reasoning contract (issue #5073). GLM-5.3-Flash cannot
# disable thinking and only accepts low/high/max effort, so the
# contract keeps thinking on for every foreground and background
# call, maps DeerFlow's generic minimal/medium presets onto the
# provider vocabulary, and serializes clear_thinking=true so the
# model does not require exact reasoning replay after summarization.
# `default` also governs callers that never choose an effort
# (summarization, title generation, subagents), so it stays at
# `high` rather than the provider's deepest `max`; `max` remains
# selectable in the composer.
"reasoning": {
"thinking": "required",
"dialect": "openai_extra_body",
"history": "clear",
"effort": {
"values": ["low", "high", "max"],
"default": "high",
"aliases": {"minimal": "low", "medium": "high"},
},
},
"extra_body": {"tool_stream": True},
# Z.AI streams terminal usage without requiring OpenAI's undocumented
# stream_options.include_usage request field.
"stream_usage": False,
},
),
LLMProvider(
name="openai",
display_name="OpenAI",
description="GPT-5, GPT-4.1, GPT-4o",
use="langchain_openai:ChatOpenAI",
models=["gpt-5", "gpt-5-mini", "gpt-4.1", "gpt-4o"],
default_model="gpt-5",
env_var="OPENAI_API_KEY",
package="langchain-openai",
extra_config={
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 4096,
"temperature": 0.7,
"supports_vision": True,
},
),
LLMProvider(
name="openai_responses",
display_name="OpenAI Responses API",
description="GPT-5 via /v1/responses",
use="langchain_openai:ChatOpenAI",
models=["gpt-5", "gpt-5-mini"],
default_model="gpt-5",
env_var="OPENAI_API_KEY",
package="langchain-openai",
extra_config={
"request_timeout": 600.0,
"max_retries": 2,
"use_responses_api": True,
"output_version": "responses/v1",
"supports_vision": True,
},
),
LLMProvider(
name="anthropic",
display_name="Anthropic",
description="Claude Sonnet 4 with extended thinking",
use="langchain_anthropic:ChatAnthropic",
models=["claude-sonnet-4-20250514", "claude-opus-4-5", "claude-sonnet-4-5"],
default_model="claude-sonnet-4-20250514",
env_var="ANTHROPIC_API_KEY",
package="langchain-anthropic",
extra_config={
"default_request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 16000,
"supports_vision": True,
**ANTHROPIC_THINKING_CONFIG,
},
),
LLMProvider(
name="deepseek",
display_name="DeepSeek",
description="DeepSeek V4 with thinking support",
use="deerflow.models.patched_deepseek:PatchedChatDeepSeek",
models=["deepseek-v4-pro", "deepseek-v4-flash"],
default_model="deepseek-v4-pro",
env_var="DEEPSEEK_API_KEY",
package="langchain-deepseek",
extra_config={
"timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"supports_vision": False,
**OPENAI_COMPAT_THINKING_CONFIG,
},
),
LLMProvider(
name="google",
display_name="Google Gemini",
description="Native Gemini SDK, no thinking support",
use="langchain_google_genai:ChatGoogleGenerativeAI",
models=["gemini-2.5-pro", "gemini-2.0-flash"],
default_model="gemini-2.5-pro",
env_var="GEMINI_API_KEY",
package="langchain-google-genai",
api_key_field="gemini_api_key",
extra_config={
"timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"supports_vision": True,
},
),
LLMProvider(
name="gemini_openai_gateway",
display_name="Gemini OpenAI-compatible",
description="Gemini thinking via an OpenAI-compatible gateway",
use="deerflow.models.patched_openai:PatchedChatOpenAI",
models=["google/gemini-2.5-pro-preview"],
default_model="google/gemini-2.5-pro-preview",
env_var="GEMINI_API_KEY",
package="langchain-openai",
extra_config={
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 16384,
"supports_vision": True,
**OPENAI_COMPAT_THINKING_CONFIG,
},
base_url_prompt="Gateway base URL (e.g. https://your-gateway.example/v1)",
),
LLMProvider(
name="ollama_qwen",
display_name="Ollama Qwen3",
description="Native local Ollama provider with thinking support",
use="langchain_ollama:ChatOllama",
models=["qwen3:32b"],
default_model="qwen3:32b",
env_var=None,
package="langchain-ollama",
extra_config={
"base_url": "http://localhost:11434",
"num_predict": 8192,
"temperature": 0.7,
"reasoning": True,
"supports_thinking": True,
"supports_vision": False,
},
auth_hint="No API key is required. Ensure Ollama is running and the model is pulled.",
),
LLMProvider(
name="ollama_gemma",
display_name="Ollama Gemma",
description="Native local Ollama provider with vision support",
use="langchain_ollama:ChatOllama",
models=["gemma4:27b"],
default_model="gemma4:27b",
env_var=None,
package="langchain-ollama",
extra_config={
"base_url": "http://localhost:11434",
"num_predict": 8192,
"temperature": 0.7,
"reasoning": True,
"supports_thinking": True,
"supports_vision": True,
},
auth_hint="No API key is required. Ensure Ollama is running and the model is pulled.",
),
LLMProvider(
name="mimo",
display_name="Xiaomi MiMo",
description="MiMo thinking models with reasoning replay",
use="deerflow.models.patched_mimo:PatchedChatMiMo",
models=["mimo-v2.5-pro", "mimo-v2.5", "mimo-v2-pro", "mimo-v2-omni", "mimo-v2-flash"],
default_model="mimo-v2.5-pro",
env_var="MIMO_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.xiaomimimo.com/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"supports_vision": False,
**OPENAI_COMPAT_THINKING_CONFIG,
},
),
LLMProvider(
name="kimi",
display_name="Moonshot Kimi",
description="Kimi K2.5 with thinking support",
use="deerflow.models.patched_deepseek:PatchedChatDeepSeek",
models=["kimi-k2.5"],
default_model="kimi-k2.5",
env_var="MOONSHOT_API_KEY",
package="langchain-deepseek",
extra_config={
"api_base": "https://api.moonshot.cn/v1",
"timeout": 600.0,
"max_retries": 2,
"max_tokens": 32768,
"supports_vision": True,
**OPENAI_COMPAT_THINKING_CONFIG,
},
),
LLMProvider(
name="novita",
display_name="Novita AI",
description="DeepSeek V3.2 via OpenAI-compatible API",
use="langchain_openai:ChatOpenAI",
models=["deepseek/deepseek-v3.2"],
default_model="deepseek/deepseek-v3.2",
env_var="NOVITA_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.novita.ai/openai",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 4096,
"temperature": 0.7,
"supports_vision": True,
**OPENAI_COMPAT_THINKING_CONFIG,
},
),
LLMProvider(
name="minimax",
display_name="MiniMax",
description="International OpenAI-compatible endpoint",
use="langchain_openai:ChatOpenAI",
models=["MiniMax-M3", "MiniMax-M2.7", "MiniMax-M2.7-highspeed"],
default_model="MiniMax-M3",
env_var="MINIMAX_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.minimax.io/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 4096,
"temperature": 1.0,
"supports_vision": True,
"supports_thinking": True,
},
model_vision_overrides={
"MiniMax-M2.7": False,
"MiniMax-M2.7-highspeed": False,
},
),
LLMProvider(
name="minimax_cn",
display_name="MiniMax CN",
description="China OpenAI-compatible endpoint",
use="langchain_openai:ChatOpenAI",
models=["MiniMax-M3", "MiniMax-M2.7", "MiniMax-M2.7-highspeed"],
default_model="MiniMax-M3",
env_var="MINIMAX_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.minimaxi.com/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 4096,
"temperature": 1.0,
"supports_vision": True,
"supports_thinking": True,
},
model_vision_overrides={
"MiniMax-M2.7": False,
"MiniMax-M2.7-highspeed": False,
},
),
LLMProvider(
name="openrouter",
display_name="OpenRouter",
description="OpenAI-compatible gateway with broad model catalog",
use="langchain_openai:ChatOpenAI",
models=["google/gemini-2.5-flash-preview", "openai/gpt-5-mini", "anthropic/claude-sonnet-4"],
default_model="google/gemini-2.5-flash-preview",
env_var="OPENROUTER_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://openrouter.ai/api/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"temperature": 0.7,
},
),
LLMProvider(
name="orcarouter",
display_name="OrcaRouter",
description="OpenAI-compatible adaptive routing gateway",
use="langchain_openai:ChatOpenAI",
models=["openai/gpt-5.5", "anthropic/claude-opus-4.8", "google/gemini-3.5-flash", "orcarouter/auto"],
default_model="openai/gpt-5.5",
env_var="ORCAROUTER_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.orcarouter.ai/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"temperature": 0.7,
},
),
LLMProvider(
name="opper",
display_name="Opper",
description="EU-hosted AI gateway, 700+ models behind one OpenAI-compatible API",
use="langchain_openai:ChatOpenAI",
models=["claude-sonnet-4-6", "gpt-5.5", "gemini-3.8-flash", "gpt-5.4-mini"],
default_model="claude-sonnet-4-6",
env_var="OPPER_API_KEY",
package="langchain-openai",
extra_config={
"base_url": "https://api.opper.ai/v3/compat",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"temperature": 0.7,
},
),
LLMProvider(
name="vllm",
display_name="vLLM",
description="Self-hosted OpenAI-compatible serving",
use="deerflow.models.vllm_provider:VllmChatModel",
models=["Qwen/Qwen3-32B", "Qwen/Qwen2.5-Coder-32B-Instruct"],
default_model="Qwen/Qwen3-32B",
env_var="VLLM_API_KEY",
package=None,
extra_config={
"base_url": "http://localhost:8000/v1",
"request_timeout": 600.0,
"max_retries": 2,
"max_tokens": 8192,
"supports_thinking": True,
"supports_vision": False,
"when_thinking_enabled": {
"extra_body": {
"chat_template_kwargs": {
"enable_thinking": True,
}
}
},
"when_thinking_disabled": {
"extra_body": {
"chat_template_kwargs": {
"enable_thinking": False,
}
}
},
},
),
LLMProvider(
name="mindie",
display_name="MindIE",
description="Qwen3-Coder on MindIE Engine",
use="deerflow.models.mindie_provider:MindIEChatModel",
models=["Qwen3-Coder-480B-A35B-Instruct-Client"],
default_model="Qwen3-Coder-480B-A35B-Instruct-Client",
env_var="OPENAI_API_KEY",
package=None,
extra_config={
"base_url": "http://localhost:8989/v1",
"temperature": 0,
"max_retries": 1,
"supports_thinking": False,
"supports_vision": False,
"supports_reasoning_effort": False,
"read_timeout": 900.0,
"connect_timeout": 30.0,
"write_timeout": 60.0,
"pool_timeout": 30.0,
},
),
LLMProvider(
name="codex",
display_name="Codex CLI",
description="Uses Codex CLI local auth (~/.codex/auth.json)",
use="deerflow.models.openai_codex_provider:CodexChatModel",
models=["gpt-5.4", "gpt-5-mini"],
default_model="gpt-5.4",
env_var=None,
package=None,
api_key_field="api_key",
extra_config={"supports_thinking": True, "supports_reasoning_effort": True},
auth_hint="Uses existing Codex CLI auth from ~/.codex/auth.json",
),
LLMProvider(
name="claude_code",
display_name="Claude Code OAuth",
description="Uses Claude Code local OAuth credentials",
use="deerflow.models.claude_provider:ClaudeChatModel",
models=["claude-sonnet-4-6", "claude-opus-4-1"],
default_model="claude-sonnet-4-6",
env_var=None,
package=None,
extra_config={"max_tokens": 4096, "supports_thinking": True},
auth_hint="Uses Claude Code OAuth credentials from your local machine",
),
LLMProvider(
name="other",
display_name="Other OpenAI-compatible",
description="Custom gateway with base_url and model name",
use="langchain_openai:ChatOpenAI",
models=["gpt-4o"],
default_model="gpt-4o",
env_var="OPENAI_API_KEY",
package="langchain-openai",
base_url_prompt="Base URL (e.g. https://api.openai.com/v1)",
model_prompt="Model name",
ask_thinking_support=True,
),
]
SEARCH_PROVIDERS: list[SearchProvider] = [
SearchProvider(
name="ddg",
display_name="DuckDuckGo (free, no key needed)",
description="No API key required",
use="deerflow.community.ddg_search.tools:web_search_tool",
env_var=None,
extra_config={"max_results": 5},
),
SearchProvider(
name="tavily",
display_name="Tavily",
description="Recommended, free tier available",
use="deerflow.community.tavily.tools:web_search_tool",
env_var="TAVILY_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="infoquest",
display_name="InfoQuest",
description="Higher quality vertical search, API key required",
use="deerflow.community.infoquest.tools:web_search_tool",
env_var="INFOQUEST_API_KEY",
extra_config={"search_time_range": 10},
),
SearchProvider(
name="exa",
display_name="Exa",
description="Neural + keyword web search, API key required",
use="deerflow.community.exa.tools:web_search_tool",
env_var="EXA_API_KEY",
extra_config={
"max_results": 5,
"search_type": "auto",
"contents_max_characters": 1000,
},
),
SearchProvider(
name="firecrawl",
display_name="Firecrawl",
description="Search + crawl via Firecrawl API",
use="deerflow.community.firecrawl.tools:web_search_tool",
env_var="FIRECRAWL_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="fastcrw",
display_name="fastCRW",
description="Firecrawl-compatible web scraper, single binary, self-host or cloud",
use="deerflow.community.fastcrw.tools:web_search_tool",
env_var="CRW_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="brave",
display_name="Brave Search",
description="Independent index, official API, API key required",
use="deerflow.community.brave.tools:web_search_tool",
env_var="BRAVE_SEARCH_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="serply",
display_name="Serply",
description="Google Search, News and Scholar results, API key required",
use="deerflow.community.serply.tools:web_search_tool",
env_var="SERPLY_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="sofya",
display_name="Sofya",
description="Search results include page content, API key required",
use="deerflow.community.sofya.tools:web_search_tool",
env_var="SOFYA_API_KEY",
extra_config={"max_results": 5, "contents_max_characters": 2000},
),
SearchProvider(
name="groundroute",
display_name="GroundRoute",
description="One key across six engines, price-routed with failover, API key required",
use="deerflow.community.groundroute.tools:web_search_tool",
env_var="GROUNDROUTE_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="webz",
display_name="Webz.io News Search",
description="Recent news search (not general web search), API key required",
use="deerflow.community.webz.tools:web_search_tool",
env_var="WEBZ_API_KEY",
extra_config={"max_results": 5},
),
SearchProvider(
name="youcom",
display_name="You.com",
description="Own index with extracted page snippets, free tier needs no key",
use="deerflow.community.youcom.tools:web_search_tool",
# No env_var: the provider falls back to You.com's keyless endpoint, so the
# wizard should not demand a key. Operators who have one set YDC_API_KEY in
# .env (or api_key in config.yaml) and the provider switches endpoints.
env_var=None,
extra_config={"max_results": 5},
),
]
WEB_FETCH_PROVIDERS: list[WebProvider] = [
WebProvider(
name="jina_ai",
display_name="Jina AI Reader",
description="Good default reader, no API key required",
use="deerflow.community.jina_ai.tools:web_fetch_tool",
env_var=None,
tool_name="web_fetch",
extra_config={"timeout": 10},
),
WebProvider(
name="exa",
display_name="Exa",
description="API key required",
use="deerflow.community.exa.tools:web_fetch_tool",
env_var="EXA_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="infoquest",
display_name="InfoQuest",
description="API key required",
use="deerflow.community.infoquest.tools:web_fetch_tool",
env_var="INFOQUEST_API_KEY",
tool_name="web_fetch",
extra_config={"timeout": 10, "fetch_time": 10, "navigation_timeout": 30},
),
WebProvider(
name="firecrawl",
display_name="Firecrawl",
description="Search-grade crawl with markdown output, API key required",
use="deerflow.community.firecrawl.tools:web_fetch_tool",
env_var="FIRECRAWL_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="groundroute",
display_name="GroundRoute",
description="Page fetch via routed engines, API key required",
use="deerflow.community.groundroute.tools:web_fetch_tool",
env_var="GROUNDROUTE_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="fastcrw",
display_name="fastCRW",
description="Firecrawl-compatible web scraper with markdown output, self-host or cloud",
use="deerflow.community.fastcrw.tools:web_fetch_tool",
env_var="CRW_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="sofya",
display_name="Sofya",
description="Page as markdown, API key required",
use="deerflow.community.sofya.tools:web_fetch_tool",
env_var="SOFYA_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="unbrowse",
display_name="Unbrowse",
description="Page as markdown, cloud browser for JavaScript pages, API key required",
use="deerflow.community.unbrowse.tools:web_fetch_tool",
env_var="UNBROWSE_API_KEY",
tool_name="web_fetch",
),
WebProvider(
name="crawl4ai",
display_name="Crawl4AI",
description="Self-hosted headless Chromium with markdown output, no API key required",
use="deerflow.community.crawl4ai.tools:web_fetch_tool",
env_var=None,
tool_name="web_fetch",
extra_config={"base_url": "http://localhost:11235", "timeout": 30},
),
]
Reference in New Issue View Git Blame Copy Permalink