mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 05:06:59 +00:00
f889ce7517
* feat(community): add Unbrowse web fetch provider Add a community web_fetch provider backed by Unbrowse (https://unbrowse.ai), a hosted service that returns a page as markdown, over plain HTTP when that is enough and through a cloud browser when the page needs JavaScript. The tool sends one JSON-RPC tools/call POST (unbrowse.scrape) to Unbrowse's MCP endpoint, so it needs no extra Python package and no MCP client (uses httpx, already a dependency). Unbrowse is not a general web search engine, so only web_fetch is added. Changes: - backend/packages/harness/deerflow/community/unbrowse/{__init__,tools}.py web_fetch_tool with the same output shape as the Sofya, Exa and Firecrawl fetch tools (# <title> plus markdown, 4KB cap). API key is read from the config.yaml api_key field or the UNBROWSE_API_KEY env var. Optional render: auto|never|always; unknown values log a warning and fall back to auto. JSON-RPC errors, isError tool results, HTTP errors and malformed payloads come back as "Error: ..." strings. - backend/tests/test_unbrowse_tools.py: key resolution, render handling, request shape, structuredContent vs text payloads, error paths. - scripts/doctor.py, scripts/wizard/providers.py: register the key and the web_fetch provider; backend/tests/test_doctor.py covers both doctor paths. - config.example.yaml, .env.example, backend/docs/CONFIGURATION.md, tools/AGENTS.md, EN/ZH harness/tools.mdx: document the provider. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(changelog): add Unbrowse web_fetch provider entry Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
119 lines
5.5 KiB
Plaintext
119 lines
5.5 KiB
Plaintext
# Serper API Key (Google Search) - https://serper.dev
|
|
SERPER_API_KEY=your-serper-api-key
|
|
|
|
# Serply API Key (Google Search, News and Scholar) - https://serply.io
|
|
SERPLY_API_KEY=your-serply-api-key
|
|
|
|
# TAVILY API Key
|
|
TAVILY_API_KEY=your-tavily-api-key
|
|
|
|
# Jina API Key
|
|
JINA_API_KEY=your-jina-api-key
|
|
|
|
# InfoQuest API Key
|
|
INFOQUEST_API_KEY=your-infoquest-api-key
|
|
|
|
# Sofya API Key (web search and fetch) - https://sofya.co
|
|
SOFYA_API_KEY=your-sofya-api-key
|
|
|
|
# Unbrowse API Key (web fetch) - https://unbrowse.ai/app
|
|
UNBROWSE_API_KEY=your-unbrowse-api-key
|
|
|
|
# Browser CORS allowlist for split-origin or port-forwarded deployments (comma-separated exact origins).
|
|
# Leave unset when using the unified nginx endpoint, e.g. http://localhost:2026.
|
|
# GATEWAY_CORS_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
|
|
|
|
# Host interface the Docker stack publishes its entry port on. Defaults to
|
|
# 127.0.0.1 (loopback only), matching the local-trusted-environment deployment
|
|
# model documented in README.md -- the agent can execute commands.
|
|
# Set 0.0.0.0 only when the host is protected by your own TLS/auth front door
|
|
# or firewall, and complete first-run setup before it becomes reachable.
|
|
# BIND_HOST=0.0.0.0
|
|
# PORT=2026
|
|
|
|
# Optional:
|
|
# FIRECRAWL_API_KEY=your-firecrawl-api-key
|
|
# VOLCENGINE_API_KEY=your-volcengine-api-key
|
|
# OPENAI_API_KEY=your-openai-api-key
|
|
# OPENVIKING_API_KEY=your-openviking-user-api-key
|
|
# GEMINI_API_KEY=your-gemini-api-key
|
|
# DEEPSEEK_API_KEY=your-deepseek-api-key
|
|
# NOVITA_API_KEY=your-novita-api-key # OpenAI-compatible, see https://novita.ai
|
|
# MINIMAX_API_KEY=your-minimax-api-key # OpenAI-compatible, see https://platform.minimax.io
|
|
# OpenAI-compatible Images API (used by the image-generation skill)
|
|
# IMAGE_GENERATION_PROVIDER=openai # openai-compatible is also accepted
|
|
# IMAGE_GENERATION_API_KEY=your-image-generation-api-key
|
|
# IMAGE_GENERATION_BASE_URL=https://api.openai.com/v1
|
|
# IMAGE_GENERATION_MODEL=gpt-image-2.5-flare
|
|
# IMAGE_GENERATION_SIZE=1536x1024
|
|
# STEPFUN_API_KEY=your-stepfun-api-key # OpenAI-compatible, see https://platform.stepfun.com
|
|
# VLLM_API_KEY=your-vllm-api-key # OpenAI-compatible
|
|
|
|
# E2B cloud sandbox API key — required only when using E2BSandboxProvider.
|
|
# Sign up at https://e2b.dev/dashboard
|
|
# E2B_API_KEY=your-e2b-api-key
|
|
# FEISHU_APP_ID=your-feishu-app-id
|
|
# FEISHU_APP_SECRET=your-feishu-app-secret
|
|
|
|
# SLACK_BOT_TOKEN=your-slack-bot-token
|
|
# SLACK_APP_TOKEN=your-slack-app-token
|
|
# TELEGRAM_BOT_TOKEN=your-telegram-bot-token
|
|
# DISCORD_BOT_TOKEN=your-discord-bot-token
|
|
|
|
# Enable LangSmith to monitor and debug your LLM calls, agent runs, and tool executions.
|
|
# LANGSMITH_TRACING=true
|
|
# LANGSMITH_ENDPOINT=https://api.smith.langchain.com
|
|
# LANGSMITH_API_KEY=your-langsmith-api-key
|
|
# LANGSMITH_PROJECT=your-langsmith-project
|
|
|
|
# GitHub API Token
|
|
# GITHUB_TOKEN=your-github-token
|
|
|
|
# Database (only needed when config.yaml has database.backend: postgres)
|
|
# DATABASE_URL=postgresql://deerflow:password@localhost:5432/deerflow
|
|
#
|
|
# WECOM_BOT_ID=your-wecom-bot-id
|
|
# WECOM_BOT_SECRET=your-wecom-bot-secret
|
|
# DINGTALK_CLIENT_ID=your-dingtalk-client-id
|
|
# DINGTALK_CLIENT_SECRET=your-dingtalk-client-secret
|
|
|
|
# Set to "false" to disable Swagger UI, ReDoc, and OpenAPI schema in production
|
|
# GATEWAY_ENABLE_DOCS=false
|
|
|
|
# Shared internal Gateway auth token for multi-worker deployments.
|
|
# `make up` honors a value set here (as it does for BETTER_AUTH_SECRET) and
|
|
# otherwise generates one and persists it under DEER_FLOW_HOME; set it
|
|
# manually when Gateway workers also run outside the bundled deploy script.
|
|
# DEER_FLOW_INTERNAL_AUTH_TOKEN=your-shared-internal-token
|
|
|
|
# Provisioner API key for sandbox authentication (required when using provisioner/K8s sandbox mode).
|
|
# The same value must be set on the provisioner container and in config.yaml sandbox.provisioner_api_key.
|
|
# Generate: openssl rand -hex 32
|
|
# PROVISIONER_API_KEY=your-provisioner-api-key
|
|
|
|
# ── Frontend SSR → Gateway wiring ─────────────────────────────────────────────
|
|
# The Next.js server uses these to reach the Gateway during SSR (auth checks,
|
|
# /api/* rewrites). They default to localhost values that match `make dev` and
|
|
# `make start`, so most local users do not need to set them.
|
|
#
|
|
# Override only when the Gateway is not on localhost:8001 (e.g. when the
|
|
# frontend and gateway run on different hosts, in containers with a service
|
|
# alias, or behind a different port). docker-compose already sets these.
|
|
# DEER_FLOW_INTERNAL_GATEWAY_BASE_URL=http://localhost:8001
|
|
# DEER_FLOW_TRUSTED_ORIGINS=http://localhost:3000,http://localhost:2026
|
|
|
|
# ── Claude Code / Codex CLI subscription as a model provider (optional) ───────
|
|
# If you configure a ClaudeChatModel / Codex model provider (or an ACP agent)
|
|
# that reuses your CLI subscription login, prefer passing a token via env over
|
|
# bind-mounting your whole ~/.claude / ~/.codex into the container. The Gateway
|
|
# credential loader reads these first, so no directory mount is needed.
|
|
# CLAUDE_CODE_CREDENTIALS_PATH points at a single .credentials.json (Claude)
|
|
# rather than the whole dir. docker-compose.cli-auth.yaml is the opt-in
|
|
# directory-mount fallback for adapters that need the full CLI config.
|
|
# ACP adapters often take their own env API key (e.g. ANTHROPIC_API_KEY) and
|
|
# need no mount at all — check the adapter's docs. See SECURITY.md.
|
|
# CLAUDE_CODE_OAUTH_TOKEN=your-claude-code-oauth-token
|
|
# ANTHROPIC_AUTH_TOKEN=your-anthropic-auth-token
|
|
# CLAUDE_CODE_CREDENTIALS_PATH=/path/to/.claude/.credentials.json
|
|
# CODEX_AUTH_PATH=/path/to/codex/auth.json
|