diff --git a/frontend/src/content/en/application/agents-and-threads.mdx b/frontend/src/content/en/application/agents-and-threads.mdx index 9ad5eb851..223bb2e77 100644 --- a/frontend/src/content/en/application/agents-and-threads.mdx +++ b/frontend/src/content/en/application/agents-and-threads.mdx @@ -1,3 +1,135 @@ # Agents and Threads -TBD +DeerFlow App supports multiple named agents and maintains conversation state across sessions through threads and checkpointing. + +## Agents + +### The default agent + +The default agent is the Lead Agent with no custom configuration. It loads all globally enabled skills, has access to all configured tools, and uses the first model in `config.yaml` as its default. + +### Custom agents + +Custom agents are named variants of the Lead Agent. Each one can have: + +- a **display name** and an auto-derived ASCII slug (the `name` used internally) +- a specific **model** to use by default +- a restricted set of **skills** (or all globally enabled skills if unspecified) +- a restricted set of **tool groups** +- a custom **system prompt** or agent-specific instructions + +Custom agents are created and managed through: + +- **The App UI**: open the Agents section in the settings panel. +- **The Gateway API**: `POST /api/agents` with the agent definition. + +The slug (`name`) is automatically derived from the `display_name` and must be unique. The system checks for conflicts and appends a suffix if needed (`/api/agents/check`). + +Agent configuration is stored in `agents/{name}/config.yaml` relative to the backend directory. + +### Restricting agent capabilities + +To restrict a custom agent to specific skills: + +```yaml +# agents/my-researcher/config.yaml +name: my-researcher +display_name: My Researcher +skills: + - deep-research + - academic-paper-review +# Omit skills key entirely to inherit all globally enabled skills +# Set skills: [] to disable all skills for this agent +``` + +To restrict to specific tool groups: + +```yaml +tool_groups: + - research # only tools in the 'research' group are available +``` + +## Threads + +A **thread** is a persistent conversation session. Each thread has: + +- a unique thread ID +- a message history +- accumulated artifacts (output files) +- a title (auto-generated after the first exchange) +- a reference to the agent used + +Threads are listed in the sidebar. Clicking a thread resumes the conversation from where it left off. + +### Thread lifecycle + + + +#### Create + +A new thread is created when you start a conversation. The thread ID is generated and the initial configuration (model, agent, skills) is recorded. + +#### Run + +Each user message triggers an agent run. The run streams tokens and tool calls back to the browser in real time. The thread state (messages, artifacts) is updated after each turn. + +#### Checkpoint + +If a checkpointer is configured, the thread state is persisted after each turn. This means the conversation survives process restarts. + +#### Resume + +Opening a thread from the sidebar loads its full history from the checkpointer. The agent picks up from where it left off. + + + +### Checkpointer configuration + +The checkpointer controls how thread state is persisted: + +```yaml +# In-memory (default if omitted — state lost on restart) +# checkpointer: +# type: memory + +# SQLite (survives restarts, single-process) +checkpointer: + type: sqlite + connection_string: checkpoints.db + +# PostgreSQL (multi-process, production) +# checkpointer: +# type: postgres +# connection_string: postgresql://user:password@localhost:5432/deerflow +``` + + + DeerFlow App uses the checkpointer setting in{" "} + config.yaml to persist thread state through the Gateway runtime + (via make_checkpointer() in + deerflow.agents.checkpointer.async_provider). Configure this + section if you want threads to survive process restarts. + + +### Thread data storage + +Working files produced during a thread (uploads, intermediate files, output artifacts) are stored under: + +``` +backend/.deer-flow/threads/{thread_id}/user-data/ +``` + +This directory is mounted into the sandbox as `/mnt/user-data/` for the agent to read and write. + +In Docker deployments, this path is bind-mounted from the host so that thread data persists across container restarts. Set `DEER_FLOW_ROOT` to the absolute path of your deer-flow repository root to ensure the correct host path is used. + + + + + diff --git a/frontend/src/content/en/reference/api-gateway-reference.mdx b/frontend/src/content/en/reference/api-gateway-reference.mdx index 2a5922d2b..bb19835d8 100644 --- a/frontend/src/content/en/reference/api-gateway-reference.mdx +++ b/frontend/src/content/en/reference/api-gateway-reference.mdx @@ -1,3 +1,75 @@ # API / Gateway Reference -TBD + + DeerFlow Gateway is built on FastAPI and provides interactive API + documentation at http://localhost:8001/docs. + + +## Base URL + +``` +http://localhost:8001 +``` + +Via nginx proxy: + +``` +http://localhost:2026/api +``` + +## Core endpoints + +### System + +| Method | Path | Description | +| ------ | ------------- | --------------------------------- | +| `GET` | `/health` | Service health check | +| `GET` | `/api/models` | Get the list of configured models | + +### Agent management + +| Method | Path | Description | +| -------- | -------------------- | ------------------------------- | +| `GET` | `/api/agents` | List all agents | +| `POST` | `/api/agents` | Create a custom agent | +| `GET` | `/api/agents/{name}` | Get agent configuration | +| `PUT` | `/api/agents/{name}` | Update agent configuration | +| `DELETE` | `/api/agents/{name}` | Delete an agent | +| `POST` | `/api/agents/check` | Check/suggest unique agent slug | + +### Threads and memory + +| Method | Path | Description | +| -------- | ----------------------------- | -------------------------------------------------- | +| `GET` | `/api/threads` | List threads | +| `DELETE` | `/api/threads/{thread_id}` | Delete a thread | +| `GET` | `/api/memory` | Get global memory data (context + facts) | +| `DELETE` | `/api/memory` | Clear all memory data | +| `POST` | `/api/memory/reload` | Reload memory from storage file | +| `GET` | `/api/memory/facts` | (included in `/api/memory` response `facts` array) | +| `POST` | `/api/memory/facts` | Create a memory fact | +| `PATCH` | `/api/memory/facts/{fact_id}` | Update a memory fact | +| `DELETE` | `/api/memory/facts/{fact_id}` | Delete a memory fact | +| `GET` | `/api/memory/export` | Export memory data as JSON | +| `POST` | `/api/memory/import` | Import and overwrite memory data | +| `GET` | `/api/memory/config` | Get memory configuration | +| `GET` | `/api/memory/status` | Get memory config + data in one request | + +### Extensions + +| Method | Path | Description | +| ------ | --------------------------------------- | -------------------------------------------- | +| `GET` | `/api/extensions` | List all extensions (MCP servers and skills) | +| `POST` | `/api/extensions/mcp/{name}/enable` | Enable an MCP server | +| `POST` | `/api/extensions/mcp/{name}/disable` | Disable an MCP server | +| `POST` | `/api/extensions/skills/{name}/enable` | Enable a skill | +| `POST` | `/api/extensions/skills/{name}/disable` | Disable a skill | + +### File uploads + +| Method | Path | Description | +| ------ | ------------------------------------- | ------------------------------------- | +| `POST` | `/api/uploads/{thread_id}` | Upload a file to the thread workspace | +| `GET` | `/api/uploads/{thread_id}/{filename}` | Retrieve an uploaded file | + +For the full interactive API documentation visit `http://localhost:8001/docs` (Swagger UI). diff --git a/frontend/src/content/en/reference/configuration-reference.mdx b/frontend/src/content/en/reference/configuration-reference.mdx index cbc914dda..9312b4539 100644 --- a/frontend/src/content/en/reference/configuration-reference.mdx +++ b/frontend/src/content/en/reference/configuration-reference.mdx @@ -1,3 +1,124 @@ # Configuration Reference -TBD +This page is the complete reference for all top-level fields in `config.yaml`. + + + See config.example.yaml in the repository root for a fully + commented example with all available options. + + +## Top-level fields + +| Field | Type | Description | +| ---------------------- | ------------- | ----------------------------------------------- | +| `config_version` | `int` | Config schema version (current: 6) | +| `log_level` | `str` | Log verbosity: `debug`/`info`/`warning`/`error` | +| `models` | `list` | Available LLM model configurations | +| `image_generate_model` | `str \| list` | Model name to use for image generation | +| `token_usage` | `object` | Token usage tracking config | +| `tools` | `list` | Available tool configurations | +| `tool_groups` | `list` | Named groupings of tools | +| `tool_search` | `object` | Deferred tool loading config | +| `sandbox` | `object` | Sandbox provider and options | +| `skills` | `object` | Skills directory and container path | +| `skill_evolution` | `object` | Agent-managed skill creation | +| `subagents` | `object` | Subagent timeouts and max turns | +| `acp_agents` | `dict` | External ACP agent configurations | +| `memory` | `object` | Cross-session memory storage | +| `summarization` | `object` | Conversation summarization | +| `title` | `object` | Automatic thread title generation | +| `checkpointer` | `object` | Thread state persistence | +| `guardrails` | `object` | Tool call authorization | +| `uploads` | `object` | File upload settings | +| `channels` | `list` | IM channel integrations | + +## models + +```yaml +models: + - name: gpt-4o # Model identifier (referenced in requests) + use: langchain_openai:ChatOpenAI # Python class path + model: gpt-4o # Model name passed to the LLM + api_key: $OPENAI_API_KEY # API key (env var interpolation supported) + base_url: null # Optional: custom endpoint URL + request_timeout: 600.0 # Request timeout in seconds + max_retries: 2 # Number of retries on failure + supports_vision: true # Whether the model accepts image inputs + supports_thinking: false # Whether the model supports extended thinking + # thinking: {} # Optional thinking config (passed when thinking is active) + # when_thinking_enabled: {} # Optional overrides applied when thinking is enabled + # when_thinking_disabled: {} # Optional overrides applied when thinking is disabled + # Any additional fields are passed through to the model constructor +``` + +## sandbox + +```yaml +sandbox: + # Local (default, no container isolation) + use: deerflow.sandbox.local:LocalSandboxProvider + allow_host_bash: false + bash_output_max_chars: 20000 + read_file_output_max_chars: 50000 + ls_output_max_chars: 20000 + + # Container-based + # use: deerflow.community.aio_sandbox:AioSandboxProvider + # image: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest + # replicas: 3 + # idle_timeout: 600 +``` + +## memory + +```yaml +memory: + enabled: true + storage_path: memory.json + debounce_seconds: 30 + max_facts: 100 + fact_confidence_threshold: 0.7 + injection_enabled: true + max_injection_tokens: 2000 + model_name: null +``` + +## summarization + +```yaml +summarization: + enabled: true + model_name: null + trigger: + - type: tokens + value: 15564 + keep: + type: messages + value: 10 + trim_tokens_to_summarize: 15564 + summary_prompt: null +``` + +## checkpointer + +```yaml +checkpointer: + type: sqlite # sqlite | redis | postgres + connection_string: .deer-flow/checkpoints.db +``` + +## subagents + +```yaml +subagents: + timeout_seconds: 900 + agents: + general-purpose: + timeout_seconds: 1800 + max_turns: 160 + bash: + timeout_seconds: 300 + max_turns: 80 +``` + +See the dedicated documentation page for each feature for full field descriptions. diff --git a/frontend/src/content/en/reference/runtime-flags-and-modes.mdx b/frontend/src/content/en/reference/runtime-flags-and-modes.mdx index 76ce6f9ab..fe89e564e 100644 --- a/frontend/src/content/en/reference/runtime-flags-and-modes.mdx +++ b/frontend/src/content/en/reference/runtime-flags-and-modes.mdx @@ -1,3 +1,36 @@ # Runtime Flags and Modes -TBD +This page documents the runtime flags and modes that affect DeerFlow Harness and agent runtime behavior. + +## Per-request configurable options + +These options are passed via `config.configurable` (for programmatic use) or selected in the web UI (for application use): + +| Flag | Type | Default | Description | +| -------------------------- | ------------- | ---------------------- | ------------------------------------------------ | +| `model_name` | `str \| None` | First configured model | Model to use for the request | +| `agent_name` | `str \| None` | `None` | Load a custom agent configuration | +| `thinking_enabled` | `bool` | `True` | Enable extended thinking (model must support it) | +| `reasoning_effort` | `str \| None` | `None` | Reasoning effort level (model-specific) | +| `is_plan_mode` | `bool` | `False` | Enable TodoList middleware | +| `subagent_enabled` | `bool` | `False` | Allow subagent delegation | +| `max_concurrent_subagents` | `int` | `3` | Maximum parallel subagent calls per turn | + +## Environment variables + +| Variable | Default | Description | +| ----------------------- | --------------- | ------------------------------------------------ | +| `DEER_FLOW_CONFIG_PATH` | Auto-discovered | Absolute path to `config.yaml` | +| `LOG_LEVEL` | `info` | Log level override | +| `DEER_FLOW_ROOT` | Repo root | Base path for Docker bind mounts | +| `BETTER_AUTH_SECRET` | — | Frontend session secret (required in production) | +| `BETTER_AUTH_URL` | — | Public URL (for callbacks and CORS) | + +## Model capability flags + +Set in the model configuration in `config.yaml`: + +| Flag | Type | Description | +| ------------------- | ------ | ------------------------------------- | +| `supports_vision` | `bool` | Model accepts image inputs | +| `supports_thinking` | `bool` | Model supports extended thinking mode | diff --git a/frontend/src/content/en/reference/source-map.mdx b/frontend/src/content/en/reference/source-map.mdx index f5a4823da..7e39fa846 100644 --- a/frontend/src/content/en/reference/source-map.mdx +++ b/frontend/src/content/en/reference/source-map.mdx @@ -1,3 +1,88 @@ # Source Map -TBD +This page maps DeerFlow's core concepts to where they are implemented in the codebase, helping you quickly locate specific features. + +## Backend core paths + +``` +backend/ +├── app/ +│ └── gateway/ # FastAPI Gateway API +│ ├── routers/ +│ │ ├── agents.py # Custom agent CRUD +│ │ ├── extensions.py # MCP/skill enable/disable +│ │ ├── memory.py # Memory read/clear +│ │ ├── threads.py # Thread management +│ │ └── uploads.py # File uploads +│ └── app.py # FastAPI app entry point (create_app()) +│ +└── packages/harness/deerflow/ + ├── agents/ + │ ├── lead_agent/ + │ │ ├── agent.py # make_lead_agent() factory + │ │ └── prompt.py # System prompt templates + │ ├── middlewares/ # All middleware implementations + │ ├── memory/ + │ │ ├── middleware.py # MemoryMiddleware + │ │ └── storage.py # Memory file storage + │ └── thread_state.py # ThreadState dataclass + │ + ├── config/ + │ ├── app_config.py # AppConfig (top-level config) + │ ├── model_config.py # ModelConfig + │ ├── paths.py # Path resolution utilities + │ └── *.py # Per-module config classes + │ + ├── mcp/ + │ ├── cache.py # mtime-based tool cache + │ └── oauth.py # MCP OAuth support + │ + ├── models/ + │ └── factory.py # create_chat_model() LLM factory + │ + ├── sandbox/ + │ ├── local/ # LocalSandboxProvider + │ └── sandbox.py # Sandbox base class + │ + ├── skills/ + │ ├── loader.py # load_skills() (hot reload) + │ ├── parser.py # SKILL.md parsing + │ ├── installer.py # Dependency installation + │ └── manager.py # Skill lifecycle management + │ + ├── subagents/ + │ └── registry.py # Subagent lookup and config override + │ + └── tools/ + └── builtins/ # Built-in tool implementations +``` + +## Frontend core paths + +``` +frontend/src/ +├── app/ # Next.js routes and pages +├── components/ +│ └── workspace/ # Workspace UI components +├── core/ +│ ├── agents/ # Agent types and API client +│ ├── messages/ # Message types and tool call handling +│ └── threads/ # Thread state management +└── content/ # Documentation content (MDX) + ├── en/ # English documentation + └── zh/ # Chinese documentation +``` + +## Quick index + +| Goal | File | +| -------------------------- | ----------------------------- | +| Lead agent creation | `agents/lead_agent/agent.py` | +| System prompt template | `agents/lead_agent/prompt.py` | +| All middleware | `agents/middlewares/` | +| Config loading | `config/app_config.py` | +| Model factory | `models/factory.py` | +| Skill loading (hot reload) | `skills/loader.py` | +| MCP tool cache | `mcp/cache.py` | +| File upload handling | `uploads/manager.py` | +| Gateway app factory | `app/gateway/app.py` |