diff --git a/config.example.yaml b/config.example.yaml index bad64c6fd..b0ff3cfb5 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -316,6 +316,35 @@ models: # For Docker deployments, use host.docker.internal instead of localhost: # base_url: http://host.docker.internal:11434 + # Example: llama.cpp (OpenAI-compatible local server) + # + # llama-server exposes an OpenAI-compatible /v1/chat/completions endpoint, + # so use ChatOpenAI and point base_url at the server's /v1 root. --alias + # gives the served model a stable API name independent of the GGUF filename. + # + # Start llama.cpp, for example: + # llama-server -m /path/to/model.gguf --host 0.0.0.0 --port 8080 --alias qwen-local + # + # Tool calling depends on the model's chat template supporting tools. Current + # llama.cpp enables its Jinja template engine by default; if you override the + # template, use one that supports function/tool calls. + # + # - name: qwen-llamacpp + # display_name: Qwen (llama.cpp) + # use: langchain_openai:ChatOpenAI + # model: qwen-local + # api_key: no-key # Non-empty placeholder when llama-server auth is disabled + # base_url: http://localhost:8080/v1 + # request_timeout: 600.0 + # max_retries: 2 + # supports_vision: false + # + # If llama-server was started with --api-key, replace the placeholder with + # that key (prefer an environment variable). In Docker, localhost is the + # DeerFlow container itself; use an address reachable from the container, + # for example on Docker Desktop: + # base_url: http://host.docker.internal:8080/v1 + # Example: Anthropic Claude model (with extended thinking) # supports_thinking: true is required — without it, DeerFlow silently falls # back to non-thinking mode even when the UI thinking toggle is on. diff --git a/frontend/src/content/en/application/configuration.mdx b/frontend/src/content/en/application/configuration.mdx index c0ec7b140..c33a178a9 100644 --- a/frontend/src/content/en/application/configuration.mdx +++ b/frontend/src/content/en/application/configuration.mdx @@ -32,7 +32,7 @@ The most important sections for application configuration are: Configure the LLM providers the agent can use. At least one model is required. - + ```yaml models: @@ -108,6 +108,33 @@ Install Ollama provider: `cd backend && uv add 'deerflow-harness[ollama]'` +```bash +llama-server -m /path/to/model.gguf --host 0.0.0.0 --port 8080 --alias qwen-local +``` + +```yaml +models: + - name: qwen-llamacpp + use: langchain_openai:ChatOpenAI + model: qwen-local + api_key: no-key + base_url: http://localhost:8080/v1 + request_timeout: 600.0 + max_retries: 2 + supports_vision: false +``` + +`llama-server` exposes an OpenAI-compatible `/v1/chat/completions` endpoint, so point `base_url` at its `/v1` root. `ChatOpenAI` expects a non-empty API key; `no-key` is only a placeholder when llama.cpp authentication is disabled. If the server uses `--api-key`, configure that value instead. + +For Docker deployments, `localhost` refers to the DeerFlow container. Use a host address reachable from the container, such as `http://host.docker.internal:8080/v1` on Docker Desktop. + + + DeerFlow relies on tool calling. Make sure the GGUF model's chat template supports + function/tool calls. Current llama.cpp enables its Jinja chat-template engine by + default; custom templates must preserve tool-call support. + + + ```yaml models: - name: gemini-2.5-pro diff --git a/frontend/src/content/zh/application/configuration.mdx b/frontend/src/content/zh/application/configuration.mdx index 019d80159..8ea3dbbdd 100644 --- a/frontend/src/content/zh/application/configuration.mdx +++ b/frontend/src/content/zh/application/configuration.mdx @@ -17,7 +17,7 @@ import { Callout, Tabs } from "nextra/components"; ### 模型提供商 - + ```yaml models: @@ -106,6 +106,32 @@ ollama pull llama3.3 ``` + +```bash +llama-server -m /path/to/model.gguf --host 0.0.0.0 --port 8080 --alias qwen-local +``` + +```yaml +models: + - name: qwen-llamacpp + use: langchain_openai:ChatOpenAI + model: qwen-local + api_key: no-key + base_url: http://localhost:8080/v1 + request_timeout: 600.0 + max_retries: 2 + supports_vision: false +``` + +`llama-server` 提供 OpenAI 兼容的 `/v1/chat/completions` 接口,因此 `base_url` 应指向它的 `/v1` 根路径。`ChatOpenAI` 需要一个非空 API key;当 llama.cpp 未启用鉴权时,`no-key` 只是占位值。如果启动服务时使用了 `--api-key`,请改为对应的真实密钥。 + +Docker 部署时,`localhost` 指向 DeerFlow 容器本身。请改用容器能够访问的宿主机地址;例如 Docker Desktop 通常可使用 `http://host.docker.internal:8080/v1`。 + + + DeerFlow 依赖工具调用。请确认 GGUF 模型的 chat template 支持 function/tool calls。 + 当前 llama.cpp 默认启用 Jinja chat-template 引擎;如果自定义模板,也必须保留工具调用支持。 + + ### 沙箱