deer-flow/backend/docs/CONFIGURATION.md
Totoro 35cdcab1c5
feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks

- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
  streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
  stored prompt never contains it. Wording adapts when scheduler.tool_enabled
  is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
  previous-run reference and own-schedule stop; request_stop and the
  scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
  ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
  condition/note changes (not on resume); goal-check failures neither count
  nor reset the streak; pause of a finished task returns "finished"; trials
  on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
  list_manageable_from_thread, thread_relation/thread_run on thread tasks,
  task_ids_for_thread, run_number, and run_number/total_tokens/summary on
  run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
  clears, naive end_at = task timezone), resume recomputes from now with an
  optional renewal body, once_time_passed, task_finished, limits_exhausted,
  scheduler_not_running on create, trigger returns outcome/existing/thread_id,
  task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
  chat capability (scheduled_task_errors.py), pinned by
  contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
  check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
  min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
  (breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
  over HTTP, repository, service and capability cases; updated tests for the
  new error shape, finished-task pause and head revision.

* feat(scheduler): conversation update/resume, run-conversation authority and readable launches

- schedule_task: update and resume actions, stop_condition and clear_fields
  parameters, plain parameter types (value rules come back as coded results),
  rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
  interactive turns in a task's run conversation may manage that task;
  scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
  last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
  lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
  start_run and never forwarded to the run config) > not needed for intervals
  and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
  particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
  deerflow_scheduled_origin metadata with the user-language parts, run thread
  title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
  covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.

* feat(frontend): scheduled-task core types, formatting, errors and glossary

- throwGatewayApiError now throws GatewayApiError (status, code, params,
  rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
  active_run_status, origin/notes, run_number/total_tokens/summary, thread
  relation, trigger result, renewal and scheduled origin; isTaskBusy and
  hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
  result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
  history), useScheduledTask with live gating and final 404, useInView,
  localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
  (relative task times, viewer time, display zone), status.ts, actions.ts,
  run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
  goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
  tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
  type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
  recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
  shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
  hooks, run context, thread titles and features; scheduled e2e copy updated

* feat(frontend): list-first scheduled tasks page with outcomes and editable goals

- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
  status tabs (All/Active/Paused/Finished) and search over the task list,
  ?task_id= deep link with router.replace on selection, ?thread_id= chip
  with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
  running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
  More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
  Runs / Stops when / Each run's goal / Does / Notes from chat / History,
  origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
  continuations, tokens, Open chat link; raw errors and the evaluator
  reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
  advanced agent/context options; PATCH sends only changed fields, null
  clears a cap
- renew dialog raises or removes caps and resumes in one request; a
  limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
  the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note

* feat(frontend): live schedule card, run conversation provenance and header entry

- schedule_task results (create, update, pause, resume, trial, delete) become
  an assistant:scheduled-task message group rendered as a live card: schedule,
  next run, stop condition with the safety cap, goal, where results go, Run
  once now, Pause/Resume and Open task; the card polls the task only while it
  is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
  for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
  scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
  task instructions, stop condition and notes collapsed; the launched text is
  never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
  shows the count, links to the task (or the chat filter), and a run's chat
  links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
  e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
  run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
  frontend AGENTS.md notes

* fix(scheduler): address verification findings

Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.

* fix(scheduler): address review findings

* fix(gateway): no-transform on SSE responses

All four text/event-stream responses (thread run create stream, join,
existing-run GET/POST stream, stateless create stream) now take their
headers from app/gateway/sse_headers.py and send
Cache-Control: no-cache, no-transform, so compressing proxies such as
Next's rewrite proxy stop buffering streams. Content-Location and
X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep
stream_mode=None.

Tests pin the headers on every SSE route and fail on a literal no-cache
header left in routers. Docs: API.md SSE headers note, CHANGELOGs.

* feat(scheduler): per-owner run cap and fair queue selection

New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective
min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task
owner. claim_queued_run checks it right after the global count under the same
budget lock, so it is atomic across workers and Pods. list_queued_runs ranks
same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner)
and orders the batch by that rank, so the drain is owner-fair even with the
cap off; owners at their cap drop out before the batch limit. Orphan rows
(task deleted while queued) are never held back by the cap.

Gateway lifespan passes the startup value; the service constructor default
stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md,
the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged).
The live-task quota is untouched.

* feat(scheduler): durable lifecycle events for the originating chat

When a chat-created schedule is paused by the agent, auto-paused or finishes,
finalization now reports task_stopped / task_paused / task_finished, only on a
live -> paused/finished transition. The new finish_task_at_end_condition
replaces the three inline "mark completed" blocks (admission rejected as ended,
claim-time skip, complete_if_ended), so every site emits exactly once.

The scheduler's finalization observer is always installed. It writes
scheduled_task_events rows for the originating chat in the same transaction as
the state change, deduplicated on (task_id, anchor, event), with a title
snapshot, stop condition, run number, last run outcome and the chat's newest
run as placement anchor. Lookups fall back on bad stored data; only database
errors propagate. Manual trials drop only run_* events. The IM step keeps
PR1's selection until B4.

New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's
rows, oldest first); rows survive task deletion and go with the chat.
Migration 0032_activity_and_task_events adds the events table plus the
run-origin column/index and read-state tables used later, each object guarded
on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary.
Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs.

* feat(scheduler): honest, localized and self-contained IM notices

Every scheduled-task IM notice is now enqueued by the finalization observer
in the transaction that records the outcome, one message per occurrence
(task_stopped > task_paused > task_finished > run_failed > run_unmet >
run_completed; a once task sends only its run event). The merged auto-pause
replaces the separate unmet + pause messages, agent stops and finishes now
notify, and recovery-finalized runs notify exactly once. Idle finishes use a
deterministic idle: key derived from the lifecycle anchor. The old
non-transactional enqueue in handle_run_completion is removed.

New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a
proactive_notifications flag (adds the missing qq entry); only push-capable
providers (WeCom) get rows, /api/channels/providers returns the flag, and the
worker ends legacy rows for other providers as terminal failed.

New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what
happened (plus the last run's outcome), the tasks page's one-line run summary
(redacted) when the agent replied, and a closing line. No IDs, no links, no
public_base_url. Locale is the owner's new `locale` preference, else the new
channel_connections.notification_locale (default en-US).

Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md,
channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged).

* feat(gateway): thread origin, activity feed and read markers for server-created threads

Runs started by the server for a user (scheduler, IM channels, GitHub
agents, extension handles, MCP notifications) now carry a server-owned
metadata.deerflow_origin {kind, provider?, namespace?} (new harness module
deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json).
It is stamped only from request.state.run_origin or by the internal channel
caller; run admission drops client copies from both metadata and
config.metadata, and thread create/patch strip it. RunRepository derives
runs.origin_kind on put and on create_thread_operation_atomic (agent runs
only), and gains latest_change().

New ThreadReadRepository keeps a per-user, never-decreasing read position
(thread_read_markers) and a per-user read clock (thread_read_versions).
POST /api/threads/{id}/read marks a thread read, thread search items carry
unread, and thread delete removes the markers. GET /api/thread-activity pages
the caller's run changes over the run-change clock, returns only threads
changed by server-originated runs plus read_version, and 503s without SQL;
/api/features reports thread_activity.available.

Docs: API.md, gateway AGENTS.md (condensed to stay under its budget),
CHANGELOGs.

* fix(scheduler): adapt PR2 to PR1 follow-ups

PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into
the persistence guide and grew the sandbox AGENTS.md chain to within 267
bytes of its hard limit. The per-owner queue-fairness detail now lives in
the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps
the cap and a link), so check_agent_guidance passes again.

PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2;
PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged).

* fix(scheduled-tasks): polish from live acceptance

- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
  and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
  days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
  the interpolated time and the Chinese after it; a copy test guards every
  scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
  (ends with ":" or ":"), the list items follow it as plain text (bullets,
  numbering, checkboxes, bold, code and links stripped), joined with ";" for
  CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
  arrives in a chat invalidates that chat's task list, the tasks list and the
  named task, so the header button and cards update without waiting for the
  idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
  Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
  pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
  and run 2, paused by the agent), fetched with GET from the demo Gateway and
  trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
  run-thread and tasks-page specs run in both locales over the recorded data

* docs(changelog): reference #6378 for the scheduled-tasks entries

* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit

* test(scheduled-tasks): pin the live takes to their recorded time zone

The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.

* feat(frontend): thread origin, activity polling, read markers and task events core

Frontend core for server-created threads and schedule lifecycle events
(PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it.

- core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin,
  then channel_source (github -> GitHub origin), then a legacy
  scheduled_task_id; key and kinds pinned to
  contracts/thread_origin_contract.json. labelOfThreadOrigin and the now
  exported labelOfChannelProvider(provider, t) use the new localized
  threads.origin.providers names (飞书, 企业微信, ...), with the English map
  (gaining GitHub) only as the fallback.
- core/threads/activity.ts: useThreadActivity polls /api/thread-activity
  every 15 s (focus refetch, never in the background, off unless
  features.thread_activity.available). The first response only seeds the
  cursor; server-origin threads, a truncated page or a higher read_version
  invalidate the thread lists, a scheduled thread also ["scheduled-tasks"];
  an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s,
  skips a thread whose cached unread is false, patches the search and
  infinite caches optimistically and stores the returned read_version so a
  tab's own read does not refetch its lists. AgentThread gains unread.
- core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query
  (["scheduled-tasks","thread-events",id], one fetch per saved chat, not
  gated on the task list so a deleted task's lines stay, refetched when a
  task of the chat changes status or finishes a run), describeTaskEvent
  (en/zh copy, title and condition as segments, last-run suffix, See that
  run / Open task) and placeTaskEvents (end of the after_run_id turn, else
  the tail, ordered by created_at); lifecycle names pinned to contract v3.
- features: thread_activity (default unavailable); channels:
  proactive_notifications on providers.
- Locale sync: the I18nProvider reports the UI language and a signed-in
  preference session saves it as the new `locale` preference when the
  account's differs (and on every switch), so IM notices follow the UI
  language; nothing is written with auth disabled.
- i18n: threads.* (unread, origin labels, provider names),
  scheduledTasks.events.*, history.waitingForSlot (now shown for queued run
  rows), channels.scheduledUpdates.*.
- e2e mocks: thread activity feed and controller, POST /read, the events
  route, providers with proactive_notifications, search unread, features
  thread_activity (off by default). Preference e2e mocks return the UI
  locale so they keep asserting only their own edits.
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in

- The edit, duplicate and renew dialogs keep a stored end time's instant
  while its field still shows it. A minute-precision wall value cannot
  keep seconds or name the later 01:30 of a DST fall-back, so a
  title-only edit moved the cap an hour earlier, and raising the run
  limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
  filter when the new task does not belong to that chat, instead of
  showing "This task isn't one of this chat's tasks".

* feat(frontend): origin markers and unread state in the thread lists

Server-created threads (PR2 F2) now say who created them and whether they
are new, in the sidebar and on the Chats page, and reach an open sidebar
without a reload.

- ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run,
  the provider mark for IM threads, the GitHub mark (new in
  ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions;
  the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an
  extension" label is its accessible name and tooltip. ThreadChannelIcon
  stays exported as a wrapper, and the provider chip tooltip is localized
  (no more "Feishu channel").
- Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the
  row aria-label becomes "{title}, unread" / "{title},未读". Never shown for
  the open thread.
- WorkspaceSidebar mounts useThreadActivity once.
- useMarkOpenThreadRead (components/workspace/chats): the default and
  custom-agent chat pages mark the thread read once its metadata has
  loaded, when its cached unread flips to true while open, when the tab
  becomes visible, and from onFinish (a send or a joined run ended). Each
  trigger goes through useMarkThreadRead, so it is debounced and a no-op
  while the cached unread is false.
- Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active
  row, no raw identifiers), use-mark-open-thread-read DOM tests,
  e2e thread-activity.spec.ts (new scheduled run appears unread within one
  fake-clock poll without a reload, opening it posts /read and clears the
  dot, a read on another device clears it, IM and GitHub markers, zh
  labels, the Chats page, no polling when unavailable);
  thread-history.spec.ts now finds the localized origin icon.
- Docs: frontend AGENTS.md, CHANGELOGs.

* feat(frontend): schedule event line in the originating chat and channel notice support

The chat that created a schedule now says when the schedule paused or
ended, and the channel cards say which apps receive scheduled task updates
(PR2 F3).

- ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with
  the describeTaskEvent text (title in <strong>, stop condition clamped to
  one line with the full text in its tooltip, last-run suffix), the time
  via formatTaskTime and a right-aligned "See that run" (run chat) or
  "Open task" link. PauseCircle for paused, CircleCheck for finished and
  once-done, CircleX for once-failed. role="note", aria-label "Scheduled
  task update" / "定时任务通知", data-task-id and data-event-id only.
- MessageList takes scheduledTaskEvents and renders placeTaskEvents output
  through a new VirtualMessageList.renderAfterGroup, so each line sits at
  the end of its after_run_id turn inside the group's measured row (also
  when the list virtualizes); lines with nowhere to go render at the tail.
- Both chat pages load useThreadScheduledTaskEvents (not gated on the task
  list, so the lines stay after a task is deleted). The custom-agent page
  passes its agent so "See that run" opens the run on that agent's route.
- ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not
  available for this app yet" (定时任务通知:会发送到这里 / 此应用暂不支持)
  on the Settings provider cards and under each sidebar channel row; nothing
  when an older backend does not report proactive_notifications.
- e2e mock controller: setScheduledTasks.
- Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests
  (every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese
  title), MessageList placement DOM tests; e2e
  scheduled-task-chat-events.spec.ts (the line appears with the card's
  "Paused by agent" without a reload, See that run opens the run chat,
  anchored vs tail placement, two events in time order, the line survives
  task deletion and Open task shows the task is gone; en-3 and zh-1) and
  channels.spec.ts (WeCom sent here, Feishu not yet; en and zh).
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): tie pause notices to the pause they describe

- Paused by agent links and times only the run named in last_error. When
  that run is not loaded, the notice shows no link and the task's own time
  instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
  shown, so browsing older runs no longer swaps the auto-pause reason and
  "Open latest run" to an earlier miss. It shares that page's request.

* fix(scheduler): address PR2 review findings

- Event lines: hold an event whose anchor run is on an unloaded history
  page instead of placing it at the bottom (hasMoreHistory); tail fallback
  only for fully loaded (branched/pruned) history or no anchor.
- Idle-finish anchor follows its reason: end:<iso> only for an end_at
  finish, seq:<n> otherwise, so a later end-time finish is not deduplicated
  away by an earlier max_runs finish.
- Observer takes FOR SHARE on the originating threads_meta row, so a racing
  thread delete cannot leave an orphan event line.
- One taskPagePath (core), re-exported by the card.
- Notification locales derived from NotificationLocale; /preferences uses it.
- IM notices drop generic reasons (goal_not_met_yet, unknown) instead of
  repeating the sentence.
- Localized provider names on Settings cards, sidebar rows and setup dialog.
- zh event lines: no space after a Chinese title (Latin-aware gap).
- Unread row label keeps the origin marker; chats page keeps the time as a
  description.
- Sidebar shows the scheduled-updates line only for connected apps;
  Settings says "available after you connect" for unconnected push apps.
- Event line keeps a real space before the time.

* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes

- The edit form compares a cron in the form its schedule input emits, so an
  untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
  edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
  land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
  manual pause no longer reads as "Paused by agent".

* test(scheduler): consolidate PR2 backend test setup

- Fold the repeated task, origin-chat and observer setup of the
  lifecycle-event repository tests into one lifecycle_task builder.
- Build the per-owner-cap services through one _capped_service builder
  that keeps every cap explicit at the call site.
- Table the five SSE route header checks into one parametrized test with
  the same routes, bodies and Content-Location expectations.
- Reuse bind_wecom and hoist repeated local imports in the event tests.

Every case and assertion is kept; collected cases are unchanged.

* test(frontend): consolidate PR2 thread activity and origin test setup

Extract shared builders and helpers in the frontend tests PR2 adds, with
every case, input and assertion kept:

- activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant
  replace the per-test seed, spy and apply boilerplate.
- mark-read.dom.test.tsx: newClient() and advance() replace the repeated
  QueryClient wrapper and fake-timer act blocks.
- origin.test.ts: the English and Chinese label checks become tables; the
  no-English-name check now runs on every Chinese row.
- thread-activity.spec.ts: a scheduledRun() builder replaces four inline
  copies of the scheduled run thread.
- channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper.

Collected cases are unchanged (unit 211, e2e 60); no test was removed.

* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages

useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.

* fix(scheduler): two gaps found in live acceptance

- A fresh scheduled run thread is named when it launches. The worker
  copied the title to the thread list only when the run ended, so the
  sidebar, which now shows the new run thread at once, listed it as
  untitled while it ran.
- A task capped at one run finishes with "its one run is done" /
  "唯一一次运行已完成" in the chat event line and the IM notice, instead
  of "all 1 runs are done".

* test(scheduled-tasks): let the viewer's time carry its own day label

Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.

* test(scheduler): expect the one-run finish copy in service notices

* fix(gateway): create a scheduled run thread with its title

Pre-setting the name after start_run raced the thread row, which the run
worker creates asynchronously: live, one of two run threads still reached
the sidebar untitled. Run admission now creates a new thread with the title
its run input carries, which the worker would copy only at run end.

* docs(changelog): reference #6393 for the scheduled-task signal entries

* fix(scheduled-tasks): route event-line run links by the agent that ran them

The originating chat's agent was used for "See that run", but a task's
agent can change after the chat created it, so the link could open the run
chat on another agent's route. Lifecycle events now record run_agent_name,
the task's agent when the run finished (edits are refused while a run is
active), and the line routes by it.

* fix(scheduled-tasks): load up to the route's maximum of chat event lines

The chat loads its lifecycle events once, and the route's default of 50
would silently drop the oldest lines of a chat with many schedules. Ask
for the maximum, 200.

* fix(threads): keep activity reads and the first cursor in step with the lists

- Accepting the first activity cursor refreshes the thread lists once. The
  lists load on their own, possibly before a run the seed already counts,
  and later polls start after it, so such a thread never appeared.
- A local read whose returned read clock jumps past that one read also
  covers reads on other devices; the lists are refreshed instead of the
  clock silently skipping those threads' stale unread dots.
- An invalid cursor re-seeds through the same path, so it refreshes once.

* docs(agents): bring the middlewares guidance chain back under the hard limit

#6378 and #6363 each fit the budget, but together the middlewares AGENTS.md
chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint
Check fails on main. Move the scheduled-task recovery, launch-update and
queue-timeout rules from backend/AGENTS.md into the persistence guide's
lifecycle section, which backend/AGENTS.md already points to. The chain is
now 98,067 bytes.

* fix(helm): bump the chart's config_version to 55

config.example.yaml moved to 55 in this PR, so the Helm chart's
validate-chart drift check failed. The chart's embedded config needs no
field changes: the new scheduler and channel keys have defaults.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-07 10:22:15 +08:00

93 KiB
Raw Blame History

Configuration Guide

This guide explains how to configure DeerFlow for your environment.

Prompt overlays

Operators can add instructions around existing system prompts in config.yaml. Empty or omitted extensions preserve the original prompt exactly. The built-in instructions remain present; extensions do not change tool authorization, memory admission gates, or runtime limits.

lead_prompt_overlay:
  prepend: "Use our organization's terminology in reports."
  append: "Conclude with decisions and unresolved questions."
subagents:
  agents:
    general-purpose:
      prompt_overlay:
        append: "Include evidence for research claims."
    bash:
      prompt_overlay:
        prepend: "Prefer reproducible, non-interactive commands."
memory:
  backend_config:
    prompt_prepend: "Prefer concise summaries of lasting preferences."
    prompt_append: "Preserve explicit corrections without redundant wording."

Merge these into existing sections rather than duplicating YAML keys. Extensions are joined with two newlines around the assembled system text. They are literal: {conversation} and JSON braces are not interpolated. The configuration loader's existing $ENV_VAR resolution still applies to values beginning with $. Use YAML block scalars for multiline instructions.

These are trusted operator instructions, never request/body context or per-user agent data. Lead extensions apply to the assembled default or custom lead agent; its SOUL, skills and runtime guidance remain intact. Subagent extensions also support configured custom subagents and never mutate the shared built-in registry. Explicit application snapshots remain isolated; subsequent assemblies use new settings, while already-built graphs retain their prompt.

DeerMem's current memory_update chat prompt handles both summary updates and fact extraction; the historical FACT_EXTRACTION_PROMPT constant is not a separate live extraction call. Its extensions wrap the first system message after template rendering, leaving memory and conversation data in their original human message. They also work with prompts_dir templates, which must include a system message when overlays are used. Restart the memory backend/Gateway after changing its configuration. Other memory backends do not consume these DeerMem-specific fields. Existing external memory templates remain the mechanism for complete template replacement; this feature adds no editing endpoint or UI.

Model request admission

For request-per-minute limits, opt into pacing on each relevant models[] entry. For example, add this alongside its name, use, and model fields:

request_admission:
  requests_per_minute: 60
  group: shared-provider-account
  max_wait_seconds: 300
  max_queue_size: 256

Like every other field, these accept $VAR environment references; an integer field such as requests_per_minute: $RPM validates when the variable holds a decimal integer.

Calls wait in a bounded FIFO before dispatch. At 60 RPM, admissions are spaced at least one second apart, even after idle periods. The first call can proceed immediately. Async waiting is cancellable; a cancelled or expired waiter spends no admission. Queue overflow and wait expiry fail locally before dispatch. The deadline covers admission waiting only, not the provider's response time.

An explicit group shares the budget across model profiles using the same provider quota. Omit it for a separate budget per configured model name. All profiles in a group must have identical settings. The limiter is shared by factory-created model instances across threads and event loops, including lead agents, subagents and auxiliary models using the standard LangChain BaseChatModel invoke/stream hooks. Restart the Gateway after changing, disabling or regrouping active policies; conflicting settings fail model construction instead of resetting a live budget.

When enabled, exposed SDK max_retries settings are set to zero: SDK retries would bypass the admission hook. Agent middleware retries still work and each new attempt is paced. Calls outside that middleware no longer get SDK retries. Custom providers that bypass BaseChatModel admission hooks or perform hidden retries need their own integration. A caller-supplied rate_limiter cannot be combined with request_admission.

This is process-local RPM pacing, not TPM accounting or a distributed quota service. Divide the provider allowance among Gateway workers/replicas and allow headroom for other applications. Provider token limits, external consumption, billing failures and permanent errors can still fail a task. Existing llm_call.max_concurrent_calls remains independent: when enabled, its slot is held while the underlying model waits for admission, so use shared groups and concurrency settings deliberately. Waiting contributes to model-call latency and remains subject to the enclosing run's timeout. This option is off by default and does not promise unlimited retries or eventual task completion.

Config Versioning

config.example.yaml contains a config_version field that tracks schema changes. When the example version is higher than your local config.yaml, the application emits a startup warning:

WARNING - Your config.yaml (version 0) is outdated — the latest version is 1.
Run `make config-upgrade` to merge new fields into your config.
  • Missing config_version in your config is treated as version 0.
  • Run make config-upgrade to auto-merge missing fields (your existing values are preserved, a .bak backup is created). It upgrades the file the Gateway loads, resolved as in Configuration Priority.
  • When changing the config schema, bump config_version in config.example.yaml.

Configuration Sections

Extensions

MCP servers and skill enabled states live in extensions_config.json, separate from config.yaml. Use mcpServers.<server>.routing to add soft MCP tool preference hints for requests that should prefer a specific MCP server or tool. See MCP Server Configuration for the schema, example, and soft-vs-hard routing boundary.

Runtime edits to extensions_config.json (the MCP and skills APIs, the web UI, DeerFlowClient, or an editor) are picked up by every Gateway process that reads the same file: the process cache revalidates the file's path and content signature on each read, so uvicorn workers and multi-instance Pods sharing one volume converge without a restart or a per-Pod reload call. A change made through an API call on one instance is visible to the others on their next request. A missing, partially written or invalid file keeps the previously loaded configuration until a complete revision lands, including when it disappears during a reload; the Gateway logs one warning per such revision.

Recursion Limits

Gateway runs use the top-level recursion_limit as their LangGraph super-step budget when the request does not include an explicit value. It defaults to 100; raise it for deployments whose normal tasks need longer agent loops. Valid request values take precedence, while invalid values fall back to the configured default. max_recursion_limit (default 1000) caps both sources to limit runaway LLM cost. Both settings are read per run, so changes apply to the next request without a Gateway restart.

These settings apply to Gateway API runs. IM channel runs and embedded DeerFlowClient runs retain their own defaults and can be overridden through their channel/client-specific configuration or per-call options.

Models

Configure the LLM models available to the agent:

models:
  - name: gpt-4                    # Internal identifier
    display_name: GPT-4            # Human-readable name
    use: langchain_openai:ChatOpenAI  # LangChain class path
    model: gpt-4                   # Model identifier for API
    api_key: $OPENAI_API_KEY       # API key (use env var)
    max_tokens: 4096               # Max tokens per request
    temperature: 0.7               # Sampling temperature

Supported Providers:

  • OpenAI (langchain_openai:ChatOpenAI)
  • Anthropic (langchain_anthropic:ChatAnthropic)
  • DeepSeek (langchain_deepseek:ChatDeepSeek)
  • Xiaomi MiMo (deerflow.models.patched_mimo:PatchedChatMiMo)
  • Claude Code OAuth (deerflow.models.claude_provider:ClaudeChatModel)
  • Codex CLI (deerflow.models.openai_codex_provider:CodexChatModel)
  • Any LangChain-compatible provider

CLI-backed provider examples:

models:
  - name: gpt-5.4
    display_name: GPT-5.4 (Codex CLI)
    use: deerflow.models.openai_codex_provider:CodexChatModel
    model: gpt-5.4
    supports_thinking: true
    supports_reasoning_effort: true

  - name: claude-sonnet-4.6
    display_name: Claude Sonnet 4.6 (Claude Code OAuth)
    use: deerflow.models.claude_provider:ClaudeChatModel
    model: claude-sonnet-4-6
    max_tokens: 4096
    supports_thinking: true

Auth behavior for CLI-backed providers:

  • CodexChatModel loads Codex CLI auth from ~/.codex/auth.json
  • The Codex Responses endpoint currently rejects max_tokens and max_output_tokens, so CodexChatModel does not expose a request-level token cap
  • ClaudeChatModel accepts CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_AUTH_TOKEN, CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR, CLAUDE_CODE_CREDENTIALS_PATH, or plaintext ~/.claude/.credentials.json
  • A CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR handoff is drained on first use and the token is kept for the life of the process, so every ClaudeChatModel instance reuses it
  • On macOS, DeerFlow does not probe Keychain automatically. Use scripts/export_claude_code_oauth.py to export Claude Code auth explicitly when needed

To use OpenAI's /v1/responses endpoint with LangChain, keep using langchain_openai:ChatOpenAI and set:

models:
  - name: gpt-5-responses
    display_name: GPT-5 (Responses API)
    use: langchain_openai:ChatOpenAI
    model: gpt-5
    api_key: $OPENAI_API_KEY
    use_responses_api: true
    output_version: responses/v1

For OpenAI-compatible gateways (for example Novita or OpenRouter), keep using langchain_openai:ChatOpenAI and set base_url:

Note: for langchain_openai:ChatOpenAI the endpoint override key is base_url (not api_base). If you write api_base it is automatically normalized to base_url, and unrecognized keys are logged with a warning at model-build time. Some other model classes (e.g. PatchedChatDeepSeek) do use api_base — match the key to the class you configured.

models:
  - name: novita-deepseek-v3.2
    display_name: Novita DeepSeek V3.2
    use: langchain_openai:ChatOpenAI
    model: deepseek/deepseek-v3.2
    api_key: $NOVITA_API_KEY
    base_url: https://api.novita.ai/openai
    supports_thinking: true
    when_thinking_enabled:
      extra_body:
        thinking:
          type: enabled

  - name: minimax-m3
    display_name: MiniMax M3
    use: langchain_openai:ChatOpenAI
    model: MiniMax-M3
    api_key: $MINIMAX_API_KEY
    base_url: https://api.minimax.io/v1
    max_tokens: 4096
    temperature: 1.0  # MiniMax requires temperature in (0.0, 1.0]
    supports_vision: true

  - name: minimax-m2.7
    display_name: MiniMax M2.7
    use: langchain_openai:ChatOpenAI
    model: MiniMax-M2.7
    api_key: $MINIMAX_API_KEY
    base_url: https://api.minimax.io/v1
    max_tokens: 4096
    temperature: 1.0  # MiniMax requires temperature in (0.0, 1.0]
    supports_vision: false  # M2.7 is text-only; M3 supports vision

  - name: minimax-m2.7-highspeed
    display_name: MiniMax M2.7 Highspeed
    use: langchain_openai:ChatOpenAI
    model: MiniMax-M2.7-highspeed
    api_key: $MINIMAX_API_KEY
    base_url: https://api.minimax.io/v1
    max_tokens: 4096
    temperature: 1.0  # MiniMax requires temperature in (0.0, 1.0]
    supports_vision: false  # M2.7 is text-only; M3 supports vision
  - name: openrouter-gemini-2.5-flash
    display_name: Gemini 2.5 Flash (OpenRouter)
    use: langchain_openai:ChatOpenAI
    model: google/gemini-2.5-flash-preview
    api_key: $OPENAI_API_KEY
    base_url: https://openrouter.ai/api/v1

If your OpenRouter key lives in a different environment variable name, point api_key at that variable explicitly (for example api_key: $OPENROUTER_API_KEY).

Thinking Models: Some models support "thinking" mode for complex reasoning:

models:
  - name: deepseek-v3
    supports_thinking: true
    when_thinking_enabled:
      extra_body:
        thinking:
          type: enabled

Gemini via Google's OpenAI-compatible endpoint

When routing Gemini through an OpenAI-compatible proxy (Vertex AI OpenAI compat endpoint, AI Studio, or third-party gateways) with thinking enabled, the API attaches a thought_signature to each tool-call object returned in the response. Every subsequent request that replays those assistant messages must echo those signatures back on the tool-call entries or the API returns:

HTTP 400 INVALID_ARGUMENT: function call `<tool>` in the N. content block is
missing a `thought_signature`.

Standard langchain_openai:ChatOpenAI silently drops thought_signature when serialising messages. Use deerflow.models.patched_openai:PatchedChatOpenAI instead — it re-injects the tool-call signatures (sourced from AIMessage.additional_kwargs["tool_calls"]) into every outgoing payload:

models:
  - name: gemini-3.1-pro-preview
    display_name: Gemini 3.1 Pro (Thinking)
    use: deerflow.models.patched_openai:PatchedChatOpenAI
    model: gemini-3.1-pro-preview
    api_key: $GEMINI_API_KEY
    base_url: https://generativelanguage.googleapis.com/v1beta/openai/
    max_tokens: 16384
    supports_vision: true
    reasoning:
      thinking: required
      dialect: none
      effort:
        values: [minimal, low, medium, high]

This example targets Google's official endpoint. Its OpenAI compatibility API accepts reasoning_effort; extra_body.thinking becomes an unsupported top-level thinking field and causes HTTP 400. Gemini 3.1 Pro cannot disable thinking, so thinking: required keeps it enabled even when a caller requests otherwise, while dialect: none prevents a provider-specific thinking toggle. Omitting effort uses the model's default.

If you copied the previous example, replace its supports_thinking, when_thinking_enabled, and when_thinking_disabled settings with the reasoning block above. Third-party gateways may require different model IDs and reasoning parameters; follow that gateway's documentation instead of reusing Google's profile unchanged.

MiMo with thinking via OpenAI-compatible API:

MiMo returns reasoning_content on assistant messages in thinking mode. In multi-turn agent conversations with tool calls, subsequent requests must preserve that historical reasoning_content on assistant messages or the MiMo API can return HTTP 400. Standard langchain_openai:ChatOpenAI drops this provider-specific field, so use deerflow.models.patched_mimo:PatchedChatMiMo:

For pay-as-you-go API keys (sk-...), use https://api.xiaomimimo.com/v1. For Token Plan keys (tp-...), use the regional Token Plan Base URL shown in the MiMo console, such as https://token-plan-cn.xiaomimimo.com/v1. MiMo documents these key types as separate and non-interchangeable.

PatchedChatMiMo is model-id agnostic. Use it for every MiMo thinking model entry you configure, including model entries referenced by subagents.*.model overrides (for example mimo-v2.5-pro, mimo-v2.5, mimo-v2-pro, mimo-v2-omni, or mimo-v2-flash).

models:
  - name: mimo-v2.5-pro
    display_name: MiMo V2.5 Pro
    use: deerflow.models.patched_mimo:PatchedChatMiMo
    model: mimo-v2.5-pro
    api_key: $MIMO_API_KEY
    base_url: https://api.xiaomimimo.com/v1
    max_tokens: 8192
    supports_thinking: true
    supports_vision: false
    when_thinking_enabled:
      extra_body:
        thinking:
          type: enabled
    when_thinking_disabled:
      extra_body:
        thinking:
          type: disabled

PatchedChatMiMo preserves MiMo's choices[].message.reasoning_content, streaming delta.reasoning_content, and request-history assistant reasoning_content fields. It does not reuse the DeepSeek provider.

RAGFlow Knowledge Retrieval

RAGFlow integration is disabled by default. It adds one read-only Agent tool, knowledge_search. DeerFlow does not persist a copy of dataset or document metadata; RAGFlow is the sole source of truth. The configured API key is tenant-scoped. An optional operator-controlled datasets list restricts every Agent on this deployment to the same dataset-ID allowlist; omitting it searches all datasets visible to that tenant API key. An explicitly empty datasets: [] is rejected rather than being treated as tenant-wide access.

tool_groups:
  - name: knowledge

tools:
  - name: knowledge_search
    group: knowledge
    use: deerflow.community.ragflow.tools:knowledge_search_tool
    base_url: http://localhost:9380
    api_key: $RAGFLOW_API_KEY
    datasets:
      - 0123456789abcdef0123456789abcdef
      - fedcba9876543210fedcba9876543210
    timeout: 30
    page_size: 8
    similarity_threshold: 0.2
    vector_similarity_weight: 0.3
    top_k: 256
    max_chars_per_chunk: 800
    max_total_chars: 8000

The tool is opt-in through the normal tools: list. datasets is optional but, when present, must contain at least one ID. If it contains RAGFlow dataset IDs selected by the deployment operator, DeerFlow does not validate their existence while loading configuration; on each search it verifies them with ID-filtered requests. If datasets is omitted, each search paginates through the tenant-visible dataset catalog. Both paths resolve current names, embedding models, and chunk counts. Empty datasets are ignored; an empty dataset that has no embedding-model metadata is also skipped with a server warning. The remaining datasets are grouped by the exact embedding-model identifier and each group is sent to RAGFlow with a non-empty dataset_ids list. At most four groups are retrieved concurrently. Because raw similarity scores from different embedding spaces are not globally comparable, DeerFlow preserves each group's RAGFlow ranking, interleaves equal rank positions, omits score labels when more than one group is searched, and applies page_size as a single global chunk limit. If any searchable group fails, the whole tool call fails rather than silently omitting part of the configured scope. A deleted or inaccessible configured dataset identifies its ordinal entry in knowledge_search.datasets and produces guidance to check config.yaml. Dataset IDs and catalog listing are not exposed to the Agent.

Use an allowlist to narrow the tenant-wide scope; compatible embedding models are no longer required across selected datasets. base_url must not contain embedded username or password information. For Docker or Kubernetes, it must be reachable from the Gateway container or Pod; localhost refers to that container or Pod, not the host machine.

This integration is retrieval-only. Dataset creation, uploads, parsing, and deletion remain in RAGFlow and are not exposed as Agent tools, workspace pages, or DeerFlow APIs. The authenticated /api/knowledge/retrieval-catalog routes exist only to populate the custom-agent chat selector. The provider-neutral knowledge_base block only gates DeerFlow's knowledge capability and selector; configure the RAGFlow connection and retrieval defaults on the tools[].name: knowledge_search entry shown above:

knowledge_base:
  enabled: true
  scope_selection_enabled: true

When enabled, include the list_knowledge_bases tool entry shown above if the model should be able to discover configured dataset names. The frontend uses GET /api/features -> knowledge_base only to gate the custom-agent chat selector. RAGFlow API keys and dataset UUIDs are never returned to the browser or model. Do not put RAGFlow-specific connection, allowlist, or retrieval parameters in knowledge_base; they are read only from the provider tool entry, so different knowledge providers can use their own settings.

LightRAG Knowledge Retrieval

LightRAG integration is disabled by default. It is an alternative provider for the same read-only knowledge_search tool: an operator picks RAGFlow or LightRAG by which entry appears in the tools: list — the two entries share one name, and on duplicate names DeerFlow keeps the first configured entry, so configure exactly one. Requires LightRAG v1.4.9 or newer: v1.4.8 introduced the data-retrieval endpoint but returned a pre-envelope response shape, and the status/data envelope plus the citation fields consumed here shipped in v1.4.9. DeerFlow does not persist any index metadata; LightRAG stays the sole source of truth, and the deployment's single indexed workspace is always searched.

tool_groups:
  - name: knowledge

tools:
  - name: knowledge_search
    group: knowledge
    use: deerflow.community.lightrag.tools:knowledge_search_tool
    base_url: http://localhost:9621
    api_key: $LIGHTRAG_API_KEY
    mode: mix
    timeout: 30
    top_k: 60
    chunk_top_k: 8
    max_chars_per_chunk: 800
    max_total_chars: 8000

The tool is opt-in through the normal tools: list. Retrieval uses LightRAG's POST /query/data endpoint, which performs no LLM generation and returns structured entities, relationships, chunks, and references; DeerFlow keeps the chunks — the document text the selected mode already ranked as relevant — and formats them as citation-numbered text, dropping the graph objects to stay compact and keep the citation shape shared with the RAGFlow provider. mode selects the retrieval strategy (naive, local, global, hybrid, or mix; default mix, matching the LightRAG API's own QueryRequest default; bypass is rejected because it skips the index entirely). top_k bounds the entities retrieved in local mode or relationships in global mode, and the optional chunk_top_k bounds the text chunks retrieved and kept after reranking; both are capped at 1000 by the LightRAG server. Short queries that fail LightRAG's minimum-length validation surface the server's readable message. max_chars_per_chunk / max_total_chars bound the model-visible output size.

api_key is optional because LightRAG may run without authentication. Only omit it for loopback or trusted-network deployments — a network-exposed LightRAG must have authentication enabled, and then the key is sent as the X-API-Key header and redacted from every model-visible error and from server logs. Blank values are treated as unauthenticated. base_url must not contain embedded username or password information, and for Docker or Kubernetes it must be reachable from the Gateway container or Pod.

Internal identifiers (chunk IDs and the response-local reference IDs) are never exposed to the Agent; citations use the operator-readable file_path. This integration is retrieval-only. Document insertion, indexing, and graph mutation remain in LightRAG and are not exposed as Agent tools or DeerFlow APIs.

Tool Groups

Organize tools into logical groups:

tool_groups:
  - name: web          # Web browsing and search
  - name: file:read    # Read-only file operations
  - name: file:write   # Write file operations
  - name: bash         # Shell command execution

Scheduler

The scheduled-task MVP adds a scheduler section to config.yaml:

scheduler:
  enabled: false
  tool_enabled: false
  multi_instance: false
  poll_interval_seconds: 5
  lease_seconds: 120
  max_concurrent_runs: 3
  max_concurrent_runs_per_user: 2
  queue_timeout_seconds: 3600
  min_once_delay_seconds: 60
  recursion_limit: 1000

Notes:

  • enabled: false keeps background polling off by default.
  • tool_enabled: false keeps conversation schedule tools off. Set it together with enabled: true and restart Gateway to offer schedule_task in authorized interactive conversations and stop_scheduled_task to every scheduled run of a task, whether a chat or the tasks page created it (see "Create schedules in a conversation" in the README). A run can stop only its own task's schedule. Managing tasks from chat (create, update, resume, pause, delete, trial, notes) is interactive-only: a conversation manages the tasks created in it and, when it is a task's run conversation, that task; scheduled runs get only their own stop. The tools are offered only while this Gateway process's scheduler is running. With tool_enabled off, a task's stop condition is still sent to its runs, phrased so the run reports a met rule instead of calling a tool it does not have. Each launch reads tool_enabled from the live config, the same value that decides whether the run gets the tool, so the phrasing always matches.
  • The tasks page and REST API have parity with conversations for the per-run goal (goal_objective), safety cap (max_runs, end_at) and stop condition (stop_condition, stored in its own column and appended to the run message only at launch). Resume computes the next run from now without a catch-up run and refuses a one-time task whose time passed; reactivating a task whose cap is used up returns 409 limits_exhausted unless the same request (for example the optional Resume body {"max_runs", "end_at"}) renews or clears the cap. Goal-check failures neither count toward nor reset the three-miss automatic pause, and changing the goal, instructions or stop condition, or adding a note, starts a new count (Resume keeps it). Creating a task while this Gateway process's poller is not running returns 409 scheduler_not_running; /api/features reports scheduled_tasks.running. See backend/docs/API.md#scheduled-tasks.
  • multi_instance: true opts into lease-aware scheduler recovery across Gateway instances. It requires Postgres, run_ownership.heartbeat_enabled: true, and run_events.backend: db; otherwise startup fails fast. Leave it false for the default single-instance scheduler.
  • max_concurrent_runs is a shared global execution cap in multi-instance mode. Waiting queued rows do not consume capacity; an atomic queued → launching claim counts launching/running rows under a Postgres advisory lock so concurrent Pods cannot exceed the cap.
  • max_concurrent_runs_per_user (default 2, 0 = off, at most 32) caps how many scheduled runs one task owner may have launching/running at a time, whatever started them (manual "run now" included). The effective cap is min(max_concurrent_runs_per_user, max_concurrent_runs), so with the defaults one owner holds at most 2 of the 3 global slots and the third stays free for other owners. The check runs in the same atomic claim, under the same database-wide lock, as the global cap, so it holds across workers and Pods. A row rejected by either cap stays queued.
  • The waiting queue is drained fairly whatever the per-owner cap: eligible rows are ranked within their owner and the drain batch takes every owner's oldest row before any owner's second one, so one owner's backlog cannot push another owner's run out of the batch. Owners already at their cap drop out of the batch before its size limit applies. Same-thread FIFO and the order within one owner (fewest launch attempts, then oldest) are unchanged; with a single owner the order is the same as before.
  • queue_timeout_seconds limits how long a persisted occurrence may wait for capacity or a reused thread to become available. Expired occurrences are marked failed; queued rows otherwise survive Gateway restarts. With the per-owner cap on, an owner whose runs take longer than this can see its waiting runs expire: the run history then reads "Skipped: it waited too long for a free slot", and no chat event or IM notice is sent for it. Keep queue_timeout_seconds well above an owner's typical run length.
  • A task definition is immutable while an occurrence is queued, launching, or running. This prevents a durable occurrence from mixing its admitted thread with a later prompt or schedule edit. Transitioning a task to paused or deleting it cancels a waiting row; PATCH and resume return a conflict until the active occurrence finishes or is cancelled.
  • A manual trigger remains explicit even while the recurring schedule is paused: it may wait in the durable queue and run later, while the task itself stays paused. Transitioning an enabled task to paused still cancels its waiting occurrence atomically.
  • Queue admission, PATCH/resume, pause, and delete serialize on the parent task row. Per-thread FIFO spans all active states, so an older launching or running occurrence blocks a newer queued occurrence on the same reused thread as well as an older queued occurrence.
  • Multi-instance reconciliation uses the run ownership lease: a live peer run is preserved, an expired lease is atomically taken over before its scheduled row is interrupted, and a stale Pod cannot overwrite a newer Pod's parent-task bookkeeping.
  • recursion_limit is the LangGraph super-step cap for scheduler-launched runs (default 1000, matching the web UI's interactive budget). Values above max_recursion_limit (default 1000) are clamped. This field is read at dispatch, so a YAML edit applies to the next scheduled run without a Gateway restart.
  • Poller fields (enabled, multi_instance, poll_interval_seconds, lease_seconds, max_concurrent_runs, max_concurrent_runs_per_user, queue_timeout_seconds, min_once_delay_seconds) are restart-required; edits need a Gateway restart.
  • Upgrade note (behavior change): one owner now runs at most 2 scheduled runs at a time by default (before, up to max_concurrent_runs). Set scheduler.max_concurrent_runs_per_user: 0 to restore the old behavior; the drain stays fair to owners either way.
  • Upgrade note: before upgrading a deployment with GATEWAY_WORKERS > 1 and scheduler.enabled: true, either run the scheduler on exactly one Gateway worker or enable scheduler.multi_instance: true with shared Postgres, run_ownership.heartbeat_enabled: true, and run_events.backend: db. The startup gate now rejects the unsafe combination instead of allowing it to start silently.
  • Upgrade note: in multi-instance mode, max_concurrent_runs is cluster-wide rather than per Pod and counts launching/running occurrences. Waiting queued rows remain outside the execution cap; capacity does not multiply with the replica count.
  • Upgrade note: scheduler.multi_instance and its related scheduler, ownership, and run-event settings are startup-only. Restart all Gateway Pods together after changing them; a ConfigMap update without a coordinated restart leaves the running service on its previous mode.
  • Multi-worker deployments (GATEWAY_WORKERS > 1) and declared multi-instance deployments (deployment.multi_instance: true / DEER_FLOW_MULTI_INSTANCE=1, see Deployment topology) must use the Postgres database backend, enable run ownership heartbeats, set run_events.backend: db, and use the Redis stream bridge. SQLite silently ignores row-level locks, while memory and JSONL run-event stores are process-local and cannot enforce singleton delivery receipts across workers; startup rejects these combinations. The process-local agentic browser tool group is incompatible with multiple Gateway workers; keep the Gateway to one worker process while browser_navigate is enabled — GATEWAY_WORKERS=1, plus WEB_CONCURRENCY unset or 1 on launches that pass uvicorn no worker count (backend/Dockerfile, scripts/serve.sh), where uvicorn takes the process count from it. Browser control also requires the backend browser extra (cd backend && uv sync --extra browser && uv run playwright install chromium); startup detects enabled browser config and fails fast when Playwright is missing, and /api/features reports browser_control.enabled=false until the runtime is available.
  • The MVP supports thread reuse and fresh-thread-per-run execution modes.
  • Create/update accept optional assistant_id (lead_agent by default, or an existing custom agent for the task owner).
  • Create/update accept once, cron, and interval. Interval uses schedule_spec.every_seconds (UTC now + N, no missed-beat catch-up). N is at least min_once_delay_seconds (default 60) and at most 30 days.
  • Manual trigger uses the same scheduled-task resource and run lifecycle.
  • Scheduled task definitions and task-run history are persisted in the application database.
  • With channel_connections.enabled: true, the scheduler sends IM notices ("scheduled task updates") to the task owner's connected identities. A delivery worker (same poll cadence as the scheduler) pushes them.
    • Which apps: only providers whose channel implements proactive push get notices; today that is WeCom (proactive_notifications in app/channels/capabilities.py, also returned by GET /api/channels/providers, where Settings shows "sent here" or "not available for this app yet"). Other providers get no outbox rows at all, and rows queued for them before the upgrade end once as failed without retries.
    • Events: run_completed, run_failed, run_unmet (goal missed), task_paused (automatic pause after three missed goals), task_stopped (paused by the agent because its stop condition was met) and task_finished (all max_runs done or end_at reached).
    • One message per occurrence: when one occurrence produces several events, only the first of task_stopped > task_paused > task_finished > run_failed > run_unmet > run_completed is sent, and that message still says how the last run went ("The last run failed." / "didn't meet the goal"). A one-time task sends only its run's outcome, never a separate task_finished.
    • Transactional: notices are queued by the finalization observer in the same database transaction that records the outcome and are deduplicated per occurrence (or per end time for a task that ends between runs), so an occurrence finalized by crash or lease recovery notifies exactly once. Plain manual "run now" trials and interrupted runs stay silent; a trial that pauses or finishes the task still sends that notice. Occurrences that end without a finished run (launch error, queue timeout, interrupted by a restart) stay silent.
    • Text: the task title, what happened, a one-line result (the same run summary the tasks page shows, redacted, only when the agent replied) and "Open DeerFlow → Scheduled tasks for details." Notices carry no task, run or thread IDs, no timestamps and no links: most deployments run on localhost or a LAN, where a link would be dead on the phone that receives it.
    • Language: the owner's web UI language (the locale user preference, which the web app keeps in sync), else channel_connections.notification_locale (en-US or zh-CN, default en-US; startup-scoped). Auth-disabled installs always use the config value.
    • Delivery: channel/transport outages park rows without consuming the retry budget, for up to about a day; platform rejections exhaust ~15 minutes of counted retries then settle failed. The worker re-checks the binding right before sending: a target the owner has disconnected since enqueue is dropped as failed, never pushed.

Deployment topology (multi-instance)

deployment:
  multi_instance: false

Notes:

  • GATEWAY_WORKERS / WEB_CONCURRENCY only count the uvicorn workers of one process tree. A Kubernetes Deployment with replicas > 1 runs one worker per Pod, so every Pod reports a single worker and the multi-worker startup gate stays inert — while each Pod's startup orphan reconciliation still writes the other Pods' lease-less runs off as crashed on every rolling update.
  • Set multi_instance: true (or export DEER_FLOW_MULTI_INSTANCE=1, which lets deploy tooling such as a Helm chart set it from its replica count) on every instance that shares one database. Startup then enforces the same prerequisites as GATEWAY_WORKERS > 1: database.backend: postgres, run_events.backend: db, run_ownership.heartbeat_enabled: true, and a Redis stream bridge (stream_bridge.type: redis or DEER_FLOW_STREAM_BRIDGE_REDIS_URL). It refuses an explicit sandbox.ownership.type: memory, process-local browser tools, and scheduler.enabled: true without scheduler.multi_instance: true.
  • DEER_FLOW_MULTI_INSTANCE treats blank, 0, false, no and off as "not declared"; any other value declares a multi-instance deployment, so a typo fails closed.
  • The declaration also drives the agent_storage.backend: file divergence warning, the inbound webhook dedupe warning, and the WeChat QR-login guard, which otherwise only look at the worker count.
  • Restart-required: the gate runs once at startup. Restart all Gateway instances together after changing it.

Agent Storage

Custom agent definitions (config.yaml + SOUL.md) are stored per-user on local disk by default. This is separate from the database backend (which holds run/thread/event data) and from agent memory.

agent_storage:
  backend: file   # file (default) | db
  • backend: file — the historical layout under {base_dir}/users/{user_id}/agents/. Single-node by construction: an agent created on one node is not visible to other nodes without a shared mount.
  • backend: db — one row per agent in the shared SQL persistence layer (a new agents table), so every node in a multi-instance deployment sees the same agents. Requires database.backend to be sqlite or postgres; the Gateway fails fast at startup if it is memory (a per-process database cannot share definitions).
  • agent_storage is restart-required (the backend is captured at Gateway lifespan startup).
  • In a multi-worker Postgres deployment (GATEWAY_WORKERS > 1), leaving agent_storage.backend: file logs a startup warning — agents written to one node's local disk are invisible to the others, which is exactly the divergence the db backend fixes.

Migrating an existing install from file to db:

python backend/scripts/migrate_agents_to_db.py            # copy on-disk agents into the db
python backend/scripts/migrate_agents_to_db.py --dry-run  # preview without writing

The importer is idempotent (already-present agents are skipped) and leaves the source files untouched, so reverting agent_storage.backend to file is a clean rollback. Agent memory (memory.json) is unaffected by this switch.

Tools

Configure specific tools available to the agent:

tools:
  - name: web_search
    group: web
    use: deerflow.community.tavily.tools:web_search_tool
    max_results: 5
    include_domains:             # Optional: limit search sources to these domains
      - docs.python.org
      - developer.mozilla.org
    exclude_domains: []         # Optional: domains to exclude from search results
    # api_key: $TAVILY_API_KEY  # Optional

For Tavily, include_domains and exclude_domains are deployment-only options read from the web_search tool entry and passed directly to TavilyClient.search. For a non-empty include_domains, DeerFlow also sends include_domains_mode: filter so Tavily restricts results to those domains rather than merely boosting them. Either list may be configured independently. Omitted options are not added to the SDK call; explicit empty lists are forwarded as [], meaning no inclusion restriction or no excluded domains, respectively. No include_domains_mode is sent for an empty or omitted include_domains. These filters compose with max_results and the model's optional time_range. The model-visible arguments remain query and time_range; the filters do not apply to web_fetch or other search providers.

Jina fetch retries

Jina's web_fetch keeps one attempt by default. Configure retries on its existing tools entry; model-facing arguments remain unchanged:

tools:
  - name: web_fetch
    group: web
    use: deerflow.community.jina_ai.tools:web_fetch_tool
    timeout: 10
    max_retries: 2              # Additional attempts; default 0 (disabled)
    retry_budget_seconds: 30   # Total request + backoff budget; default 30

max_retries must be a non-negative integer and retry_budget_seconds a finite, positive number (YAML numbers, not strings or booleans). Invalid values return an Error: without sending a request. The budget applies only when retries are enabled, starts before HTTP client creation, and covers all attempts and waits. Each HTTP request timeout is capped by the remaining budget; the outer deadline also bounds responses that keep delivering data. The existing timeout remains Jina's X-Timeout header and the per-request HTTP timeout limit.

HTTP 502/503/504 and HTTPX connection-establishment errors (ConnectError, ConnectTimeout) are retryable. HTTP 429 is retried only with a valid Retry-After header; missing or malformed hints leave it terminal. HTTP 503 uses the same hints, falling back to local backoff when they are absent or invalid. Authentication, payment/credit and other statuses remain terminal even with hints; error-body prose never enables retries. Empty successful responses, read/write timeouts and arbitrary exceptions are not retried.

Retry-After accepts non-negative ASCII integer seconds or an HTTP-date (including obsolete HTTP date forms); past dates mean a zero server floor. Signed/fractional delays and non-HTTP dates are invalid. Local backoff ceilings start at 0.5 seconds, double to 1 and 2, then stay at 4. Each wait caps the local ceiling by the remaining budget and samples a uniform factor from 0.5 to 1.0. The actual wait is the greater of that local pacing and the server floor. A server floor is never reduced by jitter or the 4-second local ceiling. If the hinted wait equals or exceeds the remaining budget (including enormous valid integers), the last HTTP status error is returned without another request. Dates use wall time to compute a delay; requests and waits share one monotonic deadline. Each attempt uses only its own hint.

Offline mocked regressions cover these policies; they do not establish that Jina's hosted service always supplies recovery hints. No paid-provider testing is required to enable this option. Cancellation propagates during requests and waits. This stops local work; it cannot cancel work already started by Jina. Enabling retries can send up to 1 + max_retries upstream requests and incur additional cost. Successful content and final Error: results retain the existing contract.

Jina response byte budget

On the same Jina web_fetch tool entry, optionally set max_response_bytes: 1048576 (for example, 1 MiB). This uses existing tool configuration extras; no model-facing argument is added. Omitted or null preserves the buffered default. An enabled value must be a positive integer; booleans, strings, fractions, zero and negative values return Error: before HTTP client creation or network activity.

Enabled fetches stream and count actual content-decoded bytes (after decompression, before text decoding), ignoring Content-Length. Exactly the limit is accepted. The first chunk exceeding it stops consumption and closes the response, returning an explicit size Error: without body content, partial success or readability extraction. This applies to all statuses, including 502/503/504, and oversize never retries. Each retry response has its own counter within the existing shared time budget. Streams close on success, errors, cancellation and read failures. Responses within the limit retain charset decoding and existing status/retry handling.

This limits response retention/consumption, not wire-byte bandwidth or allocations inside HTTPX's decompressor; it is not a hard process-memory bound. The final Markdown truncation at 4096 characters is unchanged and independent of this option.

Serper web_search also accepts the optional model argument time_range: "day" | "week" | "month" | "year". For example, {"query": "Python releases", "time_range": "week"} sends tbs: "qdr:w" to Serper. Omitting time_range or passing null omits the recency constraint from the search request. This option does not change Serper image_search.

Serper source filters

tools:
  - name: web_search
    group: web
    use: deerflow.community.serper.tools:web_search_tool
    max_results: 5
    include_domains: [example.com, bücher.de]
    exclude_domains: [ads.example.com]

Each optional list accepts at most 10 entries (before deduplication). Omitted or empty lists impose no restriction; explicit null, non-lists, or any invalid entry return a configuration error before HTTP. Validation failures are also logged without query or configured domain values. Entries must be domain names: no surrounding whitespace, scheme, path, port, wildcard, IP literal or query operator. Names are lowercased, one trailing dot is removed, and Python's IDNA codec converts Unicode names to ASCII. DNS labels must be 1–63 characters and the normalized domain at most 253 characters, with at least two labels. Duplicates are removed after normalization.

Matching uses the exact hostname or a dot-delimited subdomain; example.com does not match notexample.com or example.com.evil.com. Exclusion wins over inclusion. With either list non-empty, malformed/non-HTTP(S) result URLs, credentials in URLs, and invalid hosts or ports are discarded. No DNS lookup or redirect resolution is performed. Unconfigured/empty-filter behavior stays unchanged, including the legacy query trimming and 500-character truncation.

The adapter appends Google query operators, for example (news) (site:example.com OR site:example.org) -site:ads.example.com, in the existing Serper q field; it sends no provider-specific domain JSON fields. After the existing query cleanup, the complete filtered query must fit 500 characters or the tool returns an error before HTTP. Operators are never truncated or dropped. Model-supplied operators can affect upstream retrieval, so the local hostname check always enforces the configured scope. The returned query remains the cleaned original query, without appended restrictions, including on provider errors.

time_range still maps to tbs, and max_results caps the filtered results. total_results reports the actual remaining count (zero with results: [] when none survive). One request is made: no refill or relaxed-filter retries. These settings do not affect image_search or the model-facing tool schema. Source selection is neither a factuality guarantee nor a global URL-access policy for fetch tools, browsers, or redirects.

Provider validation: Google documents site: and subdomain behavior and search exclusion syntax. Serper describes its Google Search API, but live Serper operator handling has not been verified here. Upstream operators are best-effort; offline mocked tests verify request composition and local filtering, not provider retrieval behavior. No paid API calls are needed for the regression suite.

Built-in Tools:

  • web_search - Search the web (DuckDuckGo, Tavily, Brave, Serper, Serply, Exa, InfoQuest, Tencent Cloud WSA, Firecrawl, fastCRW, GroundRoute, Sofya)
  • web_fetch - Fetch web pages (Jina AI, Crawl4AI, Exa, InfoQuest, Firecrawl, fastCRW, GroundRoute, Browserless, Sofya, Unbrowse)
  • web_capture - Capture rendered webpage screenshots as artifacts (Browserless)
  • image_search - Search for reference images (DuckDuckGo, InfoQuest, Serper, Brave)
  • ls - List directory contents
  • read_file - Read file contents
  • write_file - Write file contents
  • str_replace - String replacement in files
  • bash - Execute bash commands

Browserless can be configured as an opt-in visual capture tool:

tools:
  - name: web_capture
    group: web
    use: deerflow.community.browserless.tools:web_capture_tool
    base_url: http://localhost:3032
    # token: $BROWSERLESS_TOKEN
    output_format: png
    full_page: true
    viewport_width: 1280
    viewport_height: 720
    # allow_private_addresses: false  # SSRF guard; keep false in production

web_capture writes screenshots to the current thread's /mnt/user-data/outputs directory and presents the image path through the standard artifact mechanism. By default it refuses URLs that resolve to private, loopback, link-local, shared (100.64.0.0/10, used by CGNAT and Tailscale), other non-global, or cloud-metadata addresses; set allow_private_addresses: true only when you intentionally point the tool at an internal target.

Both web_fetch (Browserless provider) and web_capture need a running Browserless instance. You can point base_url at Browserless Cloud (set BROWSERLESS_TOKEN) or run one locally with Docker:

# Browserless listens on port 3000 inside the container; map it to 3032 to
# match the default base_url (http://localhost:3032). Recent Browserless
# images always require a token — if you don't pass one, a random token is
# generated and requests without it are rejected — so set it explicitly.
docker run -d --name browserless -p 3032:3000 -e "TOKEN=local-dev-token" ghcr.io/browserless/chromium

Then set the same token so the tool sends it (uncomment token: $BROWSERLESS_TOKEN in the config above):

export BROWSERLESS_TOKEN=local-dev-token

Verify the instance is reachable before enabling the tool:

curl -sS "http://localhost:3032/screenshot?token=local-dev-token" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "options": {"type": "png"}}' \
  -o /tmp/browserless-check.png  # writes a PNG on success

For Docker Compose deployments, run Browserless as a service and point base_url at the service name (e.g. http://browserless:3000) instead of localhost. See the Browserless project for full deployment and configuration options.

Reading Referenced Conversations

Enable the read-only Gateway tool through the existing tools list:

tools:
  - name: read_conversation
    group: conversation
    use: deerflow.tools.conversation:read_conversation

It is off by default. A run must explicitly submit conversation_references and have runs:read permission before the lead agent receives this tool. Custom agents must also permit the conversation tool group where they restrict groups. References are limited to owned threads and the current run; they do not enable history discovery, memory extraction or cross-user access. See the request contract and limits. Once the tool is listed, GET /api/features reports conversation_references.enabled: true and the per-run cap, so a client can show an entry point only where the tool exists; SDK clients that cannot add top-level request fields pass the list as context.conversation_references.

Reader pages are sized to stay within the tool_output budget for read_conversation (12,000 serialized characters by default), so they are not externalized to .tool-results. To allow larger pages, raise tool_output.tool_overrides.read_conversation; a page still holds at most 20,000 text characters.

Sandbox

DeerFlow supports multiple sandbox execution modes. Configure your preferred mode in config.yaml:

Local Execution (runs sandbox code directly on the host machine):

sandbox:
   use: deerflow.sandbox.local:LocalSandboxProvider # Local execution
   allow_host_bash: false # default; host bash is disabled unless explicitly re-enabled

Docker Execution (runs sandbox code in isolated Docker containers):

sandbox:
   use: deerflow.community.aio_sandbox:AioSandboxProvider # Docker-based sandbox

For AIO images on the supported semver line (1.9.3 through the recommended 1.11.0 image), sandbox.bash_command_timeout is enforced server-side through the hard_timeout API when the image exposes it. DeerFlow's legacy frozen all-in-one-sandbox:latest image predates that API, so only the host-side request is bounded there. On supported semver AIO images, list_dir uses a 60 second server-side hard timeout with a 65 second no-retry host envelope; the frozen legacy image only gets the bounded host wait. Timed-out or otherwise ambiguous commands are never replayed, and a partial list_dir result is never returned as a complete listing.

Explicit AIO shell/bash session creation is a separate control-plane operation. DeerFlow bounds those create requests to 5 seconds with SDK retries disabled. If a response cannot prove whether creation committed, DeerFlow does not replay the create or execute on that session id. The affected creation plane is quarantined, bounded best-effort session cleanup is attempted, and the container is recycled instead of being returned to the warm pool. Session-level cleanup does not clear that quarantine because a timed-out create may commit after cleanup has already returned.

BoxLite micro-VM Sandbox (runs sandbox code in daemonless OCI micro-VMs):

sandbox:
   use: deerflow.community.boxlite:BoxliteProvider
   image: python:3.12-slim
   memory_mib: 1024                 # optional per-box memory cap
   cpus: 2                          # optional per-box vCPUs
   replicas: 3                      # max active + warm VMs per gateway process
   idle_timeout: 600                # warm VM idle seconds before stop; 0 disables idle reaping
   environment:
      PYTHONUNBUFFERED: "1"

Install the optional runtime before selecting this provider:

pip install "deerflow-harness[boxlite]"

BoxLite boxes are named from the effective (user_id, thread_id) scope and are released into an in-process warm pool after each turn. The same user/thread can reclaim its warm VM on the next acquire; different threads cannot share a VM. replicas caps active plus warm VMs. When the cap is reached only warm VMs are evicted; active VMs continue and the provider may temporarily exceed the cap if all boxes are active.

Docker Execution with Kubernetes (runs sandbox code in Kubernetes pods via provisioner service):

This mode runs each sandbox in an isolated Kubernetes Pod on your host machine's cluster. Requires Docker Desktop K8s, OrbStack, or similar local K8s setup.

sandbox:
   use: deerflow.community.aio_sandbox:AioSandboxProvider
   provisioner_url: http://provisioner:8002

When using Docker development (make docker-start), DeerFlow starts the provisioner service only if this provisioner mode is configured. In local or plain Docker sandbox modes, provisioner is skipped.

Remote/provisioner backends default to explicit file synchronization because DeerFlow cannot infer whether their /mnt/user-data mount points reference the same storage as the Gateway. When the deployment guarantees that both sides use the same thread user-data directories, opt out of that extra transfer:

sandbox:
  use: deerflow.community.aio_sandbox:AioSandboxProvider
  provisioner_url: http://provisioner:8002
  thread_data_mounts: true

Leave thread_data_mounts unset to retain backend auto-detection. Set it to false to force explicit synchronization even for a local container backend. Only set it to true after verifying the Gateway's users/{user_id}/threads/{thread_id}/user-data directory and the sandbox's /mnt/user-data are the same storage; a false positive skips synchronization and makes newly uploaded files unavailable inside the sandbox.

See Provisioner Setup Guide for detailed configuration, prerequisites, and troubleshooting.

E2B Cloud Sandbox (runs sandbox code in E2B cloud micro-VMs):

sandbox:
   use: deerflow.community.e2b_sandbox:E2BSandboxProvider
   api_key: $E2B_API_KEY            # required; or set the E2B_API_KEY env var
   template: code-interpreter-v1     # e2b sandbox template id
   # domain: e2b.dev                # optional; for self-hosted e2b deployments
   home_dir: /home/user             # /mnt/user-data is remapped under this directory
   idle_timeout: 600                # forwarded to e2b's server-side set_timeout()
   replicas: 3                      # max concurrent sandboxes per gateway process
   mount_upload_deadline_seconds: 120  # per-sandbox time budget for mount uploads (seconds)
   ownership:                       # use Redis when more than one gateway shares E2B
     type: redis
     redis_url: $REDIS_URL
   reconciliation_interval_seconds: 60
   reconciliation_grace_seconds: 120
   reconciliation_orphan_ttl_seconds: 3600
   reconciliation_max_pages: 10
   reconciliation_max_items: 200
   reconciliation_max_seconds: 15
   mounts:                          # one-shot upload of host files at sandbox start
     - host_path: /path/on/host
       container_path: /home/user/shared
       read_only: false
   environment:                     # forwarded to the sandbox at create time
     OPENAI_API_KEY: $OPENAI_API_KEY

e2b-code-interpreter is bundled as a core dependency of deerflow-harness, so no extra install step is needed; just supply your API key and switch the provider in config.yaml.

Notes specific to E2BSandboxProvider:

  • Each DeerFlow thread is bound to its E2B sandbox via metadata (deer_flow_user, deer_flow_thread, deer_flow_skills_root). Startup and periodic reconciliation probe every bounded candidate, adopt one healthy canonical sandbox, and reap duplicates after a grace period. A sandbox whose skills root differs from the provider's startup snapshot is never adopted and is reaped after the same grace period once no live peer owns it. Provider-tagged entries without a complete user/thread identity are reaped only after the orphan TTL.
  • Ownership leases prevent one gateway from adopting or destroying a sandbox another live gateway is responsible for. The default in-memory store is safe only for one gateway process. Multi-worker/load-balanced deployments must use sandbox.ownership.type: redis; an existing Redis stream bridge configuration is inferred automatically.
  • Reconciliation is bounded by page, item, and wall-clock limits. Its summary log exposes discovered, adopted, duplicate, deferred, killed, dead, and budget-exhausted counts for operational monitoring.
  • Idle expiry is enforced server-side by e2b's set_timeout(). The provider refreshes the timeout on every release so warm sandboxes stay alive long enough for the next acquire.
  • mounts are uploaded once when the sandbox starts; e2b cannot host bind-mount the gateway filesystem, so changes inside the sandbox are not reflected back on disk automatically. Use the download_file tool or write outputs under /mnt/user-data/outputs/ (which is mapped to home_dir/outputs/ inside the sandbox and surfaced through the standard artifact pipeline) to ship files back to the gateway.
  • mount_upload_deadline_seconds sets the per-sandbox time budget for mount uploads. The provider checks it before each mount, during directory preflight, and before each SDK write. The deadline does not interrupt active filesystem or E2B SDK calls. Omitting the key preserves the 120-second default. Values below 1 are clamped to 1; non-numeric or null values fall back to the default.

OpenSandbox Remote Sandbox (runs code through an OpenSandbox deployment):

sandbox:
   use: deerflow.community.opensandbox:OpenSandboxProvider
   image: python:3.11
   api_key: $OPEN_SANDBOX_API_KEY     # optional when the SDK env var is set
   domain: localhost:8080             # OPEN_SANDBOX_DOMAIN fallback
   protocol: http
   request_timeout: 30                # management request timeout seconds
   ready_timeout: 30                  # create/readiness timeout seconds
   use_server_proxy: false            # proxy execd/file traffic through server
   sandbox_timeout: 14400             # remote lifetime; 0 = explicit cleanup
   bash_command_timeout: 600          # default remote command timeout seconds
   replicas: 3                        # active + warm cap per gateway process
   idle_timeout: 600                  # warm seconds before destroy; 0 disables
   environment:
      PYTHONUNBUFFERED: "1"

Install the optional SDK before selecting this provider:

pip install "deerflow-harness[opensandbox]"

The provider creates a sandbox per effective user/thread scope and parks it in an in-process warm pool after each turn. The same scope can reclaim it after a health check; another user or thread cannot. Create-time readiness and /mnt/user-data/{workspace,uploads,outputs} bootstrap failures are cleaned up before acquire() returns. Each remote owns an independent SDK transport. Operations renew the configured server-side lifetime, and commands without an explicit timeout use bash_command_timeout; a longer explicit timeout extends the renewal horizon to cover the command. Operations on one remote are serialized so a shorter renewal cannot overwrite an in-flight command's horizon. File transfer uses OpenSandbox's native filesystem API; bounded find/grep commands implement the directory and content-search surface. Downloads are restricted to /mnt/user-data and all file paths reject traversal. Multi-process discovery and ownership coordination are not yet implemented, so replicas is a per-Gateway-process soft cap.

Choose between local execution or Docker-based isolation:

Option 1: Local Sandbox (default, simpler setup):

sandbox:
  use: deerflow.sandbox.local:LocalSandboxProvider
  allow_host_bash: false

allow_host_bash is intentionally false by default. DeerFlow's local sandbox is a host-side convenience mode, not a secure shell isolation boundary. If you need bash, prefer AioSandboxProvider. Only set allow_host_bash: true for fully trusted single-user local workflows.

When LocalSandboxProvider runs under make up, it runs inside the deer-flow-gateway container. In that mode, sandbox.mounts[].host_path is resolved from the gateway container's filesystem, not from your Docker host. If you need a local-sandbox custom mount in production Docker, bind the host directory into the gateway service first, then use the in-container path in config.yaml:

# docker/docker-compose.yaml or an override file
services:
  gateway:
    volumes:
      - ${DEER_FLOW_REPO_ROOT}/.deer-flow/knowledge:/app/.deer-flow/knowledge:ro
sandbox:
  use: deerflow.sandbox.local:LocalSandboxProvider
  mounts:
    - host_path: /app/.deer-flow/knowledge
      container_path: /mnt/knowledge
      read_only: true

If the configured host_path is not visible to the gateway process, DeerFlow logs an error and ignores that mount.

Option 2: Docker Sandbox (isolated, more secure):

sandbox:
  use: deerflow.community.aio_sandbox:AioSandboxProvider
  port: 8080
  container_prefix: deer-flow-sandbox

  # Optional: Additional mounts
  mounts:
    - host_path: /path/on/host
      container_path: /path/in/container
      read_only: false

When you configure sandbox.mounts, DeerFlow exposes those container_path values in the agent prompt so the agent can discover and operate on mounted directories directly instead of assuming everything must live under /mnt/user-data.

Sandbox network policy

Local Docker AIO sandboxes can opt into an outbound policy:

sandbox:
  use: deerflow.community.aio_sandbox:AioSandboxProvider
  network:
    mode: allowlist
    allow_domains:
      - pypi.org
      - files.pythonhosted.org
      - registry.npmjs.org
    approval: prompt
    temporary_grant_ttl: 300

open (the compatibility default) keeps normal Docker egress. isolated places each sandbox on a per-sandbox internal bridge and denies all outbound traffic. allowlist uses the same bridge and a trusted sidecar that supports HTTP and HTTPS CONNECT only. Exact domains and leading wildcards such as *.pythonhosted.org are accepted; URLs, ports, and a catch-all * are rejected. Traffic that ignores proxy environment variables still has no route out of the internal bridge. DeerFlow also sets the upstream AIO image's PROXY_SERVER/PROXY_EXCLUDE variables so its Chromium service uses the same policy sidecar; standard upper/lower-case HTTP, HTTPS, and ALL proxy variables cover shell and package-manager clients.

The sidecar is dual-homed between the sandbox's internal bridge and a separate per-sandbox egress bridge with inter-container communication disabled. It is never attached to Docker's shared default bridge, so unrelated containers cannot reach its container address directly. Its published sandbox-API relay also requires a cryptographically random per-sandbox token, so containers on other bridge networks cannot use Docker's host-port mapping to bypass that separation. Only the sidecar is attached to the egress bridge; outbound traffic still goes through Docker NAT.

Plain HTTP connections carry exactly one fully framed, policy-checked request; the sidecar closes the upstream connection afterward so a pipelined request cannot reuse the first request's decision. HTTP field names are parsed once, must use the RFC token grammar with the colon immediately following the name, and malformed or ambiguous fields are rejected before any policy lookup or forwarding. HTTPS CONNECT validates both the CONNECT authority and the TLS ClientHello SNI without intercepting TLS. Because the encrypted HTTP Host/:authority remains invisible, a deliberately malicious client may still reach another virtual host co-located on an allowed endpoint if that server accepts a mismatched inner authority. Deployments that require strict origin-level HTTPS isolation should use isolated mode or an operator-managed TLS-inspecting egress gateway.

With approval: prompt, a denied public domain becomes a Human Input card with Deny, Allow temporarily, and Allow for this sandbox choices. DeerFlow does not replay the failed command after approval because it may already have performed local side effects; the agent must retry it explicitly. Non-interactive runs auto-deny without opening a card or waiting for input. The sidecar rejects hostnames that policy does not allow before DNS resolution; allowed hostnames that resolve to loopback, private, link-local, multicast, IPv6 ULA/site-local, or cloud metadata destinations are rejected and can never be approved. Raw TCP/UDP, Git-over-SSH, and other non-HTTP protocols remain unavailable in restricted modes.

Restricted modes currently require the local Docker backend and Docker Engine 28 or newer. They fail closed on Apple Container, provisioner mode, and older engines. DeerFlow applies Engine 28's isolated bridge gateway mode to both IPv4 and IPv6 so the sandbox cannot reach services bound to either host-side bridge address. The sandbox, sidecar, internal network, and egress network carry a digest of the effective policy, proxy source, and image reference; startup and reconciliation destroy and recreate a persisted resource set when that identity or its required network properties no longer match. Docker sandboxes in every mode also carry stable identity and mode labels. After a Gateway restart, changing between open and a restricted mode is therefore reported as an incompatible persisted sandbox and replaced only after the normal ownership, orphan-grace, and teardown fences. Unlabelled open containers from older DeerFlow versions are recognized when they use the configured image and retain their published API port. Docker Desktop is detected from the daemon, not the Gateway process, so Docker-outside-of-Docker deployments handle its synthetic DNS range correctly. The policy sidecar publishes only its fixed sandbox-API relay back to the Gateway; the sandbox API itself is not published. DeerFlow generates a separate relay token for each sandbox, requires it on every new relay connection, reconstructs it from Docker during discovery, and injects it only into Gateway control-plane clients. The token is excluded from SandboxInfo serialization, representations, and command logs. Mirror or digest-pin network.proxy_image in production environments that require supply-chain pinning.

Sandbox container network exposure and hardening

The sandbox HTTP API (/v1/shell/* and friends) has no authentication: anyone who can reach a published sandbox port can execute arbitrary commands in that sandbox. For bare-metal Docker sandbox runs that use localhost, DeerFlow binds the sandbox port to 127.0.0.1 so it is not exposed on other host interfaces. For Docker-outside-of-Docker deployments that connect through host.docker.internal, the port is bound to the address that hostname actually resolves to — the daemon's host-gateway-ip mapping (customizable, possibly IPv6) — so the published port and the address the gateway connects to always match, and the port is no longer published on external network interfaces (previously it was bound to 0.0.0.0). On Docker Desktop, resolving host.docker.internal yields an internal VM gateway address that the host OS cannot bind; because Docker Desktop forwards host.docker.internal to host loopback, DeerFlow defaults to 127.0.0.1 for host.docker.internal on Desktop daemons. Custom non-loopback sandbox hosts continue to bind their resolved address. If resolution fails, the Docker default bridge gateway (via docker network inspect bridge, falling back to 172.17.0.1) is used as a best-effort bind and a warning is logged. Set DEER_FLOW_SANDBOX_BIND_HOST explicitly if your deployment needs a different bind address; setting it to 0.0.0.0 restores the legacy broad bind, which re-exposes the unauthenticated exec API on every interface and should be paired with an external firewall.

Local Docker sandbox containers are also hardened by default: all Linux capabilities are dropped (--cap-drop=ALL) except a five-capability compatibility allowlist — CHOWN, FOWNER, SETUID, SETGID, and DAC_OVERRIDE — while privilege escalation across exec stays blocked with no-new-privileges and CPU/memory/PID resources are bounded. CHOWN/SETUID/SETGID support the runtime user handoff and DAC_OVERRIDE supports the root nginx master's writes to gem-owned logs. FOWNER is specifically required by the newer AIO 1.11.x startup path (regression-tested against the recommended 1.11.0 image), which runs chmod /run/user/1000 after capabilities are dropped. Images that do not perform that chmod do not need FOWNER; DeerFlow deliberately does not guess a smaller set from mutable tags, digests, or arbitrary custom images, so the default compatibility allowlist remains version-agnostic.

A custom image that is already fully initialized as a non-root user and needs none of those compatibility capabilities should set DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS=0 to drop the whole set. This is an all-or-nothing opt-out, not a per-capability selector: an older or custom root-initialized image that does not need FOWNER may still require CHOWN, SETUID, SETGID, or DAC_OVERRIDE and should therefore leave the compatibility set enabled. Retained capabilities remain available for the container's lifetime and can let sandboxed code change ownership or mode on accessible bind-mounted paths, impersonate mounted-file UIDs/GIDs, or bypass discretionary access checks. no-new-privileges does not mitigate that existing-capability risk — it only blocks gaining new privileges across exec. One hardening knob is relaxed by default: the shipped AIO image runs with seccomp=unconfined because its Chromium browser does not start under Docker's default seccomp profile (syscall filtering is disabled — see the two seccomp variables below to change that). The following environment variables (set them in the gateway process, e.g. via .env loaded by docker-compose, or the gateway service environment:) tune or disable each knob:

Environment variable Default Purpose
DEER_FLOW_SANDBOX_BIND_HOST loopback (localhost or Docker Desktop with host.docker.internal) / host-gateway-ip / bridge gateway Host interface for the sandbox -p publish. Must be an IP literal (bare or bracketed IPv6) or a hostname, which is resolved to an address first — Docker publish specs do not accept hostnames. 0.0.0.0 restores the legacy broad bind (risky).
DEER_FLOW_SANDBOX_SECCOMP_UNCONFINED on The shipped AIO image's Chromium browser does not start under Docker's default seccomp profile (see the upstream agent-infra sandbox FAQ), so seccomp=unconfined remains the default. Set to 0 to run with the built-in profile — passed explicitly as seccomp=builtin, so a daemon configured with a different default cannot weaken the opt-out — and only for images verified to start and pass browser checks with it.
DEER_FLOW_SANDBOX_IMAGE_STARTUP_CAPS on Keeps the five-capability compatibility set (CHOWN/FOWNER/SETUID/SETGID/DAC_OVERRIDE). FOWNER specifically covers the newer AIO 1.11.x startup chmod /run/user/1000 path (tested with 1.11.0); images without that step do not need FOWNER, but DeerFlow does not infer per-image capability subsets from tags/digests/custom images. Set to 0 only for images that need none of the five — the switch drops the entire set.
DEER_FLOW_SANDBOX_SECCOMP_PROFILE unset Path to a custom seccomp profile (e.g. a restricted, Chromium-compatible one built from Docker's default plus the namespace syscalls Chromium needs). Takes precedence over the unconfined default.
DEER_FLOW_SANDBOX_MEMORY 2g --memory limit per sandbox container. 0/none disables the limit.
DEER_FLOW_SANDBOX_CPUS 2 --cpus limit per sandbox container. 0/none disables the limit.
DEER_FLOW_SANDBOX_PIDS_LIMIT 512 --pids-limit per sandbox container (fork-bomb guard). 0/none disables the limit.
DEER_FLOW_SANDBOX_CONTAINER_USER unset (image default) Passed through as --user (e.g. 1000:1000). The default AIO image's user is upstream-controlled, so DeerFlow does not force one; set this only if you know your image's runtime user.
DEER_FLOW_SANDBOX_NETWORK unset (daemon default network) Legacy open-mode escape hatch passed through as --network. Prefer sandbox.network for managed isolation. host, container:<name>, and none are rejected at startup. Restricted modes ignore this variable and use their own per-sandbox internal network.

These hardening flags are Docker-only; Apple Container (container runtime) keeps its previous, unhardened invocation and therefore supports only network.mode: open. On macOS, an open Gateway normally prefers Apple Container, but it keeps using Docker while the configured sandbox prefix has managed Docker sandboxes so startup reconciliation can safely replace resources left by a restricted-mode deployment before the runtime changes.

Sandbox control-plane HTTP calls to loopback/private IPs, single-label cluster hosts, and Docker/Podman internal hostnames bypass HTTP_PROXY/HTTPS_PROXY inside the client. This prevents an inherited proxy from returning a misleading 502 for a healthy local sandbox. Externally hosted sandbox FQDNs and public IPs continue to use the normal environment proxy configuration.

AIO shell-session capacity

Each concurrently running native subagent uses one persistent AIO shell session. The semver AIO images from 1.9.3 through 1.11.0 default MAX_SHELL_SESSIONS to 10 and evict the oldest idle session when an eleventh is created. If subagent_runtime.max_running is greater than nine, DeerFlow sets the container limit to max_running + 1; the extra slot leaves room for the lead agent's shell. This applies to both locally created containers and provisioner Pods. Lower concurrency keeps the image's own default unchanged.

The separate bash.exec API uses its own AIO_BASH_MAX_SESSIONS pool rather than MAX_SHELL_SESSIONS. DeerFlow nevertheless creates and closes an explicit transient bash session around every env-bearing command, so request-scoped secrets and completed command sessions are not retained.

You may set sandbox.environment.MAX_SHELL_SESSIONS explicitly. It must be a positive integer at least as large as subagent_runtime.max_running + 1, or the provider fails at startup with the conflicting values. The setting is applied when a sandbox is created. Persisted local containers and provisioner Pods report their effective value; DeerFlow replaces one whose capacity is below the current requirement instead of reusing it. Reuse checks also apply when no explicit override is needed for new containers: a previously configured lower limit must still fit the current concurrency. The Gateway waits for existing ownership and the orphan recovery grace before replacing an incompatible sandbox. A create request that encounters a lower-capacity Pod returns HTTP 409 without deleting it; a later acquisition can discover and replace it through that same ownership check.

If a Service survives deletion of its old Pod, a later create repairs the missing Pod. A failed capacity read other than a Pod-not-found response remains an error and does not authorize replacement.

Building a Custom AIO Sandbox Image

AioSandboxProvider talks to the sandbox container through the agent-sandbox SDK. The Dockerfile for the default enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest image is not part of this repository; DeerFlow treats that image as an upstream AIO sandbox runtime.

For persistent system or language dependencies, extend the published image and keep its startup command intact:

FROM enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest

USER root
# Example user dependency; not required by DeerFlow itself.
RUN apt-get update \
    && apt-get install -y --no-install-recommends graphviz \
    && rm -rf /var/lib/apt/lists/*

# Example Python dependency for work done inside the sandbox.
RUN python -m pip install --no-cache-dir pandas

# Do not override ENTRYPOINT or CMD; keep the upstream sandbox server startup.

Use the custom image in local Docker or Apple Container mode with sandbox.image:

sandbox:
  use: deerflow.community.aio_sandbox:AioSandboxProvider
  image: your-registry/your-aio-sandbox:tag

In provisioner mode, sandbox Pods are created by the provisioner service, so configure the provisioner SANDBOX_IMAGE environment variable instead of sandbox.image. See the Provisioner Setup Guide.

If you rebuild the runtime from scratch instead of extending the published image, it must expose the same HTTP API used by agent-sandbox. DeerFlow currently depends on:

  • sandbox.get_context(), including home_dir
  • shell.exec_command(...)
  • bash.exec(...) — only exercised for per-command environment injection (skills that declare required-secrets). The /v1/bash/* routes exist since upstream all-in-one-sandbox 1.9.3; on older images (including a latest tag still frozen on the 1.0.0.x line) DeerFlow fails fast with an actionable error instead of surfacing the raw 404. Pin sandbox.image to 1.9.3 or newer (e.g. 1.11.0) and recreate the sandbox container to use required-secrets with the AIO sandbox.
  • file.read_file(...)
  • file.write_file(...), including base64 writes for binary content
  • streamed file.download_file(...)
  • file.find_files(...)
  • file.list_path(...)
  • file.search_in_file(...)

Custom images must also keep these compatibility constraints:

  • The container should listen on the configured sandbox port, 8080 by default.
  • /mnt/user-data must remain writable because DeerFlow mounts thread workspace, uploads, and outputs there.
  • home_dir comes from the sandbox context endpoint; do not assume DeerFlow hardcodes it.
  • Shell command handling must remain compatible with serialized exec_command calls. DeerFlow serializes shell access on the host side to avoid corrupting the sandbox's persistent shell session.

Skills

Configure the skills directory for specialized workflows:

skills:
  # Host path (optional, default: ../skills)
  path: /custom/path/to/skills

  # Container mount path (default: /mnt/skills)
  container_path: /mnt/skills

For the AIO provider (including the Kubernetes provisioner) and E2B, skills.container_path is captured when the provider starts and must be one canonical absolute, non-root POSIX path. Do not use redundant separators, ./.., or a path that contains or sits below DeerFlow's reserved mounts (/mnt/user-data, /mnt/acp-workspace, or /mnt/integrations/lark-cli). Restart the Gateway after changing it so sandbox identities and mounts use the same root. E2B also records the root in remote metadata and refuses to adopt a VM created for another root.

How Skills Work:

  • Skills are stored in deer-flow/skills/{public,custom}/
  • Each skill has a SKILL.md file with metadata
  • Skills are automatically discovered and loaded
  • Available in both local and Docker sandbox via path mapping

Skill installs and agent-managed skill writes also run through native deterministic SkillScan before the LLM scanner:

skill_scan:
  enabled: true

Set skill_scan.enabled: false to disable only the deterministic analyzers. Safe archive extraction and the LLM-based skill scanner still run.

Per-Agent Skill Filtering: Custom agents can restrict which skills they discover and activate by defining a skills field in their config.yaml (located at workspace/agents/<agent_name>/config.yaml):

  • Omitted or null: Makes all globally enabled skills available (default fallback).
  • [] (empty list): Disables all skills for this specific agent.
  • ["skill-name"]: Makes only the explicitly specified skills available.

This field is a discovery and activation allowlist; it does not activate every listed skill's allowed-tools policy when the agent is constructed. Use tool_groups to define the agent's baseline tools. A listed skill's policy applies only after slash activation or an actual SKILL.md load.

The same semantics apply to subagents.agents.<name>.skills and subagents.custom_agents.<name>.skills: omitted or null exposes all enabled skills, [] exposes none, and a list limits discovery and activation. A passive subagent skill never removes baseline tools; its allowed-tools declaration becomes active only after slash activation or a completed SKILL.md read.

LocalSandboxProvider enforces this filesystem view through its managed virtual path mappings only. Explicit per-Agent skill policies therefore fail closed when sandbox.allow_host_bash is enabled, because host subprocesses can bypass those mappings. Keep host bash disabled (the default), or use AIO/provisioner/E2B when shell access and filesystem isolation are both required.

Title Generation

Automatic conversation title generation:

title:
  enabled: true
  max_words: 6
  max_chars: 60
  model_name: null  # null = fast local fallback; set a model name to use LLM title generation

GitHub API Token (Optional for GitHub Deep Research Skill)

The default GitHub API rate limits are quite restrictive. For frequent project research, we recommend configuring a personal access token (PAT) with read-only permissions.

Configuration Steps:

  1. Uncomment the GITHUB_TOKEN line in the .env file and add your personal access token
  2. Restart the DeerFlow service to apply changes

Environment Variables

DeerFlow supports environment variable substitution using the $ prefix:

models:
  - api_key: $OPENAI_API_KEY  # Reads from environment

Common Environment Variables:

  • OPENAI_API_KEY - OpenAI API key
  • ANTHROPIC_API_KEY - Anthropic API key
  • DEEPSEEK_API_KEY - DeepSeek API key
  • MIMO_API_KEY - Xiaomi MiMo API key
  • NOVITA_API_KEY - Novita API key (OpenAI-compatible endpoint)
  • TAVILY_API_KEY - Tavily search API key
  • BRAVE_SEARCH_API_KEY - Brave Search API key for web_search and image_search
  • SERPER_API_KEY - Serper (Google Search/Images API) key for web_search and image_search
  • SERPLY_API_KEY - Serply key for web_search (Google Search, plus Google News and Google Scholar via vertical)
  • GROUNDROUTE_API_KEY - GroundRoute meta-search API key for web_search and web_fetch (routes across Serper, Brave, Exa, Tavily, Firecrawl, Perplexity with gain-share pricing)
  • SOFYA_API_KEY - Sofya key for web_search and web_fetch
  • UNBROWSE_API_KEY - Unbrowse key for web_fetch
  • BROWSERLESS_TOKEN - Browserless Cloud token for web_capture (optional for self-hosted Browserless)
  • DEER_FLOW_PROJECT_ROOT - Project root for relative runtime paths
  • DEER_FLOW_CONFIG_PATH - Custom config file path
  • DEER_FLOW_EXTENSIONS_CONFIG_PATH - Custom extensions config file path
  • DEER_FLOW_HOME - Runtime state directory (defaults to .deer-flow under the project root)
  • DEER_FLOW_SKILLS_PATH - Skills directory when skills.path is omitted
  • GATEWAY_ENABLE_DOCS - Set to false to disable Swagger UI (/docs), ReDoc (/redoc), and OpenAPI schema (/openapi.json) endpoints (default: true)

Backend dotenv selection

Set DEER_FLOW_ENV_FILE in the backend process environment before startup to load one explicit UTF-8 dotenv file instead of default dotenv discovery. Use an absolute path for launches from unrelated directories. Relative paths are resolved against the backend process working directory, not the YAML file, project root or this documentation's directory. No automatic ENV profile naming or config-relative dotenv lookup is added.

Existing process variables, including empty values, take precedence. An unset selector keeps the existing default lookup; a set but empty selector, missing file, directory or unreadable file raises an actionable startup error without printing file contents. An empty file is valid and loads no defaults. Explicit selection also raises when PYTHON_DOTENV_DISABLED is 1, true, t, yes or y (case-insensitive), even for an empty file. Unset either option to resolve the conflict. Without a selector, python-dotenv's normal disable behavior is unchanged. Restart after changing the selector or file contents.

DEER_FLOW_CONFIG_PATH continues to select YAML independently. For example, from backend/:

DEER_FLOW_ENV_FILE=/srv/deer-flow/stage.env DEER_FLOW_CONFIG_PATH=/srv/deer-flow/stage.yaml make gateway

This option covers backend Python startup (including auth and debug.py). It does not change shell launcher, Docker Compose or frontend dotenv handling; values already injected by those layers remain process variables and win. For containers, explicitly pass the selector and mount the selected file at a container-visible path. Database, runtime-home, storage and tenant isolation must be configured separately; selecting a dotenv file does not provide them.

Configuration Location

The configuration file should be placed in the project root directory (deer-flow/config.yaml). Set DEER_FLOW_PROJECT_ROOT when the process may start from another working directory, or set DEER_FLOW_CONFIG_PATH to point at a specific file.

Configuration Priority

DeerFlow searches for configuration in this order:

  1. Path specified in code via config_path argument
  2. Path from DEER_FLOW_CONFIG_PATH environment variable
  3. config.yaml under DEER_FLOW_PROJECT_ROOT, or under the current working directory when DEER_FLOW_PROJECT_ROOT is unset
  4. Legacy backend/repository-root locations for monorepo compatibility

Security Notes

Sandbox Isolation and the Docker Socket (DooD)

DeerFlow executes agent-generated shell/code through a configurable sandbox (sandbox.use in config.yaml). The isolation guarantees differ by mode, and one mode requires mounting the host Docker socket. Understand the trade-offs before exposing an instance to untrusted input.

Mode config.yaml Host Docker socket Isolation
local (default) deerflow.sandbox.local:LocalSandboxProvider Not mounted Commands run inside the gateway container on its filesystem. Not a strong boundary — allow_host_bash is false by default and should stay off for untrusted workloads.
aio (pure DooD) deerflow.community.aio_sandbox:AioSandboxProvider (no provisioner_url) Mounted (opt-in overlay) Sandbox containers are started via the host Docker daemon.
provisioner (Kubernetes) AioSandboxProvider + provisioner_url Not mounted Sandbox pods are created through the provisioner's K8s API over HTTP. Strongest isolation.

The Docker socket is host root

Mounting /var/run/docker.sock into a container grants that container root-equivalent control of the host: anything able to reach the socket can start a new container that bind-mounts the host filesystem and escape. This matters for DeerFlow because the gateway executes model-generated commands, so a prompt injection or any in-container code-execution primitive could pivot to the host through the socket.

To keep this off the default attack surface:

  • The host Docker socket is not mounted by the default Compose stack. It is added only for aio mode through the opt-in docker/docker-compose.dood.yaml overlay, which scripts/deploy.sh and scripts/docker.sh append automatically when detect_sandbox_mode() returns aio.
  • Prefer provisioner/Kubernetes mode for multi-tenant or internet-exposed deployments — it isolates sandboxes without handing the gateway the host daemon.
  • If you must use aio/DooD, treat the host as part of the gateway's trust boundary: run it on a dedicated host, and consider a scoped Docker API proxy instead of the raw socket.

Note: the gateway bind-mounts $HOME/.claude and $HOME/.codex (read-only) for CLI auto-auth in all modes. These hold long-lived CLI credentials; scope or omit them when the gateway runs untrusted workloads.

CLI Credential Mounts (Claude Code / Codex / MiniMax Code)

DeerFlow can reuse your Claude Code / Codex CLI subscription login as a model provider (ClaudeChatModel, the Codex provider) or for ACP agents that run the CLI in-container. The Compose stack used to bind-mount the entire ~/.claude and ~/.codex directories (read-only) into the gateway container in every configuration — exposing not just credentials but full conversation history, per-project session data, and global CLI config. A gateway compromise (prompt injection, tool/MCP misuse, RCE) would leak all of it.

These directories are no longer mounted by default. Supply CLI credentials with the least exposure that fits your setup:

Need How Exposure
Claude model provider env CLAUDE_CODE_OAUTH_TOKEN / ANTHROPIC_AUTH_TOKEN (via .env), or CLAUDE_CODE_CREDENTIALS_PATH → a single mounted .credentials.json none / one file
Codex model provider env CODEX_AUTH_PATH pointing at a single mounted auth.json one file
ACP agent the adapter's own auth — many ACP adapters take an env API key (e.g. ANTHROPIC_API_KEY / OPENAI_API_KEY) and need no mount; use the opt-in docker/docker-compose.cli-auth.yaml overlay only if your adapter reads the full CLI config dir none / full dir

The Gateway credential loader checks environment variables before the default credential files, so the env-token paths need no bind mount at all. ACP adapters authenticate independently of DeerFlow via their own documented env — for example the common claude-code-acp adapter starts as ANTHROPIC_API_KEY=… claude-code-acp and honors CLAUDE_CONFIG_DIR to redirect its config directory, so it needs no ~/.claude mount at all. Prefer the adapter's documented env auth, and reach for the docker-compose.cli-auth.yaml overlay only as a fallback for an adapter that genuinely reads the full CLI config directory.

MiniMax Code is a native ACP agent, so it does not need an adapter. For local Gateway runs, install it with npm install --global @minimax-ai/code, run mcode login, and configure acp_agents.mcode with command: mcode and args: ["acp"]. The executable and its authenticated runtime must be available inside the Gateway environment; a host-only installation is not visible to a Docker container. DeerFlow forwards enabled MCP servers to the MCode session. Leave auto_approve_permissions disabled for untrusted tasks, and enable it only when the agent is expected to edit files or run commands for a trusted task.

Best Practices

  1. Place config.yaml in project root - Set DEER_FLOW_PROJECT_ROOT if the runtime starts elsewhere
  2. Never commit config.yaml - It's already in .gitignore
  3. Use environment variables for secrets - Don't hardcode API keys
  4. Keep config.example.yaml updated - Document all new options
  5. Test configuration changes locally - Before deploying
  6. Use Docker sandbox for production - Better isolation and security

Troubleshooting

"Config file not found"

  • Ensure config.yaml exists in the project root directory (deer-flow/config.yaml)
  • If the runtime starts outside the project root, set DEER_FLOW_PROJECT_ROOT
  • Alternatively, set DEER_FLOW_CONFIG_PATH environment variable to custom location

"Invalid API key"

  • Verify environment variables are set correctly
  • Check that $ prefix is used for env var references

"Skills not loading"

  • Check that deer-flow/skills/ directory exists
  • Verify skills have valid SKILL.md files
  • Check skills.path or DEER_FLOW_SKILLS_PATH if using a custom path

"Docker sandbox fails to start"

  • Ensure Docker is running
  • Check port 8080 (or configured port) is available
  • Verify Docker image is accessible

Examples

See config.example.yaml for complete examples of all configuration options.