Totoro 35cdcab1c5
feat(scheduled-tasks): signals, honest notices and fair queueing for server-started runs (RFC #6340, 2/2) (#6393)
* feat(scheduler): lifecycle rules, goal/limit parity and coded errors for scheduled tasks

- Migration 0031: nullable scheduled_tasks.unmet_streak_after_seq (internal
  streak boundary) and stop_condition (the user's stop rule, own column).
- Stop rule is normalized (stop_rule.py) and appended only at launch; the
  stored prompt never contains it. Wording adapts when scheduler.tool_enabled
  is off.
- Willem 1: page-created tasks get goal evaluation, <standing_notes>, the
  previous-run reference and own-schedule stop; request_stop and the
  scheduled capability no longer require an origin conversation.
- Repository: single reactivation choke point raising
  ScheduledTaskLimitsExhausted; streak boundary moves on goal/prompt/stop
  condition/note changes (not on resume); goal-check failures neither count
  nor reset the streak; pause of a finished task returns "finished"; trials
  on a paused task keep the host pause marker.
- New repository reads: automatic_runs_used_for, active_run_status_for,
  list_manageable_from_thread, thread_relation/thread_run on thread tasks,
  task_ids_for_thread, run_number, and run_number/total_tokens/summary on
  run history rows.
- REST: goal/max_runs/end_at/stop_condition on create and PATCH (null
  clears, naive end_at = task timezone), resume recomputes from now with an
  optional renewal body, once_time_passed, task_finished, limits_exhausted,
  scheduler_not_running on create, trigger returns outcome/existing/thread_id,
  task responses add automatic_runs_used and active_run_status.
- Coded errors {code, message, params} for routes, shared validation and the
  chat capability (scheduled_task_errors.py), pinned by
  contracts/scheduled_task_errors_contract.json; goal notes contract v2 with
  check_failure_codes, scheduled_origin_key and host_run_errors (host_notes.py).
- /api/features adds scheduled_tasks {available, running, tool_enabled,
  min_interval_seconds}; ScheduledTaskService.is_running.
- Docs: six README scheduler sections, API.md scheduled tasks section
  (breaking error shape), CONFIGURATION.md, AGENTS notes, CHANGELOGs.
- Tests: migration 0031, stop rule, error contract, run listing, lifecycle
  over HTTP, repository, service and capability cases; updated tests for the
  new error shape, finished-task pause and head revision.

* feat(scheduler): conversation update/resume, run-conversation authority and readable launches

- schedule_task: update and resume actions, stop_condition and clear_fields
  parameters, plain parameter types (value rules come back as coded results),
  rewritten docstrings for schedule_task and stop_scheduled_task.
- Capability: update/resume share the REST validation and error codes;
  interactive turns in a task's run conversation may manage that task;
  scheduled runs keep only own-stop; grants require a running scheduler.
- Results: compact JSON-safe task views with local times, display card/text,
  last run in list, timezone_source/browser_timezone, now_local; no ISO, user,
  lease or origin fields; stop_instructions removed.
- Timezone: explicit > browser (context.client_timezone, validated in
  start_run and never forwarded to the run config) > not needed for intervals
  and offset run_at > timezone_required.
- Trial gate accepts natural direct requests after acknowledgements and
  particles; trial results report thread_id and existing.
- Scheduled launches: stable message id scheduled-<task_run_id>, server-owned
  deerflow_scheduled_origin metadata with the user-language parts, run thread
  title for fresh-thread tasks.
- Worker records goal continuations in the verdict; lead prompt language rule
  covers stored task text and scheduled runs.
- Docs: six READMEs, CONFIGURATION, API, runtime/tools AGENTS, CHANGELOGs.

* feat(frontend): scheduled-task core types, formatting, errors and glossary

- throwGatewayApiError now throws GatewayApiError (status, code, params,
  rawMessage); message stays the string detail or detail.message
- scheduled-task types gain stop_condition, automatic_runs_used,
  active_run_status, origin/notes, run_number/total_tokens/summary, thread
  relation, trigger result, renewal and scheduled origin; isTaskBusy and
  hasActiveRun helpers
- api: fetchScheduledTask, resume with optional renewal body, typed trigger
  result and update payload with null clears
- hooks: polling at 3 s while a run is active (list, thread list, run
  history), useScheduledTask with live gating and final 404, useInView,
  localized error toasts; run history refreshes tasks when a run finishes
- new pure modules: errors.ts (contract code to localized copy), format.ts
  (relative task times, viewer time, display zone), status.ts, actions.ts,
  run-error.ts; describeTaskSchedule in cron.ts; goal-outcome adds unchecked
  goal checks, describeTaskOutcome and parseStopRunId
- features: scheduled_tasks block with defaults when missing
- buildRunContext sends client_timezone
- titleOfThread takes a localized untitled label; breadcrumb names scheduled
  tasks and agents; reference chips use the localized untitled label
- schedule input: shared browserTimeZone, labelled selects, aria-pressed
  type toggles, interval minimum from the hint template
- i18n: all PR1 scheduled-task keys in en-US and zh-CN, glossary fixes,
  recipe instructions moved into i18n
- e2e mocks follow the new task, trigger, resume, thread-task and features
  shapes; expectNoRawIdentifiers helpers for e2e and DOM tests
- tests for errors, format, cron, status, actions, goal outcome, run errors,
  hooks, run context, thread titles and features; scheduled e2e copy updated

* feat(frontend): list-first scheduled tasks page with outcomes and editable goals

- page.tsx is now a thin composition of components/workspace/scheduled-tasks:
  status tabs (All/Active/Paused/Finished) and search over the task list,
  ?task_id= deep link with router.replace on selection, ?thread_id= chip
  with Show all, empty state, scheduler-off notice and unavailable state
- task list rows show the schedule in words and one status line (next run,
  running now, paused by agent, auto-paused, finished by its cap)
- task detail: status badge, actions from availableActions with reasons,
  More actions menu (Duplicate, Copy task ID, Delete), outcome notice,
  Runs / Stops when / Each run's goal / Does / Notes from chat / History,
  origin chat link; success toasts for every action
- run rows: run number or Trial run, summary, goal and assumption badges,
  continuations, tokens, Open chat link; raw errors and the evaluator
  reason only behind Details
- create/edit/duplicate dialog with goal, stop condition, safety caps and
  advanced agent/context options; PATCH sends only changed fields, null
  clears a cap
- renew dialog raises or removes caps and resumes in one request; a
  limits_exhausted Resume opens it
- removed i18n keys the old page used; added feedback.resumedNoTime
- recorded en-3 and zh-1 page fixtures; scheduled e2e specs rewritten for
  the new page; DOM tests for the detail and the form dialog
- CHANGELOG entries and the frontend AGENTS.md scheduled-task note

* feat(frontend): live schedule card, run conversation provenance and header entry

- schedule_task results (create, update, pause, resume, trial, delete) become
  an assistant:scheduled-task message group rendered as a live card: schedule,
  next run, stop condition with the safety cap, goal, where results go, Run
  once now, Pause/Resume and Open task; the card polls the task only while it
  is on screen and shows deleted, scheduler-off and limits-exhausted states
- one card per task per assistant turn; the card group is not a turn boundary
  for usage, branching, editing, outline or skill attribution
- a human message with deerflow_scheduled_origin stays a human group with
  scheduledOrigin and renders as a "Scheduled run / Trial run" block with the
  task instructions, stop condition and notes collapsed; the launched text is
  never shown and the message is not editable
- readable step labels for schedule_task actions and stop_scheduled_task
- ThreadScheduledTasksButton replaces the header link: hidden without tasks,
  shows the count, links to the task (or the chat filter), and a run's chat
  links to its task
- recorded en-3 / zh-1 chat and run-thread fixtures; chat-card and run-thread
  e2e specs; unit and DOM tests for parsing, grouping, turn boundaries, card,
  run block, header button, step labels and reconnect hydration
- i18n: header.countLabelOne added, unused card.next removed; CHANGELOG and
  frontend AGENTS.md notes

* fix(scheduler): address verification findings

Move the scheduled-task lifecycle guidance from backend/AGENTS.md into the persistence guide and leave a one-line pointer, so the effective AGENTS.md chains for agents/middlewares, sandbox and subagents stay under the 98304-byte hard limit enforced by scripts/check_agent_guidance.py in lint-check.

* fix(scheduler): address review findings

* fix(gateway): no-transform on SSE responses

All four text/event-stream responses (thread run create stream, join,
existing-run GET/POST stream, stateless create stream) now take their
headers from app/gateway/sse_headers.py and send
Cache-Control: no-cache, no-transform, so compressing proxies such as
Next's rewrite proxy stop buffering streams. Content-Location and
X-Accel-Buffering are unchanged; scheduled and MCP-notification runs keep
stream_mode=None.

Tests pin the headers on every SSE route and fail on a literal no-cache
header left in routers. Docs: API.md SSE headers note, CHANGELOGs.

* feat(scheduler): per-owner run cap and fair queue selection

New scheduler.max_concurrent_runs_per_user (default 2, 0 = off, effective
min(cap, max_concurrent_runs)) caps launching/running scheduled runs per task
owner. claim_queued_run checks it right after the global count under the same
budget lock, so it is atomic across workers and Pods. list_queued_runs ranks
same-thread FIFO heads per owner with ROW_NUMBER() OVER (PARTITION BY owner)
and orders the batch by that rank, so the drain is owner-fair even with the
cap off; owners at their cap drop out before the batch limit. Orphan rows
(task deleted while queued) are never held back by the cap.

Gateway lifespan passes the startup value; the service constructor default
stays 0. Docs: config.example.yaml (config_version 54), CONFIGURATION.md,
the six READMEs, backend AGENTS.md, CHANGELOGs (behavior change flagged).
The live-task quota is untouched.

* feat(scheduler): durable lifecycle events for the originating chat

When a chat-created schedule is paused by the agent, auto-paused or finishes,
finalization now reports task_stopped / task_paused / task_finished, only on a
live -> paused/finished transition. The new finish_task_at_end_condition
replaces the three inline "mark completed" blocks (admission rejected as ended,
claim-time skip, complete_if_ended), so every site emits exactly once.

The scheduler's finalization observer is always installed. It writes
scheduled_task_events rows for the originating chat in the same transaction as
the state change, deduplicated on (task_id, anchor, event), with a title
snapshot, stop condition, run number, last run outcome and the chat's newest
run as placement anchor. Lookups fall back on bad stored data; only database
errors propagate. Manual trials drop only run_* events. The IM step keeps
PR1's selection until B4.

New GET /api/threads/{thread_id}/scheduled-task-events (owner check, caller's
rows, oldest first); rows survive task deletion and go with the chat.
Migration 0032_activity_and_task_events adds the events table plus the
run-origin column/index and read-state tables used later, each object guarded
on its own. Contract scheduled_goal_notes v3 pins the lifecycle vocabulary.
Docs: API.md, migrations chain, backend AGENTS.md, six READMEs, CHANGELOGs.

* feat(scheduler): honest, localized and self-contained IM notices

Every scheduled-task IM notice is now enqueued by the finalization observer
in the transaction that records the outcome, one message per occurrence
(task_stopped > task_paused > task_finished > run_failed > run_unmet >
run_completed; a once task sends only its run event). The merged auto-pause
replaces the separate unmet + pause messages, agent stops and finishes now
notify, and recovery-finalized runs notify exactly once. Idle finishes use a
deterministic idle: key derived from the lifecycle anchor. The old
non-transactional enqueue in handle_run_completion is removed.

New app/channels/capabilities.py holds CHANNEL_CAPABILITIES with a
proactive_notifications flag (adds the missing qq entry); only push-capable
providers (WeCom) get rows, /api/channels/providers returns the flag, and the
worker ends legacy rows for other providers as terminal failed.

New app/scheduler/notification_text.py renders en-US/zh-CN text: title, what
happened (plus the last run's outcome), the tasks page's one-line run summary
(redacted) when the agent replied, and a closing line. No IDs, no links, no
public_base_url. Locale is the owner's new `locale` preference, else the new
channel_connections.notification_locale (default en-US).

Docs: config.example.yaml (config_version 55), CONFIGURATION.md, API.md,
channels AGENTS.md, six READMEs, CHANGELOGs (behavior change flagged).

* feat(gateway): thread origin, activity feed and read markers for server-created threads

Runs started by the server for a user (scheduler, IM channels, GitHub
agents, extension handles, MCP notifications) now carry a server-owned
metadata.deerflow_origin {kind, provider?, namespace?} (new harness module
deerflow.runtime.run_origin, pinned by contracts/thread_origin_contract.json).
It is stamped only from request.state.run_origin or by the internal channel
caller; run admission drops client copies from both metadata and
config.metadata, and thread create/patch strip it. RunRepository derives
runs.origin_kind on put and on create_thread_operation_atomic (agent runs
only), and gains latest_change().

New ThreadReadRepository keeps a per-user, never-decreasing read position
(thread_read_markers) and a per-user read clock (thread_read_versions).
POST /api/threads/{id}/read marks a thread read, thread search items carry
unread, and thread delete removes the markers. GET /api/thread-activity pages
the caller's run changes over the run-change clock, returns only threads
changed by server-originated runs plus read_version, and 503s without SQL;
/api/features reports thread_activity.available.

Docs: API.md, gateway AGENTS.md (condensed to stay under its budget),
CHANGELOGs.

* fix(scheduler): adapt PR2 to PR1 follow-ups

PR1 moved the scheduled-task lifecycle rules out of backend/AGENTS.md into
the persistence guide and grew the sandbox AGENTS.md chain to within 267
bytes of its hard limit. The per-owner queue-fairness detail now lives in
the persistence guide next to the lifecycle rules (backend/AGENTS.md keeps
the cap and a link), so check_agent_guidance passes again.

PR1's goal-outcome unit test pinned scheduled_goal_notes_contract version 2;
PR2 bumps it to 3 (lifecycle vocabulary only, the v2 keys are unchanged).

* fix(scheduled-tasks): polish from live acceptance

- zh schedule wording: an interval of one unit reads 每分钟 / 每小时 / 每秒,
  and a saved interval of whole days reads 每天 / 每 2 天 (Every day / Every 2
  days) instead of 每 1 分钟 or 每 24 小时; several units keep the number
- zh spacing: 已于{time} 满足 and 在{time} 的运行中 now leave a space between
  the interpolated time and the Chinese after it; a copy test guards every
  scheduled-task zh string against a placeholder glued to Chinese
- run summary: when the first line of the final reply only leads in to a list
  (ends with ":" or ":"), the list items follow it as plain text (bullets,
  numbering, checkboxes, bold, code and links stripped), joined with ";" for
  CJK text and "; " otherwise, within the 160-character cap
- a schedule_task result (create, update, pause, resume, trial, delete) that
  arrives in a chat invalidates that chat's task list, the tasks list and the
  named task, so the header button and cards update without waiting for the
  idle poll; results already in the loaded history do not refetch
- e2e and unit fixtures are now the live acceptance recordings (DeepSeek
  Flash, demo data, fictional names): live-weekday.json (create, trial, edit,
  pause, resume in one zh chat) and live-minute.json (per-minute chat, run 1
  and run 2, paused by the agent), fetched with GET from the demo Gateway and
  trimmed; the hand-shaped en-3 / zh-1 fixtures are removed and the chat-card,
  run-thread and tasks-page specs run in both locales over the recorded data

* docs(changelog): reference #6378 for the scheduled-tasks entries

* docs(agents): shorten the scheduled-task pointer to stay under the guidance limit

* test(scheduled-tasks): pin the live takes to their recorded time zone

The live fixtures were recorded in Asia/Shanghai, and an interval task
reads in the viewer's zone, so the 16:48 assertions failed on UTC CI
runners. Pin the browser zone in the two live-take e2e specs and the
zh live-take unit test.

* feat(frontend): thread origin, activity polling, read markers and task events core

Frontend core for server-created threads and schedule lifecycle events
(PR2 F1); the sidebar markers (F2) and the chat event line (F3) build on it.

- core/threads/origin.ts: threadOriginOf reads metadata.deerflow_origin,
  then channel_source (github -> GitHub origin), then a legacy
  scheduled_task_id; key and kinds pinned to
  contracts/thread_origin_contract.json. labelOfThreadOrigin and the now
  exported labelOfChannelProvider(provider, t) use the new localized
  threads.origin.providers names (飞书, 企业微信, ...), with the English map
  (gaining GitHub) only as the fallback.
- core/threads/activity.ts: useThreadActivity polls /api/thread-activity
  every 15 s (focus refetch, never in the background, off unless
  features.thread_activity.available). The first response only seeds the
  cursor; server-origin threads, a truncated page or a higher read_version
  invalidate the thread lists, a scheduled thread also ["scheduled-tasks"];
  an invalid cursor re-seeds. useMarkThreadRead debounces POST /read by 1 s,
  skips a thread whose cached unread is false, patches the search and
  infinite caches optimistically and stores the returned read_version so a
  tab's own read does not refetch its lists. AgentThread gains unread.
- core/scheduled-tasks/events.ts: ScheduledTaskEvent, the events query
  (["scheduled-tasks","thread-events",id], one fetch per saved chat, not
  gated on the task list so a deleted task's lines stay, refetched when a
  task of the chat changes status or finishes a run), describeTaskEvent
  (en/zh copy, title and condition as segments, last-run suffix, See that
  run / Open task) and placeTaskEvents (end of the after_run_id turn, else
  the tail, ordered by created_at); lifecycle names pinned to contract v3.
- features: thread_activity (default unavailable); channels:
  proactive_notifications on providers.
- Locale sync: the I18nProvider reports the UI language and a signed-in
  preference session saves it as the new `locale` preference when the
  account's differs (and on every switch), so IM notices follow the UI
  language; nothing is written with auth disabled.
- i18n: threads.* (unread, origin labels, provider names),
  scheduledTasks.events.*, history.waitingForSlot (now shown for queued run
  rows), channels.scheduledUpdates.*.
- e2e mocks: thread activity feed and controller, POST /read, the events
  route, providers with proactive_notifications, search unread, features
  thread_activity (off by default). Preference e2e mocks return the UI
  locale so they keep asserting only their own edits.
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): keep untouched end times and drop a chat filter the new task is not in

- The edit, duplicate and renew dialogs keep a stored end time's instant
  while its field still shows it. A minute-precision wall value cannot
  keep seconds or name the later 01:30 of a DST fall-back, so a
  title-only edit moved the cap an hour earlier, and raising the run
  limit re-sent the end time without its seconds.
- Creating or duplicating a task from a chat's task view drops the chat
  filter when the new task does not belong to that chat, instead of
  showing "This task isn't one of this chat's tasks".

* feat(frontend): origin markers and unread state in the thread lists

Server-created threads (PR2 F2) now say who created them and whether they
are new, in the sidebar and on the Chats page, and reach an open sidebar
without a reload.

- ThreadOriginIcon (thread-channel-source.tsx): clock for a scheduled run,
  the provider mark for IM threads, the GitHub mark (new in
  ChannelProviderIcon) for GitHub threads, a puzzle piece for extensions;
  the localized "Scheduled run" / "From Feishu" / "From GitHub" / "From an
  extension" label is its accessible name and tooltip. ThreadChannelIcon
  stays exported as a wrapper, and the provider chip tooltip is localized
  (no more "Feishu channel").
- Unread dot after the title (aria-hidden dot plus sr-only "Unread"); the
  row aria-label becomes "{title}, unread" / "{title},未读". Never shown for
  the open thread.
- WorkspaceSidebar mounts useThreadActivity once.
- useMarkOpenThreadRead (components/workspace/chats): the default and
  custom-agent chat pages mark the thread read once its metadata has
  loaded, when its cached unread flips to true while open, when the tab
  becomes visible, and from onFinish (a send or a joined run ended). Each
  trigger goes through useMarkThreadRead, so it is debounced and a no-op
  while the cached unread is false.
- Tests: thread-origin-icon and recent-chat-list DOM tests (en/zh, active
  row, no raw identifiers), use-mark-open-thread-read DOM tests,
  e2e thread-activity.spec.ts (new scheduled run appears unread within one
  fake-clock poll without a reload, opening it posts /read and clears the
  dot, a read on another device clears it, IM and GitHub markers, zh
  labels, the Chats page, no polling when unavailable);
  thread-history.spec.ts now finds the localized origin icon.
- Docs: frontend AGENTS.md, CHANGELOGs.

* feat(frontend): schedule event line in the originating chat and channel notice support

The chat that created a schedule now says when the schedule paused or
ended, and the channel cards say which apps receive scheduled task updates
(PR2 F3).

- ScheduledTaskEventLine (mockup m2, marker 4): a dashed, muted row with
  the describeTaskEvent text (title in <strong>, stop condition clamped to
  one line with the full text in its tooltip, last-run suffix), the time
  via formatTaskTime and a right-aligned "See that run" (run chat) or
  "Open task" link. PauseCircle for paused, CircleCheck for finished and
  once-done, CircleX for once-failed. role="note", aria-label "Scheduled
  task update" / "定时任务通知", data-task-id and data-event-id only.
- MessageList takes scheduledTaskEvents and renders placeTaskEvents output
  through a new VirtualMessageList.renderAfterGroup, so each line sits at
  the end of its after_run_id turn inside the group's measured row (also
  when the list virtualizes); lines with nowhere to go render at the tail.
- Both chat pages load useThreadScheduledTaskEvents (not gated on the task
  list, so the lines stay after a task is deleted). The custom-agent page
  passes its agent so "See that run" opens the run on that agent's route.
- ChannelScheduledUpdates: "Scheduled task updates: sent here" / "... not
  available for this app yet" (定时任务通知:会发送到这里 / 此应用暂不支持)
  on the Settings provider cards and under each sidebar channel row; nothing
  when an older backend does not report proactive_notifications.
- e2e mock controller: setScheduledTasks.
- Tests: scheduled-task-event-line and channel-scheduled-updates DOM tests
  (every kind in en and zh, hrefs, clamp, no raw identifiers, Chinese
  title), MessageList placement DOM tests; e2e
  scheduled-task-chat-events.spec.ts (the line appears with the card's
  "Paused by agent" without a reload, See that run opens the run chat,
  anchored vs tail placement, two events in time order, the line survives
  task deletion and Open task shows the task is gone; en-3 and zh-1) and
  channels.spec.ts (WeCom sent here, Feishu not yet; en and zh).
- Docs: frontend AGENTS.md, CHANGELOGs.

* fix(scheduled-tasks): tie pause notices to the pause they describe

- Paused by agent links and times only the run named in last_error. When
  that run is not loaded, the notice shows no link and the task's own time
  instead of an earlier run that also asked to stop.
- The outcome notice reads the latest history page whichever page is
  shown, so browsing older runs no longer swaps the auto-pause reason and
  "Open latest run" to an earlier miss. It shares that page's request.

* fix(scheduler): address PR2 review findings

- Event lines: hold an event whose anchor run is on an unloaded history
  page instead of placing it at the bottom (hasMoreHistory); tail fallback
  only for fully loaded (branched/pruned) history or no anchor.
- Idle-finish anchor follows its reason: end:<iso> only for an end_at
  finish, seq:<n> otherwise, so a later end-time finish is not deduplicated
  away by an earlier max_runs finish.
- Observer takes FOR SHARE on the originating threads_meta row, so a racing
  thread delete cannot leave an orphan event line.
- One taskPagePath (core), re-exported by the card.
- Notification locales derived from NotificationLocale; /preferences uses it.
- IM notices drop generic reasons (goal_not_met_yet, unknown) instead of
  repeating the sentence.
- Localized provider names on Settings cards, sidebar rows and setup dialog.
- zh event lines: no space after a Chinese title (Latin-aware gap).
- Unread row label keeps the origin marker; chats page keeps the time as a
  description.
- Sidebar shows the scheduled-updates line only for connected apps;
  Settings says "available after you connect" for unconnected push apps.
- Event line keeps a real space before the time.

* fix(scheduled-tasks): three lifecycle gaps found while re-checking review classes

- The edit form compares a cron in the form its schedule input emits, so an
  untouched "0 09 * * 0,6" is not re-sent as "0 9 * * 6,0" and a title-only
  edit no longer re-arms or refuses a finished task.
- Chat resume asks for a timezone, as update does, when a naive end_at would
  land in the UTC placeholder of a zone-free task.
- Re-arming a task clears a host pause marker, as resume does, so a later
  manual pause no longer reads as "Paused by agent".

* test(scheduler): consolidate PR2 backend test setup

- Fold the repeated task, origin-chat and observer setup of the
  lifecycle-event repository tests into one lifecycle_task builder.
- Build the per-owner-cap services through one _capped_service builder
  that keeps every cap explicit at the call site.
- Table the five SSE route header checks into one parametrized test with
  the same routes, bodies and Content-Location expectations.
- Reuse bind_wecom and hoist repeated local imports in the event tests.

Every case and assertion is kept; collected cases are unchanged.

* test(frontend): consolidate PR2 thread activity and origin test setup

Extract shared builders and helpers in the frontend tests PR2 adds, with
every case, input and assertion kept:

- activity.test.ts: seededClient/apply helpers and one LIST_KEYS constant
  replace the per-test seed, spy and apply boilerplate.
- mark-read.dom.test.tsx: newClient() and advance() replace the repeated
  QueryClient wrapper and fake-timer act blocks.
- origin.test.ts: the English and Chinese label checks become tables; the
  no-English-name check now runs on every Chinese row.
- thread-activity.spec.ts: a scheduledRun() builder replaces four inline
  copies of the scheduled run thread.
- channels.spec.ts: the PR2 block uses setLocaleCookie and one card helper.

Collected cases are unchanged (unit 211, e2e 60); no test was removed.

* fix(scheduled-tasks): keep the notice's latest-runs query from refreshing older pages

useLatestScheduledTaskRuns kept React Query's default refetch on mount,
focus and reconnect, so going offline and back while an older history page
was shown fetched the latest page again. Refreshing stays with the history,
which refreshes the latest page only while it is shown.

* fix(scheduler): two gaps found in live acceptance

- A fresh scheduled run thread is named when it launches. The worker
  copied the title to the thread list only when the run ended, so the
  sidebar, which now shows the new run thread at once, listed it as
  untitled while it ran.
- A task capped at one run finishes with "its one run is done" /
  "唯一一次运行已完成" in the chat event line and the IM notice, instead
  of "all 1 runs are done".

* test(scheduled-tasks): let the viewer's time carry its own day label

Between 16:00 and 24:00 UTC, UTC and Asia/Shanghai are on different days,
so the run prompt shows "Tomorrow 01:00 · Today 09:00 your time" and the
fixed "01:00 · 09:00 your time" assertion failed depending on the hour.

* test(scheduler): expect the one-run finish copy in service notices

* fix(gateway): create a scheduled run thread with its title

Pre-setting the name after start_run raced the thread row, which the run
worker creates asynchronously: live, one of two run threads still reached
the sidebar untitled. Run admission now creates a new thread with the title
its run input carries, which the worker would copy only at run end.

* docs(changelog): reference #6393 for the scheduled-task signal entries

* fix(scheduled-tasks): route event-line run links by the agent that ran them

The originating chat's agent was used for "See that run", but a task's
agent can change after the chat created it, so the link could open the run
chat on another agent's route. Lifecycle events now record run_agent_name,
the task's agent when the run finished (edits are refused while a run is
active), and the line routes by it.

* fix(scheduled-tasks): load up to the route's maximum of chat event lines

The chat loads its lifecycle events once, and the route's default of 50
would silently drop the oldest lines of a chat with many schedules. Ask
for the maximum, 200.

* fix(threads): keep activity reads and the first cursor in step with the lists

- Accepting the first activity cursor refreshes the thread lists once. The
  lists load on their own, possibly before a run the seed already counts,
  and later polls start after it, so such a thread never appeared.
- A local read whose returned read clock jumps past that one read also
  covers reads on other devices; the lists are refreshed instead of the
  clock silently skipping those threads' stale unread dots.
- An invalid cursor re-seeds through the same path, so it refreshes once.

* docs(agents): bring the middlewares guidance chain back under the hard limit

#6378 and #6363 each fit the budget, but together the middlewares AGENTS.md
chain on main is 98,425 bytes, 121 over the 98,304 hard limit, so Lint
Check fails on main. Move the scheduled-task recovery, launch-update and
queue-timeout rules from backend/AGENTS.md into the persistence guide's
lifecycle section, which backend/AGENTS.md already points to. The chain is
now 98,067 bytes.

* fix(helm): bump the chart's config_version to 55

config.example.yaml moved to 55 in this PR, so the Helm chart's
validate-chart drift check failed. The chart's embedded config needs no
field changes: the new scheduler and channel keys have defaults.

---------

Co-authored-by: Totoro-qaq <279883115+Totoro-qaq@users.noreply.github.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
2026-10-07 10:22:15 +08:00

1873 lines
75 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# API Reference
This document provides a complete reference for the DeerFlow backend APIs.
## Overview
DeerFlow backend exposes two sets of APIs:
1. **LangGraph-compatible API** - Agent interactions, threads, and streaming (`/api/langgraph/*`)
2. **Gateway API** - Models, MCP, skills, uploads, and artifacts (`/api/*`)
All APIs are accessed through the Nginx reverse proxy at port 2026.
For agent conversations, clients can either pre-create a thread
(`POST /api/langgraph/threads`) or start immediately with the stateless stream
endpoint (`POST /api/langgraph/runs/stream`). The latter auto-creates a thread
and returns `thread_id` and `run_id` in the response `Content-Location` header.
## Authentication
Browser sessions authenticate with the `access_token` session cookie issued at
login. Programmatic clients can instead use a **personal access token (PAT)**
sent as a Bearer credential:
```http
POST /api/threads/search
Authorization: Bearer dfp_...
Content-Type: application/json
{}
```
PATs require a configured database backend (SQLite/PostgreSQL) — on the
memory-only backend, Bearer credentials are rejected and PAT management routes
return `503`.
### Account Preferences
`GET /api/v1/auth/preferences` returns the signed-in browser user's five
preferences. `PATCH` updates only explicitly supplied fields and returns `204`.
Both require `X-Expected-User-Id` matching the session user; PATCH also requires
the normal `X-CSRF-Token` header. The expected ID is a stale-tab guard, not an
authorization credential. PAT, internal, and auth-disabled callers receive
`403`; a different session user receives `409`.
```json
{
"notification_enabled": false,
"model_name": "my-model",
"mode": "pro",
"reasoning_effort": "high",
"locale": "zh-CN"
}
```
All five fields accept `null` to restore the default. `mode` accepts `flash`,
`thinking`, `pro`, or `ultra`; `reasoning_effort` accepts `minimal`, `low`,
`medium`, or `high`; `locale` accepts `en-US` or `zh-CN`; model names are at
most 200 characters. Unknown fields and
invalid values return `422`. Missing preferences read as `null`. Separate-field
patches preserve each other's changes, and same-field writes are last-commit-wins.
Storage requires SQLite or PostgreSQL (`503` when unavailable). Browser
notification permission remains device-local and is not changed by this API.
`locale` is the web UI language; the web app writes it after sign-in and on
every language switch, and scheduled-task IM notices are written in it (without
it they use `channel_connections.notification_locale`).
### Personal Access Tokens
Base URL: `/api/v1/auth`
PAT management requires an **interactive session** (a PAT cannot manage PATs
or change passwords, so a leaked automation token cannot mint fresh
credentials). The raw token is returned **exactly once** at creation; only its
SHA-256 digest is stored server-side.
#### Create Token
```http
POST /api/v1/auth/pats
Content-Type: application/json
```
**Request Body:**
```json
{
"name": "ci-runner",
"scopes": ["threads:read", "runs:create", "runs:read"],
"expires_in_days": 90
}
```
- `scopes` — subset of the route permissions: `threads:read`, `threads:write`,
`threads:delete`, `runs:create`, `runs:read`, `runs:cancel`. A PAT can only
*narrow* its owning user's permissions, never widen them.
- `expires_in_days` — optional (`1`–`365`); omitted means the token never expires.
**Response (`201`):**
```json
{
"id": "0f0c6e6a-...",
"name": "ci-runner",
"scopes": ["runs:create", "runs:read", "threads:read"],
"expires_at": "2026-11-25T10:30:00Z",
"created_at": "2026-08-27T10:30:00Z",
"token": "dfp_..."
}
```
Save `token` immediately — it cannot be retrieved again.
#### List Tokens
```http
GET /api/v1/auth/pats
```
Returns the caller's tokens with `last_used_at` / `revoked_at` audit fields;
never returns digests or raw tokens.
#### Revoke Token
```http
DELETE /api/v1/auth/pats/{pat_id}
```
Revocation is immediate.
### PAT Constraints
- A request carrying an `Authorization` header that fails validation gets a
hard `401` — it never falls back to the session cookie.
- **Cancel capability requires `runs:cancel` on every request dimension that
carries it**, not just the dedicated cancel route: `?action=interrupt|rollback`
on `POST /api/threads/{thread_id}/runs/{run_id}/stream` (action-less joins
stay at `runs:read`), and `multitask_strategy=interrupt|rollback` on run
creation (the default `reject` stays at `runs:create`). Joining a run's
stream is pure observation — an observer disconnecting never cancels the run.
- **Route-level default-deny:** PAT requests are admitted only to the
thread/run lifecycle routes the v1 scopes govern — `POST /api/threads`
(create), `POST /api/threads/search` (list), `GET/PATCH/DELETE
/api/threads/{thread_id}`, the thread `goal`/`state`/`compact`/`history`/
`branches` subroutes, and exactly the implemented `/runs` subroutes
(`GET|POST /api/threads/{thread_id}/runs`, the POST-only `stream`, `wait`,
`regenerate/prepare`, and `edit-regenerate/prepare` collection endpoints,
`GET /api/threads/{thread_id}/runs/{run_id}` plus its `cancel` (POST),
`join`/`messages`/`events`/`workspace-changes` (GET), and
`GET|POST .../runs/{run_id}/stream`), plus `POST /api/runs/stream|wait` and
`GET /api/runs/{run_id}/messages|feedback`. A route added under `/runs` is
denied until explicitly added to the policy.
Every other authenticated route — memory, agents, models, MCP/skills
config, integrations, channels, uploads — answers `403` to PAT callers
regardless of scopes. Scope enforcement alone only constrains
permission-decorated routes, so the allowlist is the outer boundary;
session-cookie callers are unaffected.
- PAT credentials never carry admin capability, even when the owning user is
an admin. This includes extension-contributed admin routes: the extension
principal projection suppresses every admin signal for PAT callers.
- Revoking or deleting the owning user invalidates their PATs on the next
request.
## LangGraph-compatible API
Base URL: `/api/langgraph`
The public LangGraph-compatible API follows LangGraph SDK conventions. In the unified nginx deployment, Gateway owns `/api/langgraph/*` and translates those paths to its native `/api/*` run, thread, and streaming routers.
### Threads
#### Create Thread
```http
POST /api/langgraph/threads
Content-Type: application/json
```
**Request Body:**
```json
{
"metadata": {}
}
```
**Response:**
```json
{
"thread_id": "abc123",
"created_at": "2024-01-15T10:30:00Z",
"metadata": {}
}
```
#### Get Thread State
```http
GET /api/langgraph/threads/{thread_id}/state
```
**Response:**
```json
{
"values": {
"messages": [...],
"sandbox": {...},
"artifacts": [...],
"thread_data": {...},
"title": "Conversation Title"
},
"next": [],
"config": {...}
}
```
### Runs
#### Create Run
Execute the agent with input.
```http
POST /api/langgraph/threads/{thread_id}/runs
Content-Type: application/json
Idempotency-Key: <unique key for this logical request> # optional
```
The thread-scoped create, stream, and wait endpoints accept an optional
`Idempotency-Key` header. Retrying with the same authenticated user, `thread_id`,
and key reuses the existing run instead of executing the input again. The key is
shared across `/runs`, `/runs/stream`, and `/runs/wait` for a given user and
thread, so the same key string cannot back two different calls even across those
endpoints. Reuse is bound to the original `input`, `assistant_id` and
`conversation_references`; a retry that changes them returns 409. Generate a new key for every intentional user
action; reuse a key only when retrying that same action after an uncertain HTTP
result. Keys may be at most 255 characters. Stateless `/api/langgraph/runs/*`
endpoints do not support this header because requests without an explicit thread
create a new temporary conversation.
Retrying a still-running run that this worker cannot stream returns 409 from
`/runs/stream` (`Run ... is not active on this worker and cannot be streamed`)
with no `Retry-After`. The same shape on `/runs/wait` returns 200
`{"status": "<durable status>", "error": ...}` without blocking for a final
state. Retrying a finished run through `/runs/wait` also returns that durable
status payload rather than the latest thread checkpoint: a later run on the
same thread may have advanced the head, and `/wait` does not claim that head
as this run's result. That status is the durable row after completion, not
the hydrated record from admission time. The original creating `/wait` still
returns this run's checkpoint even if a retry overlaps while it is waiting. Retrying a finished run whose SSE log is gone emits a `gap` frame
(`stream_replay_gap`, `recovery: reload_durable_state`) on the creating
`/runs/stream` endpoint and closes without an `end` frame; reload durable
thread/run state instead of treating the stream as empty. Observer joins of
that same run still end with `end`. Stateless `/api/langgraph/runs/stream`
does not accept this header and keeps the existing missing-stream close of
`end`; the `gap` signal is only on a thread-scoped creating retry.
**Request Body:**
```json
{
"input": {
"messages": [
{
"role": "user",
"content": "Hello, can you help me?"
}
]
},
"config": {
"recursion_limit": 100,
"configurable": {
"model_name": "gpt-4",
"thinking_enabled": false,
"is_plan_mode": false
}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}
```
**Stream Mode Compatibility:**
- Use: `values`, `messages-tuple`, `custom`, `updates`, `debug`, `tasks`, `checkpoints`
- Unsupported modes, including `messages`, `events`, and `tools`, return `422` before a run is created. DeerFlow never substitutes `values` for an unsupported mode.
**Run Option Compatibility:**
- Supported concurrency strategies: `reject`, `rollback`, and `interrupt`
- Compatibility default: `if_not_exists="create"`; this matches DeerFlow's current behavior
- Artifact delivery is enforced automatically when a run creates or modifies regular files under `/mnt/user-data/outputs`. `present_files` must present at least one path produced by the current run (or a directory containing it), and the terminal receipt must be persisted; presenting only an unrelated file does not satisfy delivery. Runs without changed outputs retain ordinary conversational behavior. `artifact_delivery` is not a client-settable run option.
- Unsupported options return `422`: `webhook`, `stream_resumable=true`, `after_seconds`, `feedback_keys`, any non-null `on_completion` value (including the SDK values `"complete"` and `"continue"`), `if_not_exists="reject"`, and `multitask_strategy="enqueue"`
- `stream_resumable=false` is accepted: it is the LangGraph SDK's default and requests the non-resumable stream DeerFlow already serves
- Undeclared SDK options, including `checkpoint_during` and `durability`, also return `422` instead of being silently discarded
When outputs changed during the run, `run.delivery` events retain the Slice 1
facts (`presented`, `paths`, and `by_tool`) and add `produced_paths`,
`presented_paths`, `matched_paths`, plus an explicit verdict: `verification`,
`stage` (`presented`, `mismatched`, or `not_started`), and `satisfied`. Receipts
for runs without changed outputs keep their existing shape.
**Recursion Limit:**
`config.recursion_limit` caps the number of graph steps LangGraph will execute
in a single run. The unified Gateway path uses the top-level `recursion_limit`
from `config.yaml` (default `100`) when a request does not provide one. Clients
can still set `recursion_limit` explicitly in the request body, and a valid
request value takes precedence. Scheduled-task launches do not take a client body: they
use `scheduler.recursion_limit` from `config.yaml` (default `1000`, matching
the web UI). For safety, the Gateway clamps any supplied
or configured value to a server ceiling (`max_recursion_limit` in `config.yaml`,
default `1000`) so a single run cannot execute unbounded graph steps (runaway
LLM cost / DoS); invalid or non-positive request values fall back to the
configured default. Both top-level fields are read per run, so edits apply to
the next request without restarting the Gateway. This top-level setting applies
to Gateway API runs only; IM channel and embedded `DeerFlowClient` runs retain
their own defaults and override paths.
**Configurable Options:**
- `model_name` (string): Override the default model
- `thinking_enabled` (boolean): Enable extended thinking for supported models
- `is_plan_mode` (boolean): Enable TodoList middleware for task tracking
**Response:** Server-Sent Events (SSE) stream
```
event: values
data: {"messages": [...], "title": "..."}
event: messages
data: {"content": "Hello! I'd be happy to help.", "role": "assistant"}
event: end
data: {}
```
#### Referencing a previous conversation
With `read_conversation` enabled in `config.yaml` (see [configuration](CONFIGURATION.md#reading-referenced-conversations)),
Gateway API callers can attach up to three explicit references to create/stream/wait requests:
```json
{
"input": {"messages": [{"role": "user", "content": "Use the requirements agreed in the referenced conversation."}]},
"conversation_references": ["https://deerflow.example/workspace/chats/source-thread"]
}
```
A reference is a valid thread ID or an absolute `/workspace/chats/{thread_id}` URL
(also `/workspace/agents/{agent_name}/chats/{thread_id}` for custom agents)
with the same scheme and authority as the run request, without query or fragment.
URLs are parsed as local selectors and are never fetched. For split-origin clients
or internal proxies, pass the thread ID. The field is separate from message text:
links in pasted documents, tool results, or previous messages grant no access.
The server supplies source IDs to the model as background user-role data and
binds the reader to this run's references and authenticated identity.
Clients that cannot add top-level fields to a run request (the LangGraph JS SDK
builds a fixed body and drops unknown keys) may send the same list as
`context.conversation_references`:
```json
{
"input": {"messages": [{"role": "user", "content": "Use the requirements agreed in the referenced conversation."}]},
"context": {"conversation_references": ["https://deerflow.example/workspace/chats/source-thread"]}
}
```
The Gateway lifts the key out of `context` before the run context is assembled,
so it has the same bounds and error locations as the top-level field, is
recorded on the run in the same way, and never reaches the merged run context
or the checkpointed `configurable`. Sending the top-level field and the context
key together returns 422. `GET /api/features` reports
`conversation_references.enabled` (the tool is configured) and `max_references`,
so a client can hide its entry point on deployments without the tool.
The request requires `runs:read` as well as the normal run-creation permission.
The tool rechecks source ownership on each read; foreign, deleted and unowned
legacy threads are unavailable. `read_conversation(thread_id, cursor?, limit?)`
reads newest-first pages (messages within each page are chronological), at most
50 visible user/assistant messages, 4,000 characters per message and 20,000 text
characters per page. Each page also stays within the tool-output budget that
applies to `read_conversation` (`tool_output.tool_overrides.read_conversation`,
else `externalize_min_chars`, and `fallback_max_chars`; 12,000 serialized
characters by default), so results reach the model inline instead of being
externalized to a file. A message that does not fit starts the next page intact.
Only a message longer than 4,000 characters, or one whose serialized form alone
exceeds the budget, is truncated. Such a message carries
`continuation: {"message_seq", "offset"}`; `read_conversation(thread_id,
message_seq=..., offset=...)` without a cursor returns the next part of that one
message (at most 20,000 text characters, sized to the same budget) with its
`offset`, `text_length` and, while text remains, a new continuation. Offsets
refer to the source's current text: an offset past its end returns
`invalid_request`, and a message that is no longer visible is unavailable. If the
`read_conversation` budget is too small to return any text (below roughly 800
serialized characters), the result is `output_budget_too_small` rather than a
continuation that makes no progress.
Results include message IDs, sequence numbers, continuation, truncation and
unavailability. Hidden messages, reasoning blocks, raw tool
results and subagent internals are excluded. Source data is not changed.
**Live reads and retained copies.** Each call reads the source's current visible
history. Editing or regenerating the source can change subsequent reads, including
later pages; a reference does not pin an immutable transcript. Text already returned
to the destination is a copy and is not automatically refreshed by source changes.
Read permission lasts only for this run, including its internal continuation steps.
Every new run, including resume, regenerate or edit replay, must submit references
again; checkpoints and old hints never restore permission. A resume can reuse
IDs already visible in the interrupted conversation, but needs the explicit
request field again. Missing/expired transcripts are not reconstructed from
checkpoints or memory.
Permission expiry does not erase excerpts already stored in the destination
conversation or conclusions derived from them. Deleting the source does not
retroactively erase those copies either; they follow the destination's own
retention and deletion behavior. Once the source is unavailable, further source
reads report unavailability rather than reconstructing it from destination copies.
**Incomplete requirements.** When `truncated` is true, the tool's notice tells
the agent to read the rest through each cut message's continuation before relying
on it, and to acknowledge the omission and request the missing material if that
read is unavailable. `has_more: false` means there are no older messages to page
through, not that every returned message is complete. This is model guidance, not
a new confirmation mechanism or a guarantee of model compliance.
This first version adds no frontend picker or link-to-reference conversion. The
tool is unavailable to bootstrap agents, subagents and embedded clients without
a host-provided reader. Active tool/skill policies continue to apply.
#### Get Run History
```http
GET /api/langgraph/threads/{thread_id}/runs
```
**Response:**
```json
{
"runs": [
{
"run_id": "run123",
"status": "success",
"created_at": "2024-01-15T10:30:00Z"
}
]
}
```
#### Stream Run
Stream responses in real-time.
```http
POST /api/langgraph/threads/{thread_id}/runs/stream
Content-Type: application/json
Idempotency-Key: <unique key for this logical request> # optional
```
Same request body as Create Run. Returns SSE stream.
#### Stateless Stream Run
Start a conversation without creating a thread first. Gateway auto-creates a
thread when `config.configurable.thread_id` is omitted, and returns both
identifiers in the response `Content-Location` header.
```http
POST /api/langgraph/runs/stream
Content-Type: application/json
Accept: text/event-stream
```
Through Nginx, `/api/langgraph/runs/stream` is rewritten to the native Gateway
path `POST /api/runs/stream`.
**Request Body:** Same as [Create Run](#create-run). Omit `thread_id` to start a
new conversation; include it to continue an existing one:
```json
{
"input": {
"messages": [
{
"role": "user",
"content": "Hello, can you help me?"
}
]
},
"config": {
"recursion_limit": 100,
"configurable": {
"model_name": "gpt-4",
"thinking_enabled": false,
"is_plan_mode": false
}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}
```
**Response:** Server-Sent Events (SSE) stream with a `Content-Location` header:
```http
Content-Location: /api/threads/{thread_id}/runs/{run_id}
```
Clients should parse `thread_id` and `run_id` from this header (the path ends
with `/runs/{run_id}`). Persist `thread_id` and send it back on the next turn
via `config.configurable.thread_id` to keep conversation history.
**Continuing a conversation:**
```json
{
"input": {
"messages": [
{
"role": "user",
"content": "What did I just ask?"
}
]
},
"config": {
"configurable": {
"thread_id": "abc123",
"model_name": "gpt-4"
}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}
```
---
## Gateway API
Base URL: `/api`
### Custom Agent portability
`GET /api/agents/{name}/export` downloads a version-1 JSON package for a
caller-owned Custom Agent. The package uses `format: "deerflow.custom-agent"`
and contains the portable Agent configuration plus SOUL. It excludes memory
contents, conversations, credentials, and deployment-owned GitHub bindings.
`POST /api/agents/import` creates the packaged Agent for the current user.
Pass `?name=<new-name>` to choose a different local identifier. The document
schema rejects unknown fields and unsupported format/version values; invalid
names or models return 422, and an existing name returns 409 without changing
the existing Agent. Import is create-only and never restores runtime state.
```json
{
"format": "deerflow.custom-agent",
"version": 1,
"agent": {
"name": "research-lead",
"description": "Coordinates parallel research",
"model": "deepseek-v3",
"tool_groups": ["web"],
"skills": ["literature-review"],
"allowed_subagents": ["researcher", "reporter"],
"memory_enabled": true,
"soul": "Delegate independent searches, then synthesize evidence."
}
}
```
### Models
#### List Models
Get all available LLM models from configuration.
```http
GET /api/models
```
**Response:**
```json
{
"models": [
{
"name": "gpt-4",
"display_name": "GPT-4",
"supports_thinking": false,
"supports_reasoning_effort": false,
"reasoning": {"thinking": "unsupported", "effort": null, "history": null, "source": "legacy"}
},
{
"name": "deepseek-v3",
"display_name": "DeepSeek V3",
"supports_thinking": true,
"supports_reasoning_effort": true,
"reasoning": {
"thinking": "optional",
"effort": {"values": ["minimal", "low", "medium", "high"], "default": null, "aliases": {}},
"history": null,
"source": "legacy"
}
},
{
"name": "glm-5.3-flash",
"display_name": "GLM-5.3-Flash",
"supports_thinking": true,
"supports_reasoning_effort": true,
"reasoning": {
"thinking": "required",
"effort": {"values": ["low", "high", "max"], "default": "high", "aliases": {"minimal": "low", "medium": "high"}},
"history": "clear",
"source": "contract"
}
}
]
}
```
`supports_thinking` and `supports_reasoning_effort` are deprecated projections of
`reasoning`. `reasoning.source` is `legacy` when the profile declares no
`reasoning:` block (the booleans were normalized) and `contract` when it does.
#### Get Model Details
```http
GET /api/models/{model_name}
```
**Response:**
```json
{
"name": "gpt-4",
"display_name": "GPT-4",
"model": "gpt-4",
"max_tokens": 4096,
"supports_thinking": false,
"supports_reasoning_effort": false,
"reasoning": {"thinking": "unsupported", "effort": null, "history": null, "source": "legacy"}
}
```
### MCP Configuration
#### Get MCP Config
Get current MCP server configurations.
```http
GET /api/mcp/config
```
Requires an authenticated admin session. Sensitive env/header/OAuth secret
values are masked in the response. Environment placeholders outside secret
containers are returned in their raw form so editing cannot expose or persist
their expanded values. Invalid operator-authored JSON/config shapes return
`400` instead of being reported as a Gateway fault.
**Response:**
```json
{
"mcp_servers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "***"
},
"description": "GitHub operations"
}
}
}
```
#### Update MCP Config
Update MCP server configurations.
```http
PUT /api/mcp/config
Content-Type: application/json
```
Requires an authenticated admin session. API-managed `stdio` MCP servers may
only use allowed executable names for `command` (default: `npx`, `uvx`). Set
`DEER_FLOW_MCP_STDIO_COMMAND_ALLOWLIST` to a comma-separated list when a
deployment needs additional trusted launchers.
**Request Body:**
```json
{
"mcp_servers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "$GITHUB_TOKEN"
},
"description": "GitHub operations"
}
}
}
```
**Response:**
```json
{
"mcp_servers": {
"github": {
"enabled": true,
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "***"
},
"description": "GitHub operations"
}
}
}
```
#### Update One MCP Server State
Enable or disable one configured MCP server without replacing the full
extensions configuration.
```http
PATCH /api/mcp/config
Content-Type: application/json
```
Requires an authenticated admin session. Enabling a `stdio` server validates
that server's `command` against the same allowlist used by the full `PUT`
endpoint. Disabling a server does not require its command to be allowlisted, and
invalid commands on other servers do not block the update. The endpoint
preserves secrets, environment-variable placeholders, skills, custom server
fields, and other top-level extensions config. SSE/HTTP targets may use either
DeerFlow's `type` field or the MCP-spec `transport` field.
**Request Body:**
```json
{
"server_name": "semantic-scholar",
"enabled": false
}
```
The response is the full masked MCP configuration, matching `GET` and `PUT`.
An unknown `server_name` returns `404`; attempting to enable a server with a
disallowed `stdio` command returns `400`.
#### Add MCP Servers
Add one or more servers without replacing existing entries. The Gateway
re-reads the file under the shared configuration lock, so concurrent sibling
changes are preserved. Existing names return `409`.
```http
POST /api/mcp/config/servers
Content-Type: application/json
```
The request body uses the same `mcp_servers` map as the full `PUT` endpoint.
#### Replace One MCP Server
Completely replace one existing server while preserving sibling entries.
Omitted ordinary fields are deleted or reset; explicit `***` placeholders
restore the corresponding stored secret.
A disabled `stdio` replacement may keep a syntactically valid command outside
the allowlist for offline editing. Command-shape and code-injecting environment
variable checks still run when saving; the allowlist and executable-argument
policy run when the server is enabled.
```http
PUT /api/mcp/config/server
Content-Type: application/json
```
```json
{
"server_name": "github",
"server": {
"enabled": true,
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {"GITHUB_TOKEN": "***"}
}
}
```
#### Delete One MCP Server
Delete one server without replacing sibling entries. The server name is a
path parameter and the DELETE request has no body. Percent-encode names before
placing them in the URL; the path converter also keeps legacy empty and
slash-containing names addressable.
```http
DELETE /api/mcp/config/servers/{server_name}
```
All targeted mutations return the full masked MCP configuration. Before any
write, the Gateway resolves environment variables in a copy and validates the
same expanded document the runtime will load while persisting the original raw
placeholders.
#### Reset MCP Tools Cache
Publish a shared cache generation, then clear cached MCP tools and persistent
MCP sessions in the handling process. Every Gateway worker sharing the writable
extensions-config directory observes that generation and reloads tools on its
next agent run or tool lookup. This also refreshes remote `tools/list` changes
that did not modify `extensions_config.json`.
```http
POST /api/mcp/cache/reset
```
Requires an authenticated admin session.
**Response:**
```json
{
"success": true,
"scope": "shared_config",
"message": "MCP tools cache reset published through the shared config directory. Tools will reload on next use."
}
```
`shared_config` means every worker mounting that same directory observes the
generation; it does not claim a deployment-wide broadcast when replicas use
independent filesystems. When no extensions-config path can be resolved, the
request still resets the current worker and returns `"scope": "process"`.
### Skills
#### List Skills
Get all available skills.
```http
GET /api/skills
```
**Response:**
```json
{
"skills": [
{
"name": "pdf-processing",
"display_name": "PDF Processing",
"description": "Handle PDF documents efficiently",
"enabled": true,
"license": "MIT",
"path": "public/pdf-processing"
},
{
"name": "frontend-design",
"display_name": "Frontend Design",
"description": "Design and build frontend interfaces",
"enabled": false,
"license": "MIT",
"path": "public/frontend-design"
}
]
}
```
#### Get Skill Details
```http
GET /api/skills/{skill_name}
```
**Response:**
```json
{
"name": "pdf-processing",
"display_name": "PDF Processing",
"description": "Handle PDF documents efficiently",
"enabled": true,
"license": "MIT",
"path": "public/pdf-processing",
"allowed_tools": ["read_file", "write_file", "bash"],
"content": "# PDF Processing\n\nInstructions for the agent..."
}
```
#### Enable or Disable Skill
```http
PUT /api/skills/{skill_name}
Content-Type: application/json
```
Requires an authenticated admin session.
**Request Body:**
```json
{
"enabled": false
}
```
**Response:** the updated skill. An unknown `skill_name` returns `404`.
```json
{
"name": "pdf-processing",
"description": "Handle PDF documents efficiently",
"license": "MIT",
"category": "public",
"enabled": false,
"editable": false
}
```
#### Install Skill
Install a skill from a `.skill` file.
```http
POST /api/skills/install
Content-Type: multipart/form-data
```
**Request Body:**
- `file`: The `.skill` file to install
**Response:**
```json
{
"success": true,
"message": "Skill 'my-skill' installed successfully",
"skill": {
"name": "my-skill",
"display_name": "My Skill",
"path": "custom/my-skill"
}
}
```
#### Export a Custom Skill
Admin session authentication is required for both requests. PAT credentials cannot export. Only the current user's custom skill is eligible; public, legacy and integration fallback is never used. A disabled custom skill remains eligible.
1. `GET /api/skills/custom/{skill_name}/export-manifest` returns `skill_name`, `revision` (SHA-256 or null), `can_export`, `file_count`, `directory_count`, `total_bytes`, `files` (`path`, `type`, `size`, `executable`), `requirements` (`compatibility`, `allowed_tools`, `required_secrets` names and optional flags), and structured `warnings`/`blockers`. Paths are relative; `.` is the package root, counted in directory/entry totals. Structural blockers return a non-downloadable manifest. Declarations are not credential values or dependency verification.
2. `GET /api/skills/custom/{skill_name}/export?expected_revision=<64 lowercase hex characters>` recaptures content and rejects stale previews with 409 before sending ZIP headers. Successful responses carry `application/zip`, attachment `<skill_name>.skill`, accurate `Content-Length`, `Cache-Control: private, no-store`, and `X-Content-Type-Options: nosniff`.
Error `detail` contains a safe `code`, `message`, and optional relative `path`. Codes/statuses: `skill_not_found` 404, `skill_changed` 409, `skill_export_limit_exceeded` 413, `skill_export_unsupported` 422, `skill_export_busy` 429, `skill_export_timeout` 503, `skill_export_failed` 500; existing 401/403 auth behavior applies. Limits are 4096 entries including directories, 64 MiB/file, 100 MiB raw/ZIP, 1 MiB frontmatter, 1024 UTF-8 bytes per ZIP path and depth 32. Frontmatter preflight rejects YAML aliases and bounds structure to 32 nesting levels / 16384 parser events before constructing YAML objects. A 5-second lock wait and 60-second cooperative worker deadline bound work; blocking OS calls cannot be forcibly interrupted. Two export slots are shared across all users in each Gateway process; both previews and downloads use them, and 429 means that process-wide capacity is occupied. Slots remain held through worker drain and temporary-file cleanup. The streaming phase has a separate 120-second inactivity deadline, reset after each successful ASGI send. A continuously progressing transfer may exceed 120 seconds overall; a stalled send does not reset the deadline. Expiry aborts the incomplete download (no replacement JSON after ZIP headers); clients must retry. Client disconnect during preparation cancels and drains the worker, then exits the handler normally rather than leaking a synthetic task cancellation. No export cache, persistent job or sharing URL is created.
Raw skill files, sidecars and empty directories are preserved. No hooks/scripts run during export and no secrets are redacted from package files. Import still uses normal security scanning and conflict checks. Export requires no-follow descriptor-relative host filesystem operations; unsupported platforms receive 422 rather than following links unsafely.
#### Reload Skills
Invalidate the skill prompt caches for every user in the current Gateway
process. Subsequent runs rescan the configured public, custom, and legacy skill
directories; runs that have already started keep their existing skill snapshot.
```http
POST /api/skills/reload
```
The request has no body and requires an authenticated administrator. For a
cookie-authenticated request, send the CSRF cookie value in the matching header:
```bash
curl -X POST http://localhost:2026/api/skills/reload \
-b cookies.txt \
-H "X-CSRF-Token: <csrf_token-cookie-value>"
```
**Response:**
```json
{
"success": true,
"scope": "process",
"message": "Skill caches invalidated; subsequent runs in this Gateway process will rescan the latest skills."
}
```
`success` confirms cache invalidation, not that every file on disk was valid:
malformed skills retain the existing parser behavior of being skipped and
logged. The endpoint returns `401` for unauthenticated callers, `403` for
non-admin users, and a generic `500` if the invalidation mechanism itself
fails or the process-local background scan does not finish within the cache
refresh timeout. A loader-level failure, such as an unavailable mounted root,
does not publish an empty catalog: the last successfully loaded process cache
remains available. A timed-out scan continues in its daemon worker and can
still populate the process cache when it finishes.
The scope is deliberately process-local. Each Uvicorn worker or Kubernetes Pod
must be called directly; repeated requests through a load-balanced Service do
not guarantee that every instance is reached. External MinIO/NFS/CSI writes
bypass the validation, SkillScan, and history used by the install/edit APIs, so
the mounted directory must be writable only by trusted operators.
### File Uploads
#### Upload Files
Upload one or more files to a thread.
```http
POST /api/threads/{thread_id}/uploads
Content-Type: multipart/form-data
```
**Request Body:**
- `files`: One or more files to upload
**Response:**
```json
{
"success": true,
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/abc123/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf",
"markdown_file": "document.md",
"markdown_path": ".deer-flow/threads/abc123/user-data/uploads/document.md",
"markdown_virtual_path": "/mnt/user-data/uploads/document.md",
"markdown_artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.md"
}
],
"message": "Successfully uploaded 1 file(s)"
}
```
**Name collisions:** filenames are claimed unique against the thread's existing uploads and reserved atomically — a same-name upload never replaces the existing file; it lands as `document_1.pdf` (the response's `filename`/`original_filename` reflect the claimed name). Use the artifacts `PUT` endpoint for sanctioned in-place updates.
**Supported Document Formats** (auto-converted to Markdown):
- PDF (`.pdf`)
- PowerPoint (`.ppt`, `.pptx`)
- Excel (`.xls`, `.xlsx`)
- Word (`.doc`, `.docx`)
#### List Uploaded Files
```http
GET /api/threads/{thread_id}/uploads/list
```
**Response:**
```json
{
"files": [
{
"filename": "document.pdf",
"size": 1234567,
"path": ".deer-flow/threads/abc123/user-data/uploads/document.pdf",
"virtual_path": "/mnt/user-data/uploads/document.pdf",
"artifact_url": "/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf",
"extension": ".pdf",
"modified": 1705997600.0
}
],
"count": 1
}
```
#### Delete File
```http
DELETE /api/threads/{thread_id}/uploads/{filename}
```
**Response:**
```json
{
"success": true,
"message": "Deleted document.pdf"
}
```
### Thread Cleanup
Remove DeerFlow-managed local thread files under `.deer-flow/threads/{thread_id}` after the LangGraph thread itself has been deleted.
```http
DELETE /api/threads/{thread_id}
```
**Response:**
```json
{
"success": true,
"message": "Deleted local thread data for abc123"
}
```
**Error behavior:**
- `422` for invalid thread IDs
- `500` returns a generic `{"detail": "Failed to delete local thread data."}` response while full exception details stay in server logs
Deleting a thread also removes every user's read marker for it.
### Thread Origin, Activity and Unread State
Runs the server starts for a user (a schedule, an IM channel, a GitHub agent,
an extension, an MCP notification) carry a server-owned origin in their
metadata, and so does a thread the server creates for one:
```json
{ "deerflow_origin": { "kind": "im_channel", "provider": "feishu" } }
```
`kind` is one of `schedule`, `im_channel`, `github`, `extension`,
`mcp_notification` (`contracts/thread_origin_contract.json`); `provider`
(IM/GitHub) and `namespace` (extension plugin) are optional. Clients cannot
set it: `POST /api/threads` and `PATCH /api/threads/{thread_id}` strip it, run
admission drops it from `metadata` and `config.metadata`, and only server-side
launchers and the internal channel caller may stamp it. The run's kind is also
stored as `runs.origin_kind` (`null` for interactive and pre-upgrade runs). IM
threads keep `metadata.channel_source` as their marker. This key is unrelated to
the message-level `additional_kwargs.deerflow_scheduled_origin` of a scheduled
prompt.
**Activity feed.** Requires SQL persistence (`503` on the memory backend);
`GET /api/features` reports `{"thread_activity": {"available": true}}`.
```http
GET /api/thread-activity?cursor=1834:run-42&limit=200
```
```json
{
"cursor": "1840:run-57",
"threads": [{ "thread_id": "…", "origin_kind": "schedule", "status": "success" }],
"truncated": false,
"read_version": 17
}
```
- Without `cursor` the call only seeds: it returns the caller's current
position (`"0:"` for a user with no runs) and no threads.
- With a cursor it pages the caller's run changes in order (`limit` 1-500,
default 200). `threads` lists each thread with a change from a
server-originated run of the caller in the page, once, with the status of
its latest such change. The caller's own interactive runs advance the cursor
but are not listed.
- `cursor` is the position of the last change used, so a `truncated` page
continues on the next poll without gaps. Treat it as opaque.
- `read_version` is the caller's read clock; it changes when a thread is
marked read on any device.
- A malformed cursor returns `422` with `detail.code` `invalid_cursor`.
- Every query is scoped to the caller; another user's cursor reveals nothing.
**Unread state.** A thread is unread for a user while one of that user's
server-originated runs in it changed after the user last read it. The user's
own interactive runs, other users' runs in a shared thread and pre-upgrade runs
never make a thread unread.
```http
POST /api/threads/{thread_id}/read
```
```json
{ "unread": false, "read_version": 18 }
```
The read position never moves backwards. A thread already read up to its
latest run writes nothing and returns the unchanged `read_version`. Items of
`POST /api/threads/search` carry `unread` (`true`/`false`; `null` without SQL
persistence); other thread responses return `null`.
### Projects
#### Get Projects Config
```http
GET /api/projects/config
```
The `projects` config-block knobs the UI needs for client-side validation. Requires the `projects:read` scope (PATs included).
**Response:**
```json
{
"instructions_max_bytes": 8192,
"trash_retention_days": 30
}
```
Values come from `projects.instructions_max_bytes` and `projects.trash_retention_days` in `config.yaml`; the documented defaults apply when the block is absent.
### Project Documents
Per-project document shelf (Projects Phase 2). All routes fail closed: a missing or foreign project/document is `404` (never `403`); uploads and individual trash require an active project — archived projects answer `404` for those while keeping reads; a memory-backend deployment answers `503` `"Projects not available"`. Trashed rows are invisible to every route.
#### List Documents
```http
GET /api/projects/{project_id}/documents?limit=100&offset=0
```
**Query Parameters:** `limit` (default 100, 1..1000), `offset` (default 0) — out-of-bounds values are `422`.
**Response:** `{"documents": [{"id", "name", "size_bytes", "sha256", "source_thread_id", "source_kind", "source_name", "created_at", "updated_at", "content_missing"}], "total", "limit", "offset"}` in `updated_at DESC, id ASC` order. `content_missing` is read-time truth (never persisted): `true` when the document's immutable original is missing or size-mismatched (external interference); the derived `converted.md` companion is not the integrity anchor.
#### Upload Document
```http
POST /api/projects/{project_id}/documents
Content-Type: multipart/form-data
```
Exactly one file per request (`file` part), plus an optional `name` form field (defaults to the multipart filename). The name is rejected with `400` when empty after normalization, separator-bearing, or over 255 UTF-8 bytes; empty files are `400`; files over `uploads.max_file_size` are `413`.
**Response:** `201 Created` with `{"document": {...}, "deduplicated": false}`. Re-uploading identical content returns the existing row with `200 OK` and `"deduplicated": true` — the first writer's name wins. Re-upload after trash creates a fresh row.
#### Save Thread File to Shelf (from-thread)
```http
POST /api/projects/{project_id}/documents/from-thread
Content-Type: application/json
```
```json
{"thread_id": "abc123", "kind": "upload", "name": "report.pdf", "shelf_name": "q3-report.pdf"}
```
Copies one file from the thread's own uploads (`"kind": "upload"`) or outputs (`"kind": "output"`) directory into the shelf as a project-owned snapshot; the source file is never moved. `name` locates the source file inside the thread; `shelf_name` is optional and defaults to the source name, following upload-name validation (`400`). A source that does not resolve inside that thread's directory — separator-bearing names, escapes, missing files, or a missing/foreign thread — is `404`, indistinguishable from absence. Files over `uploads.max_file_size` are `413`; empty files are `400`.
**Response:** same as Upload Document — `201 Created` with `{"document": {...}, "deduplicated": false}`, or `200 OK` on a content dedup hit. The created row records `source_thread_id` / `source_kind` / `source_name` provenance.
#### Attach Document to Thread
```http
POST /api/projects/{project_id}/documents/{document_id}/attach-to-thread/{thread_id}
```
Materializes an independent copy of a live shelf document into the target thread's uploads directory through the same ingestion pipeline as an ordinary upload (filename claiming, size checks, optional conversion under `uploads.auto_convert_documents`, sandbox-readable permissions, and sandbox sync for non-mounted providers; a caller denied `sandbox:execute` keeps the host upload without allocating a sandbox). Reading an archived source project's shelf is allowed and does not mutate it; a missing/foreign document, or a target thread the caller cannot write, is `404`. A row whose original bytes are missing or size-mismatched answers `409` `"content_missing"`.
**Response:** `200 OK` with `{"filename", "size_bytes", "virtual_path", "artifact_url"}` — returned only after ingestion succeeds.
#### Get Document Content
```http
GET /api/projects/{project_id}/documents/{document_id}/content?download=false
```
Serves the converted-markdown companion when present, else inline text when the original samples as text, else an attachment; `download=true` always attaches. Active content (`text/html`, `text/xml`, `application/xml`, `text/xsl`, any `+xml` type such as XHTML/SVG) is always forced to an attachment regardless of `download`, mirroring the artifacts router, so it never executes script on the application origin. A row whose original bytes are missing or size-mismatched answers `409` `"content_missing"`.
#### Delete Document (move to trash)
```http
DELETE /api/projects/{project_id}/documents/{document_id}
```
**Response:** `204`. Recoverable trash: the row keeps its bytes and a `{project_id, project_name}` origin snapshot. Deleting a project moves its whole shelf to trash in the same transaction. Restore/purge endpoints land with the trash-completion slice.
### Project Thread Files
Read-only conversation-files view over a project's member threads (Projects Phase 2) — the discovery route for Save Thread File to Shelf. Archived projects keep read access; a missing or foreign project is `404`.
#### List Thread Files
```http
GET /api/projects/{project_id}/thread-files?offset=0&thread_limit=20&file_limit=50
```
**Query Parameters:** `offset` (member-thread cursor, default 0), `thread_limit` (default 20, 1..50), `file_limit` (per-thread file cap, default 50, 1..200) — out-of-bounds values are `422`.
**Response:** `{"groups": [{"thread_id", "display_name", "updated_at", "truncated", "files": [{"kind": "upload"|"output", "name", "size_bytes", "modified_at", "artifact_url"}]}], "next_offset", "truncated"}`. Member threads are paged in the same non-archived order as the project thread list; `next_offset` is `null` when no threads remain. Each thread contributes up to `file_limit` files across its uploads and outputs; a group's `truncated` is `true` when that thread's listing was cut, and the envelope `truncated` is the OR over the page's groups. Entries disappear when their thread is deleted — the view keeps no storage of its own.
### Trash
Recoverable deletion tier for project shelf documents (Projects Phase 2). Trashed rows keep their bytes and a `{project_id, project_name}` origin snapshot for `projects.trash_retention_days` (default 30) before the retention sweep may purge them; permanent purge is a separate action. All routes fail closed: a missing or foreign document/project is `404` (never `403`), restoring into an archived or foreign target is the same `404`, and a memory-backend deployment answers `503` `"Projects not available"`. Purge endpoints carry no confirmation parameter — the "this cannot be undone" step is a UI contract, not a server-enforced handshake.
#### List Trashed Documents
```http
GET /api/trash/documents?limit=100&offset=0
```
**Query Parameters:** `limit` (default 100, 1..1000), `offset` (default 0) — out-of-bounds values are `422`.
**Response:** `{"documents": [{"id", "name", "size_bytes", "sha256", "source_thread_id", "source_kind", "source_name", "created_at", "updated_at", "trashed_at", "trash_origin": {"project_id", "project_name"} | null}], "total", "limit", "offset"}`, most recently trashed first. The retention sweep runs lazily before the listing (a sweep failure is logged and never blocks it).
#### Restore Document
```http
POST /api/trash/documents/{document_id}/restore
Content-Type: application/json
{"project_id": "…"}
```
**Body:** `project_id` optional. Target = the body value, else `trash_origin.project_id` when that project still exists, is owned, and is active; otherwise `404` (the UI offers the project picker). A foreign or archived target is the same `404` as a missing one.
**Response:** `{"outcome": "restored" | "merged", "document": <ProjectDocumentResponse>}`. `merged` means the target already had an active row with identical bytes: the trash row is deleted and `document` is the surviving active row. Restore re-points the row without moving any file. Missing or size-mismatched content answers `409` `"content_missing"` and leaves the row trashed.
#### Purge Document
```http
POST /api/trash/documents/{document_id}/purge
```
**Response:** `204`. Permanently unlinks the original and `derived/converted.md`, then deletes the row, in one row-locked transaction. Already-absent content counts as removed; any other file-cleanup failure rolls back, keeps the trashed row, and answers `500` with a retryable message.
#### Empty Trash
```http
POST /api/trash/purge
```
**Response:** `{"purged": <int>}` — permanently deletes every trashed document of the caller, regardless of age: the confirmation covers the whole listing, so the retention cutoff never gates this route. Each row goes through the same guarded row-locked transaction as the single-document purge — bytes first, then the row. A file-cleanup failure other than already-absent content answers `500` with a retryable message, leaving that row and every row not yet visited trashed. Retention expiry is enforced only by the sweep (lazily before `GET /api/trash/documents` and once at gateway startup).
### Artifacts
#### Get Artifact
Download or view an artifact generated by the agent.
```http
GET /api/threads/{thread_id}/artifacts/{path}
```
**Path Examples:**
- `/api/threads/abc123/artifacts/mnt/user-data/outputs/result.txt`
- `/api/threads/abc123/artifacts/mnt/user-data/uploads/document.pdf`
**Query Parameters:**
- `download` (boolean): If `true`, force download with Content-Disposition header
**Response:** File content with appropriate Content-Type. HTML and XML documents (`.html`, `.xml`, `.xhtml`, `.svg`, and other `+xml` types) are always returned as attachments, regardless of `download`, so generated markup never renders in the application origin.
---
### Scheduled Tasks
Owner-scoped task management for `/workspace/scheduled-tasks`. Reads need
`threads:read`; mutations need `threads:write`, and create, update, resume and
trigger also need `runs:create`.
```http
GET /api/scheduled-tasks
POST /api/scheduled-tasks
GET /api/scheduled-tasks/{task_id}
PATCH /api/scheduled-tasks/{task_id}
DELETE /api/scheduled-tasks/{task_id}
POST /api/scheduled-tasks/{task_id}/pause
POST /api/scheduled-tasks/{task_id}/resume
POST /api/scheduled-tasks/{task_id}/trigger
GET /api/scheduled-tasks/{task_id}/runs?status=&limit=&offset=
GET /api/threads/{thread_id}/scheduled-tasks
GET /api/threads/{thread_id}/scheduled-task-events?limit=50
POST /api/scheduled-tasks/preview-cron
```
**Task fields.** Create and PATCH accept `title`, `prompt`, `schedule_type`
(`once` | `cron` | `interval`), `schedule_spec`, `timezone`, `context_mode`,
`thread_id`, `assistant_id`, plus:
| Field | Type | Notes |
|---|---|---|
| `goal_objective` | string \| null | What one run must achieve (at most 4000 characters after whitespace normalization); requires `fresh_thread_per_run`. |
| `max_runs` | integer \| null | Safety cap: automatic runs over the task's lifetime (trial runs excluded); at least 1. |
| `end_at` | date-time \| null | Safety cap: no automatic run after this time. Without a UTC offset it is wall-clock time in the task's timezone. |
| `stop_condition` | string \| null | The user's "stop when …" rule, stored in its own field. Whitespace collapses to single spaces; at most 500 characters; blank means none. It is never part of `prompt`: each launch appends it as an instruction to call `stop_scheduled_task` (or, with `scheduler.tool_enabled` off, to report a met rule). |
In a PATCH, sending `null` clears `goal_objective`, `max_runs`, `end_at` or
`stop_condition`; for every other field `null` means unchanged (`assistant_id:
null` restores `lead_agent`). PATCH may change `schedule_type` together with
`schedule_spec`. Changing the goal, `prompt` or `stop_condition` starts a new
count for the automatic pause after three unmet runs. A PATCH that changes the
schedule of a finished task re-arms it (then its safety cap applies); a PATCH
that only changes the cap leaves a finished task finished.
Task responses (list, get, create, PATCH, pause, resume) return the stored task
plus `automatic_runs_used` (integer) and `active_run_status` (`queued` |
`launching` | `running` | null). A recurring task's `status` stays `enabled`
while its run executes, so check `active_run_status` for "is a run active".
`GET /api/threads/{thread_id}/scheduled-tasks` rows add `thread_relation`
(`origin`: created in the chat, `reuse`: runs in the chat, `run`: the chat is
one of its runs) and `thread_run` (`{"run_number", "trigger", "scheduled_for",
"status"}` for `run`, else null).
`GET /api/threads/{thread_id}/scheduled-task-events` (thread owner check;
`limit` 1–200, default 50) returns the caller's lifecycle events for a chat
that created tasks, oldest first:
```json
{ "events": [ { "id": "evt-…", "task_id": "task-…", "event": "task_stopped", "reason_code": "agent_stop",
"task_title": "Check the release checklist", "stop_condition": "all items are ticked",
"run_thread_id": "…", "run_agent_name": "lead_agent", "run_number": 4, "run_status": "success", "max_runs": null, "end_at": null,
"schedule_type": "cron", "after_run_id": "…", "created_at": "2026-10-06T09:00:03+00:00" } ] }
```
`event` is `task_stopped` (the agent paused its own schedule, reason
`agent_stop`), `task_paused` (automatic pause, `consecutive_unmet`) or
`task_finished` (`max_runs`, `end_at`, or a one-time task's `once_done` /
`once_failed`); the vocabulary is pinned in
`contracts/scheduled_goal_notes_contract.json`. Each row is written in the
same transaction as the state change it reports and is unique per task,
transition and event, so recovery never adds a second one. `run_status` is the
last run's outcome (a stop or finish after a failed run says so);
`run_thread_id` is null when the occurrence never launched, and
`run_agent_name` is the agent that ran it (its run chat's route); `after_run_id` is
the newest run of the chat when the event was written (the chat shows the line
after that turn, else at the end). Rows keep a title snapshot and stay after
the task is deleted; deleting the chat removes them. Tasks created on the
Scheduled tasks page have no originating chat and no rows. IDs are for links
only.
**Create** returns `409 scheduler_not_running` (after the request validates)
while this Gateway process's scheduler is not running; the tasks page's
Duplicate is a create.
**Pause** of a `completed`, `failed` or `cancelled` task returns `409
task_finished`.
**Resume** accepts an optional renewal body:
```json
{ "max_runs": 70, "end_at": "2026-12-31T18:00:00" }
```
Each field is optional; `null` clears that cap (a chat-created task that runs
more often than hourly must keep one: `422 frequent_requires_limit`). The next
run is the stored one when still ahead, otherwise computed from now, so no
catch-up run happens; a one-time task whose time passed returns `422
once_time_passed`. Resuming an `enabled` task changes nothing. A task whose
safety cap is used up returns `409 limits_exhausted` unless the same request
renews the limit named in `params.limit`: for `max_runs`, a `max_runs` above
`params.used` or `null`; for `end_at`, a later `end_at` or `null`. A later
`end_at` alone does not renew a used-up `max_runs`.
A future `end_at` that falls before the next run returns `422
end_at_before_first_run` (also on a PATCH that changes the schedule).
**Trigger** (one trial run) returns:
```json
{ "id": "task-…", "triggered": true, "outcome": "launched", "existing": false, "thread_id": "…" }
```
`outcome` is `launched` or `queued`; `existing: true` means a run was already
waiting and no trial was added.
**Runs** rows add `run_number` (the automatic-run number counted like
`max_runs`; null for trials and runs that never launched), `total_tokens` of the
launched run (null if none), and `summary` (first line of the agent's final
reply as plain text; when that line ends with a colon, the list items that
follow it are appended, joined with `;` for CJK text and `; ` otherwise; at
most 160 characters).
**Errors (breaking change).** Every error these routes raise for a well-typed
request is coded:
```json
{ "detail": { "code": "limits_exhausted", "message": "All 5 automatic runs are used. Raise max_runs above 5 or clear it (max_runs: null) in the same request to reactivate.", "params": { "limit": "max_runs", "used": 5, "max_runs": 5, "end_at": null } } }
```
`message` stays English for API clients; `params` is omitted when empty.
Clients that read `detail` as a string must read `detail.message` instead.
Three errors keep their old shape: a `403` from route permissions is the plain
string `"Permission denied: <permission>"`, FastAPI's own `422` for malformed
JSON or wrong types (for example `"max_runs": "abc"`) keeps its list `detail`,
and the shared `503` `"Thread metadata store not available"` (a Gateway without
a thread store) stays a plain string. Codes the web UI translates: `invalid_request`, `invalid_schedule`, `invalid_schedule_type`, `invalid_timezone`, `interval_too_short`, `interval_too_long`, `once_in_past`, `once_too_soon`, `once_time_passed`, `invalid_context_mode`, `reuse_thread_requires_thread`, `thread_not_found`, `invalid_assistant`, `unknown_assistant`, `invalid_goal`, `goal_requires_fresh_thread`, `invalid_stop_condition`, `invalid_max_runs`, `end_at_in_past`, `end_at_before_first_run`, `frequent_requires_limit`, `max_runs_not_above_used`, `limits_exhausted`, `task_not_found`, `task_running`, `run_queued`, `task_changed`, `task_finished`, `task_quota_exceeded`, `scheduler_not_running`, `scheduler_unavailable` (503: this Gateway has no scheduled-task persistence), `trigger_failed`. Codes only the chat capability
returns: `timezone_required`, `authentication_required`, `permission_denied`, `scheduler_tools_disabled`, `authority_expired`, `conversation_not_found`, `interactive_run_required`, `unsupported_action`, `unsupported_fields`, `task_id_required`, `note_not_verbatim`, `note_limit_reached`, `trial_requires_direct_request`, `no_stop_authority`, `occurrence_not_active`. The machine-readable list is
`contracts/scheduled_task_errors_contract.json`.
**Feature flag.** `GET /api/features` includes:
```json
{ "scheduled_tasks": { "available": true, "running": true, "tool_enabled": false, "min_interval_seconds": 60 } }
```
`available`: the task APIs have persistence. `running`: this process's
scheduler poller runs (tasks fire and can be created). `tool_enabled`: chats
can manage tasks and scheduled runs can stop their own schedule.
`min_interval_seconds`: shortest interval and earliest one-time delay.
**Browser timezone for chat-created tasks.** A run request may carry
`context.client_timezone` (an IANA name, at most 64 characters, for example
`"Asia/Shanghai"`). It is read only as the default timezone of tasks the
`schedule_task` tool creates in that turn; an unknown or malformed value is
ignored, and it never reaches the run config, the checkpoint or the prompt.
**Scheduled run messages.** A scheduled run's prompt message has the id
`scheduled-<task_run_id>` and carries `additional_kwargs.deerflow_scheduled_origin`
(`task_id`, `task_run_id`, `trigger`, `run_number`, `scheduled_for`, `timezone`,
`schedule_type`, `task_title`, `instructions`, `stop_condition`,
`standing_notes`), the user-written parts a client shows instead of the
launched text. Show an `interval` run's time in the viewer's zone (its stored
`timezone` may be the `"UTC"` placeholder of a task created without one) and
other runs in `timezone`. A new run conversation is titled
`"{task title} · MM-DD HH:MM"` in the task's zone, or `"{task title} · #{run}"`
when that zone is only the placeholder. The key is server-owned: it is
stripped from client-supplied messages and state updates.
**IM notices ("scheduled task updates").** With `channel_connections.enabled`,
each item of `GET /api/channels/providers` carries
`proactive_notifications` (bool): whether scheduled-task updates are pushed to
that app. Only providers with proactive push (WeCom today) get notices; the
others get none, and Settings says so. An occurrence sends at most one message,
in the owner's `locale` preference (see Account Preferences), else
`channel_connections.notification_locale`. Notices carry no IDs and no links;
see `backend/docs/CONFIGURATION.md` for the events and the merge rule.
```json
{ "provider": "wecom", "display_name": "WeCom", "connection_status": "connected", "proactive_notifications": true }
```
---
## Error Responses
Most APIs return errors in this format (scheduled-task routes and skill export
return a coded object instead; see [Scheduled Tasks](#scheduled-tasks)):
```json
{
"detail": "Error message describing what went wrong"
}
```
**HTTP Status Codes:**
- `400` - Bad Request: Invalid input
- `404` - Not Found: Resource not found
- `422` - Validation Error: Request validation failed
- `500` - Internal Server Error: Server-side error
---
## Authentication
DeerFlow supports four HTTP identity sources. They share the same thread/run isolation rules but differ in whether a row is created in `users` and how external identities are mapped. See [AUTH_DESIGN.md](AUTH_DESIGN.md) for the full design.
| Model | Entry | `users` table | Isolation key |
|---|---|---|---|
| Browser session | `access_token` cookie after login/register | Yes | `users.id` |
| OIDC / SSO | OAuth callback → cookie | Yes | `users.id` (see [SSO.md](SSO.md)) |
| IM channel binding | Connect code + `channel_connections` | Bound to registered user | `channel_connections.owner_user_id` |
| **Internal Auth** | `X-DeerFlow-Internal-Token` + `X-DeerFlow-Owner-User-Id` | **No** | Owner string on `threads_meta.user_id` |
**IM channel binding** and **Internal Auth** are both *platform-trust* integrations: DeerFlow trusts the channel/platform to authenticate end users. IM bindings persist the mapping in `channel_connections` / `channel_conversations` and require a DeerFlow `users` row. Internal Auth lets a platform call the Gateway API directly with a deployment-shared token and a per-request owner header—no `users` row, but thread/run/checkpoint isolation works the same way.
### Browser session (default)
DeerFlow enforces authentication for all non-public HTTP routes. Public routes are limited to health/docs metadata and these public auth endpoints:
- `POST /api/v1/auth/initialize` creates the first admin account when no admin exists.
- `POST /api/v1/auth/login/local` logs in with email/password and sets an HttpOnly `access_token` cookie.
- `POST /api/v1/auth/register` creates a regular `user` account and sets the session cookie.
- `POST /api/v1/auth/logout` clears the session cookie.
- `GET /api/v1/auth/setup-status` reports whether the first admin still needs to be created.
The authenticated auth endpoints are:
- `GET /api/v1/auth/me` returns the current user.
- `POST /api/v1/auth/change-password` changes password, optionally changes email during setup, increments `token_version`, and reissues the cookie.
Protected state-changing requests also require the CSRF double-submit token: send the `csrf_token` cookie value as the `X-CSRF-Token` header. Login/register/initialize/logout are bootstrap auth endpoints: they are exempt from the double-submit token but still reject hostile browser `Origin` headers.
User isolation is enforced from the authenticated user context:
- Thread metadata is scoped by `threads_meta.user_id`; search/read/write/delete APIs only expose the current user's threads.
- Thread files live under `{base_dir}/users/{user_id}/threads/{thread_id}/user-data/` and are exposed inside the sandbox as `/mnt/user-data/`.
- Memory and custom agents are stored under `{base_dir}/users/{user_id}/...`.
Note: MCP outbound connections can still use OAuth for configured HTTP/SSE MCP servers; that is separate from DeerFlow API authentication.
### Internal Auth (platform HTTP integration)
For server-to-server integrations (e.g. a Feishu or WeCom/Enterprise WeChat bot backend), configure:
```bash
export DEER_FLOW_INTERNAL_AUTH_TOKEN="<long-random-secret>"
```
| Header | Required | Description |
|---|---|---|
| `X-DeerFlow-Internal-Token` | Yes | Must match `DEER_FLOW_INTERNAL_AUTH_TOKEN`; missing/invalid → `401` |
| `X-DeerFlow-Owner-User-Id` | Yes for per-user isolation | Platform user id (e.g. `feishu_ou_alice`, `wecom_user_bob`); omit → `default` bucket |
Does **not** use browser cookies or CSRF tokens. Does **not** insert into `users`; sets `threads_meta.user_id` / `runs.user_id` from the owner header. DeerFlow validates only the platform token—not whether the owner id represents a real end user; user validity is entirely the platform's responsibility. See [AUTH_DESIGN.md — Internal Auth](AUTH_DESIGN.md#internal-auth-direct-http) for trust boundaries, persistence, and security notes.
Use the standard Gateway thread/run endpoints (`POST /api/threads`, `POST /api/threads/{thread_id}/runs/stream`, etc.) with the headers above on every request.
---
## Rate Limiting
No rate limiting is implemented by default. For production deployments, configure rate limiting in Nginx:
```nginx
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
location /api/ {
limit_req zone=api burst=20 nodelay;
proxy_pass http://backend;
}
```
---
## Streaming Support
Gateway's LangGraph-compatible API streams run events with Server-Sent Events (SSE).
**Thread-scoped streaming** (thread must exist):
```http
POST /api/langgraph/threads/{thread_id}/runs/stream
Accept: text/event-stream
```
**Stateless streaming** (no pre-created thread; Gateway auto-creates one):
```http
POST /api/langgraph/runs/stream
Accept: text/event-stream
```
Both endpoints return `Content-Location: /api/threads/{thread_id}/runs/{run_id}`.
The DeerFlow web UI and LangGraph SDK clients rely on this header to discover the
assigned `thread_id` and `run_id` on the first message of a new chat.
Every SSE response (both create endpoints above, `GET /api/threads/{thread_id}/runs/{run_id}/join`
and `GET`/`POST /api/threads/{thread_id}/runs/{run_id}/stream`) carries the same headers:
```http
Content-Type: text/event-stream
Cache-Control: no-cache, no-transform
Connection: keep-alive
X-Accel-Buffering: no
```
`no-transform` keeps compressing proxies (for example the Next.js rewrite proxy
when the frontend runs with `pnpm start` and no nginx) from gzipping and therefore
buffering the stream; `X-Accel-Buffering: no` does the same for nginx. A reverse
proxy you put in front of the Gateway should not compress `text/event-stream`.
### SSE replay retention and gaps
Clients may reconnect to a run stream with `Last-Event-ID`. Replay history is
bounded by `stream_bridge.queue_maxsize` (default `256`) and, for Redis, by the
rolling `stream_ttl_seconds`. A retained cursor resumes after that event with no
additional control frame.
When a syntactically valid cursor is older than the retained watermark, the
server sends exactly one `gap` event before any retained data and closes that
subscription without an `end` event:
```text
event: gap
data: {"code":"stream_replay_gap","run_id":"run-123","requested_event_id":"1718000000000-1","earliest_available_event_id":"1718000000100-42","latest_available_event_id":"1718000000200-84","recovery":"reload_durable_state"}
```
The frame deliberately has no SSE `id:`. Both `earliest_available_event_id` and
`latest_available_event_id` are `string | null` (they are `null` when no events
are retained in the buffer). Consumers must reload durable thread state and
persisted run events/messages, then may reconnect from `latest_available_event_id`
to follow newer live events, or rejoin without a cursor when the buffer is empty
(`latest_available_event_id` is `null`). A gap does not cancel the active run.
The same signal applies when a no-cursor subscriber has already established an
empty-stream wait but the first Redis wake-up falls behind before delivery; in
that case `requested_event_id` is `null`. Malformed cursor handling is
backend-specific and is not the same as a valid cursor that was evicted.
---
## SDK Usage
### Python (LangGraph SDK)
```python
from langgraph_sdk import get_client
client = get_client(url="http://localhost:2026/api/langgraph")
run_meta: dict[str, str] = {}
def on_run_created(meta) -> None:
# langgraph-sdk 0.3.x parses Content-Location only when this callback is set.
if meta.thread_id:
run_meta["thread_id"] = meta.thread_id
run_meta["run_id"] = meta.run_id
# Option A: stateless stream — no thread pre-creation
# Gateway auto-creates a thread and returns thread_id/run_id in Content-Location.
async for event in client.runs.stream(
None,
"lead_agent",
input={"messages": [{"role": "user", "content": "Hello"}]},
config={"configurable": {"model_name": "gpt-4"}},
stream_mode=["values", "messages-tuple", "custom"],
on_run_created=on_run_created,
):
print(event)
thread_id = run_meta["thread_id"] # persist before the next turn
# Option A (continued): same thread on the next turn
async for event in client.runs.stream(
None,
"lead_agent",
input={"messages": [{"role": "user", "content": "What did I just ask?"}]},
config={"configurable": {"thread_id": thread_id, "model_name": "gpt-4"}},
stream_mode=["values", "messages-tuple", "custom"],
on_run_created=on_run_created,
):
print(event)
# Option B: thread-scoped stream — create thread first, then stream
thread = await client.threads.create()
async for event in client.runs.stream(
thread["thread_id"],
"lead_agent",
input={"messages": [{"role": "user", "content": "Hello"}]},
config={"configurable": {"model_name": "gpt-4"}},
stream_mode=["values", "messages-tuple", "custom"],
on_run_created=on_run_created,
):
print(event)
```
### JavaScript/TypeScript
```typescript
// Using fetch for Gateway API
const response = await fetch('/api/models');
const data = await response.json();
console.log(data.models);
function parseRunLocation(contentLocation: string | null) {
if (!contentLocation) return null;
const match = /\/threads\/([^/]+)\/runs\/([^/]+)/.exec(contentLocation);
if (!match) return null;
return { threadId: match[1], runId: match[2] };
}
// Option A: stateless stream — no thread pre-creation
let threadId: string | undefined;
const firstResponse = await fetch("/api/langgraph/runs/stream", {
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "text/event-stream",
},
body: JSON.stringify({
input: { messages: [{ role: "user", content: "Hello" }] },
stream_mode: ["values", "messages-tuple", "custom"],
}),
});
const created = parseRunLocation(firstResponse.headers.get("Content-Location"));
threadId = created?.threadId;
console.log("thread_id:", created?.threadId, "run_id:", created?.runId);
// Option B: continue the same thread on the next turn
const followUpResponse = await fetch("/api/langgraph/runs/stream", {
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "text/event-stream",
},
body: JSON.stringify({
input: { messages: [{ role: "user", content: "What did I just ask?" }] },
config: { configurable: { thread_id: threadId } },
stream_mode: ["values", "messages-tuple", "custom"],
}),
});
// Option C: thread-scoped stream when you already have a thread_id
const streamResponse = await fetch(`/api/langgraph/threads/${threadId}/runs/stream`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Accept: "text/event-stream",
},
body: JSON.stringify({
input: { messages: [{ role: "user", content: "Hello" }] },
stream_mode: ["values", "messages-tuple", "custom"],
}),
});
const reader = streamResponse.body?.getReader();
// Decode and parse SSE frames from reader in your client code.
```
### cURL Examples
```bash
# List models
curl http://localhost:2026/api/models
# Get MCP config
curl http://localhost:2026/api/mcp/config
# Upload file
curl -X POST http://localhost:2026/api/threads/abc123/uploads \
-F "files=@document.pdf"
# Enable skill
curl -X PUT http://localhost:2026/api/skills/pdf-processing \
-H "Content-Type: application/json" \
-d '{"enabled": true}'
# Stateless stream — no thread pre-creation
curl -s -D - -N -X POST http://localhost:2026/api/langgraph/runs/stream \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"input": {"messages": [{"role": "user", "content": "Hello"}]},
"config": {
"recursion_limit": 100,
"configurable": {"model_name": "gpt-4"}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}'
# Read Content-Location: /api/threads/{thread_id}/runs/{run_id} from the headers.
# Continue the same thread on the next turn
curl -s -N -X POST http://localhost:2026/api/langgraph/runs/stream \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"input": {"messages": [{"role": "user", "content": "What did I just ask?"}]},
"config": {
"configurable": {"thread_id": "abc123", "model_name": "gpt-4"}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}'
# Thread-scoped flow — create thread first, then stream
curl -X POST http://localhost:2026/api/langgraph/threads \
-H "Content-Type: application/json" \
-d '{}'
curl -X POST http://localhost:2026/api/langgraph/threads/abc123/runs/stream \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"input": {"messages": [{"role": "user", "content": "Hello"}]},
"config": {
"recursion_limit": 100,
"configurable": {"model_name": "gpt-4"}
},
"stream_mode": ["values", "messages-tuple", "custom"]
}'
```
> The unified Gateway path defaults `config.recursion_limit` to 100 for
> plan-mode and subagent-heavy runs. Clients may still set
> `config.recursion_limit` explicitly — see the [Create Run](#create-run)
> section for details. Scheduled-task launches use
> `scheduler.recursion_limit` from `config.yaml` instead of a client body.
## Chat archive and restore
`POST /api/threads/search` accepts `archived: true` for archived chats or
`archived: false` for recent chats (including legacy rows without an archive flag).
Omit the field or use null to include both. Filtering applies before `limit` and
`offset` and is scoped to the authenticated user. Combine it with the existing
`metadata` and `status` filters when needed.
Archive with `PATCH /api/threads/{thread_id}` and body
`{"metadata":{"deerflow_archived":true}}`; use false to restore. The flag must be
a JSON boolean. Writes containing only boolean pin/archive flags preserve
`updated_at` and all other metadata. The owner-checked endpoint returns the normal
thread metadata response; original thread and artifact URLs remain available.
Archiving does not cancel runs, pause schedules, or change retention.