deer-flow/backend/packages/harness/deerflow/subagents/AGENTS.md

Subagent System (packages/harness/deerflow/subagents/)

Built-in Agents: general-purpose (all tools except task) and bash (command specialist) Registry and managed definitions: Runtime resolution is built-in → config.yaml custom_agents → enabled administrator-managed definitions, followed by explicit subagents.agents.<name> overrides. Managed definitions are deployment-wide, persist through the same agent_storage.backend selection as Custom Agent definitions, and remain stored but are excluded from runtime when a built-in or later-added config definition owns the same name. The default Lead Agent sees the whole enabled catalog. A Custom Agent's allowed_subagents is snapshotted into run metadata (None = all, [] = hard deny, list = allowlist) and must filter both prompt discovery and task execution; never reload caller policy from mutable agent config inside the tool. Benefit-based routing policy: Enabling subagents exposes delegation as an optimization, not a default response to complexity. The lead prompt defaults to direct execution and permits task only when parallel latency, specialist capability, or context-isolation benefit clearly exceeds startup, duplicate-discovery, synthesis, state-conflict, and side-effect costs. Inter-agent output dependencies and overlapping mutable state are hard vetoes for parallel dispatch, while duplicate discovery and a cheap direct path remain costs rather than categorical vetoes; a bounded sequential chain may run in one subagent when specialist or context-isolation benefit clearly wins. Parallel scopes must be independent and non-overlapping, the lead uses the fewest useful subagents, and every later batch is re-evaluated while retaining any within-batch parallel benefit. When the enforced per-response limit is 1, the rendered prompt removes parallel and multi-batch benefit guidance and permits delegation only for material specialist or context-isolation benefit. Keep this policy aligned across lead_agent/prompt.py, the task tool description, and both built-in role descriptions; routing regressions are pinned in tests/test_subagent_routing_prompt.py, tests/test_subagent_prompt_security.py, and tests/test_lead_agent_prompt.py. User-scoped Skills: Subagents resolve their configured skills through get_or_new_user_skill_storage(user_id) using the parent runtime identity, with DEFAULT_USER_ID only when no identity is available. This keeps custom-skill shadowing and visibility aligned with the lead agent instead of reading the global-only catalog. Date context (#4781): Every built-in subagent execution registers SubagentDateContextMiddleware immediately before SystemMessageCoalescingMiddleware. Its one-time before_agent hook adds a hidden framework-owned SystemMessage containing only <current_date> before the first model call; it does not read AppConfig.memory, call the memory manager, rewrite the task HumanMessage, or inherit the lead agent's frozen-conversation/midnight lifecycle. The coalescer merges that reminder with the subagent's static prompt so strict providers still receive exactly one leading SystemMessage. The lead-only DynamicContextMiddleware registration and its date, optional-memory, and midnight-update behavior remain unchanged. Execution: Ordinary and durable-batch native subagents submit coroutines directly to one persistent isolated event loop. Gateway/embedded startup installs one process-wide async FIFO admission controller (default 3 running, bounded queue). Direct create_deerflow_agent callers can instead pass a caller-owned SubagentRuntime; reuse the same instance across graphs so its bound task, optional batch tools/service, middleware limits, and SubagentExecutor all share one controller without reading global YAML. An owned batch service must be started before graph construction and stopped at application shutdown. Waiters hold no scheduler thread, and cancellation/timeout release queue/slot ownership. Concurrency and total delegation cap: Ordinary task concurrency is resolved once as the minimum of the per-run request, the startup-frozen subagent_runtime.max_running, and the schema safety ceiling (1-64), then shared by the lead prompt and SubagentLimitMiddleware. Hot reloads must not make either layer advertise more capacity than the already-created process controller; a changed startup-only value takes effect only after restart. The same middleware separately enforces subagents.max_total_per_run (default 6, config schema 1-50, runtime override max_total_subagents clamped to the same range) against current-run entries in the durable delegation ledger, so a long lead-agent run cannot bypass concurrency limits by launching repeated legal-sized batches at each planning checkpoint, but historical delegations from previous runs in the same thread do not consume the new run's budget. Explicit batch_task work does not consume or relax that ordinary-run ledger: its persisted total/live/running limits live under subagent_batches. Gateway run_agent() and embedded DeerFlowClient.stream() both provide a per-invocation run_id in runtime context; DeerFlowClient.stream() also tags its input HumanMessage with that same id so durable-context capture can identify the current request boundary. Gateway resume paths may not append a new HumanMessage, so the worker also exposes the pre-run checkpoint's message ids in runtime context; durable-context capture uses that as the current-run boundary and never re-tags older task calls as the resumed run. When no delegation slots remain, task calls are stripped, provider raw tool-call metadata is synced, finish_reason is forced to stop, and a visible "subagent delegation limit" note is appended so the agent can synthesize already-collected results. Default subagent timeout subagents.timeout_seconds=1800 (30 min) and built-in general-purpose max_turns=150. Flow: Ordinary task() → SubagentExecutor → shared process slot → result polling/SSE. Explicit batch_task() → durable batch/item rows → lease-based batch service (subagents/batch_service.py, started by Gateway or an explicit direct runtime) → the same SubagentExecutor/process slots → bounded stored result and owner-scoped API/JSONL export. Batch mode is selected only by the explicit tool, never inferred from prompt size. Executor queue rejection/timeout occurs before model execution and therefore releases the durable lease without consuming an item attempt; real execution failure and expired leases still consume the retry budget. User cancellation terminalizes every nonterminal item immediately and clears its lease, fencing any stale worker completion. Background cancellation resolves the result/future under _background_tasks_lock but calls Future.cancel() only after releasing it, because cancellation may synchronously invoke the completion callback that reacquires the registry lock. Direct runtimes provide the tools and worker but not Gateway's HTTP/UI surface. task_started carries the resolved effective model name. The per-subagent SubagentTokenCollector publishes a cumulative usage snapshot to the shared SubagentResult after every completed LLM response; the next task_running event carries that snapshot, so collapsed workspace cards can update without re-accounting parent-run totals. Terminal ToolMessage metadata (subagent_model_name, subagent_token_usage) and the persisted subagent.end event retain the model/usage after reload; absent provider usage stays absent rather than being estimated as zero. The executor caches one resolved AppConfig snapshot (explicit or get_app_config() fallback) for agent assembly, deferred setup, and receipt harvesting, so verification.receipts_enabled=false remains authoritative on both construction paths. Terminal tool receipts are harvested before try_set_terminal and committed with the other payload fields under the same state lock, so status polling cannot observe a terminal result before its receipt metadata is available. Each yielded values chunk becomes the latest terminal-harvest state and immediately publishes its harvested receipts to the shared result before cooperative cancellation is checked. Tool-ended cancellation/failure evidence uses the current ToolMessage scan, but a completed result always uses the bounded ledger snapshot attached to the assistant text being returned—even when a max-turn partial ends on a later tool chunk—so omitted receipts cannot validate its citations; a missing/malformed completed snapshot fails closed with no receipts. Therefore direct task cancellation and both execution/polling timeouts retain the latest execution evidence even when cancellation interrupts before another stream boundary. Report contract (RFC #4651 PR3): report_contract.py owns the prompt-layer text that makes Layer 1 receipt verification non-inert. SubagentExecutor._build_initial_state appends build_report_contract_section(receipts_enabled=...) to every subagent's consolidated SystemMessage — built-in and custom alike — requiring [rN tool_name] citations (from the Tool receipts ledger) for action claims, verifiable handles (absolute path, URL, ID, HTTP status) for deliverables, and explicit reporting of failures; the citation clause follows verification.receipts_enabled, and the citation example derives from the single-owner format_citation/receipt_id so prompt text cannot drift from the verifier. The task tool hands lead-supplied acceptance_criteria to the SubagentExecutor constructor, which appends them via render_acceptance_criteria_block(...) to the task HumanMessage (stripped, capped at 20 items × 500 chars, each entry neutralized) — the untrusted channel InputSanitizationMiddleware escapes and boundary-frames, matching their model-supplied provenance. The subagent's SystemMessage never carries criterion text; it gets only the framework-owned build_acceptance_criteria_system_note(...) pointer naming the list's location and authority, so natural-language injection inside a criterion cannot gain system-channel priority over framework instructions. Deterministic leaf checking is a separate layer. Events: task_started, task_running, task_completed/task_failed/task_timed_out Handled LLM failures: LLMErrorHandlingMiddleware deliberately converts provider/model exceptions into an AIMessage so the graph can end cleanly, stamping additional_kwargs.deerflow_error_fallback=true plus error metadata. Clean graph termination does not imply subagent success: SubagentExecutor inspects the last assistant message at terminalization and maps a marked fallback to SubagentStatus.FAILED, which then emits task_failed and the existing structured subagent_error. Only the marker is authoritative — error-looking assistant prose without it remains a normal completed result, so neither the executor nor frontend parses display text as a status protocol. Guardrail caps & stop_reason (#3875 Phase 2): three independent axes can end a subagent run early, and all now surface why through one additive field rather than a new status enum. Turn axis: recursion_limit on the subagent run_config equals max_turns, so exhausting the turn budget raises GraphRecursionError from agent.astream; executor.py::_aexecute catches it specifically (before the generic except Exception). Token axis: TokenBudgetMiddleware is attached per-agent via build_subagent_runtime_middlewares from subagents.token_budget (default max_tokens coupled to summarization.enabled — 1,000,000 when subagent summarization is on, 2,000,000 when off, warn at 0.7, hard-stop at 1.0; a user-set budget always wins regardless of the switch — #3875 Phase 3; a backstop against a subagent that burns tokens on trivial work). It does not raise: at the hard-stop threshold it strips the in-flight turn's tool calls, forces finish_reason="stop", and lets the run complete naturally with a final answer. Loop axis: LoopDetectionMiddleware (attached at the same point) catches repeated identical tool-call sets — or one tool type called many times with varying args — and its hard-stop likewise strips tool_calls and forces a final answer without raising, recording loop_capped. Each guard exposes its cap on a per-run_id consume_stop_reason(run_id) accessor; _aexecute collects every middleware with that method (duck-typed via hasattr, so the executor has no import coupling to the guard classes) and surfaces the first non-None reason — adding a future guard needs no executor change. Surfacing: whichever axis fired, _aexecute stamps a normal status plus an additive reason — completed + stop_reason=token_capped|turn_capped|loop_capped when a usable final answer (or partial recovered from the last streamed chunk via _extract_final_result → utils/messages.py::message_content_to_text, returning a "No response Generated" sentinel when no text survived) was produced; failed + stop_reason=turn_capped when nothing usable survived. SubagentResult.stop_reason flows through task_tool.py::_task_result_command → format_subagent_result_message (renders Task Succeeded (capped: ...) / Task failed (capped: ...)) and make_subagent_additional_kwargs, which stamps the additive subagent_stop_reason key alongside the normal subagent_status. Why additive, not an enum: a new status value would break v1 consumers; an optional field is ignored by older frontends and ledger readers, so the cross-language contract (contracts/subagent_status_contract.json v2 + subagents/status_contract.py + frontend/.../subtask-result.ts, pinned by test_status_values_match_contract / test_stop_reason_values_match_contract) stays backward-compatible. The durable delegation ledger captures stop_reason onto the entry and renders model-facing guidance ("hit a guardrail cap with a partial result; reuse it, retry tighter, or raise the per-agent budget (max_turns / token_budget)") so the lead reuses a capped completion knowingly instead of mistaking it for a clean one. (Phase 1 shipped this surfacing as a MAX_TURNS_REACHED status enum in #3949; Phase 2 replaced that enum with the additive stop_reason field per the agreed design — the max_turns_reached status value and SubagentStatus.MAX_TURNS_REACHED are gone.) Context compaction (#3875 Phase 3, #4039): subagents inherit DeerFlowSummarizationMiddleware via build_subagent_runtime_middlewares, gated on the same summarization.enabled switch the lead reads (one config covers both chains; trigger/keep/model/prompt come from the shared summarization config so they cannot drift). The subagent builder attaches DurableContextMiddleware immediately before summarization, using the same skills path/read-tool settings as the lead chain. Compaction stores the generated summary in ThreadState.summary_text rather than as a messages item; the durable-context wrapper therefore projects it into the next model request as guarded hidden human data. This is required when a message-count keep policy preserves only an assistant tool-call plus its tool results: without the injected summary the next request begins with assistant/tool history and strict OpenAI-compatible providers can reject it. Because DurableContextMiddleware inserts a second SystemMessage(authority_contract) after the subagent's leading system prompt, the builder also appends SystemMessageCoalescingMiddleware innermost (mirroring the lead chain, appended after the optional summarization middleware so it is unconditionally last) to merge every SystemMessage into one leading system_message — otherwise the durable fix would trade #4039's assistant-first HTTP 400 for a duplicate-system 400 on the same strict backends (#4040). The factory is called with skip_memory_flush=True on the subagent path: the lead's memory_flush_hook (attached when memory.enabled) flushes pre-compaction messages into durable memory keyed by thread_id, and subagents share the parent's thread_id, so without skipping the hook a subagent's internal turns would pollute the parent thread's durable memory. Placement differs from the lead chain (lead appends summarization before the guard trio; subagent appends it after) — benign because the middleware implements only before_model (compaction) with no after_model/consume_stop_reason, so it cannot disturb the Phase 2 guard-cap stop-reason channel. Compaction rewrites the messages channel via RemoveMessage(id=REMOVE_ALL_MESSAGES), which shrinks len(messages) below the step-capture cursor mid-run; capture_new_step_messages (see Step capture below) resets the cursor to the new tail on contraction so steps appended after the compaction point are not silently dropped. Step capture & persistence (#3779): executor.py captures both assistant turns (AIMessage) and tool outputs (ToolMessage) via subagents/step_events.py::capture_new_step_messages, which walks the newly-appended tail of each stream_mode="values" chunk (not just messages[-1]) so a multi-tool-call turn — where LangGraph's ToolNode appends several ToolMessages in one super-step — keeps every tool output instead of dropping all but the last. runtime/runs/worker.py::_SubagentEventBuffer additionally persists these task_* custom events to the RunEventStore as subagent.start/subagent.step/subagent.end (category="subagent", task_id in metadata). It batches writes via put_batch (flushing on a terminal subagent.end, at FLUSH_THRESHOLD events, and in the worker's finally) rather than one put() per step, since put() is a documented low-frequency path (per-thread advisory lock per call) and a deep subagent (max_turns=150) emits hundreds of steps on the hot stream loop. subagent_run_event rejects malformed chunks that lack a non-empty task_id; running chunks additionally require a non-negative integer message_index and a message object, so persisted records always satisfy the required lifecycle envelope. build_subagent_step caps both the per-step text and each tool call's serialized args at SUBAGENT_STEP_MAX_CHARS (flagged truncated / args_truncated) so a large write_file/bash payload can't produce an unbounded row. The dedicated category keeps them out of list_messages (the thread feed) while list_events returns them for the frontend's fetch-on-expand backfill. list_events accepts task_id (filters on metadata["task_id"] — SQL-side in DbRunEventStore via event_metadata["task_id"].as_string(), in-memory in the JSONL/memory stores) plus an after_seq forward cursor, so the card pages through one subagent's steps without the run-wide limit truncating the tail (no schema migration: the filter rides the existing run-scoped index). step_events.py is a pure, unit-tested layer (build_subagent_step / subagent_run_event). History contraction (#3875 Phase 3): capture_new_step_messages assumes append-only growth, but DeerFlowSummarizationMiddleware rewrites the messages channel via RemoveMessage(id=REMOVE_ALL_MESSAGES), shrinking len(messages) below the cursor mid-run. On contraction (total < processed_count) the cursor resets to the new tail; capture_step_message's id/content dedup prevents re-emitting pre-compaction steps, so steps appended after the compaction point are still captured instead of being dropped until total overtakes the stale cursor. Deferred MCP tools (if tool_search.enabled): SubagentExecutor._build_initial_state applies the subagent name allow/deny list and assembly-time authorization before calling the shared assemble_deferred_tools, appends the tool_search tool, injects the <available-deferred-tools> section into the subagent's SystemMessage, and threads the setup to _create_agent, which attaches McpRoutingMiddleware (when PR1 routing metadata matches deferred tools) before DeferredToolFilterMiddleware through build_subagent_runtime_middlewares(...). Runtime skill policy is intentionally later and dynamic: tool_search may disclose/promote catalog metadata, but SkillToolPolicyMiddleware still removes or blocks any promoted business tool omitted by the active skill. Subagents thus withhold full MCP schemas until promotion, same as the lead agent; each task run gets a fresh ThreadState so promotion is isolated per run Checkpointer isolation: Subagent graphs are compiled with checkpointer=False to avoid inheriting the parent run's checkpointer, since subagents are one-shot and never resume. Checkpoint lineage / stream isolation: _aexecute deliberately omits checkpoint-coordinate keys (thread_id, checkpoint_ns, checkpoint_id, checkpoint_map) from the child RunnableConfig. LangGraph must inherit those coordinates from the copied parent ContextVar so the delegated graph retains a non-root subgraph namespace; explicitly re-supplying even the same parent thread_id starts a new root lineage on LangGraph 1.2.6+ and can route child AI/tool frames into the parent messages stream. DeerFlow business components still receive the parent thread_id through runtime.context, which is the preferred lookup path for sandbox, middleware, and attribution code. Regression coverage in tests/test_subagent_executor.py::TestSubagentCheckpointLineage keeps the invocation-contract assertion active on every supported version and version-gates the production-shaped parent-stream test to LangGraph 1.2.6+, where the leak exists.

Isolated-loop callback boundary: sync delegation from an active event loop and execute_async() copy the ambient ContextVars into the persistent subagent loop so checkpoint lineage, user identity, tracing context, tags, metadata, and LangGraph's namespaced message-stream handler survive. Before submission, _copy_isolated_subagent_context() copies the callback manager/list and removes only handlers marked deerflow_loop_bound; RunJournal carries that marker because it owns parent-loop tasks and a SQL store/pool. LangGraph merges inherited callbacks with the child run's explicit SubagentTokenCollector/tracing callbacks, so letting RunJournal cross loops causes duplicate accounting and Future attached to a different loop failures, while dropping the whole callback chain silently removes child token frames. Do not replace the boundary with a blank Context; the inherited checkpoint namespace and framework stream callback are required by the stream-isolation contract above.