* feat(observability): persist deferred tool promotions Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> * fix(ci): trim agent guidance chain Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> --------- Signed-off-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com> Co-authored-by: PeaceMaker-best <221849497+PeaceMaker-best@users.noreply.github.com>
36 KiB
Middleware Chain
Persisted delegation verdicts are untrusted durable context; ledger rendering revalidates them and ignores malformed values.
Lead-agent middlewares are assembled in strict order across three functions: the shared base in packages/harness/deerflow/agents/middlewares/tool_error_handling_middleware.py (_build_runtime_middlewares, exposed via build_lead_runtime_middlewares), then the lead-only middlewares appended in packages/harness/deerflow/agents/lead_agent/agent.py (build_middlewares). Items marked (optional) are appended only when their config/runtime condition holds, so the live chain length varies.
Message provenance. A middleware that injects or rewrites a message stamps
additional_kwargs with the neutral provenance keys from
deerflow_extension_api.provenance (deerflow_content_kind,
deerflow_producer_kind, and optionally deerflow_producer_entity_id) via
provenance_kwargs(). The producer is not recoverable downstream — by the
model-call boundary the message is indistinguishable from any other — so the
fact is recorded where it is known. Stamping is unconditional: a fact whose
presence depends on whether an observer is installed is not a fact. All three
keys are in _SERVER_OWNED_MESSAGE_METADATA_KEYS, so a caller cannot forge
provenance on inbound messages. Currently stamped by: DynamicContext (reminder + memory),
DurableContext (contract + data), SystemMessageCoalescing, ViewImage,
SkillActivation. Summarization, Title, and Memory are deliberately absent:
Summarization's and Title's own model calls are already attributed through
system-model-call observation (SystemOperationKind.SUMMARIZATION /
.TITLE), and the summary text they produce only ever enters a request via
DurableContextMiddleware's already-stamped durable_context_data block —
there is no separate message of theirs to stamp. Memory only reads
messages to queue them for extraction; the recalled-memory content that
actually re-enters context is DynamicContext's dynamic_context_memory
stamp, not anything Memory itself produces.
Middleware self-description. A middleware whose configuration changes agent
behaviour implements release_policy_parameters() -> dict[str, object]
(deerflow_extension_api.release.ReleasePolicyProvider, duck-typed — no base
class). Values must be JSON-serialisable; long text is hashed with
canonical_hash rather than embedded, because a declaration is an identity and
not a copy of the prompt. collect_release_policies() gathers them from an
assembled stack. Adding a behaviour-affecting field to a middleware means adding
it to that middleware's declaration in the same change.
Shared runtime base (build_lead_runtime_middlewares; subagents reuse most of this via build_subagent_runtime_middlewares):
-
InputSanitizationMiddleware - First, so it is the outermost
wrap_model_callwrapper; every inner middleware (including LLM retries) sees sanitized messages.additional_kwargs.original_user_contentis server-owned provenance: Gateway strips caller-supplied values for non-internal run requests, trusted IM calls may carry the string they captured before adding transport/file context, and the middleware replaces any non-string value before wrapping. Uploads and sanitization retain first-writer-wins only for validated strings. -
ToolOutputBudgetMiddleware - Caps tool output size (per app config) before it re-enters the model context. Oversized results are externalized to
tool_output.storage_subdir(default.tool-results, shared constantTOOL_RESULTS_DIRNAME) under the thread outputs dir with a typed synopsis +read_filereference left in context; those files are process feedback, so the workspace-changes scanner excludes that directory and run delivery verification never counts them as produced artifacts -
ToolResultSanitizationMiddleware - Neutralizes framework/injection tags (e.g.
<system-reminder>) and boundary markers in remote-content tool results (web_fetch/web_search/image_search/web_capture) so attacker-controlled fetched pages cannot forge trusted framework context. MirrorsInputSanitizationMiddleware's user-input guardrail for the other untrusted-content entry point; sits inner ofToolOutputBudgetMiddleware(neutralizes the raw output, then the budget truncates). Local tool output (bash/read_file) is left untouched. Scope is a name-based allowlist for the first-party web tools, plus every MCP-sourced tool via itsdeerflow_mcpmetadata tag, so an MCP server naming its fetcherfetch_urlis still coveredResult-rewriting middlewares between the raw callable boundary and the model-visible result append a declared entry to
additional_kwargs["deerflow_tool_transforms"]viaagents/middlewares/tool_transform_meta.py::append_tool_transform. The trail is ordered by application — the last entry produced the final visible bytes — so an observer classifies raw→visible transforms from facts rather than by sniffing output wording. -
ThreadDataMiddleware - Creates per-thread directories under the user's isolation scope (
backend/.deer-flow/users/{user_id}/threads/{thread_id}/user-data/{workspace,uploads,outputs}); resolves identity viaresolve_runtime_user_id(runtime), including Gateway runtime context and standalone LangGraph Server auth, then falls back to the request ContextVar /"default" -
UploadsMiddleware - Tracks and injects newly uploaded files into conversation (lead agent only); upload existence checks use the same runtime-resolved user bucket as thread-data creation
-
SandboxMiddleware - Acquires sandbox, stores
sandbox_idin state. The lead runtime normally owns the thread's physical Agent-skill projection; delegated subagents and the prompt-only bootstrap agent are non-owners, so their narrower discovery allowlists never rebuild the shared thread view or force eager sandbox acquisition. -
DanglingToolCallMiddleware - Injects placeholder ToolMessages for AIMessage tool_calls that lack responses (e.g., user interruption), preserving raw provider tool-call payloads in
additional_kwargs["tool_calls"]; malformed tool-call names and arguments are sanitized in the model-bound request so strict OpenAI-compatible providers do not reject the next request -
LLMErrorHandlingMiddleware - Normalizes provider/model invocation failures into recoverable assistant-facing errors before later stages run
-
Authorization / GuardrailMiddleware - Up to two independent pre-tool-call gates run here. When
authorization.enabled, theAuthorizationProviderinstance already used for Layer 1 capability filtering is wrapped byGuardrailAuthorizationAdapterand reused for Layer 2 execution checks. A generatedtool_searchbypasses the adapter's second provider call only when the current build has a concrete deferred setup; its catalog was already filtered by Layer 1, and an ordinary same-named tool without that deferred setup receives no exemption. Whenguardrails.enabled, the explicitly configuredGuardrailProvideris appended after authorization and still evaluates every call, includingtool_search. Authorization therefore runs outermost and can deny before an external guardrail call; both use the existing middleware's fail-closed, audit, sync/async, and error-ToolMessagebehavior. See the authorization RFC and docs/GUARDRAILS.md.Every guardrail decision path publishes a neutral
deerflow.authz.outcome.AuthorizationOutcomeinto the per-run runtime context, keyed bytool_call_idunder the__-prefixed__authorization_outcomekey (sobuild_run_configstrips caller-supplied forgeries). Consumers pop it; the publisher and the consumer share only that contract module. -
SandboxAuditMiddleware - Audits sandboxed shell/file operations before tool execution; command classification is defense-in-depth and audit, not a security boundary (the sandbox is the isolation boundary). Command substitution is judged by position, not the presence of
$(: command position ($(curl url),`curl url`, the word after|/&&/;, aneval/sourceargument) executes fetched content and is blocked; value position (x=$(curl url),echo $(curl url), an argument, aforword list) only captures output and passes (#4611). So_HIGH_RISK_COMMAND_POSITION_PATTERNSis matched anchored against each sub-command from_split_compound_command(split_pipes=True), never the whole string; pipe-spanning rules (| sh,base64 -d | ...) still use_classify_command's whole-command Pass 1._COMMAND_POSITION_PREFIXextends the anchor over leading assignments and exec wrappers (FOO=1 $(curl url),env/command/builtin/exec/nohup/time/sudo/doas); its assignment branch requires whitespace before the substitution, which keepsx=$(curl url)in value position. Two contexts are deliberately position-blind (matched whole-command in Pass 1, since they execute their input anywhere, e.g.xargs sh -c "$(curl url)"): aneval/sourceargument, and an interpreter code-string flag —-c(shells,python),-e(perl/ruby/node),-p(perl/node),-r(php) — plus the here-string (<<<) reaching the same place via stdin. All three substitution spellings ($(,<(,`) share one_RISKY_SUBSTITUTIONopener. An unquoted newline splits like;(elseecho hi\n$(curl url)evades the anchored rules). Heredoc bodies are data:_split_compound_commandrecords headers (<<EOF,<<-EOF,<<'EOF') and consumes their bodies verbatim, so a body line starting$(curl url)isn't promoted to a command position;<<<(here-string, needs look-ahead + look-behind) and a<<inside$(( ))/(( ))(bit shift, arithmetic depth tracked with the quote flags) must not open one. This is a heuristic, not shell parsing — an unterminated body consumes the rest of the string, an unclosed((only disables heredoc detection, and the failure direction is always toward more command positions, not fewer. Known gaps: process substitution outsideeval/source(. <(curl u)) is undetected, and two-step forms (x=$(curl u); eval "$x") need dataflow analysis. No config gate — appended unconditionally in_build_runtime_middlewares, for both lead and subagents. -
ReadBeforeWriteMiddleware - (optional, if
read_before_write.enabled, default on) Outermost write gate (issue #3857):read_filestamps a content hash onto its ToolMessage;write_file(append/overwrite-existing) andstr_replaceare blocked unless the newest mark for that path matches the file's current hash. Sits outside ToolProgressMiddleware and ToolErrorHandlingMiddleware so a blocked write returns immediately without consuming a ToolProgress slot. Blocked results callnormalize_tool_resultdirectly to stampdeerflow_tool_meta(recoverable_by_model=True) before returning, keeping the result well-formed for any outer consumer. Marks live on messages, so summarization dropping the read result invalidates the gate automatically; writes never refresh marks, forcing a re-read between consecutive edits. Gate check + tool execution are serialized per (thread, path) so same-turn parallel writes cannot reuse one stale mark; on sandboxes whoseread_filereports failures as"Error: ..."strings instead of raising (AIO/E2B), uninspectable targets fail open (creation proceeds, no mark stamped). The middleware also owns the sandbox authorization scope for these composed calls: pre-write inspection, the tool body, and post-read hashing share one sync/async provider decision, whileSandboxAuthorizationErrorbypasses the generic inspection fail-open paths and becomes an error ToolMessage. -
ToolProgressMiddleware - (optional, if
tool_progress.enabled) State-machine-based stagnation guard (RFC #3177). Outer wrapper around ToolErrorHandlingMiddleware so itswrap_tool_callreceives results already stamped withdeerflow_tool_meta. Tracks per-(thread, tool) consecutive "no-new-info" calls across three error categories: (a)recoverable_by_model=True(no_results, not_found, permission, Jaccard-duplicate success): ACTIVE → WARNED (terminal — hint re-injected on each subsequent problem); (b)recoverable_by_model=False, action≠stop(rate_limited, transient): ACTIVE → WARNED → BLOCKED afterwarn_escalation_countmore problems; (c)recoverable_by_model=False, action=stop(auth, config, internal): immediately BLOCKED on first occurrence. Division of labor with LoopDetectionMiddleware: ToolProgressMiddleware is a result-quality guard — fires after tool execution and blocks specific tools that stop producing new information; LoopDetectionMiddleware is a call-pattern guard — fires after the model responds and hard-stops the whole turn when the model repeatedly issues identical tool_calls. Both can inject HumanMessage hints in the same model call without conflict; neither reads the other's internal state. -
ToolReceiptMiddleware + ToolErrorHandlingMiddleware -
ToolReceiptMiddlewareis (optional, ifverification.receipts_enabled, default on). It is the outermostwrap_tool_calllayer — registered ahead of entries 9-12 — because Guardrail/SandboxAudit/ReadBeforeWrite/ToolProgress can short-circuit a call with their own ToolMessage (and SandboxAudit rebuilds medium-risk results); an inner receipt layer would silently gap the ledger on those results (ordering constraints indeerflow.extensions.ordering). Normal results still carry thedeerflow_tool_metastatus ToolErrorHandlingMiddleware stamps on the inner return path; short-circuit messages self-stamp meta or fall back tomessage.status. It stamps deterministic provenance (tool name, status, args/output hashes, byte count, timestamp) onto directToolMessageresults and every matchingToolMessagecarried inCommand.update.messages, including delegatedtask,present_file,view_image, andtool_searchresults; before model calls it derives a hidden receipt ledger (display ids r1..rN) from message state, and when the 2,000-character budget is exceeded the newest receipts are retained in chronological order with their original ids plus an older-receipts omission marker. Rendering returns both the text and its retained receipt subset; every response that received a ledger carries only that exact server-owned subset, never omitted receipts. Snapshot validation accepts a strictly consecutive positive original-id range (for exampler24–r30) rather than requiringr1, so subagent terminal citation verification resolves ids against evidence present in the citing turn even when later summarization drops and renumbers tool messages. Model-generated citation IDs are digit-bounded before integer conversion; oversized IDs are ignored as malformed input rather than raising through task write-back. Citation parsing deduplicates exact(id, anchor)pairs, not IDs alone, so repeated identical references stay compact while every distinct anchor claim is verified. Gateway strips delegated receipts/verdicts from external messages.ToolErrorHandlingMiddlewarereceivesAppConfig, converts tool exceptions into errorToolMessages so the run can continue instead of aborting, stamps every result withdeerflow_tool_meta(status / error_type / recoverable_by_model / recommended_next_action / source) viatool_result_meta.normalize_tool_result, stamps structured metadata for task exception wrappers, and stamps skill-read metadata for downstream durable-context capture. Task tool result text is generated from the same status/result/error inputs as the structured metadata so callers do not hand-write a second protocol string.
Authorization identity plumbing is independent of whether authorization enforcement is enabled. Gateway removes client-supplied is_internal / authz_attributes / channel_user_id, derives is_internal only from the server-owned request.state.auth_source, and accepts channel_user_id only from an internally authenticated IM caller's top-level body.context; free-form body.config can never supply it. build_principal_from_context is the shared Principal builder for assembly-time authorization and GuardrailAuthorizationAdapter; it applies default_role, strict-boolean internal provenance, and copy-on-read authz_attributes. The built-in RBAC provider validates authorization.default_role during provider resolution so an unknown fallback role fails agent construction instead of degrading into an empty tool set. Task delegation carries is_internal plus copied attributes through SubagentExecutor, while GuardrailMiddleware maps the same runtime fields into GuardrailRequest. Phase 1B applies Layer 1 before deferred-tool assembly on the lead, native-subagent, and embedded-client paths, then passes the same provider instance into Layer 2. Framework-provided describe_skill and memory tools are included in Layer 1 but restored to their legacy post-tool_search ordering afterward. DeerFlowClient.stream() treats its in-process caller as trusted and accepts the same identity fields as keyword overrides; it includes the complete Principal in its agent cache key and deep-copies nested attributes so caller mutation cannot make a stale tool set look current.
Gateway route authorization uses authz.py::resolve_route_permissions() as the single provider integration point for both AuthMiddleware and decorator-only authentication. When enabled, it evaluates the six registered threads:* / runs:* permissions as resource="route" requests whose targets are the full resource:action strings. Decisions use the async provider API and are cached for the request in AuthContext; decorators do not call the provider again. Provider resolution or decision errors follow authorization.fail_closed, scoped per permission for decision errors. When authorization is disabled, the legacy complete permission set is returned without resolving a provider. Existing owner_check enforcement and require_admin_user() management gates remain independent and unchanged. Tests: tests/test_authorization_route_permissions.py, tests/test_auth.py, and tests/test_auth_middleware.py.
Model authorization uses authz.py::resolve_model_authorization() (same cached-provider, internal-role, and principal-building path as route authorization) as the Gateway integration point for the models router: list_models filters names through filter_resources(principal, "model", ...), and get_model enforces authorize(resource="model", action="use") with a deny surfacing as 403; provider errors follow authorization.fail_closed (fail-open returns the unfiltered list / proceeds). At runtime, lead_agent/agent.py::_authorize_model_name — called from _make_lead_agent and from DeerFlowClient._ensure_agent — applies the same model:use check to the resolved model name. On deny it scans the filter_resources-visible names (excluding the denied model), re-verifying each candidate with authorize("model", "use") before falling back, because a custom provider may allow list while denying use; no usable fallback raises under fail_closed and keeps the original model under fail-open. The built-in RBAC provider maps this to the per-role models policy key. Tests: tests/test_models_authorization.py.
Sandbox authorization (sandbox:execute) gates every sandbox acquisition before provider.acquire — including this middleware's eager path (before_agent / abefore_agent skip acquisition on deny instead of raising, deferring to the lazy per-tool gate). See the sandbox module guide authorization-gate paragraph and tests/test_sandbox_authorization.py.
Before changing a later authorization phase, read the authorization RFC and its implementation notes. The notes are the cumulative handoff record for merged PR behavior, reviewer feedback, trust-boundary decisions, deferred scope, and required regression coverage.
Lead-only middlewares (build_middlewares, appended after the base):
- DynamicContextMiddleware - Injects the current date (and optionally memory) as a
<system-reminder>into the first HumanMessage, keeping the base system prompt fully static for prefix-cache reuse. The injected date follows the server-local timezone unlessDEER_FLOW_DATE_TIMEZONEnames an IANA zone (invalid values fall back to server-local). - SkillActivationMiddleware - Detects strict
/skill-name tasksyntax on the latest real user message, resolves only enabled and runtime-allowed skills, injects theSKILL.mdbody as hidden current-turn context, and records amiddleware:skill_activationaudit event - SkillToolPolicyMiddleware - Applies
allowed-toolsonly after real activation; passive enabled skills and a custom agent's configured skill allowlist do not clamp the lead toolset. A run-scoped slash activation is authoritative and suppressesskill_contextas a policy source, so reading another skill cannot widen the explicit skill's tools; without slash activation, skills captured after configuredread_fileloads retain the existing union semantics. The middleware filters model-visible schemas and blocks unauthorized execution, resolving canonical paths against the live enabled/agent-allowed registry on every model call, then stores a versioned, JSON-safe, middleware-token-bound decision signed by policy source plus active paths in run context for the resulting tool calls to reuse. The next model call always refreshes it, and malformed, foreign, stale, or unmatched decisions fall back to live resolution.tool_searchanddescribe_skillremain framework-safe discovery tools under a restrictive policy; they may reveal or promote metadata, but a deferred business tool must still be declared by the active policy before its schema or execution can survive the policy middleware. The decision's owner token is authorization-sensitive, so its reserved context key is owned byruntime.secret_contextand included inREDACTED_CONTEXT_KEYSfor observable and persisted context copies. Registry load failures and a non-empty active set with no authorized skill fail closed to framework-safe tools; an individual stale path is skipped only when at least one valid active skill remains. This is best-effort behavioral scoping rather than a hard security boundary: alternate loads such asbash catare not captured, and bounded autonomousskill_contextcan evict old entries.taskis not framework-exempt, so a restricted skill cannot delegate around its policy. The middleware must remain immediately afterSkillActivationMiddleware(which publishes the slash source throughruntime.secret_context's public path helpers authenticated by a required token shared only within the assembled middleware chain) and immediately beforeDurableContextMiddleware; assembly and compiled-graph tests pin ordering, token sharing, schema filtering, and execution blocking. - DurableContextMiddleware - Captures
taskdelegations intoThreadState.delegations(including in-progress dispatches and terminal result summaries) and loaded skill-file references (name/path/description, parsed in-memory - not the body) intoThreadState.skill_contextbefore summarization can compact the paired tool-call/result messages, then projects durable context into each model request. Static authority rules are injected as aSystemMessage; untrusted field values (summary_text, delegation results, skill descriptions) are injected separately as a hiddenHumanMessagedata block so compressed history, delegated work, and which skills are active stay visible without being stored asmessagesor promoted to system-role instructions.build_subagent_runtime_middlewaresalso attaches this middleware immediately before subagent summarization so a compactedsummary_textis projected ahead of a preserved assistant/tool tail instead of leaving strict providers with an assistant-first request. - SummarizationMiddleware - (optional, if enabled) Context reduction when approaching token limits. Compaction must preserve the latest real user request by exact message ID while allowing stale DynamicContext ID-swap
__userpeers to enter the summary; tagged dynamic-context reminders remain preserved. Do not move the cutoff backward to retain the current request, because that also keeps early AI/Tool turns active and can make first-turn long-analysis compaction a no-op.tests/test_summarization_middleware.pypins both the multi-turn stale-peer case and the first-turn long-analysis case. - TodoListMiddleware - (optional, if
is_plan_mode) Task tracking with thewrite_todostool - TokenUsageMiddleware - (optional, if
token_usage.enabled) Records token usage metrics; subagent usage is read from terminalToolMessage.additional_kwargsin the current run and merged back into the dispatching AIMessage by message position. The same state update marks the ToolMessage withsubagent_token_usage_attributed=true, so checkpoint replay or middleware re-entry cannot add the cumulative snapshot twice; missing/malformed usage or a result with no matching dispatch remains unmarked and retryable. - TitleMiddleware - Auto-generates the thread title after the first complete exchange and normalizes structured message content before prompting the title model. If a first-turn run is interrupted before this middleware can write a title,
runtime/runs/worker.pykeeps the run in a finalizing state, persists a local fallback title from the latest checkpoint or original run input, and then syncs it tothreads_meta.display_name. Replacement runs admitted bymultitask_strategy="interrupt"/"rollback"wait for older same-thread finalization before entering the graph; the interrupted run only skips the fallback title write once a later run has started and may have advanced the checkpoint. - MemoryMiddleware - Queues conversations for async memory update (filters to user + final AI responses); captures the runtime-resolved user so standalone LangGraph Server reads and writes stay in the same bucket
- ViewImageMiddleware - (optional, if the model supports vision) Appends a hidden HumanMessage with base64 image data, identified by a reserved ID prefix plus a server-owned metadata marker, to
ModelRequest.messagesinwrap_model_call/awrap_model_call. The payload lives only in that request and is never returned as a state update, so no checkpoint carries it and an interrupted run cannot strand it in history; state keeps only the lightweightviewed_imagesmetadata. It owns that context and rebuilds it per call: its own message is swept out of the request first — a thread checkpointed by the earlierbefore_model/after_modelpair (which wrote the payload into state and took it back out withRemoveMessage) can carry one that reached state but was never removed, and leaving it in would resend that base64 in every later request for the life of the thread — then a freshly built one is appended when warranted. The sweep requires both the reserved ID prefix and the server-owned marker, so a client cannot get its own message dropped; unmarked leftovers predating the marker are left in place and merely not duplicated - McpRoutingMiddleware - Auto-promotes deferred schemas matching the latest real user message before
DeferredToolFilterMiddleware, without executing tools. New names emitmiddleware:tool_promotionwithsource=routing_hint; repeat passes emit nothing - DeferredToolPromotionAuditMiddleware - Observes final
tool_searchCommands; keep it outer ofSkillToolPolicyMiddlewareso denied names are excluded. It atomically claims new names per lead run or subagent execution to dedupe parallel searches, returns the originalCommand, and omits private payloads - DeferredToolFilterMiddleware - (optional, if
tool_search.enabled) Hides deferred (MCP) tool schemas from the bound model untiltool_searchorMcpRoutingMiddlewarepromotes them (reads per-thread promotions fromThreadState.promoted, hash-scoped) - SystemMessageCoalescingMiddleware - Merges every SystemMessage into a single leading SystemMessage per request; provider-agnostic fix for strict backends (vLLM/SGLang/Qwen/Anthropic) that reject non-leading system messages. Touches the per-request payload only (checkpoint state unchanged); on midnight crossings only the latest
dynamic_context_reminderSystemMessage survives. The subagent builder places its date-only context middleware immediately before this coalescer, so the built-in subagent prompt and hidden date reminder still reach providers as one leading system block - SubagentLimitMiddleware - (optional, if
subagent_enabled) Truncates excess ordinarytasktool calls to enforce both the per-response concurrency limit (max_concurrent_subagents, resolved against startupsubagent_runtime.max_runningand the 1-64 safety range before construction) and the per-run total delegation cap (max_total_subagentsruntime override orsubagents.max_total_per_run, default 6, clamped to 1-50). The total cap counts current-run entries in the durable delegation ledger (entries are tagged withrun_idwhen captured), so repeated planning checkpoints in one run cannot keep launching legal-sized batches indefinitely, while later user turns in the same thread get a fresh run budget. Explicit durablebatch_taskcalls are a separate mode with persisted total/live/running limits and are not rewritten into ordinary ledger entries. If the ordinary cap is exhausted, the middleware strips remainingtaskcalls, forcesfinish_reason="stop", and appends a visible limit note so the run can synthesize existing results instead of ending with an empty tool-call response. - LoopDetectionMiddleware - (optional, if
loop_detection.enabled) Detects repeated tool-call loops; hard-stop clears both structuredtool_callsand raw provider tool-call metadata before forcing a final text answer; stampsloop_cappedviaconsume_stop_reason(#3875 Phase 2), symmetric toTokenBudgetMiddleware; persists warned-state transitions (first per call hash or per tool-frequency burst) and hard stops asmiddleware:loop_detection, attributed withis_subagentand the optionalagent_id, without tool arguments, message content, tool results, or argument-derived hashes. Ordinary task subagents get dedicated recorder keys through a parent-loop proxy; never passRunJournalinto their isolated loop. Durable batch subagents have no parent run journal and do not persist these transitions - TokenBudgetMiddleware - (optional, if
token_budget.enabled) Enforces per-run token limits - Custom middlewares - (optional) Any
custom_middlewarespassed tobuild_middlewaresare injected here, before config-declared extensions and the terminal-response/safety/clarification tail - Configured extension middlewares - (optional, if
extensions.middlewaresis set inconfig.yamlorextensions_config.json) Zero-argumentAgentMiddlewareclasses loaded frommodule.path:ClassNameentries viadeerflow.reflection.resolve_class. Missing packages, invalid classes, and broken modules fail loudly at agent creation. These run after built-ins/programmatic custom middleware and after the lead/subagent loop/token guards, but before the terminal-response/safety/clarification tail; subagents receive the same configured extension middleware class list before their safety tail. Treat these files as trusted operator config because middleware paths instantiate arbitrary code. Gateway skill/MCP toggle endpoints preserve this field throughto_file_dict()but must not add a write path forextensions.middlewareswithout an explicit trust-boundary review. Lead-only vs subagent-only middleware lists and per-context constructor parameters are not expressible in this MVP. - TerminalResponseMiddleware - When a provider returns an empty terminal
AIMessageafter tool execution, injects a hidden recovery prompt and retries the model once; a second empty response is replaced in checkpoint state by a visible error fallback marked for the run worker, so the run finishes as an error instead of a silent success - ModelLengthFinishReasonMiddleware - Records
stop_reason=model_length_cappedwhen provider-specific length detectors match a terminalAIMessagewithout tool-call intent (finish_reason=length/MAX_TOKENS, orstop_reason=max_tokens), preserving the original assistant content and never reparsing textual tool-call-like envelopes - SafetyFinishReasonMiddleware - (optional, if
safety_finish_reason.enabled) Suppresses tool execution when the provider safety-terminated the response (e.g.finish_reason=content_filter); registered after terminal-response/custom/configured middlewares so LangChain's reverse-orderafter_modeldispatch runs it first - ClarificationMiddleware - Intercepts
ask_clarification, writes a readableToolMessage.contentfallback plus a structuredToolMessage.artifact.human_inputpayload, and interrupts viaCommand(goto=END)(must be last).after_modeldrops same-turn sibling tool calls so they cannot run before the user answers; a malformedask_clarificationparked oninvalid_tool_callsis the same stop signal.disable_clarificationruns keep the siblings. Payloads are versioned — legacyfree_text/choice_with_otherstayversion: 1; the v2formmode (fromfields) isversion: 2so older frontends reject it and fall back to plain text. Field normalization is deterministic and lives in the middleware (it short-circuits before tool execution, so tool-arg typing gives no runtime validation), and it is atomic: any structurally broken entry — non-dict, bad/duplicate name, a name colliding with a JSObject.prototypemember (__proto__/constructor), or exceeding the caps (16 fields / 24 options per field / 200 chars per text /MAX_FORM_SERIALIZED_BYTES= 16KB UTF-8, the per-item caps alone admitting forms whose IM text fallback overruns channel limits) — degrades the whole form to the legacy option/free-text modes, so a card never renders "complete" while missing a field. Benign issues degrade locally (unknown types — incl. unhashable JSON liketype: [], which must not raise from the membership probe — and option-less selects becometext); options are trimmed/deduped with blanks dropped (form- and top-level) since the frontend rejects blank labels. XML-to-dict option payloads are recursively flattened from dict/list containers in source order, scalar leaves kept, residual XML tags stripped before that trimming. Checkboxes are booleans defaulting to "no";requiredon one means consent semantics. The response protocol is unchanged (v1text/option): form cards submit a text summary asresponse_kind: "text", so journal persistence needs no new allowlist entries. Because this middleware can short-circuit beforeon_tool_end,RunJournaldoes a root-run reconciliation forToolMessages whosetool_call_idcame from the current run, so cards survive checkpoint compaction. That reconciliation is notask_clarification-only — any middleware that answers a tool call has the same gap, and a result the user saw must not vanish on reload (#4666 —ReadBeforeWriteMiddlewareblocked-write errors reached the UI but not the event store). It is bounded by three conditions, not a name allowlist: the message is user-visible, the call belongs to this run's lead agent (_remember_current_run_tool_callsrecords lead-agent calls only; subagent results stay insubagent.step), and it is not already persisted. Human Input Card replies arehide_from_uiHumanMessages withadditional_kwargs.human_input_response;RunJournalpersists only allowlisted hidden sources (currentlyask_clarification) asllm.human.input.