mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-14 16:08:41 +00:00
* feat(agents): elide blocked write payloads from model-bound requests A write_file / str_replace call rejected by the read-before-write gate never runs, yet its payload (up to 80 KB for a non-append write, unbounded for append) stayed verbatim in every later model request: nothing in the chain rewrites AIMessage tool-call arguments, and ToolOutputBudgetMiddleware only budgets ToolMessage output. The gate demands a re-read plus a fresh call, so the model re-emits the content anyway and the original is pure dead weight. - ReadBeforeWriteMiddleware stamps `deerflow_write_block` on the blocked ToolMessage and, in wrap_model_call, replaces the paired call's payload fields (content / old_str / new_str) with a short deterministic placeholder in the model-bound request only. state["messages"], receipts, loop detection, and the run journal keep the original arguments; nothing is externalized to disk, since a file reference to content the model must re-derive after reading the target would only invite bypassing the gate. - New `tool_call_args` helper rewrites every provider surface together (structured tool_calls, raw additional_kwargs.tool_calls, tool_use content blocks, tool_call_chunks) so strict providers never see them disagree; the gate only supplies the policy (which calls, what placeholder). - `read_before_write.elide_blocked_payloads` (default on) and `read_before_write.elide_min_chars` (default 2000) configure it; the runtime builder passes the config through and the middleware declares it via release_policy_parameters. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(agents): condense middleware guide entry 11 to fit the guidance budget The agent-guidance CI check failed: the effective AGENTS.md chain for agents/middlewares was 99673 bytes against a 98304-byte hard limit. The chain already sat at 98459 on main, so the ReadBeforeWrite entry could not grow. Rewrite entry 11 so it states the same facts (gate, lock scope, fail-open, authorization scope, blocked-payload elision, shared tool_call_args helper) in 1229 bytes instead of 2640; the chain is now 98262 bytes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(config): bump config_version for the read_before_write elision keys Review follow-ups on #5329: - `read_before_write.elide_blocked_payloads` / `elide_min_chars` are new user-settable YAML keys, i.e. a config schema change, so bump `config_version` 40 -> 41 in config.example.yaml; without it an existing config.yaml gets no outdated-config warning and `make config-upgrade` has nothing to signal. - Say in the `elide_min_chars` description (and the example comment) that the threshold and the placeholder's size figure are Python character counts, not tokens: the same value spans roughly 3-4x in real context cost between ASCII and CJK text. - The builder wiring test now asserts only the wired `elide_min_chars` value instead of the whole `ReadBeforeWriteConfig` dump, so future knobs do not have to edit an unrelated test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(helm): bump chart config_version to 41 validate-chart's config_version drift check failed after config.example.yaml moved to 41 in ef9ee267. Bare bump of the chart's embedded `config:` block and the README example; the chart does not mirror the read_before_write section, so no field changes are needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(agents): rewrite Responses and v1 content-block arguments too Review finding on #5329 (P2): the content rewriter only handled Anthropic `tool_use` blocks. With `use_responses_api=true` and `output_version='responses/v1'`, AIMessage.content carries `function_call` blocks whose `arguments` still hold the full write payload, and langchain_openai's Responses input builder emits that block instead of the rewritten structured call whose `call_id` it already carries. Standard `v1` `tool_call` blocks likewise keep `extras.arguments`, which the v1->Responses translator prefers over the structured args. So the blocked payload was still sent on every later Responses API request. `tool_call_args` now rewrites every content dialect that carries its own copy of the arguments: Anthropic `tool_use` (input, drop partial_json), Responses `function_call` (arguments, matched by call_id, `fc_...` item id and status preserved), and v1 `tool_call` / `tool_call_chunk` (args plus `extras.arguments`). Tests assert against the real adapter serializers: `_construct_responses_api_input` for responses/v1, v1, and v0 messages, `_convert_message_to_dict` for chat completions, and Anthropic `_format_messages` for native and v1 content, plus an end-to-end probe through the gate's wrap_model_call. * fix(agents): pair blocked writes per call occurrence and defeat Responses chaining Two review findings on #5329: - Tool-call ids may repeat across assistant turns (DanglingToolCallMiddleware pairs them with per-id queues). The gate matched blocked results against a history-wide id set, so a successful write sharing an id with a later (or earlier) blocked one also lost its payload and was labelled as blocked. `_blocked_call_occurrences` now pairs ToolMessages with call occurrences the same FIFO-per-id way and the selector keys on (message, call id). - With `use_previous_response_id`, the OpenAI adapter sends only the messages after the last AIMessage carrying a `resp_` response id and lets the server rebuild the rest from its stored copy, which still holds the original arguments and cannot be edited; every later response chains back to it. `rewrite_messages_tool_call_args` now drops every `resp_` id from the model-bound copy whenever it rewrote anything, so the adapter replays the full rewritten history (the `use_previous_response_id=False` request shape). OpenAI bills chained input tokens as input either way, so replay costs no more; the state keeps its ids. Tests cover success-before-block and block-before-success histories through the Chat Completions serializer, and chaining through `ChatOpenAI._get_request_payload` with `use_previous_response_id=True`: unrewritten history chains and omits the call, rewritten history is replayed with the placeholder and no `previous_response_id`. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
1262 lines
59 KiB
Python
1262 lines
59 KiB
Python
import posixpath
|
|
import re
|
|
import sys
|
|
from datetime import datetime
|
|
from types import ModuleType, SimpleNamespace
|
|
|
|
import pytest
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
from langchain_core.messages import ToolMessage
|
|
from langgraph.errors import GraphInterrupt
|
|
|
|
from deerflow.agents.middlewares.tool_error_handling_middleware import (
|
|
ToolErrorHandlingMiddleware,
|
|
build_lead_runtime_middlewares,
|
|
build_subagent_runtime_middlewares,
|
|
)
|
|
from deerflow.agents.middlewares.tool_result_meta import TOOL_META_KEY
|
|
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
|
|
from deerflow.config import summarization_config
|
|
from deerflow.config.app_config import AppConfig, CircuitBreakerConfig
|
|
from deerflow.config.extensions_config import ExtensionsConfig
|
|
from deerflow.config.guardrails_config import GuardrailsConfig
|
|
from deerflow.config.model_config import ModelConfig
|
|
from deerflow.config.sandbox_config import SandboxConfig
|
|
from deerflow.subagents.status_contract import SUBAGENT_ERROR_KEY, SUBAGENT_STATUS_KEY
|
|
|
|
|
|
class ConfiguredSubagentMiddleware(AgentMiddleware):
|
|
pass
|
|
|
|
|
|
def _request(name: str = "web_search", tool_call_id: str | None = "tc-1"):
|
|
tool_call = {"name": name}
|
|
if tool_call_id is not None:
|
|
tool_call["id"] = tool_call_id
|
|
return SimpleNamespace(tool_call=tool_call)
|
|
|
|
|
|
def _module(name: str, **attrs):
|
|
module = ModuleType(name)
|
|
for key, value in attrs.items():
|
|
setattr(module, key, value)
|
|
return module
|
|
|
|
|
|
def _make_app_config(*, supports_vision: bool = False) -> AppConfig:
|
|
return AppConfig(
|
|
models=[
|
|
ModelConfig(
|
|
name="test-model",
|
|
display_name="test-model",
|
|
description=None,
|
|
use="langchain_openai:ChatOpenAI",
|
|
model="test-model",
|
|
supports_vision=supports_vision,
|
|
)
|
|
],
|
|
sandbox=SandboxConfig(use="test"),
|
|
guardrails=GuardrailsConfig(enabled=False),
|
|
circuit_breaker=CircuitBreakerConfig(failure_threshold=7, recovery_timeout_sec=11),
|
|
)
|
|
|
|
|
|
def _stub_runtime_middleware_imports(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
class FakeMiddleware:
|
|
def __init__(self, *args, **kwargs):
|
|
self.args = args
|
|
self.kwargs = kwargs
|
|
|
|
class FakeLLMErrorHandlingMiddleware:
|
|
def __init__(self, *, app_config):
|
|
self.app_config = app_config
|
|
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.llm_error_handling_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.llm_error_handling_middleware",
|
|
LLMErrorHandlingMiddleware=FakeLLMErrorHandlingMiddleware,
|
|
),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.thread_data_middleware",
|
|
_module("deerflow.agents.middlewares.thread_data_middleware", ThreadDataMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.sandbox.middleware",
|
|
_module("deerflow.sandbox.middleware", SandboxMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.dangling_tool_call_middleware",
|
|
_module("deerflow.agents.middlewares.dangling_tool_call_middleware", DanglingToolCallMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.sandbox_audit_middleware",
|
|
_module("deerflow.agents.middlewares.sandbox_audit_middleware", SandboxAuditMiddleware=FakeMiddleware),
|
|
)
|
|
|
|
|
|
def test_build_subagent_runtime_middlewares_threads_app_config_to_llm_middleware(monkeypatch: pytest.MonkeyPatch):
|
|
captured: dict[str, object] = {}
|
|
|
|
class FakeMiddleware:
|
|
def __init__(self, *args, **kwargs):
|
|
self.args = args
|
|
self.kwargs = kwargs
|
|
|
|
class FakeLLMErrorHandlingMiddleware:
|
|
def __init__(self, *, app_config):
|
|
captured["app_config"] = app_config
|
|
|
|
app_config = _make_app_config()
|
|
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.llm_error_handling_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.llm_error_handling_middleware",
|
|
LLMErrorHandlingMiddleware=FakeLLMErrorHandlingMiddleware,
|
|
),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.thread_data_middleware",
|
|
_module("deerflow.agents.middlewares.thread_data_middleware", ThreadDataMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.sandbox.middleware",
|
|
_module("deerflow.sandbox.middleware", SandboxMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.dangling_tool_call_middleware",
|
|
_module("deerflow.agents.middlewares.dangling_tool_call_middleware", DanglingToolCallMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.sandbox_audit_middleware",
|
|
_module("deerflow.agents.middlewares.sandbox_audit_middleware", SandboxAuditMiddleware=FakeMiddleware),
|
|
)
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
InputSanitizationMiddleware=FakeMiddleware,
|
|
neutralize_untrusted_tags=lambda value: value,
|
|
),
|
|
)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, lazy_init=False)
|
|
|
|
assert captured["app_config"] is app_config
|
|
# 9 baseline (InputSanitization, ToolOutputBudget, ToolResultSanitization,
|
|
# ThreadData, Sandbox, DanglingToolCall, LLMErrorHandling, SandboxAudit,
|
|
# ToolErrorHandling)
|
|
# + 1 ReadBeforeWriteMiddleware + 1 LoopDetectionMiddleware
|
|
# + 1 TokenBudgetMiddleware (subagents.token_budget enabled by default, #3875 Phase 2)
|
|
# + 1 SkillActivationMiddleware + 1 SkillToolPolicyMiddleware
|
|
# + 1 SafetyFinishReasonMiddleware + 1 DurableContextMiddleware
|
|
# + 1 SubagentDateContextMiddleware
|
|
# + 1 SystemMessageCoalescingMiddleware + 1 ToolReceiptMiddleware
|
|
# (all enabled by default).
|
|
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
|
|
from deerflow.agents.middlewares.dynamic_context_middleware import SubagentDateContextMiddleware
|
|
from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
|
|
from deerflow.agents.middlewares.skill_activation_middleware import SkillActivationMiddleware
|
|
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
|
|
from deerflow.agents.middlewares.system_message_coalescing_middleware import SystemMessageCoalescingMiddleware
|
|
from deerflow.agents.middlewares.token_budget_middleware import TokenBudgetMiddleware
|
|
from deerflow.agents.middlewares.tool_output_budget_middleware import ToolOutputBudgetMiddleware
|
|
from deerflow.agents.middlewares.tool_receipt_middleware import ToolReceiptMiddleware
|
|
|
|
assert len(middlewares) == 19
|
|
assert isinstance(middlewares[0], FakeMiddleware) # InputSanitizationMiddleware stub
|
|
assert isinstance(middlewares[1], ToolOutputBudgetMiddleware)
|
|
assert any(isinstance(m, ToolErrorHandlingMiddleware) for m in middlewares)
|
|
# The receipt layer wraps ToolErrorHandlingMiddleware so receipts read the
|
|
# deerflow_tool_meta status it stamps (guard-enforced, like ToolProgress).
|
|
receipt_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, ToolReceiptMiddleware))
|
|
error_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, ToolErrorHandlingMiddleware))
|
|
assert receipt_idx < error_idx
|
|
# The token-budget backstop is attached by default so the cap engages (#3875).
|
|
assert any(isinstance(m, TokenBudgetMiddleware) for m in middlewares)
|
|
assert any(isinstance(m, SafetyFinishReasonMiddleware) for m in middlewares)
|
|
activation_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SkillActivationMiddleware))
|
|
policy_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SkillToolPolicyMiddleware))
|
|
assert policy_idx == activation_idx + 1
|
|
assert middlewares[activation_idx]._slash_source_owner_token == middlewares[policy_idx]._slash_source_owner_token
|
|
# DurableContextMiddleware is present but not last: the coalescer (#4040) is
|
|
# appended innermost so it can merge the SystemMessage DurableContext injects.
|
|
# The coalescer is appended unconditionally (after the optional summarization
|
|
# middleware), so it is the last element regardless of summarization.enabled —
|
|
# unlike DurableContextMiddleware, which is only last when summarization is off.
|
|
durable_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, DurableContextMiddleware))
|
|
date_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SubagentDateContextMiddleware))
|
|
assert isinstance(middlewares[-1], SystemMessageCoalescingMiddleware)
|
|
assert policy_idx < durable_idx < date_idx == len(middlewares) - 2
|
|
|
|
|
|
def test_subagent_runtime_sandbox_does_not_own_lead_skill_projection() -> None:
|
|
from deerflow.extensions.registry import ExtensionRegistry
|
|
from deerflow.sandbox.middleware import SandboxMiddleware
|
|
|
|
middlewares = build_subagent_runtime_middlewares(
|
|
app_config=_make_app_config(),
|
|
available_skills={"allowed"},
|
|
extensions=ExtensionRegistry().build(),
|
|
)
|
|
|
|
sandbox_middleware = next(middleware for middleware in middlewares if isinstance(middleware, SandboxMiddleware))
|
|
assert sandbox_middleware._owns_agent_skill_projection is False
|
|
|
|
|
|
def test_tool_progress_middleware_is_outer_relative_to_error_handling(monkeypatch: pytest.MonkeyPatch):
|
|
# ToolProgressMiddleware must have a lower index than ToolErrorHandlingMiddleware
|
|
# so that the framework's "first in list = outermost" rule makes it outer.
|
|
# Only then can it read deerflow_tool_meta stamped by ToolErrorHandlingMiddleware.
|
|
from deerflow.agents.middlewares.tool_progress_middleware import ToolProgressMiddleware
|
|
from deerflow.config.tool_progress_config import ToolProgressConfig
|
|
|
|
app_config = AppConfig(
|
|
models=[
|
|
ModelConfig(
|
|
name="test-model",
|
|
display_name="test-model",
|
|
description=None,
|
|
use="langchain_openai:ChatOpenAI",
|
|
model="test-model",
|
|
)
|
|
],
|
|
sandbox=SandboxConfig(use="test"),
|
|
guardrails=GuardrailsConfig(enabled=False),
|
|
circuit_breaker=CircuitBreakerConfig(failure_threshold=7, recovery_timeout_sec=11),
|
|
tool_progress=ToolProgressConfig(enabled=True),
|
|
)
|
|
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, lazy_init=False)
|
|
|
|
progress_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, ToolProgressMiddleware))
|
|
error_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, ToolErrorHandlingMiddleware))
|
|
assert progress_idx < error_idx, f"ToolProgressMiddleware (index {progress_idx}) must be outer (lower index) than ToolErrorHandlingMiddleware (index {error_idx}); order: {[type(m).__name__ for m in middlewares]}"
|
|
|
|
|
|
def test_middleware_ordering_guard_moved_to_declarative_constraints(monkeypatch: pytest.MonkeyPatch):
|
|
"""_build_runtime_middlewares no longer hand-validates ordering; the invariant is now
|
|
declared in deerflow.extensions.ordering (core_ordering_constraints / assert_ordering) and
|
|
is checked once the composing builder merges extension contributions in (Task 9).
|
|
|
|
This test previously monkeypatched SandboxAuditMiddleware to a ToolErrorHandlingMiddleware
|
|
instance to force the wrong-order condition and asserted that the builder itself raised.
|
|
That in-builder guard was deleted on purpose: validating here would check a stack that
|
|
hasn't received extension contributions yet. Building under the same wrong-order condition
|
|
must no longer raise inside this builder; deerflow.extensions.ordering has the equivalent
|
|
coverage (see test_extension_ordering.py and test_core_constraints_are_declared).
|
|
"""
|
|
from deerflow.agents.middlewares.tool_error_handling_middleware import (
|
|
ToolErrorHandlingMiddleware,
|
|
build_lead_runtime_middlewares,
|
|
)
|
|
from deerflow.config.tool_progress_config import ToolProgressConfig
|
|
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
# Override the SandboxAuditMiddleware stub with a real ToolErrorHandlingMiddleware so it
|
|
# becomes the FIRST ToolErrorHandlingMiddleware in the list, appearing before
|
|
# ToolProgressMiddleware — the same wrong-order condition the deleted guard used to catch.
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.sandbox_audit_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.sandbox_audit_middleware",
|
|
SandboxAuditMiddleware=ToolErrorHandlingMiddleware,
|
|
),
|
|
)
|
|
|
|
app_config = _make_app_config()
|
|
app_config = app_config.model_copy(update={"tool_progress": ToolProgressConfig(enabled=True)})
|
|
|
|
# No raise here: the invariant is enforced by assert_ordering at the composing builder,
|
|
# not inside _build_runtime_middlewares.
|
|
build_lead_runtime_middlewares(app_config=app_config, lazy_init=False)
|
|
|
|
|
|
def test_lead_runtime_middlewares_thread_app_config_to_tool_error_handling(monkeypatch: pytest.MonkeyPatch):
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
_module("deerflow.agents.middlewares.input_sanitization_middleware", InputSanitizationMiddleware=object),
|
|
)
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_lead_runtime_middlewares(app_config=app_config)
|
|
|
|
tool_middleware = next(mw for mw in middlewares if isinstance(mw, ToolErrorHandlingMiddleware))
|
|
assert tool_middleware._app_config is app_config
|
|
|
|
|
|
def test_lead_runtime_middlewares_pass_agent_skills_to_sandbox(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
):
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
InputSanitizationMiddleware=object,
|
|
neutralize_untrusted_tags=lambda value: value,
|
|
),
|
|
)
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_lead_runtime_middlewares(
|
|
app_config=app_config,
|
|
available_skills={"allowed-skill"},
|
|
)
|
|
|
|
sandbox_middleware = next(middleware for middleware in middlewares if getattr(middleware, "kwargs", {}).get("available_skills") == {"allowed-skill"})
|
|
assert sandbox_middleware.kwargs == {
|
|
"lazy_init": True,
|
|
"available_skills": {"allowed-skill"},
|
|
"owns_agent_skill_projection": True,
|
|
}
|
|
|
|
|
|
def test_lead_runtime_middlewares_can_delegate_skill_projection_ownership(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
):
|
|
monkeypatch.setitem(
|
|
sys.modules,
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
_module(
|
|
"deerflow.agents.middlewares.input_sanitization_middleware",
|
|
InputSanitizationMiddleware=object,
|
|
neutralize_untrusted_tags=lambda value: value,
|
|
),
|
|
)
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_lead_runtime_middlewares(
|
|
app_config=app_config,
|
|
available_skills={"bootstrap"},
|
|
owns_agent_skill_projection=False,
|
|
)
|
|
|
|
sandbox_middleware = next(middleware for middleware in middlewares if getattr(middleware, "kwargs", {}).get("available_skills") == {"bootstrap"})
|
|
assert sandbox_middleware.kwargs == {
|
|
"lazy_init": True,
|
|
"available_skills": {"bootstrap"},
|
|
"owns_agent_skill_projection": False,
|
|
}
|
|
|
|
|
|
def test_build_lead_runtime_middlewares_orders_thread_data_before_uploads():
|
|
"""ThreadDataMiddleware must run before UploadsMiddleware so the uploads
|
|
directory is guaranteed to exist when UploadsMiddleware scans it under
|
|
lazy_init=False. This is the narrow functional concern the chain order
|
|
protects; a regression here would silently drop historical files on the
|
|
first run of a thread when the directory has not been pre-created by the
|
|
upload endpoint.
|
|
"""
|
|
from deerflow.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
|
|
from deerflow.agents.middlewares.uploads_middleware import UploadsMiddleware
|
|
|
|
app_config = _make_app_config()
|
|
middlewares = build_lead_runtime_middlewares(app_config=app_config)
|
|
|
|
td_indices = [i for i, m in enumerate(middlewares) if isinstance(m, ThreadDataMiddleware)]
|
|
um_indices = [i for i, m in enumerate(middlewares) if isinstance(m, UploadsMiddleware)]
|
|
|
|
assert td_indices and len(td_indices) == 1, f"expected exactly one ThreadDataMiddleware, got {td_indices}"
|
|
assert um_indices and len(um_indices) == 1, f"expected exactly one UploadsMiddleware, got {um_indices}"
|
|
assert td_indices[0] < um_indices[0], f"ThreadDataMiddleware (idx {td_indices[0]}) must come before UploadsMiddleware (idx {um_indices[0]}) so the uploads directory exists when UploadsMiddleware scans it under lazy_init=False."
|
|
|
|
|
|
def test_build_lead_runtime_middlewares_chain_order_matches_agents_md():
|
|
"""Pin the AGENTS.md middleware numbering for the shared runtime base.
|
|
|
|
The existing tests stub most middlewares as a single ``FakeMiddleware``,
|
|
which cannot detect a reorder. This test uses the real classes so an
|
|
index swap between any pair (e.g. Uploads vs ThreadData, Sandbox vs
|
|
DanglingToolCall) is caught. If a future refactor legitimately reorders
|
|
these, update backend/AGENTS.md "Middleware Chain" in the same change.
|
|
"""
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
from deerflow.agents.middlewares.input_sanitization_middleware import InputSanitizationMiddleware
|
|
from deerflow.agents.middlewares.llm_error_handling_middleware import LLMErrorHandlingMiddleware
|
|
from deerflow.agents.middlewares.read_before_write_middleware import ReadBeforeWriteMiddleware
|
|
from deerflow.agents.middlewares.sandbox_audit_middleware import SandboxAuditMiddleware
|
|
from deerflow.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
|
|
from deerflow.agents.middlewares.tool_output_budget_middleware import ToolOutputBudgetMiddleware
|
|
from deerflow.agents.middlewares.tool_result_sanitization_middleware import ToolResultSanitizationMiddleware
|
|
from deerflow.agents.middlewares.uploads_middleware import UploadsMiddleware
|
|
from deerflow.sandbox.middleware import SandboxMiddleware
|
|
|
|
app_config = _make_app_config()
|
|
middlewares = build_lead_runtime_middlewares(app_config=app_config)
|
|
|
|
def idx_of(cls, *, label: str) -> int:
|
|
matches = [i for i, m in enumerate(middlewares) if isinstance(m, cls)]
|
|
assert matches, f"{label} missing from chain"
|
|
assert len(matches) == 1, f"expected exactly one {label}, got indices {matches}"
|
|
return matches[0]
|
|
|
|
# Mirrors AGENTS.md "Shared runtime base" items 1-10 (non-optional spine).
|
|
expected_order: list[tuple[str, type]] = [
|
|
("InputSanitizationMiddleware", InputSanitizationMiddleware),
|
|
("ToolOutputBudgetMiddleware", ToolOutputBudgetMiddleware),
|
|
("ToolResultSanitizationMiddleware", ToolResultSanitizationMiddleware),
|
|
("ThreadDataMiddleware", ThreadDataMiddleware),
|
|
("UploadsMiddleware", UploadsMiddleware),
|
|
("SandboxMiddleware", SandboxMiddleware),
|
|
("DanglingToolCallMiddleware", DanglingToolCallMiddleware),
|
|
("LLMErrorHandlingMiddleware", LLMErrorHandlingMiddleware),
|
|
("SandboxAuditMiddleware", SandboxAuditMiddleware),
|
|
("ReadBeforeWriteMiddleware", ReadBeforeWriteMiddleware),
|
|
("ToolErrorHandlingMiddleware", ToolErrorHandlingMiddleware),
|
|
]
|
|
actual = [(label, idx_of(cls, label=label)) for label, cls in expected_order]
|
|
|
|
for (name_a, idx_a), (name_b, idx_b) in zip(actual, actual[1:]):
|
|
assert idx_a < idx_b, f"{name_a} (idx {idx_a}) must come before {name_b} (idx {idx_b}); full chain: {actual}"
|
|
|
|
|
|
def test_wrap_tool_call_passthrough_on_success():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request()
|
|
expected = ToolMessage(content="ok", tool_call_id="tc-1", name="web_search")
|
|
|
|
result = middleware.wrap_tool_call(req, lambda _req: expected)
|
|
|
|
assert result is expected
|
|
|
|
|
|
def test_read_file_skill_read_stamps_compact_skill_metadata():
|
|
app_config = _make_app_config()
|
|
app_config.skills.container_path = "/mnt/skills"
|
|
app_config.summarization.skill_file_read_tool_names = ["read_file"]
|
|
middleware = ToolErrorHandlingMiddleware(app_config=app_config)
|
|
req = _request(name="read_file", tool_call_id="read-1")
|
|
req.tool_call["args"] = {"path": "/mnt/skills/public/data-analysis/SKILL.md"}
|
|
|
|
result = middleware.wrap_tool_call(
|
|
req,
|
|
lambda _req: ToolMessage(
|
|
content="---\nname: data-analysis\ndescription: Analyze data.\n---\nBODY",
|
|
tool_call_id="read-1",
|
|
name="read_file",
|
|
),
|
|
)
|
|
|
|
assert result.additional_kwargs["skill_context_entry"] == {
|
|
"path": "/mnt/skills/public/data-analysis/SKILL.md",
|
|
"description": "Analyze data.",
|
|
}
|
|
|
|
|
|
def test_skill_read_config_is_cached_on_middleware_instance():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
default_names = getattr(summarization_config, "DEFAULT_SKILL_FILE_READ_TOOL_NAMES", None)
|
|
|
|
assert default_names is not None
|
|
assert middleware._skill_read_tool_names == frozenset(default_names)
|
|
assert middleware._skills_root == "/mnt/skills"
|
|
|
|
|
|
def test_skill_metadata_respects_custom_skills_root():
|
|
app_config = _make_app_config()
|
|
app_config.skills.container_path = "/custom/skills"
|
|
app_config.summarization.skill_file_read_tool_names = ["read_file"]
|
|
middleware = ToolErrorHandlingMiddleware(app_config=app_config)
|
|
req = _request(name="read_file", tool_call_id="read-1")
|
|
req.tool_call["args"] = {"path": "/custom/skills/public/x/SKILL.md"}
|
|
|
|
result = middleware.wrap_tool_call(
|
|
req,
|
|
lambda _req: ToolMessage("---\ndescription: X\n---\nBody", tool_call_id="read-1", name="read_file"),
|
|
)
|
|
|
|
assert result.additional_kwargs["skill_context_entry"]["path"] == "/custom/skills/public/x/SKILL.md"
|
|
|
|
|
|
def test_skill_metadata_disabled_when_read_tool_names_empty():
|
|
app_config = _make_app_config()
|
|
app_config.summarization.skill_file_read_tool_names = []
|
|
middleware = ToolErrorHandlingMiddleware(app_config=app_config)
|
|
req = _request(name="read_file", tool_call_id="read-1")
|
|
req.tool_call["args"] = {"path": "/mnt/skills/public/x/SKILL.md"}
|
|
|
|
result = middleware.wrap_tool_call(
|
|
req,
|
|
lambda _req: ToolMessage("---\ndescription: X\n---\nBody", tool_call_id="read-1", name="read_file"),
|
|
)
|
|
|
|
assert "skill_context_entry" not in result.additional_kwargs
|
|
|
|
|
|
def test_wrap_tool_call_returns_error_tool_message_on_exception():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="web_search", tool_call_id="tc-42")
|
|
|
|
def _boom(_req):
|
|
raise RuntimeError("network down")
|
|
|
|
result = middleware.wrap_tool_call(req, _boom)
|
|
|
|
assert isinstance(result, ToolMessage)
|
|
assert result.tool_call_id == "tc-42"
|
|
assert result.name == "web_search"
|
|
assert result.status == "error"
|
|
assert "Tool 'web_search' failed" in result.text
|
|
assert "network down" in result.text
|
|
|
|
|
|
def test_wrap_tool_call_stamps_tool_meta_on_exception():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="web_search", tool_call_id="tc-42")
|
|
|
|
def _boom(_req):
|
|
raise ConnectionError("connection refused")
|
|
|
|
result = middleware.wrap_tool_call(req, _boom)
|
|
|
|
assert isinstance(result, ToolMessage)
|
|
assert TOOL_META_KEY in result.additional_kwargs
|
|
meta = result.additional_kwargs[TOOL_META_KEY]
|
|
assert meta["status"] == "error"
|
|
assert meta["source"] == "exception"
|
|
assert meta["error_type"] == "transient"
|
|
|
|
|
|
def test_task_exception_wrapper_uses_subagent_result_formatter():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="task", tool_call_id="tc-task")
|
|
|
|
def _boom(_req):
|
|
raise RuntimeError("network down")
|
|
|
|
result = middleware.wrap_tool_call(req, _boom)
|
|
|
|
assert isinstance(result, ToolMessage)
|
|
assert result.tool_call_id == "tc-task"
|
|
assert result.name == "task"
|
|
assert result.status == "error"
|
|
assert result.content == "Task failed. Error: RuntimeError: network down. Continue with available context, or choose an alternative tool."
|
|
assert result.additional_kwargs[SUBAGENT_STATUS_KEY] == "failed"
|
|
assert result.additional_kwargs[SUBAGENT_ERROR_KEY] == "RuntimeError: network down"
|
|
|
|
|
|
def test_wrap_tool_call_uses_fallback_tool_call_id_when_missing():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="mcp_tool", tool_call_id=None)
|
|
|
|
def _boom(_req):
|
|
raise ValueError("bad request")
|
|
|
|
result = middleware.wrap_tool_call(req, _boom)
|
|
|
|
assert isinstance(result, ToolMessage)
|
|
assert result.tool_call_id == "missing_tool_call_id"
|
|
assert result.name == "mcp_tool"
|
|
assert result.status == "error"
|
|
|
|
|
|
def test_wrap_tool_call_reraises_graph_interrupt():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="ask_clarification", tool_call_id="tc-int")
|
|
|
|
def _interrupt(_req):
|
|
raise GraphInterrupt(())
|
|
|
|
with pytest.raises(GraphInterrupt):
|
|
middleware.wrap_tool_call(req, _interrupt)
|
|
|
|
|
|
@pytest.mark.anyio
|
|
async def test_awrap_tool_call_returns_error_tool_message_on_exception():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="mcp_tool", tool_call_id="tc-async")
|
|
|
|
async def _boom(_req):
|
|
raise TimeoutError("request timed out")
|
|
|
|
result = await middleware.awrap_tool_call(req, _boom)
|
|
|
|
assert isinstance(result, ToolMessage)
|
|
assert result.tool_call_id == "tc-async"
|
|
assert result.name == "mcp_tool"
|
|
assert result.status == "error"
|
|
assert "request timed out" in result.text
|
|
|
|
|
|
@pytest.mark.anyio
|
|
async def test_awrap_tool_call_reraises_graph_interrupt():
|
|
middleware = ToolErrorHandlingMiddleware()
|
|
req = _request(name="ask_clarification", tool_call_id="tc-int-async")
|
|
|
|
async def _interrupt(_req):
|
|
raise GraphInterrupt(())
|
|
|
|
with pytest.raises(GraphInterrupt):
|
|
await middleware.awrap_tool_call(req, _interrupt)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_include_view_image_for_vision_model(monkeypatch):
|
|
app_config = _make_app_config(supports_vision=True)
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
assert any(isinstance(middleware, ViewImageMiddleware) for middleware in middlewares)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_include_view_image_for_default_vision_model(monkeypatch):
|
|
app_config = _make_app_config(supports_vision=True)
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name=None)
|
|
|
|
assert any(isinstance(middleware, ViewImageMiddleware) for middleware in middlewares)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_skip_view_image_for_text_model(monkeypatch):
|
|
app_config = _make_app_config(supports_vision=False)
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
assert not any(isinstance(middleware, ViewImageMiddleware) for middleware in middlewares)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_attach_deferred_filter_when_setup_has_names(monkeypatch):
|
|
"""A subagent built with deferred MCP tools gets DeferredToolFilterMiddleware, positioned before SafetyFinishReasonMiddleware (mirrors the lead ordering)."""
|
|
from langchain_core.tools import tool as as_tool
|
|
|
|
from deerflow.agents.middlewares.deferred_tool_filter_middleware import DeferredToolFilterMiddleware
|
|
from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
|
|
from deerflow.agents.middlewares.skill_tool_policy_middleware import SkillToolPolicyMiddleware
|
|
from deerflow.agents.middlewares.tool_promotion_audit_middleware import DeferredToolPromotionAuditMiddleware
|
|
from deerflow.tools.builtins.tool_search import build_deferred_tool_setup
|
|
from deerflow.tools.mcp_metadata import tag_mcp_tool
|
|
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
@as_tool
|
|
def mcp_thing(x: str) -> str:
|
|
"deferred mcp tool"
|
|
return x
|
|
|
|
setup = build_deferred_tool_setup([tag_mcp_tool(mcp_thing)], enabled=True)
|
|
assert setup.deferred_names # sanity: populated setup
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, deferred_setup=setup)
|
|
|
|
filters = [m for m in middlewares if isinstance(m, DeferredToolFilterMiddleware)]
|
|
audits = [m for m in middlewares if isinstance(m, DeferredToolPromotionAuditMiddleware)]
|
|
assert len(filters) == 1
|
|
assert len(audits) == 1
|
|
audit_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, DeferredToolPromotionAuditMiddleware))
|
|
policy_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SkillToolPolicyMiddleware))
|
|
filter_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, DeferredToolFilterMiddleware))
|
|
safety_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SafetyFinishReasonMiddleware))
|
|
assert audit_idx < policy_idx < filter_idx
|
|
assert filter_idx < safety_idx
|
|
|
|
|
|
def test_subagent_runtime_middlewares_inject_configured_extension_middlewares(monkeypatch):
|
|
from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
|
|
|
|
app_config = _make_app_config()
|
|
app_config.extensions = ExtensionsConfig(middlewares=[f"{__name__}:ConfiguredSubagentMiddleware"])
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config)
|
|
|
|
extension_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, ConfiguredSubagentMiddleware))
|
|
safety_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SafetyFinishReasonMiddleware))
|
|
for guard_name in ("LoopDetectionMiddleware", "TokenBudgetMiddleware"):
|
|
guard_idx = next((i for i, m in enumerate(middlewares) if type(m).__name__ == guard_name), None)
|
|
if guard_idx is not None:
|
|
assert guard_idx < extension_idx
|
|
assert extension_idx < safety_idx
|
|
|
|
|
|
def test_subagent_runtime_middlewares_place_mcp_routing_before_deferred_filter(monkeypatch):
|
|
from deerflow.agents.middlewares.deferred_tool_filter_middleware import DeferredToolFilterMiddleware
|
|
from deerflow.agents.middlewares.mcp_routing_middleware import McpRoutingMiddleware
|
|
from deerflow.tools.builtins.tool_search import DeferredToolSetup
|
|
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
routing = McpRoutingMiddleware({"mcp_thing": {"priority": 100, "keywords": ["orders"]}}, "hash123", 3)
|
|
setup = DeferredToolSetup(object(), frozenset({"mcp_thing"}), "hash123")
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, deferred_setup=setup, mcp_routing_middleware=routing)
|
|
|
|
routing_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, McpRoutingMiddleware))
|
|
filter_idx = next(i for i, middleware in enumerate(middlewares) if isinstance(middleware, DeferredToolFilterMiddleware))
|
|
assert routing_idx < filter_idx
|
|
|
|
|
|
def test_subagent_runtime_middlewares_skip_deferred_filter_without_names(monkeypatch):
|
|
"""No deferred setup (disabled / no MCP tool) -> no DeferredToolFilterMiddleware."""
|
|
from deerflow.agents.middlewares.deferred_tool_filter_middleware import DeferredToolFilterMiddleware
|
|
from deerflow.agents.middlewares.tool_promotion_audit_middleware import DeferredToolPromotionAuditMiddleware
|
|
from deerflow.tools.builtins.tool_search import DeferredToolSetup
|
|
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
for setup in (None, DeferredToolSetup(None, frozenset(), None)):
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, deferred_setup=setup)
|
|
assert not any(isinstance(m, DeferredToolFilterMiddleware) for m in middlewares)
|
|
assert not any(isinstance(m, DeferredToolPromotionAuditMiddleware) for m in middlewares)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_attach_loop_detection_when_enabled(monkeypatch):
|
|
"""Subagents must inherit the lead's LoopDetectionMiddleware so a degenerate
|
|
tool loop is broken instead of burning tokens until ``max_turns`` (#3875).
|
|
``loop_detection.enabled`` defaults to True, so the default subagent chain
|
|
carries the guard. Phase 1 of #3875."""
|
|
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
|
|
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
loop = [m for m in middlewares if isinstance(m, LoopDetectionMiddleware)]
|
|
assert len(loop) == 1
|
|
|
|
|
|
def test_subagent_runtime_middlewares_omit_loop_detection_when_disabled(monkeypatch):
|
|
"""``loop_detection.enabled=False`` must drop the guard from the subagent
|
|
chain, mirroring the lead's gate (``lead_agent/agent.py``)."""
|
|
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
|
|
from deerflow.config.loop_detection_config import LoopDetectionConfig
|
|
|
|
app_config = _make_app_config().model_copy(update={"loop_detection": LoopDetectionConfig(enabled=False)})
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
assert not any(isinstance(m, LoopDetectionMiddleware) for m in middlewares)
|
|
|
|
|
|
def test_subagent_runtime_middlewares_place_loop_detection_before_safety_finish(monkeypatch):
|
|
"""LoopDetectionMiddleware must be registered before SafetyFinishReasonMiddleware
|
|
(earlier in the middleware list). LangChain dispatches after_model hooks in
|
|
reverse registration order, so SafetyFinishReasonMiddleware (registered
|
|
later) executes first — the placement its docstring requires and the lead
|
|
chain (``lead_agent/agent.py``) uses. The assertion pins registration order,
|
|
not execution order."""
|
|
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
|
|
from deerflow.agents.middlewares.safety_finish_reason_middleware import SafetyFinishReasonMiddleware
|
|
|
|
app_config = _make_app_config()
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
loop_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, LoopDetectionMiddleware))
|
|
safety_idx = next(i for i, m in enumerate(middlewares) if isinstance(m, SafetyFinishReasonMiddleware))
|
|
assert loop_idx < safety_idx
|
|
|
|
|
|
def test_subagent_runtime_middlewares_attach_durable_context_before_summarization(monkeypatch):
|
|
"""Subagents must project ``summary_text`` back into model requests after
|
|
compaction, just like the lead agent does.
|
|
|
|
Without ``DurableContextMiddleware``, a message-count keep policy can
|
|
retain only an assistant tool-call plus its tool results. The summary is
|
|
stored in ``ThreadState.summary_text`` but never reaches the next request,
|
|
so strict providers reject the assistant-first history. The durable
|
|
context layer must use the same skill settings as the lead chain and run
|
|
before summarization.
|
|
"""
|
|
from deerflow.agents.middlewares import summarization_middleware as sm
|
|
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
|
|
|
|
sentinel = object()
|
|
captured: dict[str, object] = {}
|
|
|
|
def fake_create_summarization_middleware(
|
|
*,
|
|
app_config=None,
|
|
keep=None,
|
|
skip_memory_flush=False,
|
|
run_model_name=None,
|
|
extensions=None,
|
|
):
|
|
captured["app_config"] = app_config
|
|
captured["keep"] = keep
|
|
captured["skip_memory_flush"] = skip_memory_flush
|
|
captured["run_model_name"] = run_model_name
|
|
captured["extensions"] = extensions
|
|
return sentinel
|
|
|
|
# summarization is enabled by default False; flip it on so the factory path
|
|
# is taken (the factory early-returns None when disabled).
|
|
from deerflow.config.summarization_config import SummarizationConfig
|
|
|
|
app_config = _make_app_config().model_copy(update={"summarization": SummarizationConfig(enabled=True)})
|
|
monkeypatch.setattr(sm, "create_summarization_middleware", fake_create_summarization_middleware)
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
# The shared factory received the same app_config the builder did (no lead
|
|
# wrapper, no config drift between the two chains).
|
|
assert captured["app_config"] is app_config
|
|
# skip_memory_flush=True so subagent-internal turns are not flushed into the
|
|
# PARENT thread's durable memory (#3875 Phase 3 review).
|
|
assert captured["skip_memory_flush"] is True
|
|
# Model ownership: the subagent's own resolved model is threaded into the factory
|
|
# so a distinct-model subagent summarizes with its model, not the parent's — the
|
|
# subagent context/configurable never carries the child model.
|
|
assert captured["run_model_name"] == "test-model"
|
|
assert captured["extensions"] is not None
|
|
durable = [middleware for middleware in middlewares if isinstance(middleware, DurableContextMiddleware)]
|
|
assert len(durable) == 1
|
|
# ``_skills_root`` is ``posixpath.normpath(container_path)``, so compare against
|
|
# the normalized form — a trailing slash / ``.`` / ``..`` in config would fail
|
|
# a raw equality even though the wiring is correct.
|
|
assert durable[0]._skills_root == posixpath.normpath(app_config.skills.container_path)
|
|
assert durable[0]._skill_read_tool_names == frozenset(app_config.summarization.skill_file_read_tool_names)
|
|
assert middlewares.index(durable[0]) < middlewares.index(sentinel)
|
|
|
|
|
|
def test_subagent_compaction_injects_summary_before_assistant_tool_tail(monkeypatch):
|
|
"""A three-tool turn with ``keep=4`` must remain provider-valid.
|
|
|
|
This reproduces the production failure shape: compaction preserves an
|
|
assistant tool-call plus three tool results while removing the original
|
|
system/user messages. The subagent chain must inject the generated summary
|
|
as durable human context before that tail reaches the model.
|
|
"""
|
|
from langchain.agents import create_agent
|
|
from langchain_core.language_models import BaseChatModel
|
|
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
|
|
from langchain_core.outputs import ChatGeneration, ChatResult
|
|
|
|
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
|
|
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware
|
|
from deerflow.agents.middlewares.system_message_coalescing_middleware import SystemMessageCoalescingMiddleware
|
|
from deerflow.agents.thread_state import ThreadState
|
|
from deerflow.config.summarization_config import ContextSize, SummarizationConfig
|
|
|
|
class _StaticModel(BaseChatModel):
|
|
text: str
|
|
require_durable_summary: bool = False
|
|
|
|
@property
|
|
def _llm_type(self) -> str:
|
|
return "static"
|
|
|
|
def bind_tools(self, tools, **kwargs):
|
|
return self
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
if self.require_durable_summary:
|
|
first_ai = next(i for i, message in enumerate(messages) if isinstance(message, AIMessage))
|
|
durable = [(i, message) for i, message in enumerate(messages) if isinstance(message, HumanMessage) and message.additional_kwargs.get("durable_context_data")]
|
|
assert durable, "compacted summary must be injected into the subagent request"
|
|
assert durable[0][0] < first_ai, "durable summary must precede the assistant/tool tail"
|
|
assert "COMPRESSED_SUBAGENT_HISTORY" in durable[0][1].content
|
|
# DurableContext injects a SystemMessage(authority); without the
|
|
# coalescer the request would carry it as a second/non-leading
|
|
# system message, which strict providers reject (#4040). Assert the
|
|
# outgoing request is provider-valid: a single leading SystemMessage.
|
|
system_indices = [i for i, message in enumerate(messages) if isinstance(message, SystemMessage)]
|
|
assert system_indices == [0], f"request must have exactly one leading SystemMessage, got {system_indices}"
|
|
return ChatResult(generations=[ChatGeneration(message=AIMessage(content=self.text))])
|
|
|
|
summary_model = _StaticModel(text="COMPRESSED_SUBAGENT_HISTORY")
|
|
strict_model = _StaticModel(text="final answer", require_durable_summary=True)
|
|
monkeypatch.setattr(
|
|
"deerflow.agents.middlewares.summarization_middleware.create_chat_model",
|
|
lambda **kwargs: summary_model,
|
|
)
|
|
|
|
app_config = _make_app_config().model_copy(
|
|
update={
|
|
"summarization": SummarizationConfig(
|
|
enabled=True,
|
|
trigger=ContextSize(type="messages", value=5),
|
|
keep=ContextSize(type="messages", value=4),
|
|
)
|
|
}
|
|
)
|
|
runtime_middlewares = build_subagent_runtime_middlewares(
|
|
app_config=app_config,
|
|
model_name="test-model",
|
|
agent_name="general-purpose",
|
|
)
|
|
compaction_middlewares = [middleware for middleware in runtime_middlewares if isinstance(middleware, (DurableContextMiddleware, DeerFlowSummarizationMiddleware, SystemMessageCoalescingMiddleware))]
|
|
agent = create_agent(
|
|
model=strict_model,
|
|
tools=[],
|
|
middleware=compaction_middlewares,
|
|
state_schema=ThreadState,
|
|
)
|
|
|
|
tool_calls = [{"name": "web_search", "args": {"query": f"q{i}"}, "id": f"call_{i}", "type": "tool_call"} for i in range(3)]
|
|
seed = [
|
|
SystemMessage(content="subagent instructions", id="system"),
|
|
HumanMessage(content="research three regions", id="human"),
|
|
AIMessage(content="searching", tool_calls=tool_calls, id="assistant"),
|
|
*[ToolMessage(content=f"result {i}", tool_call_id=f"call_{i}", id=f"tool_{i}") for i in range(3)],
|
|
]
|
|
|
|
result = agent.invoke({"messages": seed})
|
|
|
|
assert result["summary_text"] == "COMPRESSED_SUBAGENT_HISTORY"
|
|
assert result["messages"][-1].content == "final answer"
|
|
|
|
|
|
def test_subagent_chain_coalesces_durable_authority_system_message(monkeypatch):
|
|
"""The durable-context authority SystemMessage must not survive as a second one.
|
|
|
|
Subagents carry their system prompt as a leading ``SystemMessage`` in
|
|
``messages`` (``create_agent(system_prompt=None)``), and
|
|
``DurableContextMiddleware`` inserts ``SystemMessage(authority_contract)``
|
|
directly after it whenever durable data (summary / delegations / skills) is
|
|
present. That leaves two adjacent system messages — the exact non-leading /
|
|
duplicate-system shape strict OpenAI-compatible providers reject and the
|
|
same #4039 failure class the durable fix set out to avoid.
|
|
|
|
``build_subagent_runtime_middlewares`` must therefore pair durable context
|
|
with ``SystemMessageCoalescingMiddleware`` (#4040). This drives the real
|
|
builder output through a strict model and asserts the outgoing request keeps
|
|
exactly one leading ``SystemMessage``. Remove the coalescer from the builder
|
|
and the model sees ``[System(base), System(authority), ...]`` and this fails.
|
|
"""
|
|
from langchain.agents import create_agent
|
|
from langchain_core.language_models import BaseChatModel
|
|
from langchain_core.messages import AIMessage, SystemMessage, ToolMessage
|
|
from langchain_core.outputs import ChatGeneration, ChatResult
|
|
|
|
from deerflow.agents.middlewares.durable_context_middleware import DurableContextMiddleware
|
|
from deerflow.agents.middlewares.system_message_coalescing_middleware import SystemMessageCoalescingMiddleware
|
|
from deerflow.agents.thread_state import ThreadState
|
|
|
|
seen: dict[str, list[int]] = {}
|
|
|
|
class _StrictModel(BaseChatModel):
|
|
@property
|
|
def _llm_type(self) -> str:
|
|
return "strict"
|
|
|
|
def bind_tools(self, tools, **kwargs):
|
|
return self
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
seen["system_indices"] = [i for i, message in enumerate(messages) if isinstance(message, SystemMessage)]
|
|
return ChatResult(generations=[ChatGeneration(message=AIMessage(content="ok"))])
|
|
|
|
app_config = _make_app_config()
|
|
runtime_middlewares = build_subagent_runtime_middlewares(
|
|
app_config=app_config,
|
|
model_name="test-model",
|
|
agent_name="general-purpose",
|
|
)
|
|
# Isolate the two middlewares under test, preserving builder order. The
|
|
# coalescer must come after (inner of) durable context to observe the
|
|
# injected system message.
|
|
chain = [m for m in runtime_middlewares if isinstance(m, (DurableContextMiddleware, SystemMessageCoalescingMiddleware))]
|
|
assert [type(m).__name__ for m in chain] == ["DurableContextMiddleware", "SystemMessageCoalescingMiddleware"]
|
|
|
|
agent = create_agent(model=_StrictModel(), tools=[], middleware=chain, state_schema=ThreadState)
|
|
|
|
# A leading system prompt plus an assistant tool-call tail, with a summary
|
|
# already in state so durable context injects its authority SystemMessage.
|
|
seed = [
|
|
SystemMessage(content="subagent instructions", id="system"),
|
|
AIMessage(content="searching", tool_calls=[{"name": "web_search", "args": {"query": "x"}, "id": "call_0", "type": "tool_call"}], id="assistant"),
|
|
ToolMessage(content="result", tool_call_id="call_0", id="tool_0"),
|
|
]
|
|
agent.invoke({"messages": seed, "summary_text": "COMPRESSED_SUBAGENT_HISTORY"})
|
|
|
|
assert seen["system_indices"] == [0], f"request must have a single leading SystemMessage, got {seen['system_indices']}"
|
|
|
|
|
|
def test_subagent_chain_injects_date_without_memory_and_coalesces_for_strict_provider(monkeypatch):
|
|
"""A built-in subagent's first model request gets hidden date-only context.
|
|
|
|
The date must be framework-owned and independent of the lead agent's
|
|
memory path, even when memory injection is enabled globally. Subagents
|
|
carry their static prompt in ``messages``, so the outgoing strict-provider
|
|
payload must also retain exactly one leading ``SystemMessage`` after the
|
|
date reminder is added.
|
|
"""
|
|
from langchain.agents import create_agent
|
|
from langchain_core.language_models import BaseChatModel
|
|
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage
|
|
from langchain_core.outputs import ChatGeneration, ChatResult
|
|
|
|
from deerflow.agents.middlewares import dynamic_context_middleware as dynamic_context
|
|
from deerflow.agents.middlewares.system_message_coalescing_middleware import SystemMessageCoalescingMiddleware
|
|
from deerflow.agents.thread_state import ThreadState
|
|
|
|
class _FrozenDateTime:
|
|
@classmethod
|
|
def now(cls):
|
|
return datetime(2026, 5, 8)
|
|
|
|
def _unexpected_memory_lookup(*args, **kwargs):
|
|
raise AssertionError("subagent date context must not look up user memory")
|
|
|
|
seen: list[list] = []
|
|
task = "Find releases published today."
|
|
|
|
class _StrictModel(BaseChatModel):
|
|
@property
|
|
def _llm_type(self) -> str:
|
|
return "strict"
|
|
|
|
def bind_tools(self, tools, **kwargs):
|
|
return self
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
captured = list(messages)
|
|
seen.append(captured)
|
|
|
|
system_indices = [i for i, message in enumerate(captured) if isinstance(message, SystemMessage)]
|
|
assert system_indices == [0], f"strict provider must receive one leading SystemMessage, got {system_indices}"
|
|
|
|
system_message = captured[0]
|
|
reminder_blocks = re.findall(r"<system-reminder>\s*(.*?)\s*</system-reminder>", system_message.content, re.DOTALL)
|
|
assert reminder_blocks == ["<current_date>2026-05-08, Friday</current_date>"]
|
|
assert "<memory>" not in system_message.content
|
|
assert system_message.additional_kwargs.get("hide_from_ui") is True
|
|
|
|
human_messages = [message for message in captured if isinstance(message, HumanMessage)]
|
|
assert len(human_messages) == 1
|
|
assert human_messages[0].content == task
|
|
assert human_messages[0].id == "task"
|
|
return ChatResult(generations=[ChatGeneration(message=AIMessage(content="ok"))])
|
|
|
|
app_config = _make_app_config()
|
|
assert app_config.memory.injection_enabled is True
|
|
monkeypatch.setattr(dynamic_context, "datetime", _FrozenDateTime)
|
|
monkeypatch.setattr("deerflow.agents.lead_agent.prompt._get_memory_context", _unexpected_memory_lookup)
|
|
|
|
runtime_middlewares = build_subagent_runtime_middlewares(
|
|
app_config=app_config,
|
|
model_name="test-model",
|
|
agent_name="general-purpose",
|
|
)
|
|
# Exercise the builder-owned date/coalescing slice without unrelated
|
|
# sandbox, tool, or skill middleware side effects.
|
|
chain = [middleware for middleware in runtime_middlewares if type(middleware).__module__ == dynamic_context.__name__ or isinstance(middleware, SystemMessageCoalescingMiddleware)]
|
|
agent = create_agent(model=_StrictModel(), tools=[], middleware=chain, state_schema=ThreadState)
|
|
|
|
agent.invoke(
|
|
{
|
|
"messages": [
|
|
SystemMessage(content="subagent instructions", id="system"),
|
|
HumanMessage(content=task, id="task"),
|
|
]
|
|
}
|
|
)
|
|
|
|
assert len(seen) == 1
|
|
|
|
|
|
def test_subagent_runtime_middlewares_omit_summarization_when_factory_returns_none(monkeypatch):
|
|
"""When ``summarization.enabled`` is False the shared factory returns None and
|
|
the subagent chain must NOT carry a summarization middleware — the default
|
|
state, since SummarizationConfig.enabled defaults to False."""
|
|
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware
|
|
|
|
app_config = _make_app_config() # summarization.enabled defaults to False
|
|
_stub_runtime_middleware_imports(monkeypatch)
|
|
|
|
middlewares = build_subagent_runtime_middlewares(app_config=app_config, model_name="test-model")
|
|
|
|
assert not any(isinstance(m, DeerFlowSummarizationMiddleware) for m in middlewares)
|
|
|
|
|
|
def test_lead_runtime_chain_finds_historical_uploads_under_lazy_init_false(tmp_path, monkeypatch):
|
|
"""Integration anchor for the ThreadData → Uploads ordering.
|
|
|
|
Under lazy_init=False, ThreadDataMiddleware eagerly creates the thread
|
|
directories in before_agent. UploadsMiddleware then only injects
|
|
<current_uploads> for new files — historical uploads are discovered
|
|
on demand via list_uploaded_files, not injected every turn.
|
|
|
|
This complements the static order contract
|
|
(test_build_lead_runtime_middlewares_orders_thread_data_before_uploads):
|
|
that test pins the chain position; this test pins the observable behavior
|
|
at that position.
|
|
"""
|
|
from langchain_core.messages import HumanMessage
|
|
from langgraph.runtime import Runtime
|
|
|
|
from deerflow.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
|
|
from deerflow.agents.middlewares.uploads_middleware import UploadsMiddleware
|
|
from deerflow.config.paths import Paths
|
|
from deerflow.runtime.user_context import get_effective_user_id
|
|
|
|
thread_id = "thread-historical-files"
|
|
user_id = get_effective_user_id()
|
|
|
|
paths = Paths(str(tmp_path))
|
|
uploads_dir = paths.sandbox_uploads_dir(thread_id, user_id=user_id)
|
|
uploads_dir.mkdir(parents=True, exist_ok=True)
|
|
(uploads_dir / "prior-report.txt").write_bytes(b"historical payload")
|
|
|
|
td = ThreadDataMiddleware(base_dir=str(tmp_path), lazy_init=False)
|
|
um = UploadsMiddleware(base_dir=str(tmp_path))
|
|
|
|
runtime = Runtime(context={"thread_id": thread_id, "run_id": "run-1"})
|
|
state = {"messages": [HumanMessage(content="please summarise the prior upload")]}
|
|
|
|
td_result = td.before_agent(state, runtime)
|
|
assert td_result is not None, "ThreadDataMiddleware must run and produce state updates"
|
|
# Sanity: under lazy_init=False the directories were created (not just computed).
|
|
assert uploads_dir.exists(), "ThreadDataMiddleware should have ensured the uploads directory exists"
|
|
|
|
# ThreadDataMiddleware rewrites the last HumanMessage (annotating run_id/timestamp);
|
|
# carry its updated messages into the UploadsMiddleware input state, mirroring
|
|
# how LangGraph chains before_agent outputs into the next middleware.
|
|
um_input = {**state, "messages": td_result["messages"]}
|
|
um_result = um.before_agent(um_input, runtime)
|
|
|
|
# Historical files are NO LONGER injected — only new (current-run) uploads.
|
|
# The prior-report.txt file exists in the uploads dir from a previous turn,
|
|
# so UploadsMiddleware must NOT inject it into the prompt.
|
|
# It MUST however clear uploaded_files so list_uploaded_files doesn't
|
|
# incorrectly exclude files that just became historical.
|
|
assert um_result == {"uploaded_files": []}, "UploadsMiddleware must NOT inject context for historical files, but MUST clear uploaded_files to prevent cross-turn state leakage into list_uploaded_files"
|
|
|
|
|
|
def test_subagent_summarization_fires_mid_run_and_produces_usable_result(monkeypatch):
|
|
"""Integration coverage for #3875 Phase 3 review gap: drive the REAL
|
|
``DeerFlowSummarizationMiddleware`` (the exact instance the subagent chain
|
|
gets via ``create_summarization_middleware(skip_memory_flush=True)``) through
|
|
a ``create_agent`` run, and assert that (a) compaction actually fires mid-run
|
|
(messages channel contracts via ``RemoveMessage``) and (b) the run still
|
|
completes with a usable final answer — not just wiring.
|
|
|
|
The builder-wiring test above proves the middleware lands on the chain; this
|
|
proves the live middleware triggers and the run survives it. We bypass the
|
|
full ``build_subagent_runtime_middlewares`` chain (whose sandbox/thread-data
|
|
stubs aren't AgentMiddleware-compatible for a live run) and use the factory
|
|
directly — the same instance the builder appends."""
|
|
from langchain.agents import create_agent
|
|
from langchain_core.language_models import BaseChatModel
|
|
from langchain_core.messages import AIMessage, HumanMessage, RemoveMessage
|
|
from langchain_core.outputs import ChatGeneration, ChatResult
|
|
|
|
from deerflow.agents.middlewares.summarization_middleware import (
|
|
DeerFlowSummarizationMiddleware,
|
|
create_summarization_middleware,
|
|
)
|
|
from deerflow.agents.thread_state import ThreadState
|
|
from deerflow.config.memory_config import MemoryConfig
|
|
from deerflow.config.summarization_config import ContextSize, SummarizationConfig
|
|
|
|
# A model that always emits a plain AIMessage — no tools, so the run is a
|
|
# single turn but the input already exceeds the trigger threshold, forcing
|
|
# before_model compaction on the first (and only) model call.
|
|
class _StaticModel(BaseChatModel):
|
|
text: str = "final answer after compaction"
|
|
|
|
@property
|
|
def _llm_type(self) -> str:
|
|
return "static"
|
|
|
|
def bind_tools(self, tools, **kwargs):
|
|
return self
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
return ChatResult(generations=[ChatGeneration(message=AIMessage(content=self.text))])
|
|
|
|
static_model = _StaticModel()
|
|
# The factory resolves its summary model via create_chat_model; point it at
|
|
# the same static model so no real provider is contacted.
|
|
monkeypatch.setattr(
|
|
"deerflow.agents.middlewares.summarization_middleware.create_chat_model",
|
|
lambda **kwargs: static_model,
|
|
)
|
|
|
|
app_config = SimpleNamespace(
|
|
summarization=SummarizationConfig(
|
|
enabled=True,
|
|
trigger=ContextSize(type="messages", value=4),
|
|
keep=ContextSize(type="messages", value=2),
|
|
),
|
|
# memory disabled + skip_memory_flush=True mirrors the subagent path:
|
|
# no memory_flush_hook is attached.
|
|
memory=MemoryConfig(enabled=False),
|
|
)
|
|
middleware = create_summarization_middleware(
|
|
app_config=app_config,
|
|
skip_memory_flush=True,
|
|
)
|
|
assert isinstance(middleware, DeerFlowSummarizationMiddleware), "the real middleware must be built"
|
|
# Subagent invariant: skip_memory_flush means no durable-memory hook.
|
|
assert not middleware._before_summarization_hooks
|
|
|
|
agent = create_agent(
|
|
model=static_model,
|
|
tools=[],
|
|
middleware=[middleware],
|
|
state_schema=ThreadState,
|
|
)
|
|
|
|
# 6 messages > trigger(4) → compaction must fire in before_model.
|
|
seed = [
|
|
HumanMessage(content="q1", id="h1"),
|
|
AIMessage(content="a1", id="a1"),
|
|
HumanMessage(content="q2", id="h2"),
|
|
AIMessage(content="a2", id="a2"),
|
|
HumanMessage(content="q3", id="h3"),
|
|
AIMessage(content="a3", id="a3"),
|
|
]
|
|
chunks = list(agent.stream({"messages": seed}, stream_mode="updates"))
|
|
|
|
# (a) Compaction fired: the middleware's before_model emitted a summary + RemoveMessage.
|
|
before_model_chunks = [c for c in chunks if "DeerFlowSummarizationMiddleware.before_model" in c]
|
|
assert before_model_chunks, "summarization before_model must fire when messages exceed the trigger"
|
|
summary_update = before_model_chunks[0]["DeerFlowSummarizationMiddleware.before_model"]
|
|
assert summary_update.get("summary_text"), "a summary must be produced"
|
|
emitted = summary_update["messages"]
|
|
assert isinstance(emitted[0], RemoveMessage), "compaction must lead with RemoveMessage"
|
|
|
|
# (b) The run completed with a usable final AIMessage despite compaction.
|
|
# The model's output surfaces under the "model" node key in updates mode.
|
|
final_messages: list = []
|
|
for chunk in chunks:
|
|
node_msg = chunk.get("model") or chunk.get("agent") or {}
|
|
final_messages = node_msg.get("messages", final_messages)
|
|
ai_finals = [m for m in final_messages if isinstance(m, AIMessage)]
|
|
assert ai_finals, "the run must produce a final AIMessage after compaction"
|
|
assert ai_finals[-1].content == "final answer after compaction"
|
|
|
|
|
|
def test_build_lead_runtime_middlewares_passes_read_before_write_config():
|
|
"""The gate's model-bound payload elision is configured from app_config.read_before_write."""
|
|
from deerflow.agents.middlewares.read_before_write_middleware import ReadBeforeWriteMiddleware
|
|
from deerflow.config.read_before_write_config import ReadBeforeWriteConfig
|
|
|
|
app_config = _make_app_config().model_copy(update={"read_before_write": ReadBeforeWriteConfig(elide_min_chars=321)})
|
|
middlewares = build_lead_runtime_middlewares(app_config=app_config)
|
|
|
|
gates = [m for m in middlewares if isinstance(m, ReadBeforeWriteMiddleware)]
|
|
assert len(gates) == 1
|
|
# Only the wired value is under test; the full policy identity is covered by the middleware's own tests.
|
|
assert gates[0].release_policy_parameters()["config"]["elide_min_chars"] == 321
|