mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
* feat(memory): redact queued memory payloads (pii_redaction follow-up) Closes the remaining vector from #3190 (memory storage): request-scoped redaction leaves originals in thread state, and MemoryMiddleware queued that raw conversation into the durable extraction buffer — so the extraction model's input and the persisted facts carried raw PII even with pii_redaction.enabled. - MemoryMiddleware gains pii_redaction_config (same pattern as summarization / durable-context / title); the payload returned by _resolve_add_args() is redacted at the enqueue boundary before add()/aadd(), covering both after_agent and aafter_agent - Reuses the shared redact_text() seam from #5527 via active_pii_detectors/_Redactor/_redact_content; the buffered payload itself becomes clean, not just the extraction model's view - Wired at the lead-agent assembly site - Tests: queue-payload redaction enabled/disabled, detector toggles, message immutability, token stability across turns * fix(memory): reserve existing placeholder indices across the queued batch Review round 1 on #5577: thread state mixes raw user turns with already-redacted tool results and summaries, so a token in a later message must not collide with a new value in an earlier one. Pre-scan the whole queued batch with redactor.reserve() before allocating. Also extend the two MemoryMiddleware stubs in test_lead_agent_model_resolution.py to accept the new pii_redaction_config kwarg (CI backend-unit-tests red). * fix(memory): redact the compaction-triggered memory_flush_hook queue too Review round 2 on #5577: with memory + summarization enabled, memory_flush_hook() queues list(event.messages_to_summarize) directly into MemoryManager.add_nowait(), bypassing MemoryMiddleware entirely — and compaction removes those messages from state right after, so the after-agent redaction can never repair a raw batch queued there. Extract the batch redaction (reserve pre-scan + rebuild) into a shared redact_queued_messages() used by both MemoryMiddleware's enqueue boundary and the hook; the hook gains an optional pii_redaction_config wired via functools.partial at the summarization factory (subagents use skip_memory_flush=True and are unaffected). Regression covers the hook path enabled/disabled/absent. * test(summarization): unwrap the partial in the factory hook identity check edd916e wrapped memory_flush_hook in functools.partial to carry the pii_redaction config, but missed committing the matching update to test_factory_attaches_memory_flush_hook_by_default — a functools.partial is not equal to its wrapped function, so the identity assertion failed deterministically (caught by static review on #5577). * fix(memory): cover tool-call arguments and lenient text blocks in queued payloads Review round 4 on #5577 — two reproduced gaps still allowed raw PII into memory payloads with pii_redaction.enabled=true: - OpenViking retains the full message object, so parsed tool_calls args and provider-format arguments in additional_kwargs bypassed the content-only redaction; queued payloads now redact tool-call args recursively (and reserve them in the pre-scan). - DeerMem's format_conversation_for_update reads p.get("text") of any dict block, not only type=="text" — _redact_content now redacts every string-valued text field so a lenient consumer cannot bypass it. Both repro shapes are pinned as regressions. * fix(pii): value-derived placeholders for stable capture identity Review round 5 on #5577: sequential allocation is order-dependent — the same unchanged message re-redacted in a later batch can get a different token, which flips OpenViking's content-signature and replays already recorded turns into persistent memory sessions. Placeholders are now a pure function of the matched value (sha256-6 per category): identities stay stable across batches, enqueues, seams, and backends with no shared state and no stored mapping. - _Redactor becomes stateless; the reserve machinery (index pre-scans, _PLACEHOLDER_PATTERN) is removed along with its call sites - redact_texts keeps its signature, now a simple per-text redaction - reserve-semantics tests are superseded by cross-enqueue identity tests (two successive captures where the second batch carries higher existing placeholders); the compiled-graph echo regex matches the new format * fix(pii): 128-bit placeholder digests; redact tool-argument keys Review round 6 on #5577: - [P2] The 24-bit truncation collided distinct identities (reproduced: contact3513@ / contact3727@example.com both rendered [EMAIL_575dbb]). Retain 128 bits of the digest so separate people stay separate at any realistic volume; the compiled-graph echo regex and the AGENTS.md entry follow the new format. - [P2] _redact_strings redacted values but preserved mapping keys, so {"contacts": {"alice@example.com": "manager"}} kept the raw email as a key. Redact string keys along with values; deterministic 128-bit tokens make distinct-key collisions a hash-collision non-event, and a redacted key equal to another key merges the entries. Regression pins the email-keyed arguments shape. * docs(agents): trim PiiRedactionMiddleware entry; drop stale reserve wording The chain grew past the 98304-byte AG002 hard limit again (main growth plus the round-6 format example). Trim the entry to fit, and replace the stale reserve-indices wording with the value-derived semantics that replaced it. * docs(agents): shrink PiiRedactionMiddleware entry below the chain budget main moved (#5609) and the base comparison flipped the nearly-over chain back to an AG002 error (98441 > 98304). Trim the entry to the facts that are not documented elsewhere: seams, allowlist, order, enqueue boundary. * fix(pii): letters-only value-derived tokens; wire the factory SDK path Review round 7 on #5577: - [P2] Hex digests carry digit runs that later digit-anchored detectors re-scan, corrupting minted tokens into nested placeholders (reproduced: user280@example.com -> [EMAIL_…[PHONE_…]…], ~1/140 emails). The 128-bit digest is now encoded over a-z, so a minted token survives the full pinned detector order. - [P2] create_deerflow_agent (the SDK path) constructed MemoryMiddleware and DurableContextMiddleware without pii_redaction_config; the knob is now an explicit optional parameter threaded through _assemble_from_features (the factory reads no config files by design, so SDK callers pass it explicitly). - [Nit] hoist the Command rebuild comprehension out of the per-message loop in _redact_result. - [Nit] drop the dead _iter_tool_calls helper and Iterator import. * test(factory): pin pii_redaction_config wiring; document the parameter Review round 8 on #5577: - [P3] test_create_deerflow_agent.py had zero coverage of the pii_redaction_config parameter; two regressions now assert that create_deerflow_agent(pii_redaction_config=cfg) builds a MemoryMiddleware and a DurableContextMiddleware carrying cfg, and that the default is None (redaction off). - [Nit] document the parameter in the create_deerflow_agent docstring: None leaves redaction off; a middleware= takeover bypasses the assembly that would apply it. * fix(pii): HMAC-scoped placeholder tokens; install model-bound redaction in the SDK chain Review round 10 on #5577: - [P2] sha256(category||value) is an unkeyed, globally deterministic fingerprint: a provider or memory backend can confirm candidate values offline and link identical values across users and threads, contradicting the documented no-re-identification guarantee. Placeholders are now HMAC-derived over a deployment-scoped token_secret (new config field); the docstring documents the linkability scope for both modes. - [P2] _assemble_from_features() never appended PiiRedactionMiddleware, so the SDK knob enabled only the downstream seams while the primary model still received raw content. The middleware is now appended in the same relative position as the lead-agent base chain when the gate is on, with a model-call regression through the assembled SDK chain. * docs(pii): document token_secret; update the placeholder format examples Follow-up to the round-10 review on #5577: - config.example.yaml documents the new token_secret knob: a deployment-scoped secret makes placeholders HMAC-derived and only linkable within one deployment; leaving it unset keeps publicly computable fingerprints (documented trade). - the PiiRedactionConfig class docstring's placeholder example now matches the shipped format (128-bit HMAC digests over base-26 letters). * docs(pii): update the module docstring's placeholder format example The module docstring still showed the sequential [EMAIL_1] example; align it with the shipped value-derived format (128-bit HMAC digests over base-26 letters). * fix(pii): require a non-empty token_secret when redaction is enabled Review round 11 on #5577: with the secret optional, the default enabled configuration minted empty-key HMAC digests — publicly computable, globally linkable fingerprints of the raw values, confirmable by offline guessing for low-entropy identifiers such as phone numbers. - PiiRedactionConfig now rejects enabled=true without a non-empty token_secret (minimum 16 characters); the unkeyed mode is gone entirely. - The request-scoped redactor in _process_request now carries the configured key: it built its redactor without it, so user-message placeholders silently diverged from every other seam for the same value. The empty-key guard in _placeholder_token is the backstop that keeps every path keyed. - Module/config docstrings and config.example.yaml no longer describe the placeholders as irreversible or the secret as optional; the stale reserve-semantics wording and the "memory path not covered" note are corrected. - Regressions: validation errors for missing/blank/short secrets, and the user-message boundary minting the same keyed token as redact_text. * fix(memory): redact original_user_content on queued memory payloads Review round 12 on #5577: UploadsMiddleware preserves the raw user turn in additional_kwargs.original_user_content, but redact_queued_messages rewrote only content, tool_calls, and provider-format tool_calls — the queued copy still carried the raw value, so the memory queue and backend received raw PII with redaction enabled. The shared helper now redacts the provenance string field on the queued copy (same batch redactor, merged with the existing additional_kwargs rewrite, rebuilt via model_copy — thread-state messages stay untouched), covering both the enqueue boundary and the compaction flush hook. Regressions pin the redacted provenance field through both paths and the untouched original. * fix(memory): redact human_input_response on queued memory payloads Review round 13 on #5577: clarification replies keep the raw user answer in additional_kwargs.human_input_response.value (DeerMem reads that mapping via its read_human_input_response mirror), but redact_queued_messages rewrote only content, tool_calls, and original_user_content — the queued copy still carried the raw value into the memory queue and backend. The shared helper now rebuilds that mapping with value redacted (same batch redactor) while preserving the protocol fields (version/kind/source/ request_id/option_id/response_kind), merged with the existing additional_kwargs rewrite and rebuilt via model_copy so thread-state messages stay untouched. Regressions pin the redacted value, preserved protocol fields, the untouched original, and the flush-hook path. * chore(config): bump config_version to 47; generate token_secret on upgrade Review round 13 on #5577: existing deployments can legitimately sit at config_version 46 with pii_redaction.enabled: true and no token_secret; the mandatory-secret validator would fail their startup while config.example.yaml still said 46, so the upgrade checker considered them current and emitted no migration warning. - Bump config_version to 47 so the outdated-config warning fires for v46. - Add a v47 migration to scripts/config-upgrade.sh: when pii_redaction is enabled and token_secret is missing/blank, generate a random deployment- scoped secret (secrets.token_urlsafe) and persist it, so the upgraded deployment starts under the mandatory validation. - Regressions: a v46 enabled config is flagged outdated against the real example; the upgrade run generates a >= 16-char secret and the result passes AppConfig validation; a disabled config gets no secret. - Also switch the two v41 script tests from a hardcoded "bash" to SCRIPT_BASH/skipif, matching the documented Windows constraint and the neighboring tests (hardcoded bash resolves to the WSL stub there). * chore(helm): sync chart config_version to 47 Review round 14 on #5577: check_config_version.sh fails whenever the chart's embedded config_version lags config.example.yaml, and the chart workflow is triggered by config.example.yaml — so the v47 bump left this head failing that check. Update the values.yaml config block and the README example to 47; scripts/check_config_version.sh now passes. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
1214 lines
46 KiB
Python
1214 lines
46 KiB
Python
"""Tests for create_deerflow_agent SDK entry point."""
|
|
|
|
from typing import get_type_hints
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import pytest
|
|
from langchain.agents import AgentState
|
|
from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
|
|
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
|
|
from langchain_core.tools import tool
|
|
from langgraph.channels import DeltaChannel
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
from deerflow.agents.factory import create_deerflow_agent
|
|
from deerflow.agents.features import Next, Prev, RuntimeFeatures
|
|
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware
|
|
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
|
|
from deerflow.agents.thread_state import DeltaThreadState, ThreadState
|
|
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
|
from deerflow.config.subagent_batches_config import SubagentBatchesConfig
|
|
from deerflow.config.subagent_runtime_config import SubagentRuntimeConfig
|
|
from deerflow.subagents import SubagentRuntime
|
|
|
|
|
|
def _make_mock_model():
|
|
return MagicMock(name="mock_model")
|
|
|
|
|
|
class _FakeModel(FakeMessagesListChatModel):
|
|
def bind_tools(self, tools, **kwargs): # type: ignore[override]
|
|
return self
|
|
|
|
|
|
class _CustomState(AgentState):
|
|
custom_value: str
|
|
|
|
|
|
def _make_mock_tool(name: str = "my_tool"):
|
|
tool = MagicMock(name=name)
|
|
tool.name = name
|
|
return tool
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. Minimal creation — only model
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_minimal_creation(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
model = _make_mock_model()
|
|
|
|
result = create_deerflow_agent(model)
|
|
|
|
mock_create_agent.assert_called_once()
|
|
assert result is mock_create_agent.return_value
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["model"] is model
|
|
assert call_kwargs["system_prompt"] is None
|
|
assert call_kwargs["state_schema"] is ThreadState
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_delta_creation_selects_delta_state_and_copies_middleware(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
middleware = ViewImageMiddleware()
|
|
original_schema = middleware.state_schema
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[middleware],
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args.kwargs
|
|
assert call_kwargs["state_schema"] is DeltaThreadState
|
|
assert call_kwargs["middleware"][0] is not middleware
|
|
assert middleware.state_schema is original_schema
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_state_schema_is_preserved_in_full_mode_and_adapted_in_delta_mode(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
|
|
create_deerflow_agent(_make_mock_model(), state_schema=_CustomState)
|
|
assert mock_create_agent.call_args.kwargs["state_schema"] is _CustomState
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
state_schema=_CustomState,
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
adapted = mock_create_agent.call_args.kwargs["state_schema"]
|
|
hints = get_type_hints(adapted, include_extras=True)
|
|
assert "custom_value" in hints
|
|
assert any(isinstance(item, DeltaChannel) for item in hints["messages"].__metadata__)
|
|
|
|
|
|
def test_delta_checkpointer_combination_is_rejected_before_any_persistence():
|
|
"""Mixed-mode corruption guard: delta + checkpointer must fail loudly at
|
|
construction, before any state is read or written through the ungated graph."""
|
|
saver = InMemorySaver()
|
|
with pytest.raises(ValueError, match="checkpoint_channel_mode='delta'"):
|
|
create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(content="ok")]),
|
|
checkpoint_channel_mode="delta",
|
|
checkpointer=saver,
|
|
)
|
|
assert list(saver.list(None)) == []
|
|
|
|
|
|
def test_compiled_factory_graph_selects_full_and_delta_message_channels():
|
|
full_graph = create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(id="full-response", content="done")]),
|
|
)
|
|
delta_graph = create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(id="delta-response", content="done")]),
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
|
|
assert type(full_graph.channels["messages"]).__name__ == "BinaryOperatorAggregate"
|
|
assert isinstance(delta_graph.channels["messages"], DeltaChannel)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. With tools
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_with_tools(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
model = _make_mock_model()
|
|
tool = _make_mock_tool("search")
|
|
|
|
create_deerflow_agent(model, tools=[tool])
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "search" in tool_names
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. With system_prompt
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_with_system_prompt(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
prompt = "You are a helpful assistant."
|
|
|
|
create_deerflow_agent(_make_mock_model(), system_prompt=prompt)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["system_prompt"] == prompt
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. Features mode — auto-assemble middleware chain
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_features_mode(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=True, auto_title=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert len(middleware) > 0
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "ThreadDataMiddleware" in mw_types
|
|
assert "SandboxMiddleware" in mw_types
|
|
assert "TitleMiddleware" in mw_types
|
|
assert "ClarificationMiddleware" in mw_types
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_factory_threads_pii_redaction_config(mock_create_agent):
|
|
"""#5577 review: the SDK-path wiring must not silently drop the knob —
|
|
both MemoryMiddleware and DurableContextMiddleware carry the config."""
|
|
mock_create_agent.return_value = MagicMock()
|
|
pii = PiiRedactionConfig(enabled=True, token_secret="sdk-path-pii-redaction-secret")
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(memory=True), pii_redaction_config=pii)
|
|
|
|
middleware = mock_create_agent.call_args[1]["middleware"]
|
|
memory_mw = next(m for m in middleware if type(m).__name__ == "MemoryMiddleware")
|
|
durable_mw = next(m for m in middleware if type(m).__name__ == "DurableContextMiddleware")
|
|
pii_mw = next(m for m in middleware if type(m).__name__ == "PiiRedactionMiddleware")
|
|
assert memory_mw._pii_redaction_config is pii
|
|
assert durable_mw._pii_redaction_config is pii
|
|
|
|
# Model-call regression: the assembled SDK chain actually redacts user
|
|
# content at the model boundary (round-10 review).
|
|
class _Req:
|
|
def __init__(self, messages):
|
|
self.messages = list(messages)
|
|
|
|
def override(self, **kwargs):
|
|
copy = object.__new__(type(self))
|
|
copy.messages = kwargs.get("messages", self.messages)
|
|
return copy
|
|
|
|
captured = {}
|
|
pii_mw.wrap_model_call(_Req([HumanMessage("reach alice@example.com")]), lambda req: captured.update(messages=req.messages) or "r")
|
|
assert "alice@example.com" not in str(captured["messages"][0].content)
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_factory_defaults_to_redaction_off(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(memory=True))
|
|
|
|
middleware = mock_create_agent.call_args[1]["middleware"]
|
|
memory_mw = next(m for m in middleware if type(m).__name__ == "MemoryMiddleware")
|
|
assert memory_mw._pii_redaction_config is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 5. Middleware full takeover
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_middleware_takeover(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
custom_mw = MagicMock(name="custom_middleware")
|
|
custom_mw.name = "custom"
|
|
|
|
create_deerflow_agent(_make_mock_model(), middleware=[custom_mw])
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["middleware"] == [custom_mw]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 6. Conflict — middleware + features raises ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_middleware_and_features_conflict():
|
|
with pytest.raises(ValueError, match="Cannot specify both"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[MagicMock()],
|
|
features=RuntimeFeatures(),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 7. Vision feature auto-injects view_image_tool when thread data is available
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_injects_view_image_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(vision=True, sandbox=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" in tool_names
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_without_sandbox_does_not_inject_view_image_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(vision=True, sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" not in tool_names
|
|
|
|
|
|
def test_view_image_middleware_preserves_viewed_images_reducer():
|
|
middleware_hints = get_type_hints(ViewImageMiddleware.state_schema, include_extras=True)
|
|
thread_hints = get_type_hints(ThreadState, include_extras=True)
|
|
|
|
assert middleware_hints["viewed_images"] == thread_hints["viewed_images"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 8. Subagent feature auto-injects task_tool
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_subagent_injects_task_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(subagent=True, sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "task" in tool_names
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_explicit_subagent_runtime_aligns_factory_middleware_and_tools(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
submitter = MagicMock()
|
|
runtime = SubagentRuntime(
|
|
SubagentRuntimeConfig(max_running=7),
|
|
max_total_per_run=12,
|
|
batch_submitter=submitter,
|
|
)
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args.kwargs
|
|
limit = next(middleware for middleware in call_kwargs["middleware"] if type(middleware).__name__ == "SubagentLimitMiddleware")
|
|
assert limit.max_concurrent == 7
|
|
assert limit.max_total == 12
|
|
tool_names = {tool.name for tool in call_kwargs["tools"]}
|
|
assert {"task", "batch_task", "batch_status", "cancel_batch"} <= tool_names
|
|
|
|
|
|
def test_explicit_subagent_runtime_requires_the_subagent_feature() -> None:
|
|
runtime = SubagentRuntime(SubagentRuntimeConfig(max_running=4))
|
|
|
|
with pytest.raises(ValueError, match="subagent_runtime.*features.subagent"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=False, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
|
|
def test_factory_rejects_configured_batch_runtime_before_worker_start() -> None:
|
|
runtime = SubagentRuntime(
|
|
SubagentRuntimeConfig(max_running=4),
|
|
batch_repository=MagicMock(),
|
|
batch_config=SubagentBatchesConfig(enabled=True),
|
|
app_config=MagicMock(),
|
|
)
|
|
|
|
with pytest.raises(RuntimeError, match="await subagent_runtime.start"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 9. Middleware ordering — ClarificationMiddleware always last
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_clarification_always_last(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=True, memory=True, vision=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
last_mw = middleware[-1]
|
|
assert type(last_mw).__name__ == "ClarificationMiddleware"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 10. RuntimeFeatures default values
|
|
# ---------------------------------------------------------------------------
|
|
def test_agent_features_defaults():
|
|
f = RuntimeFeatures()
|
|
assert f.sandbox is True
|
|
assert f.memory is False
|
|
assert f.summarization is False
|
|
assert f.subagent is False
|
|
assert f.vision is False
|
|
assert f.auto_title is False
|
|
assert f.guardrail is False
|
|
assert f.loop_detection is True
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 11. Tool deduplication — user-provided tools take priority
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_tool_deduplication(mock_create_agent):
|
|
"""If user provides a tool with the same name as an auto-injected one, no duplicate."""
|
|
mock_create_agent.return_value = MagicMock()
|
|
user_clarification = _make_mock_tool("ask_clarification")
|
|
|
|
create_deerflow_agent(_make_mock_model(), tools=[user_clarification], features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
names = [t.name for t in call_kwargs["tools"]]
|
|
assert names.count("ask_clarification") == 1
|
|
# The first one should be the user-provided tool
|
|
assert call_kwargs["tools"][0] is user_clarification
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 12. Sandbox disabled — no ThreadData/Uploads/Sandbox middleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_sandbox_disabled(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "ThreadDataMiddleware" not in mw_types
|
|
assert "UploadsMiddleware" not in mw_types
|
|
assert "SandboxMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 13. Checkpointer passed through
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_checkpointer_passthrough(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
cp = MagicMock(name="checkpointer")
|
|
|
|
create_deerflow_agent(_make_mock_model(), checkpointer=cp)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["checkpointer"] is cp
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 14. Custom AgentMiddleware instance replaces default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_middleware_replaces_default(mock_create_agent):
|
|
"""Passing an AgentMiddleware instance uses it directly instead of the built-in default."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyMemoryMiddleware(AgentMiddleware):
|
|
pass
|
|
|
|
custom_memory = MyMemoryMiddleware()
|
|
feat = RuntimeFeatures(sandbox=False, memory=custom_memory)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom_memory in middleware
|
|
# Should NOT have the default MemoryMiddleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "MemoryMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 15. Custom sandbox middleware replaces the 3-middleware group
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_sandbox_replaces_group(mock_create_agent):
|
|
"""Passing an AgentMiddleware for sandbox replaces ThreadData+Uploads+Sandbox with one."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MySandbox(AgentMiddleware):
|
|
pass
|
|
|
|
custom_sb = MySandbox()
|
|
feat = RuntimeFeatures(sandbox=custom_sb)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom_sb in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "ThreadDataMiddleware" not in mw_types
|
|
assert "UploadsMiddleware" not in mw_types
|
|
assert "SandboxMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 16. Always-on error handling middlewares are present
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_always_on_error_handling(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "DanglingToolCallMiddleware" in mw_types
|
|
assert "ToolErrorHandlingMiddleware" in mw_types
|
|
tool_error_middleware = next(m for m in middleware if type(m).__name__ == "ToolErrorHandlingMiddleware")
|
|
assert tool_error_middleware._app_config is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 17. Vision with custom middleware follows thread-data availability
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_custom_middleware_without_sandbox_does_not_inject_tool(mock_create_agent):
|
|
"""Custom vision middleware without thread data does not get view_image_tool auto-injected."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyVision(AgentMiddleware):
|
|
pass
|
|
|
|
feat = RuntimeFeatures(sandbox=False, vision=MyVision())
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" not in tool_names
|
|
|
|
|
|
# ===========================================================================
|
|
# @Next / @Prev decorators and extra_middleware insertion
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 18. @Next decorator sets _next_anchor
|
|
# ---------------------------------------------------------------------------
|
|
def test_next_decorator():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Anchor(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(Anchor)
|
|
class MyMW(AgentMiddleware):
|
|
pass
|
|
|
|
assert MyMW._next_anchor is Anchor
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 19. @Prev decorator sets _prev_anchor
|
|
# ---------------------------------------------------------------------------
|
|
def test_prev_decorator():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Anchor(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(Anchor)
|
|
class MyMW(AgentMiddleware):
|
|
pass
|
|
|
|
assert MyMW._prev_anchor is Anchor
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 20. extra_middleware with @Next inserts after anchor
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_next_inserts_after_anchor(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MyAudit(AgentMiddleware):
|
|
pass
|
|
|
|
audit = MyAudit()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[audit],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
dangling_idx = mw_types.index("DanglingToolCallMiddleware")
|
|
audit_idx = mw_types.index("MyAudit")
|
|
assert audit_idx == dangling_idx + 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 21. extra_middleware with @Prev inserts before anchor
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_prev_inserts_before_anchor(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MyFilter(AgentMiddleware):
|
|
pass
|
|
|
|
filt = MyFilter()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[filt],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
clar_idx = mw_types.index("ClarificationMiddleware")
|
|
filt_idx = mw_types.index("MyFilter")
|
|
assert filt_idx == clar_idx - 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 22. Unanchored extra_middleware goes before ClarificationMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_unanchored_before_clarification(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyPlain(AgentMiddleware):
|
|
pass
|
|
|
|
plain = MyPlain()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[plain],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert mw_types[-2] == "MyPlain"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 23. Conflict: two extras @Next same anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_conflict_same_next_target():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MW1(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MW2(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Conflict"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW1(), MW2()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 24. Conflict: two extras @Prev same anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_conflict_same_prev_target():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MW1(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MW2(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Conflict"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW1(), MW2()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 25. Both @Next and @Prev on same class → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_both_next_and_prev_error():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
class MW(AgentMiddleware):
|
|
pass
|
|
|
|
MW._next_anchor = DanglingToolCallMiddleware
|
|
MW._prev_anchor = ClarificationMiddleware
|
|
|
|
with pytest.raises(ValueError, match="both @Next and @Prev"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 26. Cross-external anchoring: extra anchors to another extra
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_cross_external_anchoring(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class First(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(First)
|
|
class Second(AgentMiddleware):
|
|
pass
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[Second(), First()], # intentionally reversed
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
dangling_idx = mw_types.index("DanglingToolCallMiddleware")
|
|
first_idx = mw_types.index("First")
|
|
second_idx = mw_types.index("Second")
|
|
assert first_idx == dangling_idx + 1
|
|
assert second_idx == first_idx + 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 27. Unresolvable anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_unresolvable_anchor():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Ghost(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(Ghost)
|
|
class MW(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Cannot resolve"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 28. extra_middleware + middleware (full takeover) → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_with_middleware_takeover_conflict():
|
|
with pytest.raises(ValueError, match="full takeover"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[MagicMock()],
|
|
extra_middleware=[MagicMock()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# LoopDetection, TodoMiddleware, GuardrailMiddleware
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 29. LoopDetectionMiddleware is always present
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_always_present(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "LoopDetectionMiddleware" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30. LoopDetection before Clarification
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_before_clarification(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
loop_idx = mw_types.index("LoopDetectionMiddleware")
|
|
clar_idx = mw_types.index("ClarificationMiddleware")
|
|
assert loop_idx < clar_idx
|
|
assert loop_idx == clar_idx - 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30b. loop_detection=False skips LoopDetectionMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_disabled(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, loop_detection=False),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "LoopDetectionMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30c. loop_detection=<custom AgentMiddleware> replaces the default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_custom_middleware(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyLoopDetection(AM):
|
|
pass
|
|
|
|
custom = MyLoopDetection()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, loop_detection=custom),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
# Default LoopDetectionMiddleware must not also appear.
|
|
assert "LoopDetectionMiddleware" not in mw_types
|
|
# Custom replacement sits immediately before TokenBudgetMiddleware and ClarificationMiddleware.
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert mw_types[-2] == "MyLoopDetection"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 31. plan_mode=True adds TodoMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_plan_mode_adds_todo_middleware(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False), plan_mode=True)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "TodoMiddleware" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 32. plan_mode=False (default) — no TodoMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_plan_mode_default_no_todo(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "TodoMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 33. summarization=True without model → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_summarization_true_raises():
|
|
with pytest.raises(ValueError, match="requires a custom AgentMiddleware"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, summarization=True),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 34. guardrail=True without built-in → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_guardrail_true_raises():
|
|
with pytest.raises(ValueError, match="requires a custom AgentMiddleware"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, guardrail=True),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 34. guardrail with custom AgentMiddleware replaces default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_guardrail_custom_middleware(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyGuardrail(AM):
|
|
pass
|
|
|
|
custom = MyGuardrail()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, guardrail=custom),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "GuardrailMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 35. guardrail=False (default) — no GuardrailMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_guardrail_default_off(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "GuardrailMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 36. Full chain order matches make_lead_agent (all features on)
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_full_chain_order(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyGuardrail(AM):
|
|
pass
|
|
|
|
class MySummarization(AM):
|
|
pass
|
|
|
|
feat = RuntimeFeatures(
|
|
sandbox=True,
|
|
memory=True,
|
|
summarization=MySummarization(),
|
|
subagent=True,
|
|
vision=True,
|
|
auto_title=True,
|
|
guardrail=MyGuardrail(),
|
|
)
|
|
create_deerflow_agent(_make_mock_model(), features=feat, plan_mode=True)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
|
|
expected_order = [
|
|
"ThreadDataMiddleware",
|
|
"UploadsMiddleware",
|
|
"SandboxMiddleware",
|
|
"DanglingToolCallMiddleware",
|
|
"MyGuardrail",
|
|
"ToolErrorHandlingMiddleware",
|
|
"DurableContextMiddleware",
|
|
"SystemMessageCoalescingMiddleware",
|
|
"MySummarization",
|
|
"TodoMiddleware",
|
|
"TitleMiddleware",
|
|
"MemoryMiddleware",
|
|
"ViewImageMiddleware",
|
|
"SubagentLimitMiddleware",
|
|
"LoopDetectionMiddleware",
|
|
"ClarificationMiddleware",
|
|
]
|
|
assert mw_types == expected_order
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 37. @Next(ClarificationMiddleware) does not break tail invariant
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_next_clarification_preserves_tail_invariant(mock_create_agent):
|
|
"""Even with @Next(ClarificationMiddleware), Clarification stays last."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(ClarificationMiddleware)
|
|
class AfterClar(AgentMiddleware):
|
|
pass
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[AfterClar()],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert "AfterClar" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 38. @Next(X) + @Prev(X) on same anchor from different extras → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_opposite_direction_same_anchor_conflict():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class AfterDangling(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(DanglingToolCallMiddleware)
|
|
class BeforeDangling(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="cross-anchoring"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[AfterDangling(), BeforeDangling()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# Input validation and error message hardening
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 39. @Next with non-AgentMiddleware anchor → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_next_bad_anchor_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware subclass"):
|
|
|
|
@Next(str) # type: ignore[arg-type]
|
|
class MW:
|
|
pass
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 40. @Prev with non-AgentMiddleware anchor → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_prev_bad_anchor_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware subclass"):
|
|
|
|
@Prev(42) # type: ignore[arg-type]
|
|
class MW:
|
|
pass
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 41. extra_middleware with non-AgentMiddleware item → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_middleware_bad_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware instances"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[object()], # type: ignore[list-item]
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 42. Circular dependency among extras → clear error message
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_circular_dependency():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class MW_A(AgentMiddleware):
|
|
pass
|
|
|
|
class MW_B(AgentMiddleware):
|
|
pass
|
|
|
|
MW_A._next_anchor = MW_B # type: ignore[attr-defined]
|
|
MW_B._next_anchor = MW_A # type: ignore[attr-defined]
|
|
|
|
with pytest.raises(ValueError, match="Circular dependency"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW_A(), MW_B()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# Delegation ledger and compacted summary in a factory-built graph
|
|
# ===========================================================================
|
|
|
|
|
|
class _RecordingFakeModel(_FakeModel):
|
|
def __init__(self, **kwargs):
|
|
super().__init__(**kwargs)
|
|
object.__setattr__(self, "received", [])
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
self.received.append(list(messages))
|
|
return super()._generate(messages, stop=stop, run_manager=run_manager, **kwargs)
|
|
|
|
|
|
@tool("task")
|
|
def _fake_task(description: str, prompt: str, subagent_type: str) -> str:
|
|
"""Fake task tool."""
|
|
return f"Task Succeeded. Result: {description}"
|
|
|
|
|
|
def _task_turn(call_id: str) -> AIMessage:
|
|
return AIMessage(
|
|
content="",
|
|
tool_calls=[{"name": "task", "args": {"description": call_id, "prompt": "do it", "subagent_type": "general-purpose"}, "id": call_id, "type": "tool_call"}],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 43. The per-run subagent total holds across model turns
|
|
# ---------------------------------------------------------------------------
|
|
def test_subagent_total_per_run_holds_across_model_turns():
|
|
model = _FakeModel(responses=[_task_turn("call-1"), _task_turn("call-2"), _task_turn("call-3"), AIMessage(content="done")])
|
|
runtime = SubagentRuntime(SubagentRuntimeConfig(max_running=3), max_total_per_run=2)
|
|
graph = create_deerflow_agent(
|
|
model,
|
|
tools=[_fake_task],
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
result = graph.invoke({"messages": [HumanMessage(content="delegate three pieces of work")]}, context={"run_id": "run-1"})
|
|
|
|
ran = [message.tool_call_id for message in result["messages"] if isinstance(message, ToolMessage) and message.name == "task"]
|
|
assert ran == ["call-1", "call-2"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 44. The model still sees the summary after summarization compacts history
|
|
# ---------------------------------------------------------------------------
|
|
def test_summarization_feature_keeps_the_summary_in_model_requests():
|
|
summarizer = DeerFlowSummarizationMiddleware(
|
|
model=_FakeModel(responses=[AIMessage(content="compressed summary")]),
|
|
trigger=("messages", 4),
|
|
keep=("messages", 2),
|
|
token_counter=len,
|
|
)
|
|
model = _RecordingFakeModel(responses=[AIMessage(content="one"), AIMessage(content="two"), AIMessage(content="three")])
|
|
graph = create_deerflow_agent(
|
|
model,
|
|
system_prompt="You are a test agent.",
|
|
features=RuntimeFeatures(summarization=summarizer, sandbox=False),
|
|
checkpointer=InMemorySaver(),
|
|
)
|
|
config = {"configurable": {"thread_id": "factory-summary"}}
|
|
|
|
for text in ("first", "second", "third"):
|
|
result = graph.invoke({"messages": [HumanMessage(content=text)]}, config)
|
|
|
|
assert result["summary_text"] == "compressed summary"
|
|
assert "first" not in [message.content for message in result["messages"]]
|
|
request = model.received[-1]
|
|
assert any("compressed summary" in str(message.content) for message in request)
|
|
# The durable-context authority contract must not reach the provider as a second SystemMessage.
|
|
assert [index for index, message in enumerate(request) if isinstance(message, SystemMessage)] == [0]
|
|
assert "You are a test agent." in request[0].content
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 45. token_budget=True enforces the default budget
|
|
# ---------------------------------------------------------------------------
|
|
def test_token_budget_true_enforces_the_default_budget():
|
|
@tool("bash")
|
|
def bash(command: str) -> str:
|
|
"""Run a fake shell command."""
|
|
return "ok"
|
|
|
|
over_budget = AIMessage(
|
|
content="",
|
|
tool_calls=[{"name": "bash", "args": {"command": "ls"}, "id": "call-1", "type": "tool_call"}],
|
|
usage_metadata={"input_tokens": 250_000, "output_tokens": 0, "total_tokens": 250_000},
|
|
)
|
|
graph = create_deerflow_agent(
|
|
_FakeModel(responses=[over_budget, AIMessage(content="done")]),
|
|
tools=[bash],
|
|
features=RuntimeFeatures(token_budget=True, sandbox=False),
|
|
)
|
|
context = {"run_id": "run-1"}
|
|
|
|
result = graph.invoke({"messages": [HumanMessage(content="go")]}, context=context)
|
|
|
|
assert not any(isinstance(message, ToolMessage) for message in result["messages"])
|
|
assert "TOKEN BUDGET EXCEEDED" in result["messages"][-1].content
|
|
assert context["stop_reason"] == "token_capped"
|