mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
* fix(agents): sanitize persisted URL content blocks before model calls
Threads whose checkpoints carry URL-sourced file blocks (written before
the MCP conversion-layer fix) fail every model call at serialization: the
Chat Completions translator rejects any file block carrying a url, and the
error handler answers each later turn with a non-retriable fallback. Add a
read-time sanitizer that rewrites those blocks in the model-bound request
view only, so poisoned threads heal on their next turn with no migration
and no user action.
- ModelContentCompatibilityMiddleware wraps the model call (sync and
async) and rewrites ToolMessage list content: URL-sourced file blocks
become "[Resource ({mime or unknown type}) available at {url}]" text
blocks, and image blocks whose scheme is not http/https/data get the
same placeholder. Fetchable image URLs, base64 payloads, text, and
non-ToolMessage messages pass through. The rewrite is order-preserving,
idempotent, and never writes state or mutates checkpoints, so artifact
capture and other state readers keep seeing the original blocks.
- Register on all three chains that replay checkpointed ToolMessages:
build_middlewares (lead; outer of the MODEL_PHYSICAL extension anchor so
its final-request guarantee holds), the SDK factory
_assemble_from_features (new step 13a; docstring updated and its stale
omission of TokenBudgetMiddleware fixed), and
build_subagent_runtime_middlewares (after DurableContext/summarization,
before the date/coalescer tail).
- Subagent coverage verified by tracing every subagent agent-creation
site: executor.py _create_agent is the only caller of
build_subagent_runtime_middlewares and serves both the task tool and the
durable batch worker (SubagentRuntime itself only carries dependencies),
and a wiring test now asserts the middleware is present in that
builder's output; the lead and factory chains gained the same wiring
assertions.
- Update the chain-order contract tests for the new slot: full-chain
order, loop-detection adjacency, custom loop-detection tail positions,
and the subagent chain count (22 -> 23).
* fix(mcp): stop emitting URL-sourced file blocks from ResourceLink results
MCP ResourceLink conversion produced a URL-sourced file content block for
every non-image link, which LangChain's Chat Completions translator rejects
at serialization time. Once such a block was checkpointed in the
ToolMessage, every subsequent turn of the thread failed at model-call time,
permanently bricking the conversation.
- Downgrade every ResourceLink that is not an http(s) image to a short text
placeholder carrying the resource name, MIME type, and the
already-virtualized URL, so host paths never leak and the model still
learns a resource was produced and where it lives.
- Preserve each downgraded link as structured data in the tool-message
artifact under resource_links (merged with structured_content when
present) for artifact capture and future MCP Apps rendering.
- Keep http(s) image links as image blocks; embedded base64 payloads, text
path rewriting, and the error-to-ToolException path are unchanged.
- Re-contract the file-migration tests to the new behavior and document
the conversion contract in the MCP AGENTS.md.
* docs(middlewares): add ModelContentCompatibilityMiddleware to the chain inventory
Insert the read-time sanitizer at its actual lead-chain position (after the
token-budget guard, before the custom-middleware injection point) and renumber
the tail entries 34-39 to 35-40.
* fix(agents): rewrite file blocks on url key presence, not truthiness
langchain-core's OpenAI translator raises for ANY file block carrying a
url key — an empty string or None included — so the read-time sanitizer
must not let such blocks through just because the value is falsy. A
persisted block with url="" or url=None would still brick every later
model call in the thread.
- Gate the file-block rewrite on "url" in block instead of a truthiness
check; a non-empty string URL keeps the
"[Resource (mime) available at url]" placeholder, while an empty or
None URL gets a location-less "[Resource (mime)]" placeholder, so no
unusable location is advertised to the model.
- Pin both shapes with new middleware tests (url="" and url=None file
blocks are both rewritten to text). The image branch is unchanged: an
empty-image case fails provider-side with a 400, not a bricked thread.
* fix(mcp): keep data URIs out of model text and resource_links
A data: scheme ResourceLink fell into the generic placeholder branch,
inlining megabytes of base64 into the model-visible text and into
resource_links, which is checkpointed state — against the repo policy
that data:/blob: URIs never enter state.
- Special-case data: links in _convert_call_tool_result: the
model-visible placeholder becomes "[Resource: name (mime) embedded
inline]" and the raw URI is not added to resource_links at all, so the
payload can neither bloat the context window nor persist in the
checkpoint.
- Pin the behavior with a conversion test asserting both the placeholder
wording and the absence of the payload/URI from every emitted block and
the artifact.
- Add the missing passthrough pin for an HTTPS:// uppercase-scheme image
link: scheme matching is case-insensitive, so the link stays an image
block (the block's URL validation normalizes the scheme to lowercase).
* fix(tools): consume resource_links in extract_artifacts_from_result
The MCP conversion layer preserves downgraded ResourceLinks as
{"name", "uri", "mime_type"} entries under the tool message
artifact's resource_links key, but the artifact extractor only read
structured_content — a result whose only artifact signal was
resource_links produced no entries and the file silently vanished from
the artifacts panel.
- Add a fourth extraction path: each resource_links entry becomes a file
artifact entry, reusing _is_referenceable_url like the
structured-content and content-block paths, so data:/blob: and other
unresolvable URIs stay rejected per existing policy. The entry is named
after the link when a name is present, falling back to the URI's final
segment; a string mime_type is carried through.
- Integration-test a result whose only signal is resource_links, plus the
referenceability gate (data:, ui://, and absolute-path URIs in one
artifact).
* docs(mcp): update the ResourceLink conversion contract bullet
Mention the new artifact-registry consumer and the data: special case so
the bullet stays accurate: uppercase http(s) schemes match
case-insensitively, data: URIs get the embedded-inline placeholder and
are dropped from resource_links, and extract_artifacts_from_result turns
resource_links into registry entries through the shared
referenceability gate.
* refactor(tools): add the canonical resource placeholder formatter
Both the MCP conversion layer and the read-time
ModelContentCompatibilityMiddleware downgrade model-incompatible resource
references to a text placeholder, but each built its own literal. Extract
the single formatter they will share so a thread healed at read time shows
the model the same placeholder shape a fresh conversion produces; the name
segment is optional because persisted content blocks carry no resource name.
* fix(mcp): keep host paths out of ResourceLink placeholder text
Review follow-up on the ResourceLink conversion contract. The placeholder
embedded the resolved URL unconditionally, so a link outside the user-data
tree (or converted without thread context) leaked a raw host path — a
file:// URI or bare filesystem path — into model-visible text, violating
US-16. Compute the resolved URL's scheme and withhold the location segment
when it spells out a host filesystem location (empty scheme, file, or a
single-letter Windows drive scheme); virtual paths and remote schemes keep
their location. The unresolved URI still lands in the artifact channel,
which the model never sees.
- add _model_visible_link_location gate and route the generic placeholder
through it, re-contracting the outside-user-data and no-context tests to
the location-less shape and pinning the Windows-drive case
- treat blob: like data: as non-referenceable inline input: drop its raw
URI from checkpointed resource_links and emit the location-less
placeholder instead of the mismatched 'embedded inline' wording, and fix
the comment that claimed blob: was already excluded
- replace every inline placeholder f-string in the ResourceLink branch
with the shared resource_placeholder_text formatter so the conversion
layer and the read-time middleware emit byte-identical shapes; data:
links now render in the canonical location-less form
- sync the ResourceLink conversion contract bullet in mcp/AGENTS.md and
the _convert_call_tool_result docstring with the host-path suppression
rule and blob: handling
* fix(agents): sanitize URL content blocks on any message role
Address the second-pass review of the request sanitizer with two
changes that share the same code path.
- Adopt the canonical resource_placeholder_text formatter from
deerflow.tools.resource_placeholder instead of the module-local
placeholder helpers, so the read-time rewrite and the MCP conversion
layer emit byte-identical placeholder text from one definition.
- Widen the rewrite from ToolMessage-only to every request message with
list-form content. User uploads are verified to produce base64 blocks,
but imported or cross-client history can carry a URL-sourced file
block on a HumanMessage or AIMessage, and the Chat Completions
serializer rejects such a block regardless of the role carrying it.
The rewrite stays request-view-only: state and checkpoints are never
mutated, and base64/file-id blocks and string content pass through
untouched on any role.
* docs(agents): state the ModelContentCompatibility placement contract
The comment above the registration claimed the sanitizer sits just
before Clarification. In the lead chain it is actually followed by the
custom/extension injection point and the whole terminal tail
(terminal-response, finish-reason, and the always-last
ClarificationMiddleware). Describe the real contract: after every
built-in content transform, before extension injection and the
terminal tail.
* docs(middlewares): clarify provenance stamps for request-view-only rewrites
The message-provenance rule read as if every rewrite needs a
provenance_kwargs stamp. Add one line noting the exception: rewrites
that only modify the model-bound request view and never enter state
need no stamps, citing InputSanitizationMiddleware,
DanglingToolCallMiddleware, and ModelContentCompatibilityMiddleware as
precedents.
* fix(agents): keep host paths and data URIs out of healed placeholders
The read-time ModelContentCompatibilityMiddleware rewrote URL-sourced
file blocks and non-fetchable image blocks into placeholders that
inlined the block's URL verbatim. Pre-fix checkpoints — exactly the
threads this middleware exists to heal — can carry a file:// URI, a bare
or Windows host path (links that never resolved to virtual paths), or a
data: URI embedding megabytes of base64, so healing leaked deployment
filesystem layout into prompts (US-16) and could inject the whole inline
payload into model-visible text.
The conversion layer already suppresses those forms via
_model_visible_link_location. Move that gate into the shared
deerflow.tools.resource_placeholder module as model_visible_location
(now also collapsing data:/blob: URIs) and route both middleware
branches through it, so both layers apply identical suppression rules
with one definition. Conversion behavior is unchanged: its data:/blob:
branch short-circuits before the gate, and the gate covers those schemes
for defense in depth.
- resource_placeholder.py gains model_visible_location alongside the
shared formatter; mcp/tools.py drops its private copy and imports it.
- The middleware's file and image branches pass the URL through the gate
before building the placeholder; module docstring documents the rules.
- Tests: gate unit tests (virtual/remote visible; file://, bare/Windows
host paths, data:/blob: withheld) and middleware regression tests
pinning that a pre-fix data: payload, file:// URI, and Windows drive
path heal to location-less placeholders.
- AGENTS.md bullets in mcp/ and middlewares/ state the shared gate.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(agents): downgrade malformed image URLs instead of raising in the sanitizer
The read-time ModelContentCompatibilityMiddleware gated persisted image
blocks on urlparse(url).scheme, but urlparse raises ValueError on
structurally invalid URLs — a bad IPv6 literal, invalid netloc
characters — instead of returning an unusable scheme. Pre-fix
checkpoints and imported history can carry exactly those URLs, and the
block stays in the checkpoint, so every later model call in the thread
raised again: the sanitizer re-bricked the very threads it exists to
heal, only with a framework-side exception instead of the langchain
serialization error.
Catch ValueError and treat the URL as non-fetchable, so the block
degrades into the same text placeholder as any other unresolvable
scheme (the shared model_visible_location gate already swallows
ValueError, so the location segment is withheld as well).
- wrap the image branch's urlparse call in try/except ValueError and
fall back to an empty scheme, keeping http/https/data pass-through
behavior unchanged for well-formed URLs
- add a regression test pinning that an image block with a bad IPv6
literal heals to the location-less placeholder without raising
Co-Authored-By: Codex <noreply@openai.com>
* fix(tools): dedupe artifact refs across extraction sources
One downgraded ResourceLink produced two registry entries: the typed
resource_links channel added an entry for the URI, and the free-text
scan over the conversion layer's placeholder text — which embeds the
same virtual path or remote URL — matched it again. The two handles
pointed at one resource, spent double of the render budget, and showed
the model two handles for a single artifact.
extract_artifacts_from_result now tracks seen real_refs across all
sources and keeps only the first entry per ref. Extraction order makes
the typed sources (structured content, resource_links, content blocks)
win over the free-text scan, so the surviving entry keeps the link's
display name and mime type.
- route all six entry-producing sites through a local _add_unique
helper keyed on real_ref; per-source behavior is unchanged
- document the cross-source dedup in the function docstring
- add a regression test covering a virtual-path and a remote-URL
placeholder together with their resource_links entries
Co-Authored-By: Codex <noreply@openai.com>
* docs(mcp): state the intentional data: image downgrade asymmetry
The conversion layer downgrades image/* data: ResourceLinks to
location-less text placeholders, while the read-time middleware lets
persisted data: image blocks pass through to the model. The two
contracts looked contradictory but are deliberately layered: the
conversion layer keeps new inline payloads out of checkpointed state,
and the middleware only preserves blocks already occupying one.
- mcp/AGENTS.md's ResourceLink conversion contract and the conversion
comment in mcp/tools.py now state the image/* data: downgrade and
why it differs from the read-time pass-through
- middlewares/AGENTS.md's ModelContentCompatibilityMiddleware entry
states the symmetric read-time rule
- pin the behavior with an image-mime data: ResourceLink conversion
test (location-less placeholder, nothing in the artifact channel)
Co-Authored-By: Codex <noreply@openai.com>
* docs(backend): compact AGENTS.md guidance to fit the chain byte budget
The branch's new contract bullets pushed the middlewares and mcp
effective AGENTS.md chains past the 98304-byte AG002 hard limit (the
sandbox/subagents chains on main were already over it). Compact the
guidance without dropping contracts:
- middlewares item 34 points at the MCP ResourceLink conversion
contract and the module docstring instead of restating both
- the mcp ResourceLink conversion contract bullet is tightened
- backend/AGENTS.md merges the File Upload and Context Summarization
bullet sprawl, folds the stray ATX test note into the upload outline
rule, drops the Nginx routing block the root guide already covers,
merges the duplicate MCP-task pointer bullet, compacts Vision
Support, and scopes the Project Structure tree to the backend
subtree (the root guide maps the full repo)
Co-Authored-By: Codex <noreply@openai.com>
* fix(agents): gate the file-URL downgrade on provider support and neutralize placeholder metadata
Two review findings on the read-time sanitizer:
- The file-block downgrade ran for every provider, but only Chat
Completions rejects URL-sourced file blocks: the OpenAI Responses API
(input_file file_url) and Anthropic (document url source) accept
http(s) document URLs, so a persisted block created for those
providers lost its document from the model request. Keep the block
when the selected model serializes http(s) file URLs natively; every
other URL shape (non-http schemes, empty/None, malformed) still
degrades to the placeholder.
- The placeholder interpolated persisted MIME/location metadata
verbatim, but the middleware runs after the input/tool-result
sanitization passes and historical URL blocks never went through them
(non-text content is left untouched), so values carrying
<system-reminder>-style tags or user-input boundary markers reached
the model raw. Route the interpolated metadata through the shared
neutralize_untrusted_tags guard in both the file and image branches.
Tests: provider gating (Responses/Anthropic preserve, Chat Completions
still downgrades, non-http and malformed URLs downgrade even for
capable providers) and metadata neutralization (injection tags
HTML-escaped, boundary tokens neutralized, file and image branches).
Co-Authored-By: Codex <noreply@openai.com>
* fix(tools): require a directory boundary before exposing virtual locations
model_visible_location admitted any URL starting with the virtual
prefix, so a sibling like /mnt/user-data-backups/ops/private/report.pdf
was treated as a virtual path and its host location reached
model-visible placeholder text — defeating the US-16 host-path
suppression the gate exists for. Require the exact virtual root or a
slash-delimited descendant; siblings now collapse to the location-less
placeholder like any other host path. Both the file and image
downgrade branches share the gate, and the conversion layer applies
the same rules to fresh results.
Regressions: gate unit tests (exact root and descendants stay visible;
/mnt/user-data-backups and /mnt/user-database siblings withheld) and
middleware tests pinning the location-less placeholder for both
branches.
Co-Authored-By: Codex <noreply@openai.com>
* docs(agents): scope the placeholder artifact-visibility claim to the conversion layer
The module docstring claimed the healed placeholder keeps the location
visible to the artifact free-text scan and that the link survives in
the structured artifact channel — true for the conversion layer, whose
placeholders are checkpointed, but not on the healing path: the request
view never reaches artifact capture, and no resource_links entry is
created for a downgraded block. Spell out what actually happens here:
a referenceable original block (http(s) URL or absolute host path) is
still captured, from the capture-time scan of the state block rather
than the placeholder, while a non-referenceable one (file://, data:,
blob:) yields no entry and survives only in the untouched checkpoint.
Co-Authored-By: Codex <noreply@openai.com>
* docs(mcp): clarify healing and artifact preservation contracts
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
1219 lines
47 KiB
Python
1219 lines
47 KiB
Python
"""Tests for create_deerflow_agent SDK entry point."""
|
|
|
|
from typing import get_type_hints
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import pytest
|
|
from langchain.agents import AgentState
|
|
from langchain_core.language_models.fake_chat_models import FakeMessagesListChatModel
|
|
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
|
|
from langchain_core.tools import tool
|
|
from langgraph.channels import DeltaChannel
|
|
from langgraph.checkpoint.memory import InMemorySaver
|
|
|
|
from deerflow.agents.factory import create_deerflow_agent
|
|
from deerflow.agents.features import Next, Prev, RuntimeFeatures
|
|
from deerflow.agents.middlewares.summarization_middleware import DeerFlowSummarizationMiddleware
|
|
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
|
|
from deerflow.agents.thread_state import DeltaThreadState, ThreadState
|
|
from deerflow.config.pii_redaction_config import PiiRedactionConfig
|
|
from deerflow.config.subagent_batches_config import SubagentBatchesConfig
|
|
from deerflow.config.subagent_runtime_config import SubagentRuntimeConfig
|
|
from deerflow.subagents import SubagentRuntime
|
|
|
|
|
|
def _make_mock_model():
|
|
return MagicMock(name="mock_model")
|
|
|
|
|
|
class _FakeModel(FakeMessagesListChatModel):
|
|
def bind_tools(self, tools, **kwargs): # type: ignore[override]
|
|
return self
|
|
|
|
|
|
class _CustomState(AgentState):
|
|
custom_value: str
|
|
|
|
|
|
def _make_mock_tool(name: str = "my_tool"):
|
|
tool = MagicMock(name=name)
|
|
tool.name = name
|
|
return tool
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. Minimal creation — only model
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_minimal_creation(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
model = _make_mock_model()
|
|
|
|
result = create_deerflow_agent(model)
|
|
|
|
mock_create_agent.assert_called_once()
|
|
assert result is mock_create_agent.return_value
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["model"] is model
|
|
assert call_kwargs["system_prompt"] is None
|
|
assert call_kwargs["state_schema"] is ThreadState
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_delta_creation_selects_delta_state_and_copies_middleware(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
middleware = ViewImageMiddleware()
|
|
original_schema = middleware.state_schema
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[middleware],
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args.kwargs
|
|
assert call_kwargs["state_schema"] is DeltaThreadState
|
|
assert call_kwargs["middleware"][0] is not middleware
|
|
assert middleware.state_schema is original_schema
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_state_schema_is_preserved_in_full_mode_and_adapted_in_delta_mode(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock(name="compiled_graph")
|
|
|
|
create_deerflow_agent(_make_mock_model(), state_schema=_CustomState)
|
|
assert mock_create_agent.call_args.kwargs["state_schema"] is _CustomState
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
state_schema=_CustomState,
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
adapted = mock_create_agent.call_args.kwargs["state_schema"]
|
|
hints = get_type_hints(adapted, include_extras=True)
|
|
assert "custom_value" in hints
|
|
assert any(isinstance(item, DeltaChannel) for item in hints["messages"].__metadata__)
|
|
|
|
|
|
def test_delta_checkpointer_combination_is_rejected_before_any_persistence():
|
|
"""Mixed-mode corruption guard: delta + checkpointer must fail loudly at
|
|
construction, before any state is read or written through the ungated graph."""
|
|
saver = InMemorySaver()
|
|
with pytest.raises(ValueError, match="checkpoint_channel_mode='delta'"):
|
|
create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(content="ok")]),
|
|
checkpoint_channel_mode="delta",
|
|
checkpointer=saver,
|
|
)
|
|
assert list(saver.list(None)) == []
|
|
|
|
|
|
def test_compiled_factory_graph_selects_full_and_delta_message_channels():
|
|
full_graph = create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(id="full-response", content="done")]),
|
|
)
|
|
delta_graph = create_deerflow_agent(
|
|
_FakeModel(responses=[AIMessage(id="delta-response", content="done")]),
|
|
checkpoint_channel_mode="delta",
|
|
)
|
|
|
|
assert type(full_graph.channels["messages"]).__name__ == "BinaryOperatorAggregate"
|
|
assert isinstance(delta_graph.channels["messages"], DeltaChannel)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. With tools
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_with_tools(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
model = _make_mock_model()
|
|
tool = _make_mock_tool("search")
|
|
|
|
create_deerflow_agent(model, tools=[tool])
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "search" in tool_names
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. With system_prompt
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_with_system_prompt(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
prompt = "You are a helpful assistant."
|
|
|
|
create_deerflow_agent(_make_mock_model(), system_prompt=prompt)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["system_prompt"] == prompt
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. Features mode — auto-assemble middleware chain
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_features_mode(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=True, auto_title=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert len(middleware) > 0
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "ThreadDataMiddleware" in mw_types
|
|
assert "SandboxMiddleware" in mw_types
|
|
assert "TitleMiddleware" in mw_types
|
|
assert "ClarificationMiddleware" in mw_types
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_factory_threads_pii_redaction_config(mock_create_agent):
|
|
"""#5577 review: the SDK-path wiring must not silently drop the knob —
|
|
both MemoryMiddleware and DurableContextMiddleware carry the config."""
|
|
mock_create_agent.return_value = MagicMock()
|
|
pii = PiiRedactionConfig(enabled=True, token_secret="sdk-path-pii-redaction-secret")
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(memory=True), pii_redaction_config=pii)
|
|
|
|
middleware = mock_create_agent.call_args[1]["middleware"]
|
|
memory_mw = next(m for m in middleware if type(m).__name__ == "MemoryMiddleware")
|
|
durable_mw = next(m for m in middleware if type(m).__name__ == "DurableContextMiddleware")
|
|
pii_mw = next(m for m in middleware if type(m).__name__ == "PiiRedactionMiddleware")
|
|
assert memory_mw._pii_redaction_config is pii
|
|
assert durable_mw._pii_redaction_config is pii
|
|
|
|
# Model-call regression: the assembled SDK chain actually redacts user
|
|
# content at the model boundary (round-10 review).
|
|
class _Req:
|
|
def __init__(self, messages):
|
|
self.messages = list(messages)
|
|
|
|
def override(self, **kwargs):
|
|
copy = object.__new__(type(self))
|
|
copy.messages = kwargs.get("messages", self.messages)
|
|
return copy
|
|
|
|
captured = {}
|
|
pii_mw.wrap_model_call(_Req([HumanMessage("reach alice@example.com")]), lambda req: captured.update(messages=req.messages) or "r")
|
|
assert "alice@example.com" not in str(captured["messages"][0].content)
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_factory_defaults_to_redaction_off(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(memory=True))
|
|
|
|
middleware = mock_create_agent.call_args[1]["middleware"]
|
|
memory_mw = next(m for m in middleware if type(m).__name__ == "MemoryMiddleware")
|
|
assert memory_mw._pii_redaction_config is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 5. Middleware full takeover
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_middleware_takeover(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
custom_mw = MagicMock(name="custom_middleware")
|
|
custom_mw.name = "custom"
|
|
|
|
create_deerflow_agent(_make_mock_model(), middleware=[custom_mw])
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["middleware"] == [custom_mw]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 6. Conflict — middleware + features raises ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_middleware_and_features_conflict():
|
|
with pytest.raises(ValueError, match="Cannot specify both"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[MagicMock()],
|
|
features=RuntimeFeatures(),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 7. Vision feature auto-injects view_image_tool when thread data is available
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_injects_view_image_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(vision=True, sandbox=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" in tool_names
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_without_sandbox_does_not_inject_view_image_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(vision=True, sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" not in tool_names
|
|
|
|
|
|
def test_view_image_middleware_preserves_viewed_images_reducer():
|
|
middleware_hints = get_type_hints(ViewImageMiddleware.state_schema, include_extras=True)
|
|
thread_hints = get_type_hints(ThreadState, include_extras=True)
|
|
|
|
assert middleware_hints["viewed_images"] == thread_hints["viewed_images"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 8. Subagent feature auto-injects task_tool
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_subagent_injects_task_tool(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(subagent=True, sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "task" in tool_names
|
|
|
|
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_explicit_subagent_runtime_aligns_factory_middleware_and_tools(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
submitter = MagicMock()
|
|
runtime = SubagentRuntime(
|
|
SubagentRuntimeConfig(max_running=7),
|
|
max_total_per_run=12,
|
|
batch_submitter=submitter,
|
|
)
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args.kwargs
|
|
limit = next(middleware for middleware in call_kwargs["middleware"] if type(middleware).__name__ == "SubagentLimitMiddleware")
|
|
assert limit.max_concurrent == 7
|
|
assert limit.max_total == 12
|
|
tool_names = {tool.name for tool in call_kwargs["tools"]}
|
|
assert {"task", "batch_task", "batch_status", "cancel_batch", "read_batch_result"} <= tool_names
|
|
|
|
|
|
def test_explicit_subagent_runtime_requires_the_subagent_feature() -> None:
|
|
runtime = SubagentRuntime(SubagentRuntimeConfig(max_running=4))
|
|
|
|
with pytest.raises(ValueError, match="subagent_runtime.*features.subagent"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=False, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
|
|
def test_factory_rejects_configured_batch_runtime_before_worker_start() -> None:
|
|
runtime = SubagentRuntime(
|
|
SubagentRuntimeConfig(max_running=4),
|
|
batch_repository=MagicMock(),
|
|
batch_config=SubagentBatchesConfig(enabled=True),
|
|
app_config=MagicMock(),
|
|
)
|
|
|
|
with pytest.raises(RuntimeError, match="await subagent_runtime.start"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 9. Middleware ordering — ClarificationMiddleware always last
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_clarification_always_last(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=True, memory=True, vision=True)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
last_mw = middleware[-1]
|
|
assert type(last_mw).__name__ == "ClarificationMiddleware"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 10. RuntimeFeatures default values
|
|
# ---------------------------------------------------------------------------
|
|
def test_agent_features_defaults():
|
|
f = RuntimeFeatures()
|
|
assert f.sandbox is True
|
|
assert f.memory is False
|
|
assert f.summarization is False
|
|
assert f.subagent is False
|
|
assert f.vision is False
|
|
assert f.auto_title is False
|
|
assert f.guardrail is False
|
|
assert f.loop_detection is True
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 11. Tool deduplication — user-provided tools take priority
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_tool_deduplication(mock_create_agent):
|
|
"""If user provides a tool with the same name as an auto-injected one, no duplicate."""
|
|
mock_create_agent.return_value = MagicMock()
|
|
user_clarification = _make_mock_tool("ask_clarification")
|
|
|
|
create_deerflow_agent(_make_mock_model(), tools=[user_clarification], features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
names = [t.name for t in call_kwargs["tools"]]
|
|
assert names.count("ask_clarification") == 1
|
|
# The first one should be the user-provided tool
|
|
assert call_kwargs["tools"][0] is user_clarification
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 12. Sandbox disabled — no ThreadData/Uploads/Sandbox middleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_sandbox_disabled(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "ThreadDataMiddleware" not in mw_types
|
|
assert "UploadsMiddleware" not in mw_types
|
|
assert "SandboxMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 13. Checkpointer passed through
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_checkpointer_passthrough(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
cp = MagicMock(name="checkpointer")
|
|
|
|
create_deerflow_agent(_make_mock_model(), checkpointer=cp)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
assert call_kwargs["checkpointer"] is cp
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 14. Custom AgentMiddleware instance replaces default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_middleware_replaces_default(mock_create_agent):
|
|
"""Passing an AgentMiddleware instance uses it directly instead of the built-in default."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyMemoryMiddleware(AgentMiddleware):
|
|
pass
|
|
|
|
custom_memory = MyMemoryMiddleware()
|
|
feat = RuntimeFeatures(sandbox=False, memory=custom_memory)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom_memory in middleware
|
|
# Should NOT have the default MemoryMiddleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "MemoryMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 15. Custom sandbox middleware replaces the 3-middleware group
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_custom_sandbox_replaces_group(mock_create_agent):
|
|
"""Passing an AgentMiddleware for sandbox replaces ThreadData+Uploads+Sandbox with one."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MySandbox(AgentMiddleware):
|
|
pass
|
|
|
|
custom_sb = MySandbox()
|
|
feat = RuntimeFeatures(sandbox=custom_sb)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom_sb in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "ThreadDataMiddleware" not in mw_types
|
|
assert "UploadsMiddleware" not in mw_types
|
|
assert "SandboxMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 16. Always-on error handling middlewares are present
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_always_on_error_handling(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
feat = RuntimeFeatures(sandbox=False)
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "DanglingToolCallMiddleware" in mw_types
|
|
assert "ToolErrorHandlingMiddleware" in mw_types
|
|
tool_error_middleware = next(m for m in middleware if type(m).__name__ == "ToolErrorHandlingMiddleware")
|
|
assert tool_error_middleware._app_config is None
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 17. Vision with custom middleware follows thread-data availability
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_vision_custom_middleware_without_sandbox_does_not_inject_tool(mock_create_agent):
|
|
"""Custom vision middleware without thread data does not get view_image_tool auto-injected."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyVision(AgentMiddleware):
|
|
pass
|
|
|
|
feat = RuntimeFeatures(sandbox=False, vision=MyVision())
|
|
|
|
create_deerflow_agent(_make_mock_model(), features=feat)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
tool_names = [t.name for t in call_kwargs["tools"]]
|
|
assert "view_image" not in tool_names
|
|
|
|
|
|
# ===========================================================================
|
|
# @Next / @Prev decorators and extra_middleware insertion
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 18. @Next decorator sets _next_anchor
|
|
# ---------------------------------------------------------------------------
|
|
def test_next_decorator():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Anchor(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(Anchor)
|
|
class MyMW(AgentMiddleware):
|
|
pass
|
|
|
|
assert MyMW._next_anchor is Anchor
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 19. @Prev decorator sets _prev_anchor
|
|
# ---------------------------------------------------------------------------
|
|
def test_prev_decorator():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Anchor(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(Anchor)
|
|
class MyMW(AgentMiddleware):
|
|
pass
|
|
|
|
assert MyMW._prev_anchor is Anchor
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 20. extra_middleware with @Next inserts after anchor
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_next_inserts_after_anchor(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MyAudit(AgentMiddleware):
|
|
pass
|
|
|
|
audit = MyAudit()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[audit],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
dangling_idx = mw_types.index("DanglingToolCallMiddleware")
|
|
audit_idx = mw_types.index("MyAudit")
|
|
assert audit_idx == dangling_idx + 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 21. extra_middleware with @Prev inserts before anchor
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_prev_inserts_before_anchor(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MyFilter(AgentMiddleware):
|
|
pass
|
|
|
|
filt = MyFilter()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[filt],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
clar_idx = mw_types.index("ClarificationMiddleware")
|
|
filt_idx = mw_types.index("MyFilter")
|
|
assert filt_idx == clar_idx - 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 22. Unanchored extra_middleware goes before ClarificationMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_unanchored_before_clarification(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyPlain(AgentMiddleware):
|
|
pass
|
|
|
|
plain = MyPlain()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[plain],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert mw_types[-2] == "MyPlain"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 23. Conflict: two extras @Next same anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_conflict_same_next_target():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MW1(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class MW2(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Conflict"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW1(), MW2()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 24. Conflict: two extras @Prev same anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_conflict_same_prev_target():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MW1(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(ClarificationMiddleware)
|
|
class MW2(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Conflict"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW1(), MW2()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 25. Both @Next and @Prev on same class → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_both_next_and_prev_error():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
class MW(AgentMiddleware):
|
|
pass
|
|
|
|
MW._next_anchor = DanglingToolCallMiddleware
|
|
MW._prev_anchor = ClarificationMiddleware
|
|
|
|
with pytest.raises(ValueError, match="both @Next and @Prev"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 26. Cross-external anchoring: extra anchors to another extra
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_extra_cross_external_anchoring(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class First(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(First)
|
|
class Second(AgentMiddleware):
|
|
pass
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[Second(), First()], # intentionally reversed
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
dangling_idx = mw_types.index("DanglingToolCallMiddleware")
|
|
first_idx = mw_types.index("First")
|
|
second_idx = mw_types.index("Second")
|
|
assert first_idx == dangling_idx + 1
|
|
assert second_idx == first_idx + 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 27. Unresolvable anchor → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_unresolvable_anchor():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class Ghost(AgentMiddleware):
|
|
pass
|
|
|
|
@Next(Ghost)
|
|
class MW(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="Cannot resolve"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW()],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 28. extra_middleware + middleware (full takeover) → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_with_middleware_takeover_conflict():
|
|
with pytest.raises(ValueError, match="full takeover"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
middleware=[MagicMock()],
|
|
extra_middleware=[MagicMock()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# LoopDetection, TodoMiddleware, GuardrailMiddleware
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 29. LoopDetectionMiddleware is always present
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_always_present(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "LoopDetectionMiddleware" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30. LoopDetection before Clarification
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_before_clarification(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
loop_idx = mw_types.index("LoopDetectionMiddleware")
|
|
clar_idx = mw_types.index("ClarificationMiddleware")
|
|
assert loop_idx < clar_idx
|
|
# The read-time sanitizer owns the slot immediately before Clarification.
|
|
assert mw_types[clar_idx - 1] == "ModelContentCompatibilityMiddleware"
|
|
assert loop_idx == clar_idx - 2
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30b. loop_detection=False skips LoopDetectionMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_disabled(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, loop_detection=False),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "LoopDetectionMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 30c. loop_detection=<custom AgentMiddleware> replaces the default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_loop_detection_custom_middleware(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyLoopDetection(AM):
|
|
pass
|
|
|
|
custom = MyLoopDetection()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, loop_detection=custom),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
# Default LoopDetectionMiddleware must not also appear.
|
|
assert "LoopDetectionMiddleware" not in mw_types
|
|
# Custom replacement sits after TokenBudgetMiddleware's slot, with only the
|
|
# read-time sanitizer between it and ClarificationMiddleware.
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert mw_types[-2] == "ModelContentCompatibilityMiddleware"
|
|
assert mw_types[-3] == "MyLoopDetection"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 31. plan_mode=True adds TodoMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_plan_mode_adds_todo_middleware(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False), plan_mode=True)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "TodoMiddleware" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 32. plan_mode=False (default) — no TodoMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_plan_mode_default_no_todo(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "TodoMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 33. summarization=True without model → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_summarization_true_raises():
|
|
with pytest.raises(ValueError, match="requires a custom AgentMiddleware"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, summarization=True),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 34. guardrail=True without built-in → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_guardrail_true_raises():
|
|
with pytest.raises(ValueError, match="requires a custom AgentMiddleware"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, guardrail=True),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 34. guardrail with custom AgentMiddleware replaces default
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_guardrail_custom_middleware(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyGuardrail(AM):
|
|
pass
|
|
|
|
custom = MyGuardrail()
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False, guardrail=custom),
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
assert custom in middleware
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert "GuardrailMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 35. guardrail=False (default) — no GuardrailMiddleware
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_guardrail_default_off(mock_create_agent):
|
|
mock_create_agent.return_value = MagicMock()
|
|
create_deerflow_agent(_make_mock_model(), features=RuntimeFeatures(sandbox=False))
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
assert "GuardrailMiddleware" not in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 36. Full chain order matches make_lead_agent (all features on)
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_full_chain_order(mock_create_agent):
|
|
from langchain.agents.middleware import AgentMiddleware as AM
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
class MyGuardrail(AM):
|
|
pass
|
|
|
|
class MySummarization(AM):
|
|
pass
|
|
|
|
feat = RuntimeFeatures(
|
|
sandbox=True,
|
|
memory=True,
|
|
summarization=MySummarization(),
|
|
subagent=True,
|
|
vision=True,
|
|
auto_title=True,
|
|
guardrail=MyGuardrail(),
|
|
)
|
|
create_deerflow_agent(_make_mock_model(), features=feat, plan_mode=True)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
mw_types = [type(m).__name__ for m in call_kwargs["middleware"]]
|
|
|
|
expected_order = [
|
|
"ThreadDataMiddleware",
|
|
"UploadsMiddleware",
|
|
"SandboxMiddleware",
|
|
"DanglingToolCallMiddleware",
|
|
"MyGuardrail",
|
|
"ToolErrorHandlingMiddleware",
|
|
"DurableContextMiddleware",
|
|
"SystemMessageCoalescingMiddleware",
|
|
"MySummarization",
|
|
"TodoMiddleware",
|
|
"TitleMiddleware",
|
|
"MemoryMiddleware",
|
|
"ViewImageMiddleware",
|
|
"SubagentLimitMiddleware",
|
|
"LoopDetectionMiddleware",
|
|
"ModelContentCompatibilityMiddleware",
|
|
"ClarificationMiddleware",
|
|
]
|
|
assert mw_types == expected_order
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 37. @Next(ClarificationMiddleware) does not break tail invariant
|
|
# ---------------------------------------------------------------------------
|
|
@patch("deerflow.agents.factory.create_agent")
|
|
def test_next_clarification_preserves_tail_invariant(mock_create_agent):
|
|
"""Even with @Next(ClarificationMiddleware), Clarification stays last."""
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
|
|
|
|
mock_create_agent.return_value = MagicMock()
|
|
|
|
@Next(ClarificationMiddleware)
|
|
class AfterClar(AgentMiddleware):
|
|
pass
|
|
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[AfterClar()],
|
|
)
|
|
|
|
call_kwargs = mock_create_agent.call_args[1]
|
|
middleware = call_kwargs["middleware"]
|
|
mw_types = [type(m).__name__ for m in middleware]
|
|
assert mw_types[-1] == "ClarificationMiddleware"
|
|
assert "AfterClar" in mw_types
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 38. @Next(X) + @Prev(X) on same anchor from different extras → ValueError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_opposite_direction_same_anchor_conflict():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
|
|
|
|
@Next(DanglingToolCallMiddleware)
|
|
class AfterDangling(AgentMiddleware):
|
|
pass
|
|
|
|
@Prev(DanglingToolCallMiddleware)
|
|
class BeforeDangling(AgentMiddleware):
|
|
pass
|
|
|
|
with pytest.raises(ValueError, match="cross-anchoring"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[AfterDangling(), BeforeDangling()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# Input validation and error message hardening
|
|
# ===========================================================================
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 39. @Next with non-AgentMiddleware anchor → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_next_bad_anchor_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware subclass"):
|
|
|
|
@Next(str) # type: ignore[arg-type]
|
|
class MW:
|
|
pass
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 40. @Prev with non-AgentMiddleware anchor → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_prev_bad_anchor_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware subclass"):
|
|
|
|
@Prev(42) # type: ignore[arg-type]
|
|
class MW:
|
|
pass
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 41. extra_middleware with non-AgentMiddleware item → TypeError
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_middleware_bad_type():
|
|
with pytest.raises(TypeError, match="AgentMiddleware instances"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[object()], # type: ignore[list-item]
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 42. Circular dependency among extras → clear error message
|
|
# ---------------------------------------------------------------------------
|
|
def test_extra_circular_dependency():
|
|
from langchain.agents.middleware import AgentMiddleware
|
|
|
|
class MW_A(AgentMiddleware):
|
|
pass
|
|
|
|
class MW_B(AgentMiddleware):
|
|
pass
|
|
|
|
MW_A._next_anchor = MW_B # type: ignore[attr-defined]
|
|
MW_B._next_anchor = MW_A # type: ignore[attr-defined]
|
|
|
|
with pytest.raises(ValueError, match="Circular dependency"):
|
|
create_deerflow_agent(
|
|
_make_mock_model(),
|
|
features=RuntimeFeatures(sandbox=False),
|
|
extra_middleware=[MW_A(), MW_B()],
|
|
)
|
|
|
|
|
|
# ===========================================================================
|
|
# Delegation ledger and compacted summary in a factory-built graph
|
|
# ===========================================================================
|
|
|
|
|
|
class _RecordingFakeModel(_FakeModel):
|
|
def __init__(self, **kwargs):
|
|
super().__init__(**kwargs)
|
|
object.__setattr__(self, "received", [])
|
|
|
|
def _generate(self, messages, stop=None, run_manager=None, **kwargs):
|
|
self.received.append(list(messages))
|
|
return super()._generate(messages, stop=stop, run_manager=run_manager, **kwargs)
|
|
|
|
|
|
@tool("task")
|
|
def _fake_task(description: str, prompt: str, subagent_type: str) -> str:
|
|
"""Fake task tool."""
|
|
return f"Task Succeeded. Result: {description}"
|
|
|
|
|
|
def _task_turn(call_id: str) -> AIMessage:
|
|
return AIMessage(
|
|
content="",
|
|
tool_calls=[{"name": "task", "args": {"description": call_id, "prompt": "do it", "subagent_type": "general-purpose"}, "id": call_id, "type": "tool_call"}],
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 43. The per-run subagent total holds across model turns
|
|
# ---------------------------------------------------------------------------
|
|
def test_subagent_total_per_run_holds_across_model_turns():
|
|
model = _FakeModel(responses=[_task_turn("call-1"), _task_turn("call-2"), _task_turn("call-3"), AIMessage(content="done")])
|
|
runtime = SubagentRuntime(SubagentRuntimeConfig(max_running=3), max_total_per_run=2)
|
|
graph = create_deerflow_agent(
|
|
model,
|
|
tools=[_fake_task],
|
|
features=RuntimeFeatures(subagent=True, sandbox=False),
|
|
subagent_runtime=runtime,
|
|
)
|
|
|
|
result = graph.invoke({"messages": [HumanMessage(content="delegate three pieces of work")]}, context={"run_id": "run-1"})
|
|
|
|
ran = [message.tool_call_id for message in result["messages"] if isinstance(message, ToolMessage) and message.name == "task"]
|
|
assert ran == ["call-1", "call-2"]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 44. The model still sees the summary after summarization compacts history
|
|
# ---------------------------------------------------------------------------
|
|
def test_summarization_feature_keeps_the_summary_in_model_requests():
|
|
summarizer = DeerFlowSummarizationMiddleware(
|
|
model=_FakeModel(responses=[AIMessage(content="compressed summary")]),
|
|
trigger=("messages", 4),
|
|
keep=("messages", 2),
|
|
token_counter=len,
|
|
)
|
|
model = _RecordingFakeModel(responses=[AIMessage(content="one"), AIMessage(content="two"), AIMessage(content="three")])
|
|
graph = create_deerflow_agent(
|
|
model,
|
|
system_prompt="You are a test agent.",
|
|
features=RuntimeFeatures(summarization=summarizer, sandbox=False),
|
|
checkpointer=InMemorySaver(),
|
|
)
|
|
config = {"configurable": {"thread_id": "factory-summary"}}
|
|
|
|
for text in ("first", "second", "third"):
|
|
result = graph.invoke({"messages": [HumanMessage(content=text)]}, config)
|
|
|
|
assert result["summary_text"] == "compressed summary"
|
|
assert "first" not in [message.content for message in result["messages"]]
|
|
request = model.received[-1]
|
|
assert any("compressed summary" in str(message.content) for message in request)
|
|
# The durable-context authority contract must not reach the provider as a second SystemMessage.
|
|
assert [index for index, message in enumerate(request) if isinstance(message, SystemMessage)] == [0]
|
|
assert "You are a test agent." in request[0].content
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 45. token_budget=True enforces the default budget
|
|
# ---------------------------------------------------------------------------
|
|
def test_token_budget_true_enforces_the_default_budget():
|
|
@tool("bash")
|
|
def bash(command: str) -> str:
|
|
"""Run a fake shell command."""
|
|
return "ok"
|
|
|
|
over_budget = AIMessage(
|
|
content="",
|
|
tool_calls=[{"name": "bash", "args": {"command": "ls"}, "id": "call-1", "type": "tool_call"}],
|
|
usage_metadata={"input_tokens": 250_000, "output_tokens": 0, "total_tokens": 250_000},
|
|
)
|
|
graph = create_deerflow_agent(
|
|
_FakeModel(responses=[over_budget, AIMessage(content="done")]),
|
|
tools=[bash],
|
|
features=RuntimeFeatures(token_budget=True, sandbox=False),
|
|
)
|
|
context = {"run_id": "run-1"}
|
|
|
|
result = graph.invoke({"messages": [HumanMessage(content="go")]}, context=context)
|
|
|
|
assert not any(isinstance(message, ToolMessage) for message in result["messages"])
|
|
assert "TOKEN BUDGET EXCEEDED" in result["messages"][-1].content
|
|
assert context["stop_reason"] == "token_capped"
|