mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-10-26 13:16:24 +00:00
10bd065615
* feat(agents): persist tool artifacts as resolvable handles (#4676) MCP and sandbox tools return file paths, URLs, and task ids that are lost when context compaction summarizes the conversation. This adds a per-thread artifact handle registry so the model can reference artifacts by short deterministic handles (art_xxxxxxxx) that survive compaction and resolve to real references at the tool-call boundary. - ThreadState.tool_artifacts: reducer channel capped at 100 entries, same-handle latest-wins, survives summarization compaction - artifact_registry: deterministic handle derivation from (thread_id, tool_call_id, call_index); extraction precedence structured_content -> file/image blocks -> conservative text scan; HTML-escaped model-context rendering with char budget - ArtifactCaptureMiddleware: before_model hook captures lightweight metadata (never content) and tracks handle consumption - ArtifactResolutionMiddleware: rewrites handles in tool-call args to real references before execution; unknown handles untouched - DurableContextMiddleware: projects the registry into the hidden durable-context data block ([available]/[consumed] status) - config: new tool_artifacts section (all switches default on) - frontend: artifact badges on tool-call steps keyed by tool_call_id Fixes #4676 * fix(agents): harden artifact registry after review (#4676) Review findings on #4929: - capture/consumption updates emitted in the same before_model call are now concatenated instead of dict-merged; dict-merging clobbered the shared tool_artifacts key and permanently dropped fresh captures - handles embed a per-result ordinal so multiple references inside one tool result never collide (two paths in one text block previously collapsed into one handle) - structured_content extraction pulls concrete refs from known keys (file/path/url/task_id/..., string or list-of-string values) instead of storing a truncated JSON preview, falls back to an untruncated whole-object data entry, and no longer skips file/image content blocks - plain-string tool results (e.g. bash/write_file output) are scanned for sandbox paths / remote URLs too, gated by detect_refs_in_text - wire max_entries through configure_tool_artifact_max_entries at assembly (clamped 10..1000) and gate detect_refs_in_text scans - resolution requires both enabled and resolve_handles_in_args, matching its docs - add real ToolCallRequest.override coverage and the spec 14.4 integration cycle (capture -> inject -> resolve -> consumption) * fix(agents): address second-round artifact registry review (#4676) Follow-up findings on #4929: - drop generic `output`/`outputs` from structured ref keys: prose under those keys no longer becomes a bogus file entry (falls back to the whole-object data entry; path-shaped refs are still caught by the free-text scan) - strip closing quotes, brackets, and backticks from detected refs so markdown- and JSON-wrapped paths resolve to clean real references - remove the process-wide cap setter: the reducer enforces only an absolute ceiling (1000) and ArtifactCaptureMiddleware applies the configured max_entries per agent when emitting updates, so agents with different configs in one process never share or flap a cap - skip extraction for tool messages whose tool_call_id already has registry entries and memoize known-empty results (bounded FIFO), so steady-state capture cost drops to the new message tail instead of a full-history rescan per model call - correct the inverted ordering claim: capture is position-independent in the wrap chain because it is a before_model hook reading state messages; fixed the assembly comment, AGENTS.md item 15, and the ordering test docstring CI parity verified locally: ruff check + format --check, full offline suite, blocking-io suite, collect-only, uv lock --check, check_agent_guidance, frontend format/lint/typecheck/build/tests. * fix(agents): sliding-window cap and URI gates for artifact registry (#4676) Third-round review findings on #4929: - max_entries is now a sliding retention window: fresh captures are always registered, and when the projection exceeds the configured cap the middleware appends a trailing {"op": "trim_to", "keep": N} directive that the reducer applies by evicting the oldest entries. The previous remaining-budget emission froze the registry at the cap, permanently dropping post-compaction artifacts — the exact failure mode #4676 targets — and silently disabled the round-2 rescan optimizations at the point threads grew longest - _track_consumption memoizes settled tool-call args in a bounded FIFO alongside the existing empty-result memo, so historical AIMessage args are not regex-rescanned every model call - file/image content-block URLs must be http(s) or path-shaped; data:/blob: URIs (MCP embedded resources carrying base64 payloads) can no longer enter thread state, tool args, or crowd out the render budget; protocol-relative //host forms are rejected as well - structured-content ref values go through the same URI-shape gate as content blocks, closing the embedded-payload hole via known keys such as url/urls - tool_artifacts.enabled now gates the whole middleware including consumption tracking via an early return in before_model Docs updated to describe the sliding window, URI gates, enabled scope, and both bounded memos. * fix(agents): final evictions and pending-aware consumption for artifact registry (#4676) Fourth-round review findings on #4929: - the sliding window interacted with the seen_call_ids skip to resurrect evicted entries every round: once an entry was trimmed, its tool_call_id left the seen set while its ToolMessage stayed in context, so the next model call re-extracted it, re-registered it as newest, and evicted something else — a never-converging rotation. The empty-result memo is now a unified _processed_results memo that records every extraction regardless of outcome, making evictions final; steady state over a saturated thread is quiescent (pinned by a multi-round test asserting no updates and zero extractions) - the consumption scan built its handle map from pre-capture state, so a call referencing a handle captured in the same before_model call missed, got settled as quiet, and its consumption mark was lost forever. _track_consumption now receives the pending capture entries and resolves against state + pending; a call settles as quiet only after every referenced handle resolved, so transient misses retry next round instead of permanently suppressing the mark AGENTS.md entry 15 updated to describe both memos and the eviction-is-final semantics. * fix(artifacts): address occurrence identity, PII and URL review findings * fix(artifacts): enforce resolved policy checks and bounded registry handling --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
195 lines
7.5 KiB
Python
195 lines
7.5 KiB
Python
from langchain_core.messages import ToolMessage
|
|
from langgraph.prebuilt.tool_node import ToolCallRequest
|
|
|
|
from deerflow.agents.middlewares.artifact_resolution_middleware import ArtifactResolutionMiddleware
|
|
from deerflow.config.tool_artifact_config import ToolArtifactConfig
|
|
from deerflow.tools.artifact_registry import generate_handle
|
|
|
|
|
|
class _FakeToolCallRequest:
|
|
def __init__(self, tool_call: dict, state: dict) -> None:
|
|
self.tool_call = tool_call
|
|
self.state = state
|
|
|
|
def override(self, **overrides):
|
|
return _FakeToolCallRequest(
|
|
overrides.get("tool_call", self.tool_call),
|
|
overrides.get("state", self.state),
|
|
)
|
|
|
|
|
|
def _request(tool_call: dict, artifacts: list[dict]):
|
|
return _FakeToolCallRequest(tool_call, {"tool_artifacts": artifacts})
|
|
|
|
|
|
def _artifact(thread_id: str, call_index: int, real_ref: str) -> dict:
|
|
return {
|
|
"handle": generate_handle(thread_id, f"call_{call_index}", call_index),
|
|
"artifact_type": "file",
|
|
"real_ref": real_ref,
|
|
"display_name": real_ref.rsplit("/", 1)[-1],
|
|
"tool_name": "write_file",
|
|
"mime_type": None,
|
|
"consumed_by": [],
|
|
}
|
|
|
|
|
|
class TestResolution:
|
|
def test_resolves_bare_handle_in_args(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = _request(
|
|
{"name": "read_file", "args": {"path": handle}, "id": "call_2", "type": "tool_call"},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
seen = {}
|
|
|
|
def handler(req):
|
|
seen["args"] = req.tool_call["args"]
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
result = middleware.wrap_tool_call(request, handler)
|
|
|
|
assert seen["args"] == {"path": "/mnt/user-data/outputs/report.md"}
|
|
assert result.content == "ok"
|
|
|
|
def test_resolves_backticked_handle_in_args(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = _request(
|
|
{"name": "bash", "args": {"command": f"cat `{handle}`"}, "id": "call_2", "type": "tool_call"},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
seen = {}
|
|
|
|
def handler(req):
|
|
seen["args"] = req.tool_call["args"]
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
middleware.wrap_tool_call(request, handler)
|
|
|
|
assert seen["args"] == {"command": "cat /mnt/user-data/outputs/report.md"}
|
|
|
|
def test_resolves_nested_handle(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = _request(
|
|
{
|
|
"name": "bash",
|
|
"args": {"command": "echo hi", "cwd": None, "files": [{"path": handle, "mode": "r"}]},
|
|
"id": "call_2",
|
|
"type": "tool_call",
|
|
},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
seen = {}
|
|
|
|
def handler(req):
|
|
seen["args"] = req.tool_call["args"]
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
middleware.wrap_tool_call(request, handler)
|
|
|
|
assert seen["args"]["files"][0]["path"] == "/mnt/user-data/outputs/report.md"
|
|
|
|
def test_unknown_handle_returns_error(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
request = _request(
|
|
{"name": "read_file", "args": {"path": "art_00000000"}, "id": "call_2", "type": "tool_call"},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
|
|
def handler(req):
|
|
raise AssertionError("Unknown handle must not reach the tool")
|
|
|
|
result = middleware.wrap_tool_call(request, handler)
|
|
assert result.status == "error"
|
|
assert "art_00000000" in result.content
|
|
assert request.tool_call["args"] == {"path": "art_00000000"}
|
|
|
|
def test_empty_registry_rejects_handles_but_preserves_concrete_args(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
request = _request({"name": "read_file", "args": {"path": "art_00000000"}, "id": "call_2", "type": "tool_call"}, [])
|
|
called = []
|
|
|
|
def handler(req):
|
|
called.append(req)
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
result = middleware.wrap_tool_call(request, handler)
|
|
assert not called and result.status == "error"
|
|
concrete = _request({"name": "read_file", "args": {"path": "/tmp/a.txt"}, "id": "call_2"}, [])
|
|
assert middleware.wrap_tool_call(concrete, handler).content == "ok"
|
|
assert called == [concrete]
|
|
|
|
def test_disabled_config_skips_resolution(self):
|
|
middleware = ArtifactResolutionMiddleware(config=ToolArtifactConfig(resolve_handles_in_args=False))
|
|
request = _request(
|
|
{"name": "read_file", "args": {"path": "art_00000000"}, "id": "call_2", "type": "tool_call"},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
called = []
|
|
|
|
def handler(req):
|
|
called.append(req)
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
middleware.wrap_tool_call(request, handler)
|
|
|
|
assert called[0] is request
|
|
|
|
def test_feature_disabled_flag_blocks_resolution_even_when_args_flag_on(self):
|
|
middleware = ArtifactResolutionMiddleware(config=ToolArtifactConfig(enabled=False, resolve_handles_in_args=True))
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = _request(
|
|
{"name": "read_file", "args": {"path": handle}, "id": "call_2", "type": "tool_call"},
|
|
[_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")],
|
|
)
|
|
seen = {}
|
|
|
|
def handler(req):
|
|
seen["args"] = req.tool_call["args"]
|
|
return ToolMessage(content="ok", tool_call_id="call_2")
|
|
|
|
middleware.wrap_tool_call(request, handler)
|
|
|
|
assert seen["args"] == {"path": handle}
|
|
|
|
|
|
class TestRealToolCallRequest:
|
|
"""Exercise the real ``ToolCallRequest.override()`` dataclass path."""
|
|
|
|
def _real_request(self, args: dict) -> ToolCallRequest:
|
|
return ToolCallRequest(
|
|
tool_call={"name": "read_file", "args": args, "id": "call_real", "type": "tool_call"},
|
|
tool=None,
|
|
state={"tool_artifacts": [_artifact("thread_1", 0, "/mnt/user-data/outputs/report.md")]},
|
|
runtime=None,
|
|
)
|
|
|
|
def test_override_produces_resolved_request(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = self._real_request({"path": handle})
|
|
|
|
resolved = middleware._resolve_request(request)
|
|
|
|
assert resolved is not request
|
|
assert resolved.tool_call["args"] == {"path": "/mnt/user-data/outputs/report.md"}
|
|
assert request.tool_call["args"] == {"path": handle}, "original request must stay untouched"
|
|
|
|
def test_end_to_end_through_wrap_tool_call(self):
|
|
middleware = ArtifactResolutionMiddleware()
|
|
handle = generate_handle("thread_1", "call_0", 0)
|
|
request = self._real_request({"path": f"`{handle}`"})
|
|
seen = {}
|
|
|
|
def handler(req):
|
|
seen["args"] = req.tool_call["args"]
|
|
return ToolMessage(content="ok", tool_call_id="call_real")
|
|
|
|
result = middleware.wrap_tool_call(request, handler)
|
|
|
|
assert seen["args"] == {"path": "/mnt/user-data/outputs/report.md"}
|
|
assert result.content == "ok"
|