diff --git a/backend/packages/harness/deerflow/agents/middlewares/TOOL_ARTIFACTS.md b/backend/packages/harness/deerflow/agents/middlewares/TOOL_ARTIFACTS.md index fcd963fa41..c1bfdc5ec3 100644 --- a/backend/packages/harness/deerflow/agents/middlewares/TOOL_ARTIFACTS.md +++ b/backend/packages/harness/deerflow/agents/middlewares/TOOL_ARTIFACTS.md @@ -13,6 +13,16 @@ message history and checkpoints remain unchanged. *(optional, if `tool_artifacts.enabled`, default on)* Captures lightweight artifact references from tool results into `ThreadState.tool_artifacts` (a reducer channel, so entries survive summarization compaction). `tool_artifacts.enabled` gates the whole middleware (capture and consumption tracking); when off, `before_model` returns nothing regardless of registry state. Runs as a `before_model` hook (never wraps a `ToolMessage` in a `Command`). `ThreadState.tool_artifact_processed` stores deduplicated hashes of processed result and settled consuming-call occurrences through an additive reducer. It records empty results too. The ledger is independent of registry retention and survives checkpoint reload, graph recreation and compaction; it is not trimmed with artifacts and grows by a small identity record per processed occurrence. Never replace it with a process-local memo or infer it from currently retained handles. A missing handle gets one retry before its consuming-call occurrence settles, so permanently unknown or evicted handles are scanned at most twice, including across restarts; same-round consumption resolves against state plus pending captures. Capture and consumption updates concatenate list channels so neither is lost. Position-independent in the tool-execution wrap chain: it is a `before_model` hook reading state messages, so it sees the normalized results stored in state regardless of its wrap position (error results are skipped at extraction via `status == "error"`); it is appended after ToolErrorHandlingMiddleware for readability only. Handles are `art_` + 8 hex chars derived deterministically from (thread_id, durable ToolMessage.id, provider tool_call_id, per-result ordinal). Provider call IDs may repeat across turns; graph-assigned message IDs distinguish occurrences and remain stable after compaction. Standalone ID-less hook callers must supply distinct call indices. These short hashes are identifiers, not ownership credentials. Extraction combines all of: structured-content string values under known keys (`file`/`path`/`url`/`task_id`/...) as concrete refs (unknown or non-reference payloads fall back to complete JSON `data` only when they fit 4096 UTF-8 bytes, 1024 nodes and 32 levels; empty and oversized payloads are skipped before unbounded serialization), `file`/`image` content blocks with HTTP(S), Unix-absolute or drive-absolute references (`data:`/`blob:` URIs are rejected so embedded-resource payloads never enter state or tool args), and conservative free-text scanning of both text blocks and plain-string results such as built-in sandbox tool output (sandbox paths + remote file URLs including query/fragment suffixes, gated by `detect_refs_in_text`). The reducer enforces only an absolute ceiling (1000); the configured `tool_artifacts.max_entries` is a per-agent sliding retention window: when a capture round would exceed it, the middleware appends a trailing `{"op": "trim_to", "keep": N}` directive and the reducer evicts the oldest entries (fresh captures are always registered — nothing freezes at the cap), so agents with different configs in one process do not share a cap. Reference entries contain metadata (path/URL, name, type, MIME). The JSON compatibility fallback is an explicitly bounded exception. Concrete path/URL keys reject prose and relative paths; remote task IDs use a separate non-empty, whitespace-free, 256-character token gate. File references on all extraction paths are bounded to 4096 UTF-8 bytes. Structured traversal is bounded before fallback encoding, and valid content-block references are still captured if structured data is rejected. +Free-text remote file detection checks the complete URL path for a supported +file suffix. Query strings and fragments are retained in the reference and do +not determine its file type; a file-like prefix inside a longer path is not +registered as a shortened URL. Supported file suffixes are matched without +case sensitivity. Both URL and sandbox-path text scans strip common trailing +ASCII/CJK prose punctuation and closing quotes or brackets. CJK closers paired +with openers inside the detected reference are preserved, so a numbered glob +result such as `/mnt/user-data/workspace/项目（归档）` retains its complete name +through capture, checkpoint persistence and subsequent handle resolution. + ## Durable projection Artifact handles are projected with working notes and summary history in the hidden HumanMessage data block when `tool_artifacts.enabled` and `inject_model_context` are enabled. Every model-visible artifact label (display name, type, tool name, MIME type) is redacted with the configured PII policy before HTML escaping. Generated handles remain usable and internal `real_ref` values remain unchanged for resolution. Values remain untrusted data. Artifact capture precedes compaction; task continuity and PII redaction retain independent configuration. `inject_tool_artifacts` participates in the middleware release policy fingerprint. diff --git a/backend/packages/harness/deerflow/tools/AGENTS.md b/backend/packages/harness/deerflow/tools/AGENTS.md index faf19ab96b..ecf6bfb232 100644 --- a/backend/packages/harness/deerflow/tools/AGENTS.md +++ b/backend/packages/harness/deerflow/tools/AGENTS.md @@ -1,5 +1,12 @@ ### Tool System (`packages/harness/deerflow/tools/`) +`artifact_registry.py` checks remote file suffixes against the complete parsed +URL path without case sensitivity. Free-text URL and sandbox-path extraction +strips common trailing ASCII/CJK prose punctuation while preserving CJK +closers paired with openers inside the detected reference (including literal +glob directory names). Structured references remain literal. See +[tool artifact middleware](../agents/middlewares/TOOL_ARTIFACTS.md). + `task` and `batch_task` opt into JSON checks with `file: json-valid`. See [subagents/AGENTS.md](../subagents/AGENTS.md) for read limits and UNVERIFIED semantics. Batch readers bound by assembly use its AppConfig output budget. Fit the complete escaped response under 10K and active per-tool/fallback limits; too-small envelopes stop reading. Numeric reader inputs are strict integers. Escape `<` in outer JSON to preserve untrusted data without exposing framework tags; decoding restores the document unchanged. diff --git a/backend/packages/harness/deerflow/tools/artifact_registry.py b/backend/packages/harness/deerflow/tools/artifact_registry.py index 1c473fc7b2..c835aff6cd 100644 --- a/backend/packages/harness/deerflow/tools/artifact_registry.py +++ b/backend/packages/harness/deerflow/tools/artifact_registry.py @@ -30,8 +30,10 @@ _HANDLE_LENGTH = 8 # inside the mounted user-data tree). _SANDBOX_PATH_PATTERN = re.compile(r"/mnt/user-data/\S+") -# Conservative URL-with-file-extension match for remote references. -_REMOTE_FILE_URL_PATTERN = re.compile(r"https?://[^\s\"'`<>]+\.(?:png|jpg|jpeg|gif|html|pdf|csv|json|txt|log|md|xlsx?|docx?|zip)(?:[?#][^\s\"'`<>]*)?") +# Scan complete URLs before checking the path, so extensions in query strings +# cannot truncate a reference or turn a non-file URL into a file artifact. +_REMOTE_URL_PATTERN = re.compile(r"https?://[^\s\"'`<>]+") +_REMOTE_FILE_EXTENSION_PATTERN = re.compile(r"\.(?:png|jpg|jpeg|gif|html|pdf|csv|json|txt|log|md|xlsx?|docx?|zip)\Z", re.IGNORECASE) # Structured-content keys whose string values are treated as concrete # references (paths, URLs, remote task ids) rather than opaque payload. @@ -43,8 +45,10 @@ _STRUCTURED_TASK_KEYS = frozenset({"task_id", "job_id"}) # Characters stripped from detected refs: prose punctuation plus the closing # quotes/brackets/backticks that markdown- and JSON-formatted tool output -# commonly wraps paths in. `\S+` would otherwise consume them into `real_ref`. -_REF_TRAILING_NOISE_CHARS = ".,;:)]}\"'`" +# commonly wraps paths in, including CJK punctuation. `\S+` would otherwise +# consume them into `real_ref`. +_REF_TRAILING_NOISE_CHARS = ".,;:)]}\"'`\u3002\uff0c\uff1b\uff1a\u3001\uff09\u3011\u300b\u201d\u2019" +_CJK_CLOSING_DELIMITERS = {"）": "（", "】": "【", "》": "《", "”": "“", "’": "‘"} # Content-block and structured-key refs are trusted only in these shapes. # `data:`/`blob:` URIs can carry arbitrarily large embedded payloads (MCP @@ -126,12 +130,30 @@ def _make_entry( return entry +def _strip_ref_trailing_noise(raw: str) -> str: + """Strip prose closers while preserving paired CJK characters inside a ref.""" + end = len(raw.rstrip(_REF_TRAILING_NOISE_CHARS)) + if end == len(raw): + return raw + + unmatched_openers = dict.fromkeys(_CJK_CLOSING_DELIMITERS.values(), 0) + for index, char in enumerate(raw): + if char in unmatched_openers: + unmatched_openers[char] += 1 + elif (opener := _CJK_CLOSING_DELIMITERS.get(char)) and unmatched_openers[opener]: + unmatched_openers[opener] -= 1 + # An opener outside the detected token is prose; an opener inside + # it makes this closer part of a literal directory name or URL. + end = max(end, index + 1) + return raw[:end] + + def _detect_refs_in_text(text: str) -> list[dict[str, str]]: """Conservatively detect file paths and remote file URLs in free text.""" refs: list[dict[str, str]] = [] seen: set[str] = set() for match in _SANDBOX_PATH_PATTERN.finditer(text): - raw = match.group(0).rstrip(_REF_TRAILING_NOISE_CHARS) + raw = _strip_ref_trailing_noise(match.group(0)) if raw in seen or not _is_referenceable_url(raw): continue seen.add(raw) @@ -142,10 +164,12 @@ def _detect_refs_in_text(text: str) -> list[dict[str, str]]: "display": raw.split("/")[-1], } ) - for match in _REMOTE_FILE_URL_PATTERN.finditer(text): - raw = match.group(0).rstrip(_REF_TRAILING_NOISE_CHARS) + for match in _REMOTE_URL_PATTERN.finditer(text): + raw = _strip_ref_trailing_noise(match.group(0)) if raw in seen or not _is_referenceable_url(raw): continue + if not _REMOTE_FILE_EXTENSION_PATTERN.search(urlsplit(raw).path): + continue seen.add(raw) refs.append( { diff --git a/backend/tests/test_artifact_integration.py b/backend/tests/test_artifact_integration.py index 149c5be40a..98efc0f457 100644 --- a/backend/tests/test_artifact_integration.py +++ b/backend/tests/test_artifact_integration.py @@ -8,6 +8,8 @@ rewrites it to the real reference at the tool-call boundary. import re from typing import Any +import httpx +import pytest from _agent_e2e_helpers import FakeToolCallingModel from langchain.agents import create_agent from langchain_core.messages import AIMessage, HumanMessage @@ -122,3 +124,100 @@ def test_handle_projected_into_model_context(): durable_blocks = [message.content for message in round3 if getattr(message, "additional_kwargs", {}).get("durable_context_data")] handle = next(e["handle"] for e in result["tool_artifacts"] if e["tool_call_id"] == "call_make") assert any(handle in content for content in durable_blocks), "captured handle never reached the model-facing durable context block" + + +@pytest.mark.parametrize( + ("url", "closing_punctuation"), + [ + ("https://files.example/report.pdf?token=part.csvX", "]"), + ("https://files.example/report.pdf?token=abc&download=copy.csvX#page=2", "]"), + ("https://files.example/report.pdf?token=abc", "]"), + pytest.param("https://files.example/report.pdf?token=part.csvX#page=2", "\u3002\u201d\uff09", id="cjk-punctuation"), + pytest.param("https://files.example/report.PdF?token=part.csvX#page=2", "]", id="mixed-case-extension"), + ], +) +def test_remote_url_survives_capture_checkpoint_and_resolved_download(url, closing_punctuation): + """Exercise the real agent graph with a fake model and offline HTTP transport.""" + downloaded = [] + + @tool("make_file") + def remote_report(name: str) -> str: + """Create a remotely hosted report.""" + return f"Download [{url}{closing_punctuation}" + + def serve_report(request: httpx.Request) -> httpx.Response: + downloaded.append(request.url) + return httpx.Response(200 if request.url.query == httpx.URL(url).query else 403, content=b"report") + + @tool("read_file") + def download_report(path: str) -> str: + """Download a report by its URL or artifact handle.""" + assert path == url + with httpx.Client(transport=httpx.MockTransport(serve_report)) as client: + response = client.get(path) + response.raise_for_status() + return response.text + + agent = create_agent( + model=_cycle_model(), + tools=[remote_report, download_report], + middleware=[DurableContextMiddleware(), ArtifactCaptureMiddleware(), ArtifactResolutionMiddleware()], + state_schema=ThreadState, + checkpointer=InMemorySaver(), + ) + config = {"configurable": {"thread_id": THREAD_ID}} + + result = agent.invoke({"messages": [HumanMessage(content="make then download")]}, config, context={"thread_id": THREAD_ID}) + + assert downloaded == [httpx.URL(url)] + made = next(entry for entry in result["tool_artifacts"] if entry["tool_call_id"] == "call_make") + assert made["real_ref"] == url + assert made["consumed_by"] == ["call_read"] + persisted = next(entry for entry in agent.get_state(config).values["tool_artifacts"] if entry["tool_call_id"] == "call_make") + assert persisted["real_ref"] == url + + +def test_glob_directory_survives_capture_checkpoint_and_resolved_listing(tmp_path): + from deerflow.sandbox.tools import _format_glob_results + + directory_name = "项目（归档）" + real_ref = f"/mnt/user-data/workspace/{directory_name}" + directory = tmp_path / directory_name + directory.mkdir() + (directory / "report.txt").write_text("report", encoding="utf-8") + listed = [] + + @tool("glob") + def glob_directory(path: str, include_dirs: bool) -> str: + """Find a directory using the sandbox's numbered glob result format.""" + assert path == "/mnt" + assert include_dirs is True + return _format_glob_results(path, [real_ref], truncated=False) + + @tool("ls") + def list_directory(path: str) -> str: + """List a directory by its sandbox path or artifact handle.""" + listed.append(path) + return "\n".join(sorted(child.name for child in (tmp_path / path.rsplit("/", 1)[-1]).iterdir())) + + model = _cycle_model() + model.responses[0].tool_calls[0].update(name="glob", args={"path": "/mnt", "include_dirs": True}, id="call_glob") + model.responses[1].tool_calls[0].update(name="ls", id="call_ls") + agent = create_agent( + model=model, + tools=[glob_directory, list_directory], + middleware=[DurableContextMiddleware(), ArtifactCaptureMiddleware(), ArtifactResolutionMiddleware()], + state_schema=ThreadState, + checkpointer=InMemorySaver(), + ) + config = {"configurable": {"thread_id": THREAD_ID}} + + result = agent.invoke({"messages": [HumanMessage(content="find then list the archive directory")]}, config, context={"thread_id": THREAD_ID}) + + assert listed == [real_ref] + assert next(message.content for message in result["messages"] if getattr(message, "tool_call_id", None) == "call_ls") == "report.txt" + found = next(entry for entry in result["tool_artifacts"] if entry["tool_call_id"] == "call_glob") + assert found["real_ref"] == real_ref + assert found["consumed_by"] == ["call_ls"] + persisted = next(entry for entry in agent.get_state(config).values["tool_artifacts"] if entry["tool_call_id"] == "call_glob") + assert persisted["real_ref"] == real_ref diff --git a/backend/tests/test_artifact_registry.py b/backend/tests/test_artifact_registry.py index fcfa49f95f..226b924319 100644 --- a/backend/tests/test_artifact_registry.py +++ b/backend/tests/test_artifact_registry.py @@ -2,6 +2,7 @@ import json +import pytest from langchain_core.messages import ToolMessage from deerflow.agents.thread_state import merge_tool_artifacts @@ -41,6 +42,117 @@ def test_generate_handle_format(): assert len(handle) == len("art_") + 8 +@pytest.mark.parametrize( + "url", + [ + "https://files.example/report.pdf?token=part.csvX", + "https://files.example/report.pdf#section.md-more", + "https://files.example/report.pdf?token=abc&download=copy.csvX#page=2", + "https://files.example/report.pdf?token=abc", + "https://files.example/report.pdf#page=2", + "https://files.example/report.pdf?token=part%EF%BC%89#section%EF%BC%8C", + ], +) +@pytest.mark.parametrize("content_block", [False, True]) +def test_text_file_urls_preserve_complete_query_and_fragment(url, content_block): + text = f"Report: [{url}]" + result = ToolMessage(content=[{"type": "text", "text": text}] if content_block else text, tool_call_id="call_url", name="remote_report") + + entries = extract_artifacts_from_result(result, thread_id="thread-url") + + assert [entry["real_ref"] for entry in entries] == [url] + + +@pytest.mark.parametrize( + "suffix", + [ + pytest.param("\u3002", id="cjk-period"), + pytest.param("\uff0c", id="fullwidth-comma"), + pytest.param("\uff1b", id="fullwidth-semicolon"), + pytest.param("\uff1a", id="fullwidth-colon"), + pytest.param("\u3001", id="ideographic-comma"), + pytest.param("\uff09", id="fullwidth-parenthesis"), + pytest.param("\u3011", id="cjk-bracket"), + pytest.param("\u300b", id="cjk-angle-bracket"), + pytest.param("\u201d", id="closing-double-quote"), + pytest.param("\u2019", id="closing-single-quote"), + pytest.param("\u3002\u201d\uff09", id="combined-closers"), + ], +) +@pytest.mark.parametrize( + "ref", + [ + "https://files.example/report.pdf", + "https://files.example/report.pdf?token=part.csvX#page=2", + "/mnt/user-data/outputs/report.pdf", + ], +) +def test_text_refs_followed_by_cjk_punctuation(ref, suffix): + assert [entry["ref"] for entry in _detect_refs_in_text(f"Report: {ref}{suffix}")] == [ref] + + +@pytest.mark.parametrize("content_block", [False, True]) +@pytest.mark.parametrize("directory_name", ["项目（归档）", "项目【归档】", "项目《归档》", "项目“归档”", "项目‘归档’", "项目（归档【旧版】）", "项目）归档（完成）"]) +def test_glob_result_preserves_balanced_cjk_directory_names(directory_name, content_block): + from deerflow.sandbox.tools import _format_glob_results + + root = "/mnt/user-data/workspace" + path = f"{root}/{directory_name}" + text = _format_glob_results(root, [path], truncated=False) + result = ToolMessage(content=[{"type": "text", "text": text}] if content_block else text, tool_call_id="call_glob", name="glob") + + entries = extract_artifacts_from_result(result, thread_id="thread-glob") + + assert [entry["real_ref"] for entry in entries] == [root, path] + assert entries[-1]["display_name"] == directory_name + + +@pytest.mark.parametrize("suffix", ["", "。", "。）", "]"]) +@pytest.mark.parametrize("ref", ["/mnt/user-data/workspace/项目（归档）", "https://files.example/report.pdf?label=项目（归档）"]) +def test_text_refs_preserve_balanced_cjk_closers_while_stripping_prose(ref, suffix): + assert [entry["ref"] for entry in _detect_refs_in_text(f"Report: （{ref}{suffix}")] == [ref] + + +@pytest.mark.parametrize( + "url", + [ + "https://files.example/report.PDF", + "https://files.example/image.PNG", + "https://files.example/report.PdF?token=part.csvX#section.md-more", + "https://files.example/image.PnG?token=encoded%EF%BC%89", + ], +) +def test_text_file_url_extension_case_insensitive_preserves_reference(url): + assert [entry["ref"] for entry in _detect_refs_in_text(f"Download [{url}]")] == [url] + + +def test_structured_url_keeps_literal_cjk_punctuation(): + url = "https://files.example/report.pdf?token=literal\u3011" + result = ToolMessage(content="done", tool_call_id="call_url", name="remote_report", artifact={"structured_content": {"url": url}}) + + entries = extract_artifacts_from_result(result, thread_id="thread-url") + + assert [entry["real_ref"] for entry in entries] == [url] + + +@pytest.mark.parametrize( + "url", + [ + "https://files.example/report.pdf/download", + "https://files.example/report.pdf.gz", + "https://files.example/report.pdfx", + "https://files.example/download?name=report.pdf", + "https://files.example/download#report.pdf", + "https://reports.pdf", + "https://files.example/report.PDF/download", + "https://files.example/report.PDFx", + "https://REPORT.PDF", + ], +) +def test_text_urls_require_a_file_extension_in_the_complete_path(url): + assert _detect_refs_in_text(f"Result: {url}") == [] + + def test_extract_from_file_block(): result = ToolMessage( content=[