mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-25 14:06:18 +00:00
fix(subagents): recognize empty regular files in remote acceptance pr… (#5559)
* fix(subagents): recognize empty regular files in remote acceptance probes * test(subagents): address empty-file acceptance review feedback * docs(subagents): condense empty artifact guidance --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
This commit is contained in:
parent
e4ce26f3d2
commit
c52ad191f4
@ -941,6 +941,10 @@ This release closes that milestone with **765 merged pull requests**.
|
|||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
|
- **subagents:** Recognize zero-byte regular deliverables in remote sandbox
|
||||||
|
acceptance checks. Readable empty files now satisfy `exists` and
|
||||||
|
`file_written` and deterministically fail `non-empty`, instead of remaining
|
||||||
|
UNVERIFIED. ([#5559])
|
||||||
- **persistence:** Heal databases that silently skipped the run-change clock
|
- **persistence:** Heal databases that silently skipped the run-change clock
|
||||||
schema. `0023_run_change_seq` was inserted ahead of the already-shipped
|
schema. `0023_run_change_seq` was inserted ahead of the already-shipped
|
||||||
`0023_user_preferences` revision, so databases stamped at that revision (or
|
`0023_user_preferences` revision, so databases stamped at that revision (or
|
||||||
@ -4314,6 +4318,7 @@ with **180 merged pull requests** since the first 2.0 milestone tag.
|
|||||||
[#5504]: https://github.com/bytedance/deer-flow/pull/5504
|
[#5504]: https://github.com/bytedance/deer-flow/pull/5504
|
||||||
[#5505]: https://github.com/bytedance/deer-flow/pull/5505
|
[#5505]: https://github.com/bytedance/deer-flow/pull/5505
|
||||||
[#5524]: https://github.com/bytedance/deer-flow/pull/5524
|
[#5524]: https://github.com/bytedance/deer-flow/pull/5524
|
||||||
|
[#5559]: https://github.com/bytedance/deer-flow/pull/5559
|
||||||
[#5526]: https://github.com/bytedance/deer-flow/pull/5526
|
[#5526]: https://github.com/bytedance/deer-flow/pull/5526
|
||||||
[#5534]: https://github.com/bytedance/deer-flow/pull/5534
|
[#5534]: https://github.com/bytedance/deer-flow/pull/5534
|
||||||
[#5547]: https://github.com/bytedance/deer-flow/pull/5547
|
[#5547]: https://github.com/bytedance/deer-flow/pull/5547
|
||||||
|
|||||||
@ -1527,6 +1527,8 @@ may still appear in progress; their outcome is not inferred from their text.
|
|||||||
|
|
||||||
The lead agent can spawn sub-agents on the fly — each with its own scoped context, tools, and termination conditions — when delegation has clear net benefit from real parallel latency, specialist capability, or context isolation. It keeps interdependent scopes and overlapping side effects out of parallel dispatch; a bounded sequential chain can still run in one sub-agent when specialist or context-isolation benefit clearly wins. The lead uses the fewest useful sub-agents and re-evaluates later batches instead of fanning out solely because a task is large or multi-step. Sub-agents report back structured results, and the lead agent verifies and synthesizes them into a coherent output. Deterministic tool receipts cover both direct tool messages and state-updating `Command` results such as delegated `task` responses; when the receipt ledger reaches its context budget, it retains the newest actions and their original receipt IDs. Operators can disable this provenance layer with `verification.receipts_enabled: false`. Their configured skills are resolved from the same user-scoped catalog as the lead agent, so user-owned custom skills remain available without exposing another user's version. Their internal AI and tool messages stay scoped to the delegated graph instead of entering the parent chat stream. Reloaded thread history enforces the same boundary: callback-captured sub-agent AI responses remain available in run-event diagnostics but are excluded from the parent transcript, while the parent `task` result remains attached to its subtask card. Long-running sub-agents compact older history when summarization is enabled and re-inject the summary as guarded, hidden durable context before continuing, so recent assistant/tool activity remains grounded in the task. Their system instructions, including the role and report contract, survive compaction; if only those instructions and the current request would be summarized, compaction is skipped. Provider/model request failures are reported as failed sub-agent tasks rather than successful results, so the lead agent and Web UI can react to them correctly. Concurrent parent runs also receive independent server-side sub-agent execution IDs, so a provider that reuses a tool-call ID cannot make one run poll, cancel, or clean up another run's background task. Collapsed sub-agent cards show the effective model and, when the provider returns usage metadata, a cumulative token total that updates after each completed sub-agent LLM call and persists after a reload. When token usage tracking is enabled, completed sub-agent usage is attributed back to the dispatching step from that run's terminal tool-message metadata rather than a process-global provider-ID cache.
|
The lead agent can spawn sub-agents on the fly — each with its own scoped context, tools, and termination conditions — when delegation has clear net benefit from real parallel latency, specialist capability, or context isolation. It keeps interdependent scopes and overlapping side effects out of parallel dispatch; a bounded sequential chain can still run in one sub-agent when specialist or context-isolation benefit clearly wins. The lead uses the fewest useful sub-agents and re-evaluates later batches instead of fanning out solely because a task is large or multi-step. Sub-agents report back structured results, and the lead agent verifies and synthesizes them into a coherent output. Deterministic tool receipts cover both direct tool messages and state-updating `Command` results such as delegated `task` responses; when the receipt ledger reaches its context budget, it retains the newest actions and their original receipt IDs. Operators can disable this provenance layer with `verification.receipts_enabled: false`. Their configured skills are resolved from the same user-scoped catalog as the lead agent, so user-owned custom skills remain available without exposing another user's version. Their internal AI and tool messages stay scoped to the delegated graph instead of entering the parent chat stream. Reloaded thread history enforces the same boundary: callback-captured sub-agent AI responses remain available in run-event diagnostics but are excluded from the parent transcript, while the parent `task` result remains attached to its subtask card. Long-running sub-agents compact older history when summarization is enabled and re-inject the summary as guarded, hidden durable context before continuing, so recent assistant/tool activity remains grounded in the task. Their system instructions, including the role and report contract, survive compaction; if only those instructions and the current request would be summarized, compaction is skipped. Provider/model request failures are reported as failed sub-agent tasks rather than successful results, so the lead agent and Web UI can react to them correctly. Concurrent parent runs also receive independent server-side sub-agent execution IDs, so a provider that reuses a tool-call ID cannot make one run poll, cancel, or clean up another run's background task. Collapsed sub-agent cards show the effective model and, when the provider returns usage metadata, a cumulative token total that updates after each completed sub-agent LLM call and persists after a reload. When token usage tracking is enabled, completed sub-agent usage is attributed back to the dispatching step from that run's terminal tool-message metadata rather than a process-global provider-ID cache.
|
||||||
|
|
||||||
|
For file acceptance criteria, an empty regular file in the shared workspace can satisfy `file:<path> exists` and `file_written:<path>`, including on remote sandboxes. It fails `file:<path> non-empty` with a deterministic empty-file result.
|
||||||
|
|
||||||
Content-less sub-agent final messages report `No response generated` instead of the literal text `None`. A content-less provider-error fallback reports its structured error detail when available.
|
Content-less sub-agent final messages report `No response generated` instead of the literal text `None`. A content-less provider-error fallback reports its structured error detail when available.
|
||||||
|
|
||||||
An ordinary `task` also receives a defensive snapshot of the dispatching run's current uploads. This lets eligible sub-agents use `list_uploaded_files` to find earlier-turn files without returning same-turn attachments as historical. Delayed or recovered `batch_task` workers leave this tool disabled because they have no valid turn-local upload boundary.
|
An ordinary `task` also receives a defensive snapshot of the dispatching run's current uploads. This lets eligible sub-agents use `list_uploaded_files` to find earlier-turn files without returning same-turn attachments as historical. Delayed or recovered `batch_task` workers leave this tool disabled because they have no valid turn-local upload boundary.
|
||||||
|
|||||||
@ -1,5 +1,10 @@
|
|||||||
### Subagent System (`packages/harness/deerflow/subagents/`)
|
### Subagent System (`packages/harness/deerflow/subagents/`)
|
||||||
|
|
||||||
|
**Empty remote artifacts**: Acceptance probes treat GNU `stat -c %F` labels
|
||||||
|
`regular file` and `regular empty file` as regular files. Empty files satisfy
|
||||||
|
`exists`/`file_written` but fail `non-empty`; symlink leaves, directories, and
|
||||||
|
FIFOs remain rejected.
|
||||||
|
|
||||||
**Context**: Capture after validation, before setup. Keep genuine replies, even hidden clarifications; exclude framework state and unpaired calls. Mark unserializable media as omitted.
|
**Context**: Capture after validation, before setup. Keep genuine replies, even hidden clarifications; exclude framework state and unpaired calls. Mark unserializable media as omitted.
|
||||||
|
|
||||||
**Direct runtime shutdown**: `SubagentRuntime.stop()` holds its lifecycle lock until the owned service stop task terminates, then propagates the first caller cancellation with any service failure/cancellation as its cause. The drain is intentionally unbounded: repository awaits and child cleanup must terminate; a timeout must not detach still-owned work. Keep terminal-outcome and repeated-cancellation coverage in `tests/test_subagent_runtime.py`.
|
**Direct runtime shutdown**: `SubagentRuntime.stop()` holds its lifecycle lock until the owned service stop task terminates, then propagates the first caller cancellation with any service failure/cancellation as its cause. The drain is intentionally unbounded: repository awaits and child cleanup must terminate; a timeout must not detach still-owned work. Keep terminal-outcome and repeated-cancellation coverage in `tests/test_subagent_runtime.py`.
|
||||||
|
|||||||
@ -227,11 +227,12 @@ def _resolve_scoped_path(path: str, thread_data: Mapping[str, Any] | None, *, re
|
|||||||
#: resolves, so the probe and the later read-back stay consistent. A
|
#: resolves, so the probe and the later read-back stay consistent. A
|
||||||
#: leaf-level symlink is rejected outright by the non-dereferencing
|
#: leaf-level symlink is rejected outright by the non-dereferencing
|
||||||
#: ``stat -c %F``; an intermediate dir-link escape under a sane root still
|
#: ``stat -c %F``; an intermediate dir-link escape under a sane root still
|
||||||
#: lands outside the canonical root (ESCAPED).
|
#: lands outside the canonical root (ESCAPED). GNU stat labels zero-byte
|
||||||
|
#: regular files as ``regular empty file``; both regular-file labels qualify.
|
||||||
_SIZE_PROBE_INNER_SCRIPT = (
|
_SIZE_PROBE_INNER_SCRIPT = (
|
||||||
'[ -e "$1" ] || { echo NOFILE; exit 0; }; '
|
'[ -e "$1" ] || { echo NOFILE; exit 0; }; '
|
||||||
't=$(/usr/bin/stat -c %F -- "$1") || { echo UNREADABLE; exit 0; }; '
|
't=$(/usr/bin/stat -c %F -- "$1") || { echo UNREADABLE; exit 0; }; '
|
||||||
'[ "$t" = "regular file" ] || { echo NONREGULAR; exit 0; }; '
|
'case "$t" in "regular file"|"regular empty file") ;; *) echo NONREGULAR; exit 0 ;; esac; '
|
||||||
'r=$(/usr/bin/realpath -- "$2") || { echo UNREADABLE; exit 0; }; '
|
'r=$(/usr/bin/realpath -- "$2") || { echo UNREADABLE; exit 0; }; '
|
||||||
'p=$(/usr/bin/realpath -- "$1") || { echo UNREADABLE; exit 0; }; '
|
'p=$(/usr/bin/realpath -- "$1") || { echo UNREADABLE; exit 0; }; '
|
||||||
'case $p in "$r"/*) /usr/bin/stat -c %s -- "$p" ;; *) echo ESCAPED ;; esac'
|
'case $p in "$r"/*) /usr/bin/stat -c %s -- "$p" ;; *) echo ESCAPED ;; esac'
|
||||||
@ -321,7 +322,7 @@ def _probe_file_size(runtime: Any, resolved: str, thread_data: Mapping[str, Any]
|
|||||||
_READ_PROBE_INNER_SCRIPT = (
|
_READ_PROBE_INNER_SCRIPT = (
|
||||||
'[ -e "$1" ] || { echo NOFILE; exit 0; }; '
|
'[ -e "$1" ] || { echo NOFILE; exit 0; }; '
|
||||||
't=$(/usr/bin/stat -c %F -- "$1") || { echo UNREADABLE; exit 0; }; '
|
't=$(/usr/bin/stat -c %F -- "$1") || { echo UNREADABLE; exit 0; }; '
|
||||||
'[ "$t" = "regular file" ] || { echo NONREGULAR; exit 0; }; '
|
'case "$t" in "regular file"|"regular empty file") ;; *) echo NONREGULAR; exit 0 ;; esac; '
|
||||||
'r=$(/usr/bin/realpath -- "$2") || { echo UNREADABLE; exit 0; }; '
|
'r=$(/usr/bin/realpath -- "$2") || { echo UNREADABLE; exit 0; }; '
|
||||||
'p=$(/usr/bin/realpath -- "$1") || { echo UNREADABLE; exit 0; }; '
|
'p=$(/usr/bin/realpath -- "$1") || { echo UNREADABLE; exit 0; }; '
|
||||||
'case $p in "$r"/*) ;; *) echo ESCAPED; exit 0 ;; esac; '
|
'case $p in "$r"/*) ;; *) echo ESCAPED; exit 0 ;; esac; '
|
||||||
|
|||||||
@ -625,22 +625,24 @@ class TestProbeInnerScriptRealLayouts:
|
|||||||
command = f"/usr/bin/env -i /bin/sh -c {shlex.quote(_SIZE_PROBE_INNER_SCRIPT)} probe {shlex.quote(path)} {shlex.quote(root)}"
|
command = f"/usr/bin/env -i /bin/sh -c {shlex.quote(_SIZE_PROBE_INNER_SCRIPT)} probe {shlex.quote(path)} {shlex.quote(root)}"
|
||||||
return subprocess.run(command, shell=True, capture_output=True, text=True, check=True).stdout.strip()
|
return subprocess.run(command, shell=True, capture_output=True, text=True, check=True).stdout.strip()
|
||||||
|
|
||||||
def test_real_directory_mount_root(self, tmp_path):
|
@pytest.mark.parametrize("content", ("", "hello"), ids=("empty", "nonempty"))
|
||||||
|
def test_real_directory_mount_root(self, tmp_path, content):
|
||||||
"""AIO/BoxLite/OpenSandbox layout: a genuine mount directory."""
|
"""AIO/BoxLite/OpenSandbox layout: a genuine mount directory."""
|
||||||
outputs = tmp_path / "mnt" / "user-data" / "outputs"
|
outputs = tmp_path / "mnt" / "user-data" / "outputs"
|
||||||
outputs.mkdir(parents=True)
|
outputs.mkdir(parents=True)
|
||||||
(outputs / "report.md").write_text("hello", encoding="utf-8")
|
(outputs / "report.md").write_text(content, encoding="utf-8")
|
||||||
assert self._run_probe(str(outputs / "report.md"), str(outputs)) == "5"
|
assert self._run_probe(str(outputs / "report.md"), str(outputs)) == str(len(content))
|
||||||
|
|
||||||
def test_symlinked_mount_prefix(self, tmp_path):
|
@pytest.mark.parametrize("content", ("", "hello"), ids=("empty", "nonempty"))
|
||||||
|
def test_symlinked_mount_prefix(self, tmp_path, content):
|
||||||
"""e2b/Tenki default layout: ``/mnt/user-data`` is a symlink to the
|
"""e2b/Tenki default layout: ``/mnt/user-data`` is a symlink to the
|
||||||
home dir — the canonical root still contains the canonical file."""
|
home dir — the canonical root still contains the canonical file."""
|
||||||
home_outputs = tmp_path / "home" / "user" / "outputs"
|
home_outputs = tmp_path / "home" / "user" / "outputs"
|
||||||
home_outputs.mkdir(parents=True)
|
home_outputs.mkdir(parents=True)
|
||||||
(home_outputs / "report.md").write_text("hello", encoding="utf-8")
|
(home_outputs / "report.md").write_text(content, encoding="utf-8")
|
||||||
(tmp_path / "mnt").mkdir()
|
(tmp_path / "mnt").mkdir()
|
||||||
(tmp_path / "mnt" / "user-data").symlink_to(tmp_path / "home" / "user")
|
(tmp_path / "mnt" / "user-data").symlink_to(tmp_path / "home" / "user")
|
||||||
assert self._run_probe(str(tmp_path / "mnt" / "user-data" / "outputs" / "report.md"), str(tmp_path / "mnt" / "user-data" / "outputs")) == "5"
|
assert self._run_probe(str(tmp_path / "mnt" / "user-data" / "outputs" / "report.md"), str(tmp_path / "mnt" / "user-data" / "outputs")) == str(len(content))
|
||||||
|
|
||||||
def test_final_component_symlink_is_nonregular(self, tmp_path):
|
def test_final_component_symlink_is_nonregular(self, tmp_path):
|
||||||
outputs = tmp_path / "outputs"
|
outputs = tmp_path / "outputs"
|
||||||
@ -656,12 +658,51 @@ class TestProbeInnerScriptRealLayouts:
|
|||||||
command = f"/usr/bin/env -i /bin/sh -c {shlex.quote(_READ_PROBE_INNER_SCRIPT)} probe {shlex.quote(path)} {shlex.quote(root)}"
|
command = f"/usr/bin/env -i /bin/sh -c {shlex.quote(_READ_PROBE_INNER_SCRIPT)} probe {shlex.quote(path)} {shlex.quote(root)}"
|
||||||
return subprocess.run(command, shell=True, capture_output=True, text=True, check=True).stdout.strip()
|
return subprocess.run(command, shell=True, capture_output=True, text=True, check=True).stdout.strip()
|
||||||
|
|
||||||
def test_read_probe_regular_file_is_readable(self, tmp_path):
|
@pytest.mark.parametrize("content", ("", "hello"), ids=("empty", "nonempty"))
|
||||||
|
def test_read_probe_regular_file_is_readable(self, tmp_path, content):
|
||||||
outputs = tmp_path / "outputs"
|
outputs = tmp_path / "outputs"
|
||||||
outputs.mkdir()
|
outputs.mkdir()
|
||||||
(outputs / "report.md").write_text("hello", encoding="utf-8")
|
(outputs / "report.md").write_text(content, encoding="utf-8")
|
||||||
assert self._run_read_probe(str(outputs / "report.md"), str(outputs)) == "READABLE"
|
assert self._run_read_probe(str(outputs / "report.md"), str(outputs)) == "READABLE"
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("criterion", "holds", "detail"),
|
||||||
|
(
|
||||||
|
("file:../outputs/empty.md exists", True, "exists, 0 bytes"),
|
||||||
|
("file:../outputs/empty.md non-empty", False, "file is empty"),
|
||||||
|
("file_written:../outputs/empty.md", True, "read-back ok, 0 bytes"),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
def test_empty_remote_file_acceptance(self, tmp_path, monkeypatch, criterion, holds, detail):
|
||||||
|
"""Exercise the real size probe through the public checklist API."""
|
||||||
|
outputs = tmp_path / "outputs"
|
||||||
|
outputs.mkdir()
|
||||||
|
(outputs / "empty.md").touch()
|
||||||
|
mapping = {
|
||||||
|
"/mnt/user-data/outputs/empty.md": str(outputs / "empty.md"),
|
||||||
|
"/mnt/user-data/outputs": str(outputs),
|
||||||
|
}
|
||||||
|
|
||||||
|
class _RealShellSandbox:
|
||||||
|
def execute_command(self, command, **kwargs):
|
||||||
|
assert kwargs.get("env") == {"_DEERFLOW_SIZE_PROBE": "1"}
|
||||||
|
for virtual, host in sorted(mapping.items(), key=lambda kv: -len(kv[0])):
|
||||||
|
command = command.replace(virtual, shlex.quote(host))
|
||||||
|
return subprocess.run(command, shell=True, capture_output=True, text=True, check=True, timeout=5).stdout
|
||||||
|
|
||||||
|
monkeypatch.setattr("deerflow.sandbox.tools.ensure_sandbox_initialized", lambda runtime=None: _RealShellSandbox())
|
||||||
|
verdict = check_acceptance_criteria(
|
||||||
|
[criterion],
|
||||||
|
runtime=SimpleNamespace(state=None),
|
||||||
|
thread_data=THREAD_DATA,
|
||||||
|
content_reader=_reader({"/mnt/user-data/outputs/empty.md": ""}),
|
||||||
|
)
|
||||||
|
leaf = verdict["leaves"][0]
|
||||||
|
assert leaf["checked"] is True
|
||||||
|
assert leaf["holds"] is holds
|
||||||
|
assert leaf["detail"] == detail
|
||||||
|
assert verdict["unchecked"] == []
|
||||||
|
|
||||||
@pytest.mark.skipif(os.name == "nt" or os.geteuid() == 0, reason="mode-000 readability needs POSIX permissions and a non-root euid")
|
@pytest.mark.skipif(os.name == "nt" or os.geteuid() == 0, reason="mode-000 readability needs POSIX permissions and a non-root euid")
|
||||||
def test_read_probe_mode_000_is_unreadable(self, tmp_path):
|
def test_read_probe_mode_000_is_unreadable(self, tmp_path):
|
||||||
outputs = tmp_path / "outputs"
|
outputs = tmp_path / "outputs"
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user