mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-09-12 23:19:36 +00:00
* feat(harness): deterministic acceptance checklist for subagent delegations (RFC #4651, layer 2) PR4 of RFC #4651: check lead-supplied acceptance_criteria in code when a subagent completes, so objectively checkable requirements can never be silently passed by a self-report. - subagents/acceptance_checks.py: deterministic leaf families — file:<path> exists|non-empty and file_written:<path> read through read_current_file_content scoped to the shared thread workspace; the read uses the sandbox-native virtual path form (the local read validator and provider mount tables resolve /mnt/user-data/... paths, not host paths); the scope decision canonicalizes with realpath on the local sandbox so workspace symlinks cannot escape into uploads; a remote provider's "Error: ..." return string is normalized to a failed check (provider-typed via is_local_sandbox); a UnicodeDecodeError marks a binary deliverable as existing and non-empty; out-of-scope paths degrade to UNVERIFIED. tests_passed:<command> anchors to a matching recorded bash execution with status=success and a test-summary shape; matching is shell-structure aware with control-flow attribution (span must end at the last segment with provable execution), negating-option values are ineligible evidence and a target negated anywhere in the command degrades the match, extra flags must be selection-preserving, extra positionals widen only after a path-scoped criterion, truncated commands degrade via command_truncated, the summary shape is read only from output attributable to the matched segment (preceding segments provably silent by invocation form), and pass shapes require a nonzero passed count. Criterion text is neutralized with neutralize_untrusted_tags before storage/rendering. Anything else renders UNVERIFIED, never silently passed. - executor: accumulate bounded bash command/output evidence per streamed chunk (merged by tool_call_id, newest-capped) so subagent summarization compacting earlier messages cannot erase a recorded execution; the recorded status is the actual shell exit status parsed from the output's exit marker (signed codes included; the remote Command exited with code N form is accepted only as the whole trimmed output), falling back to deerflow_tool_meta only when no marker exists. - sandbox providers: e2b/opensandbox/tenki/boxlite append the LocalSandbox-style "Exit Code: N" marker on nonzero exit even with non-empty output; aio propagates the SDK's structured exit_code on both exec paths the same way; local timeouts append Exit Code: 124; and _truncate_bash_output always preserves a trailing exit marker (signed included) inside its budget, with a 32-char floor raising any smaller configured limit, so the actual shell outcome always survives in the output text. - task_tool: run the checklist offloaded (asyncio.to_thread) on the completed branch, failure-isolated; stamp the verdict into result metadata and render the per-criterion section into the model-visible result text. - status contract: additive subagent_acceptance_verdict transport with read-side structural validation. - delegation ledger: entry carries the verdict and renders a compact acceptance segment; gateway strips caller-forged verdicts from both ledger entries and message metadata, like the citation verdict. - blocking-IO anchor pins the offload (teeth proven red->green); leaf read errors catch only OSError/SandboxError so unexpected errors reach the task-tool-level isolation instead of being mislabeled. * fix(harness): close acceptance evidence gaps from review (RFC #4651 PR4) - negating options: overlap with a matched criterion target is now checked by path/nodeid prefix, not exact token equality — excluding a sub-path of the criterion's selection (pytest tests --deselect tests/unit/test_auth.py) degrades to UNVERIFIED instead of holds - output attribution: any redirection token in the matched final segment makes the recorded tail non-attributable (> / >> / 2> are word characters to the parser, so redirection was invisible to the matcher) - silent-source allowlist narrowed from any *activate suffix to the */bin/activate shape - status_contract docstring: restore the shared-fixture sentence and note subagent_acceptance_verdict is deliberately outside the fixture - executor: update_bash_executions publishes [] (stream carried no bash-family calls) instead of collapsing it into None, mirroring update_tool_receipts * fix(harness): close acceptance residual gaps from re-review (RFC #4651 PR4) - tests_passed: add error outcomes to the fail shapes — "4 passed, 1 error" and pytest's "ERROR <nodeid>" short summary no longer satisfy the pass shape when the exit status is swallowed (|| true) or absent; zero-error counts stay clean. - file leaves: bound the deliverable read — a "wc -c" shell size probe answers files above 50k bytes without loading ~2x their size, honoring the host-bash kill switch and falling back to the full read on any non-integer rendering, so verdicts never get less sound. - executor: record the exit marker text as status_marker on harvested bash evidence; the leaf detail now reports the marker actually seen instead of asserting a failure indistinguishable from the command's own trailing text. - extend the blocking-IO anchor to drive the probe branch inside the offload; teeth re-verified red->green. * fix(harness): close acceptance forgery and bound gaps from P2 re-review (RFC #4651 PR4) - file leaves: never read unbounded — size is established first (os.stat on the validated local host path, so the host-bash-disabled configuration needs no shell; a guarded wc -c on remote providers that renders missing/unreadable in its own words). Above the 50k cap the leaf answers from the size alone, at/below it the full read runs, and an unestablishable size degrades to UNVERIFIED instead of an unlimited fallback read. - output attribution: source/. prefixes are never provably silent — a crafted */bin/activate path shape says nothing about what the script prints, so sourced segments can no longer lend a passing summary. - executable identity: an explicitly path-spelled criterion now requires the same normalized executable path; the basename rule stays only for deliberately bare criterion commands. * fix(harness): run acceptance size probe outside subagent-controlled state (RFC #4651 PR4) - remote probe no longer runs in the sandbox's persistent shell: a fresh env -i /bin/sh with absolute-path stat/realpath (poisoned functions, aliases, PATH, exported functions, IFS, locale cannot steer it), plus a marker env routing AIO onto a fresh per-call bash.exec session. - metadata-only: stat never opens content, so a FIFO deliverable cannot block the parent for the provider's idle timeout; non-regular files (fifo/dir/symlink) degrade to UNVERIFIED. - containment canonicalized against the literal mount root: a final-component symlink or a swapped parent directory (root included) cannot redirect the check outside shared storage; unprovable layouts degrade to UNVERIFIED. * fix(harness): canonicalize probe containment against the canonical mount root (RFC #4651 PR4) Literal-root equality made every remote file leaf permanently UNVERIFIED on e2b and Tenki, which realize /mnt/user-data as a symlink to the home dir by default (e2b bootstrap 'sudo ln -sfn', Tenki best-effort symlink). Containment now compares the file's realpath against the mount root's realpath — exactly what the provider's own read path resolves, so probe and read-back stay consistent; final-component symlinks stay rejected by the non-dereferencing stat, and an intermediate dir-link escape under a sane root still lands ESCAPED. The inner script is a module constant and the suite now executes the composed probe for real against on-disk layouts (real dir, symlinked prefix, final symlink, fifo, missing, dir-link escape), which the canned-output stub could not see. * fix(harness): close bare-criterion negation and CDPATH summary channels (RFC #4651 PR4) - matching: a criterion with no positional selection target (bare pytest, make test) stands for the runner's default selection, so ANY negating option (--ignore/--deselect/...) makes the recorded run a different selection — unprovable. The overlap guard only sees consumed criterion tokens, which a bare criterion does not have; scoped criteria keep the unrelated-exclusion behavior. - attribution: cd is no longer blanket-silent — CDPATH makes cd print the resolved (subagent-chosen) destination and the pass shapes match as substrings, so one mkdir 'all tests passed' plus an export minted a pass for any quiet command. A cd argument or CDPATH= value (export or leading assignment) carrying any summary shape makes the segment non-silent; shape-free cd dir wrappers keep matching. - docs: _truncate_bash_output states the effective 32-char floor (the guarantee previously read as an unconditional max_chars bound). * fix(harness): close env-assignment and expansion channels in acceptance matching (RFC #4651 PR4) Self-audit in the shape of the last review rounds — channels the matcher classified as accounted-for that can change what runs, narrow the selection, or lend the summary text: - env assignments are no longer blanket-stripped: only an allowlist of inert display/CI knobs (CI, NO_COLOR, PY_COLORS, ...) may prefix a matched span, and a non-allowlisted assignment in any preceding segment (pure-assignment or export NAME=) is state pollution — PATH redirects the executable, LD_PRELOAD/PYTHONPATH/NODE_OPTIONS inject code, PYTEST_ADDOPTS/GOFLAGS/MAKEFILES inject selection-changing inputs, BASH_ENV runs arbitrary shell startup. All degrade to unprovable. - runtime expansions: any span token carrying /$( )/backticks, any negating-option value carrying an expansion or glob (unknown excluded set), and any extra executed token carrying glob metacharacters (crafted option-looking filenames narrow invisibly) are unprovable. Criterion-side globs stay self-consistent (literal match). - cd: an argument carrying a runtime expansion or glob is non-silent (unknown destination, unknown print); CDPATH= assignments are now handled as state pollution at the match layer, subsuming the value-shape special case. * fix(harness): persistent-shell evidence, exact env sets, option-arity scoping (RFC #4651 PR4) - tests_passed: on a persistent-shell provider (new Sandbox.persistent_shell_sessions capability, set by AioSandbox) every leaf degrades to UNVERIFIED — any earlier call in the shared session could have mutated the state the clean-looking run executed in, and only a fresh controlled session (RFC section 6 verifier) can prove otherwise. The flag is read from the provider registry without acquiring a sandbox. - env assignments: the allowlist is gone — no variable is provably inert across repositories (CI/DEBUG are routinely read by tests). The span's assignment prefix must equal the criterion's exactly (values included, order-insensitive); any assignment or export NAME= in a preceding segment is state pollution. - scoping: positional targets are now read by option arity, so a path embedded in an option (--basetemp=/tmp/p, --junitxml=/tmp/r.xml) never counts as a selection target and an extra positional after such a criterion narrows the default selection it denotes. * fix(harness): stamp shell provenance at harvest, close export/unset and arity gaps (RFC #4651 PR4) * fix(harness): split physical newlines as shell separators in acceptance matching (RFC #4651 PR4) * fix(harness): scope cd wrappers to thread data roots, pin accepted boundaries (RFC #4651 PR4) * fix(harness): preserve criterion connectors, prove file_written readable, fail-closed shell capability (RFC #4651 PR4) * fix(harness): compare only the connector prefix, tolerate trailing criterion semicolons (RFC #4651 PR4) * fix(harness): preserve continuation-line operators, keep ./-spelled executable identity (RFC #4651 PR4) * fix(harness): render criteria single-line so a multiline criterion cannot inject a forged checklist line (RFC #4651 PR4) * fix(harness): reject parent-traversal executable tokens in acceptance matching (RFC #4651 PR4) * fix(harness): reject parent-traversal negated values in acceptance matching (RFC #4651 PR4)
200 lines
7.7 KiB
Python
200 lines
7.7 KiB
Python
import re
|
|
from abc import ABC, abstractmethod
|
|
|
|
from deerflow.sandbox.search import GrepMatch
|
|
|
|
# POSIX env-var name rule: letter or underscore, then letters/digits/underscores.
|
|
# Used to validate ``env`` keys before they reach a sandbox implementation.
|
|
# No current implementation splices a key into a shell string — the local
|
|
# sandbox passes the dict to ``subprocess.run(env=...)`` (no shell), the AIO
|
|
# sandbox forwards it via the ``bash.exec`` structured ``env`` field, and e2b
|
|
# forwards it as the SDK's ``envs``. The check is defense-in-depth for the
|
|
# contract: a future shell-splicing implementation must not have to re-derive
|
|
# its own rule.
|
|
_ENV_NAME_PATTERN = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
|
|
|
|
|
def _validate_extra_env(extra_env: dict[str, str] | None) -> None:
|
|
"""Reject ``env`` keys that are not valid POSIX env-var names.
|
|
|
|
The :meth:`Sandbox.execute_command` contract accepts arbitrary ``str``
|
|
keys. Today no implementation splices a key into a shell string — the
|
|
local sandbox passes the dict to ``subprocess.run(env=...)`` (no shell),
|
|
the AIO sandbox forwards it via the ``bash.exec`` structured ``env``
|
|
field (no command-string splice), and e2b forwards it as the SDK's
|
|
``envs``. Enforcing the POSIX env-name rule in the abstract layer is
|
|
defense-in-depth for the contract: a future implementation that does
|
|
route a key through a shell must not have to re-derive its own
|
|
validation rule, and a caller passing a key derived from config /
|
|
payload / user input fails fast with ``ValueError`` instead of silently
|
|
producing an exploit should a future implementation regress to splicing.
|
|
|
|
Raises:
|
|
ValueError: When ``extra_env`` is not None and any key does not
|
|
match ``^[A-Za-z_][A-Za-z0-9_]*$``. ``None`` and empty dicts
|
|
pass through unchanged.
|
|
"""
|
|
if not extra_env:
|
|
return
|
|
for key in extra_env:
|
|
if not isinstance(key, str) or not _ENV_NAME_PATTERN.fullmatch(key):
|
|
raise ValueError(f"extra_env key {key!r} is not a valid POSIX environment variable name (must match ^[A-Za-z_][A-Za-z0-9_]*$). This protects shell-using sandbox implementations from command injection via the key.")
|
|
|
|
|
|
class Sandbox(ABC):
|
|
"""Abstract base class for sandbox environments"""
|
|
|
|
_id: str
|
|
|
|
#: Whether ``execute_command`` reuses one persistent shell session across
|
|
#: calls (shell state — exports, cwd, functions — survives from one call
|
|
#: into the next). When True, a recorded command's environment cannot be
|
|
#: proven clean from the command text alone, so evidence consumers (the
|
|
#: acceptance checklist's ``tests_passed`` matcher) must treat recorded
|
|
#: bash evidence as untrusted and degrade to UNVERIFIED.
|
|
#:
|
|
#: Tri-state, failing closed: ``None`` (the default) means the
|
|
#: implementation has NOT declared its session semantics — custom
|
|
#: providers are loaded by class path and may reuse a persistent
|
|
#: session, so silence cannot be read as fresh-shell. Consumers must
|
|
#: trust only an explicit ``False`` and degrade to UNVERIFIED on
|
|
#: ``None`` exactly as on ``True``. Every shipped implementation
|
|
#: declares explicitly (AIO: ``True``; the per-call exec providers:
|
|
#: ``False``).
|
|
persistent_shell_sessions: bool | None = None
|
|
|
|
def __init__(self, id: str):
|
|
self._id = id
|
|
|
|
@property
|
|
def id(self) -> str:
|
|
return self._id
|
|
|
|
@abstractmethod
|
|
def execute_command(
|
|
self,
|
|
command: str,
|
|
env: dict[str, str] | None = None,
|
|
timeout: float | None = None,
|
|
) -> str:
|
|
"""Execute bash command in sandbox.
|
|
|
|
Args:
|
|
command: The command to execute.
|
|
env: Optional per-call environment variables to inject into the
|
|
command's process. Used to pass request-scoped secrets (e.g. a
|
|
short-lived end-user token for skill scripts, issue #3861, or a
|
|
GitHub App installation token for ``git push`` / ``gh``) without
|
|
placing them in the prompt, tool arguments, or the command
|
|
string. When ``None`` the sandbox uses its default environment.
|
|
Keys must be valid POSIX environment-variable names
|
|
(``^[A-Za-z_][A-Za-z0-9_]*$``); implementations validate
|
|
via :func:`_validate_extra_env` before use. Values are
|
|
arbitrary strings — shell-using implementations
|
|
``shlex.quote`` them on splice.
|
|
timeout: Optional per-call wall-clock timeout in seconds. Local
|
|
sandboxes use this to bound host bash commands so long-lived
|
|
foreground processes cannot hang a turn indefinitely. Remote/AIO
|
|
implementations may ignore it when their backend does not expose
|
|
an equivalent command-timeout control separate from its own API
|
|
timeouts.
|
|
|
|
Returns:
|
|
The standard or error output of the command.
|
|
|
|
Raises:
|
|
ValueError: when an ``env`` key is not a valid env-var name.
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def read_file(
|
|
self,
|
|
path: str,
|
|
start_line: int | None = None,
|
|
end_line: int | None = None,
|
|
) -> str:
|
|
"""Read the content of a file.
|
|
|
|
Args:
|
|
path: The absolute path of the file to read.
|
|
start_line: Optional starting line number (1-indexed, inclusive).
|
|
end_line: Optional ending line number (1-indexed, inclusive).
|
|
|
|
Returns:
|
|
The content of the file.
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def download_file(self, path: str) -> bytes:
|
|
"""Download the binary content of a file.
|
|
|
|
Args:
|
|
path: The absolute path of the file to download.
|
|
|
|
Returns:
|
|
Raw file bytes.
|
|
|
|
Raises:
|
|
PermissionError: If path traversal is detected or the path is outside
|
|
the allowed virtual prefix.
|
|
OSError: If the file cannot be read or does not exist. Both local
|
|
and remote implementations must raise ``OSError`` so callers
|
|
have a single exception type to handle.
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def list_dir(self, path: str, max_depth=2) -> list[str]:
|
|
"""List the contents of a directory.
|
|
|
|
Args:
|
|
path: The absolute path of the directory to list.
|
|
max_depth: The maximum depth to traverse. Default is 2.
|
|
|
|
Returns:
|
|
The contents of the directory.
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def write_file(self, path: str, content: str, append: bool = False) -> None:
|
|
"""Write content to a file.
|
|
|
|
Args:
|
|
path: The absolute path of the file to write to.
|
|
content: The text content to write to the file.
|
|
append: Whether to append the content to the file. If False, the file will be created or overwritten.
|
|
"""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def glob(self, path: str, pattern: str, *, include_dirs: bool = False, max_results: int = 200) -> tuple[list[str], bool]:
|
|
"""Find paths that match a glob pattern under a root directory."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def grep(
|
|
self,
|
|
path: str,
|
|
pattern: str,
|
|
*,
|
|
glob: str | None = None,
|
|
literal: bool = False,
|
|
case_sensitive: bool = False,
|
|
max_results: int = 100,
|
|
) -> tuple[list[GrepMatch], bool]:
|
|
"""Search for matches inside a text file or files under a directory."""
|
|
pass
|
|
|
|
@abstractmethod
|
|
def update_file(self, path: str, content: bytes) -> None:
|
|
"""Update a file with binary content.
|
|
|
|
Args:
|
|
path: The absolute path of the file to update.
|
|
content: The binary content to write to the file.
|
|
"""
|
|
pass
|