* fix(sandbox): stop list_dir from reporting failures as empty
Remote providers swallowed find/client errors as [] and 2>/dev/null
missing paths as empty stdout. ls_tool then told the agent the
directory was (empty). Raise OSError/FileNotFoundError instead so
the tool returns Error.
* fix(sandbox): list_dir raises on missing local paths and uses find -H
Empty stdout is not a missing path when find's start point is a
symlink (E2B /mnt/acp-workspace). Dereference only the start point
with find -H. LocalSandbox now raises FileNotFoundError for a
non-directory root, matching remote providers. AIO maps a missing
result.data to OSError rather than FileNotFoundError.
* fix(sandbox): group AIO list_dir find type predicates
Without parentheses, find PATH -maxdepth N -type f -o -type d applies
-type d without maxdepth and can drop files from the listing.
* fix(sandbox): distinguish list_dir command failure from missing path
Tenki, Boxlite, and OpenSandbox treated any empty find stdout as
FileNotFoundError, so a missing find binary (exit 127) or SDK error
looked like a missing directory. Raise OSError when find status is
outside (0, 1); keep FileNotFoundError for the find-ran-but-empty case.
* fix(sandbox): apply list_dir exit-status contract to AIO and E2B
Same gap as Tenki/Boxlite/OpenSandbox: empty find stdout with exit 127
was FileNotFoundError. Raise OSError when the status is outside (0, 1).
* fix(sandbox): classify list_dir by find status not head status
find | head under sh -lc reports head's exit code, so a missing find
binary (127) became FileNotFoundError. Record find's own status after
the bounded listing, treat SIGPIPE 141 as truncation success, and add
a shell-level regression test.
* test(auth): include projects permissions in /me contract pins
#5265 added projects:read/write/delete to the registered route set.
The /auth/me tests still pinned the pre-projects list, so CI failed
after merging main.
* fix(sandbox): do not treat missing list_dir marker as success
The generated script ended on `rm -f`, so process status was 0/1 even
when find's marker never landed. Both codes are in _FIND_OK, and the
parser fallback then classified an empty listing as FileNotFoundError —
the 127 misclassification this helper was meant to close.
Exit with find's status (126 if unknown). A missing marker is now
OSError unless the process status is already a non-OK failure.
* test(sandbox): emit list_dir status marker in provider fixtures
Parser now requires __DF_FIND_STATUS__ and refuses marker-less stdout.
Update AIO/Boxlite/E2B stubs and OpenSandbox/Tenki find fakes so listings
carry :0 and missing paths carry :1 with matching exit codes.
* style(sandbox): format list dir test fixture
* style(sandbox): format remote list dir helper
* docs(sandbox): keep guidance within the tested size budget
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): stop stripping filenames when parsing find output in remote providers
The list_dir and glob parsers in the e2b, OpenSandbox, AIO, Tenki, and
BoxLite providers called .strip() on every line of find output. A
filename that legitimately ends (or begins) in whitespace was corrupted,
so the listed path never resolved on any follow-up file API call, and
the remote providers diverged from LocalSandbox, which preserves such
names via pathlib.
splitlines() already removes the line terminators, so filter empty lines
only and keep each entry verbatim. Same class of bug as the e2b
_sync_outputs_to_host fix (#4861), applied to the search parsers.
Adds a trailing-space regression test per provider at the seam each
suite already uses.
* fix(sandbox): split find output on \n only, and rename the tenki test
Review follow-ups from willem-bd:
- aio_sandbox.list_dir used str.splitlines(), which also breaks records on
\v, \f, \x1c-\x1e and \x85 - all legal inside a Linux filename, and all
contrary to this PR's own rule that the newline is the only delimiter.
find emits \n and nothing else, so split("\n") is the correct parse.
- Renamed test_search_preserves_trailing_space_in_filename to
test_list_dir_and_glob_preserve_trailing_space_in_filename, matching the
sibling tests in test_opensandbox_provider.py and test_boxlite_provider.py.
The body covers list_dir and glob; it never touches grep.
* feat(harness): deterministic acceptance checklist for subagent delegations (RFC #4651, layer 2)
PR4 of RFC #4651: check lead-supplied acceptance_criteria in code when a
subagent completes, so objectively checkable requirements can never be
silently passed by a self-report.
- subagents/acceptance_checks.py: deterministic leaf families —
file:<path> exists|non-empty and file_written:<path> read through
read_current_file_content scoped to the shared thread workspace; the
read uses the sandbox-native virtual path form (the local read
validator and provider mount tables resolve /mnt/user-data/... paths,
not host paths); the scope decision canonicalizes with realpath on the
local sandbox so workspace symlinks cannot escape into uploads; a
remote provider's "Error: ..." return string is normalized to a
failed check (provider-typed via is_local_sandbox); a
UnicodeDecodeError marks a binary deliverable as existing and
non-empty; out-of-scope paths degrade to UNVERIFIED.
tests_passed:<command> anchors to a matching recorded bash execution
with status=success and a test-summary shape; matching is
shell-structure aware with control-flow attribution (span must end at
the last segment with provable execution), negating-option values are
ineligible evidence and a target negated anywhere in the command
degrades the match, extra flags must be selection-preserving, extra
positionals widen only after a path-scoped criterion, truncated
commands degrade via command_truncated, the summary shape is read
only from output attributable to the matched segment (preceding
segments provably silent by invocation form), and pass shapes require
a nonzero passed count. Criterion text is neutralized with
neutralize_untrusted_tags before storage/rendering. Anything else
renders UNVERIFIED, never silently passed.
- executor: accumulate bounded bash command/output evidence per streamed
chunk (merged by tool_call_id, newest-capped) so subagent
summarization compacting earlier messages cannot erase a recorded
execution; the recorded status is the actual shell exit status parsed
from the output's exit marker (signed codes included; the remote
Command exited with code N form is accepted only as the whole trimmed
output), falling back to deerflow_tool_meta only when no marker
exists.
- sandbox providers: e2b/opensandbox/tenki/boxlite append the
LocalSandbox-style "Exit Code: N" marker on nonzero exit even with
non-empty output; aio propagates the SDK's structured exit_code on
both exec paths the same way; local timeouts append Exit Code: 124;
and _truncate_bash_output always preserves a trailing exit marker
(signed included) inside its budget, with a 32-char floor raising any
smaller configured limit, so the actual shell outcome always survives
in the output text.
- task_tool: run the checklist offloaded (asyncio.to_thread) on the
completed branch, failure-isolated; stamp the verdict into result
metadata and render the per-criterion section into the model-visible
result text.
- status contract: additive subagent_acceptance_verdict transport with
read-side structural validation.
- delegation ledger: entry carries the verdict and renders a compact
acceptance segment; gateway strips caller-forged verdicts from both
ledger entries and message metadata, like the citation verdict.
- blocking-IO anchor pins the offload (teeth proven red->green); leaf
read errors catch only OSError/SandboxError so unexpected errors reach
the task-tool-level isolation instead of being mislabeled.
* fix(harness): close acceptance evidence gaps from review (RFC #4651 PR4)
- negating options: overlap with a matched criterion target is now
checked by path/nodeid prefix, not exact token equality — excluding a
sub-path of the criterion's selection (pytest tests --deselect
tests/unit/test_auth.py) degrades to UNVERIFIED instead of holds
- output attribution: any redirection token in the matched final segment
makes the recorded tail non-attributable (> / >> / 2> are word
characters to the parser, so redirection was invisible to the matcher)
- silent-source allowlist narrowed from any *activate suffix to the
*/bin/activate shape
- status_contract docstring: restore the shared-fixture sentence and
note subagent_acceptance_verdict is deliberately outside the fixture
- executor: update_bash_executions publishes [] (stream carried no
bash-family calls) instead of collapsing it into None, mirroring
update_tool_receipts
* fix(harness): close acceptance residual gaps from re-review (RFC #4651 PR4)
- tests_passed: add error outcomes to the fail shapes — "4 passed, 1 error"
and pytest's "ERROR <nodeid>" short summary no longer satisfy the pass
shape when the exit status is swallowed (|| true) or absent; zero-error
counts stay clean.
- file leaves: bound the deliverable read — a "wc -c" shell size probe
answers files above 50k bytes without loading ~2x their size, honoring
the host-bash kill switch and falling back to the full read on any
non-integer rendering, so verdicts never get less sound.
- executor: record the exit marker text as status_marker on harvested bash
evidence; the leaf detail now reports the marker actually seen instead of
asserting a failure indistinguishable from the command's own trailing text.
- extend the blocking-IO anchor to drive the probe branch inside the
offload; teeth re-verified red->green.
* fix(harness): close acceptance forgery and bound gaps from P2 re-review (RFC #4651 PR4)
- file leaves: never read unbounded — size is established first (os.stat on
the validated local host path, so the host-bash-disabled configuration
needs no shell; a guarded wc -c on remote providers that renders
missing/unreadable in its own words). Above the 50k cap the leaf answers
from the size alone, at/below it the full read runs, and an
unestablishable size degrades to UNVERIFIED instead of an unlimited
fallback read.
- output attribution: source/. prefixes are never provably silent — a
crafted */bin/activate path shape says nothing about what the script
prints, so sourced segments can no longer lend a passing summary.
- executable identity: an explicitly path-spelled criterion now requires
the same normalized executable path; the basename rule stays only for
deliberately bare criterion commands.
* fix(harness): run acceptance size probe outside subagent-controlled state (RFC #4651 PR4)
- remote probe no longer runs in the sandbox's persistent shell: a fresh
env -i /bin/sh with absolute-path stat/realpath (poisoned functions,
aliases, PATH, exported functions, IFS, locale cannot steer it), plus a
marker env routing AIO onto a fresh per-call bash.exec session.
- metadata-only: stat never opens content, so a FIFO deliverable cannot
block the parent for the provider's idle timeout; non-regular files
(fifo/dir/symlink) degrade to UNVERIFIED.
- containment canonicalized against the literal mount root: a
final-component symlink or a swapped parent directory (root included)
cannot redirect the check outside shared storage; unprovable layouts
degrade to UNVERIFIED.
* fix(harness): canonicalize probe containment against the canonical mount root (RFC #4651 PR4)
Literal-root equality made every remote file leaf permanently UNVERIFIED
on e2b and Tenki, which realize /mnt/user-data as a symlink to the home
dir by default (e2b bootstrap 'sudo ln -sfn', Tenki best-effort symlink).
Containment now compares the file's realpath against the mount root's
realpath — exactly what the provider's own read path resolves, so probe
and read-back stay consistent; final-component symlinks stay rejected by
the non-dereferencing stat, and an intermediate dir-link escape under a
sane root still lands ESCAPED. The inner script is a module constant and
the suite now executes the composed probe for real against on-disk
layouts (real dir, symlinked prefix, final symlink, fifo, missing,
dir-link escape), which the canned-output stub could not see.
* fix(harness): close bare-criterion negation and CDPATH summary channels (RFC #4651 PR4)
- matching: a criterion with no positional selection target (bare pytest,
make test) stands for the runner's default selection, so ANY negating
option (--ignore/--deselect/...) makes the recorded run a different
selection — unprovable. The overlap guard only sees consumed criterion
tokens, which a bare criterion does not have; scoped criteria keep the
unrelated-exclusion behavior.
- attribution: cd is no longer blanket-silent — CDPATH makes cd print the
resolved (subagent-chosen) destination and the pass shapes match as
substrings, so one mkdir 'all tests passed' plus an export minted a pass
for any quiet command. A cd argument or CDPATH= value (export or leading
assignment) carrying any summary shape makes the segment non-silent;
shape-free cd dir wrappers keep matching.
- docs: _truncate_bash_output states the effective 32-char floor (the
guarantee previously read as an unconditional max_chars bound).
* fix(harness): close env-assignment and expansion channels in acceptance matching (RFC #4651 PR4)
Self-audit in the shape of the last review rounds — channels the matcher
classified as accounted-for that can change what runs, narrow the
selection, or lend the summary text:
- env assignments are no longer blanket-stripped: only an allowlist of
inert display/CI knobs (CI, NO_COLOR, PY_COLORS, ...) may prefix a
matched span, and a non-allowlisted assignment in any preceding segment
(pure-assignment or export NAME=) is state pollution — PATH redirects
the executable, LD_PRELOAD/PYTHONPATH/NODE_OPTIONS inject code,
PYTEST_ADDOPTS/GOFLAGS/MAKEFILES inject selection-changing inputs,
BASH_ENV runs arbitrary shell startup. All degrade to unprovable.
- runtime expansions: any span token carrying /$( )/backticks, any
negating-option value carrying an expansion or glob (unknown excluded
set), and any extra executed token carrying glob metacharacters
(crafted option-looking filenames narrow invisibly) are unprovable.
Criterion-side globs stay self-consistent (literal match).
- cd: an argument carrying a runtime expansion or glob is non-silent
(unknown destination, unknown print); CDPATH= assignments are now
handled as state pollution at the match layer, subsuming the
value-shape special case.
* fix(harness): persistent-shell evidence, exact env sets, option-arity scoping (RFC #4651 PR4)
- tests_passed: on a persistent-shell provider (new
Sandbox.persistent_shell_sessions capability, set by AioSandbox) every
leaf degrades to UNVERIFIED — any earlier call in the shared session
could have mutated the state the clean-looking run executed in, and
only a fresh controlled session (RFC section 6 verifier) can prove
otherwise. The flag is read from the provider registry without
acquiring a sandbox.
- env assignments: the allowlist is gone — no variable is provably inert
across repositories (CI/DEBUG are routinely read by tests). The span's
assignment prefix must equal the criterion's exactly (values included,
order-insensitive); any assignment or export NAME= in a preceding
segment is state pollution.
- scoping: positional targets are now read by option arity, so a path
embedded in an option (--basetemp=/tmp/p, --junitxml=/tmp/r.xml) never
counts as a selection target and an extra positional after such a
criterion narrows the default selection it denotes.
* fix(harness): stamp shell provenance at harvest, close export/unset and arity gaps (RFC #4651 PR4)
* fix(harness): split physical newlines as shell separators in acceptance matching (RFC #4651 PR4)
* fix(harness): scope cd wrappers to thread data roots, pin accepted boundaries (RFC #4651 PR4)
* fix(harness): preserve criterion connectors, prove file_written readable, fail-closed shell capability (RFC #4651 PR4)
* fix(harness): compare only the connector prefix, tolerate trailing criterion semicolons (RFC #4651 PR4)
* fix(harness): preserve continuation-line operators, keep ./-spelled executable identity (RFC #4651 PR4)
* fix(harness): render criteria single-line so a multiline criterion cannot inject a forged checklist line (RFC #4651 PR4)
* fix(harness): reject parent-traversal executable tokens in acceptance matching (RFC #4651 PR4)
* fix(harness): reject parent-traversal negated values in acceptance matching (RFC #4651 PR4)
Images older than all-in-one-sandbox 1.9.x have no /v1/bash/* routes, so
every env-bearing command (skills declaring required-secrets) surfaced a
raw nginx 404 that the model kept retrying. Detect the 404, remember the
capability gap per sandbox instance, and return an actionable error that
names the minimum image version and the sandbox.image remediation.
No fallback through the legacy shell path on purpose: /v1/shell/exec has
no env parameter, and every workaround puts the secret values back into
the command string or on disk — the exact leak surfaces the
request-scoped secrets design closed.
Closes#3921
* fix(sandbox): create shell session before retrying on a fresh id
The AIO sandbox recovery path generated a UUID and passed it straight to
exec_command(id=...). The sandbox image only auto-creates a session when
exec_command is called with *no* id; an exec carrying an unknown id returns
HTTP 404 "Session not found". So every ErrorObservation recovery itself
404'd, turning a transient session lapse into an unrecoverable tool error
that looped the run up to the LangGraph recursion limit.
Explicitly create_session(id=fresh_id) before targeting that id on retry.
create_session is idempotent (returns the existing session if the id already
exists), so this is safe under the serializing lock.
Updated the regression test to assert the retry targets exactly the
created session id rather than a fabricated, uncreated one.
* fix(sandbox): release the one-shot recovery session after retry
The fresh session created on the ErrorObservation recovery path is used for
exactly one command -- the next execute_command runs with no id and returns
to the default session. Under persistent session corruption every command
would create another session that is never reused or released, accumulating
sessions on the container.
Release it best-effort with cleanup_session() in a finally, swallowing any
cleanup error so it never masks a successful retry.
Addresses review feedback on #3577.
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* fix(sandbox): close AioSandbox HTTP client during provider teardown (#2872)
AioSandbox allocates a host-side agent_sandbox client (wrapping an
httpx.Client) in __init__, but AioSandboxProvider.release/destroy/shutdown
only popped provider state and tore down the backend container — the
client/transport owned by each cached AioSandbox was never explicitly
closed, accumulating unreclaimed sockets in long-running services.
- Add AioSandbox.close(): best-effort, idempotent close of the wrapped
httpx_client (falls back to top-level client.close()); errors are
logged but never raised so backend cleanup is never blocked.
- AioSandboxProvider.release()/destroy() now close the cached AioSandbox
before dropping it; shutdown() inherits this via destroy().
* fix(sandbox): close the real httpx.Client owned by AioSandbox (#2872)
The previous close() only walked one level (wrapper.httpx_client), which resolves to the Fern-generated HttpClient wrapper that has no close(). The real socket-owning httpx.Client lives one level deeper at _client_wrapper.httpx_client.httpx_client, so the close path never fired and host-side sockets still leaked.
Resolve the real httpx.Client with graceful degradation; clear self._client under the lock for use-after-close and concurrent double-close safety; mark provider release()/destroy() try/except as defense-in-depth; rewrite TestClose against the real nested structure to lock down the original no-op bug.
* fix(sandbox): pass no_change_timeout to exec_command to prevent 120s premature termination
The agent_sandbox library's shell API defaults no_change_timeout to 120
seconds. When AioSandbox.execute_command() called exec_command() without
this parameter, commands producing no output for 120s would return with
NO_CHANGE_TIMEOUT status even though the script was still running.
Pass no_change_timeout=600 to all exec_command calls (matching the
client-level HTTP timeout) so long-running commands are not cut short.
Fixes#2668
* test(sandbox): add assertions for no_change_timeout in execute_command and list_dir
Agent-Logs-Url: https://github.com/bytedance/deer-flow/sessions/2f37bc72-0826-4443-a6ba-e5b78c22fb5a
Co-authored-by: WillemJiang <219644+WillemJiang@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* fix: prevent concurrent subagent file write conflicts
Serialize same-path str_replace operations in sandbox tools
Guard AioSandbox write_file/update_file with the existing sandbox lock
Add regression tests for concurrent str_replace and append races
Verify with backend full tests and ruff lint checks
* fix(sandbox): Fix the concurrency issue of file operations on the same path in isolated sandboxes.
Ensure that different sandbox instances use independent locks for file operations on the same virtual path to avoid concurrency conflicts. Change the lock key from a single path to a composite key of (sandbox.id, path), and add tests to verify the concurrent safety of isolated sandboxes.
* feat(sandbox): Extract file operation lock logic to standalone module and fix concurrency issues
Extract file operation lock related logic from tools.py into a separate file_operation_lock.py module.
Fix data race issues during concurrent str_replace and write_file operations.
* fix(sandbox): serialize concurrent exec_command calls in AioSandbox
The AIO sandbox container maintains a single persistent shell session
that corrupts when multiple exec_command requests arrive concurrently
(e.g. when ToolNode issues parallel tool_calls). The corrupted session
returns 'ErrorObservation' strings as output, cascading into subsequent
commands.
Add a threading.Lock to AioSandbox to serialize shell commands. As a
secondary defense, detect ErrorObservation in output and retry with a
fresh session ID.
Fixes#1433
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(sandbox): address Copilot review findings
- Fix shell injection in list_dir: use shlex.quote(path) to escape
user-provided paths in the find command
- Narrow ErrorObservation retry condition from broad substring match
to the specific corruption signature to prevent false retries
- Improve test_lock_prevents_concurrent_execution: use threading.Barrier
to ensure all workers contend for the lock simultaneously
- Improve test_list_dir_uses_lock: assert lock.locked() is True during
exec_command to verify lock acquisition
* style: auto-format with ruff
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
