* feat(sandbox): share sandbox identity derivation and acquire serialization (#4741)
Remote providers (AIO, E2B, BoxLite, Tenki, OpenSandbox) each inlined the
same sha256(user:thread)[:16] sandbox-id expression and kept per-scope lock
dicts that grew unboundedly until shutdown. This extracts both mechanisms
into shared components without changing provider lifecycle, ids, capacity
semantics, or public tool behavior:
- sandbox/identity.py: keyword-only derive_sandbox_scope_token (byte-pinned
compatibility contract) + is_sandbox_scope_token; per-provider golden
vectors pin current behavior including BoxLite's raw-None quirk and each
provider's private user_id resolution.
- sandbox/acquire_serialization.py: AcquireSerializer — per-key lock table
with holder/waiter refcount reclamation, bounded dedicated executor
(async waits off both the event loop and the default executor),
worker-owned cancellation cleanup (no event-loop callback dependency), idempotent close().
- Each provider adopts both components; AIO/E2B key by (user_id, thread_id)
with acquire and (E2B) release serialized; BoxLite/Tenki/OpenSandbox key
by derived sandbox id and offload the whole sync acquire to the
serializer's executor so a cancelled awaiter cannot overlap a retried
same-scope body (leaked-remote-VM regression caught in review).
- thread_id=None acquires stay unserialized; provider shutdown()/reset()
close the serializer; E2B capacity/ledger/reconciliation and AIO
ownership/flock machinery untouched.
- blocking-IO anchor proves contended OpenSandbox acquire_async stays off
the event loop (teeth verified red/green); AGENTS.md documents the
shared components.
* refactor(sandbox): address review on acquire serialization (#5089)
- Replace unreachable checkin branch with an assertion: run() returns
False only after abandon(), which the except handler always re-raises;
the old _checkin would have double-decremented the refcount.
- Document the task.cancelling() == 0 assumption in hold_async.
- Drop unused thread_id/user_id kwargs from BoxLite and Tenki
_acquire_scope_locked (OpenSandbox still forwards them).
* fix(sandbox): preserve request ContextVars in acquire executor bridge (#5089)
loop.run_in_executor() does not copy contextvars, unlike the inherited
SandboxProvider.acquire_async() which used asyncio.to_thread(). The
BoxLite/OpenSandbox/Tenki acquire_async bridges introduced in this PR
therefore dropped the request trace id (logged as trace_id=-).
Add AcquireSerializer.run_on_executor(), which copies the calling
context and runs the callable through ctx.run, and route all three
providers through it. Add regression tests binding request_trace_context
and verifying the worker thread observes it.
* bench: add provider-agnostic sandbox benchmark with BoxLite warm pool results
- scripts/bench_sandbox_provider.py: CLI for measuring acquire/run/release
across providers, scenarios, workloads, and concurrency levels
- scripts/summarize_bench.py: JSONL aggregation with p50/p95/p99 tables
- bench_results.jsonl: 110 turns across 7 scenarios on real BoxLite 0.9.7
Key findings:
cold acquire: ~860ms
warm reclaim: ~14ms (60x speedup)
release: ~0ms
warm_hit_rate: 95% (warm_same_thread)
* perf(boxlite): skip health check for recently-released warm pool boxes
Boxes released within health_check_skip_seconds (default 5.0s)
are promoted directly without the ~14ms echo-ok round-trip.
A VM alive seconds ago is overwhelmingly likely to still be alive.
Add sandbox.health_check_skip_seconds config option.
Set to 0 to always health-check (old behaviour).
Benchmark (warm_same_thread, noop, 20 iters):
acquire p50: 14.9ms → 0.0ms
total p50: 29.9ms → 14.0ms
* chore: move benchmark scripts into backend/scripts/benchmark/
* fix: address BoxLite benchmark review findings
* fix(boxlite): only skip warm reclaim checks for released boxes
* fix(benchmark): keep BoxLite shim workaround off the event loop
* fix(boxlite): invalidate dead boxes from command path
* test(boxlite): cover skip window and invalidation edge cases
* fix(boxlite): treat sandbox-has-been-closed as terminal in _exec
* fix(boxlite): harden warm-pool reclaim and benchmark accounting
* fix(boxlite): validate warm-pool reclaims by default
* fix(config): expose boxlite health-check skip setting
* fix(boxlite): tighten failure classification and benchmark workaround
* Update config_version to 21 in values.yaml
---------
Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
* feat(sandbox): add BoxLite micro-VM sandbox provider (scaffold)
Community SandboxProvider backed by BoxLite, a daemonless OCI-native micro-VM
runtime. execute_command is wired end to end through a private asyncio loop that
bridges BoxLite's async SDK to DeerFlow's sync Sandbox contract; the remaining
Sandbox methods are stubbed pending approach review. Opt-in via sandbox.use
(pip install boxlite); a packaged [boxlite] extra will follow. Existing
providers are unchanged.
Refs #3936, #3439, #3213
* feat(sandbox): implement full Sandbox contract for BoxLite backend
Rename the community BoxLite integration to read as a compute backend rather
than "a sandbox": module boxlite_sandbox -> boxlite, BoxliteSandboxProvider ->
BoxliteProvider, BoxliteSandbox -> BoxliteBox.
Implement the file surface DeerFlow's default tool path assumes -- read_file,
write_file, update_file, download_file, list_dir, glob, grep -- as shell
commands inside the box (cat/find/grep/chunked base64), reusing
deerflow.sandbox.search and busybox-portable flags. Removes the reachable
NotImplementedError regression once the provider is selected. download_file
keeps the /mnt/user-data prefix + traversal guards; the provider materialises
those virtual dirs on box start.
Refs #3936, #3940
* fix(sandbox): resolve CI + review findings on BoxLite backend
- provider: import DEFAULT_SKILLS_CONTAINER_PATH instead of the "/mnt/skills"
literal (backend-unit-tests guard), and drop the redundant env-var
re-resolution -- AppConfig.resolve_env_variables already resolves $VARS and
raises on missing.
- box: grep now passes the raw pattern to grep (-F/-E); it previously handed
the re.escape'd pattern to grep -F, so a literal search of e.g. foo.bar
looked for foo\.bar and never matched.
- box: execute_command now calls _validate_extra_env(env), matching the
Sandbox POSIX env-key contract that the local/e2b/aio sandboxes enforce.
- add tests/test_boxlite_provider.py (CI-safe, no BoxLite): actionable
ImportError on the lazy import, clean acquire-failure + shutdown, the
traversal / download-prefix guards, and env-key rejection.
Refs #3936, #3940
