ac18f518c8
* feat(sandbox): add Tenki cloud sandbox provider Adds deerflow.community.tenki, a SandboxProvider backed by Tenki cloud microVMs, alongside the existing e2b_sandbox / boxlite / aio_sandbox backends. Selected via `sandbox.use: deerflow.community.tenki:TenkiSandboxProvider` (resolved by class path, so the change is purely additive). The full Sandbox contract is implemented — execute_command plus read/write/update/download_file and list_dir/glob/grep — with file ops run as busybox-portable shell commands (cat / find / grep / chunked base64), reusing deerflow.sandbox.search, mirroring e2b_sandbox and boxlite. Tenki's SDK is synchronous, so unlike boxlite there is no event-loop bridge. Tenki sandboxes run as an unprivileged user with /mnt root-owned, so the /mnt/user-data virtual prefix is remapped under the writable home dir (like e2b_sandbox); the provider also best-effort sudo-symlinks /mnt/user-data to that home dir so agent shell commands using the literal path still work. Sandboxes are pooled per (user, thread) with warm reclaim, a replica cap, and an idle reaper via the shared WarmPoolLifecycleMixin. Transient transport blips get one bounded retry; terminal session errors evict and recreate. Only the stable Tenki surface is used (create/terminate + exec/shell/fs) — no volumes, snapshots, or template builds — so any stock base image works. The tenki-sandbox SDK is an optional extra (deerflow-harness[tenki]) and is imported lazily, so a default install and every other provider are unaffected. Tested: unit suite runs in CI without tenki-sandbox installed; a live integration test and full-surface e2e were verified against real Tenki sandboxes. * fix(sandbox): remove unsafe auto-retry from Tenki exec Pre-merge review caught that the transient-transport retry sat at the universal _exec layer, so it retried every operation — execute_command and base64 file-write chunks included. gRPC has no exactly-once guarantee: a "socket closed" ack-drop after the server already ran the op means the retry runs it twice, double-firing command side effects and duplicating a write chunk mid-file (silent binary corruption on multi-chunk writes). exec is not idempotent, so it must not be auto-retried. Reverts to the boxlite/e2b behavior: a transient error surfaces to the caller (returned as text by execute_command, raised by the file ops); a terminal session error still evicts the sandbox so the next acquire rebuilds it. Verified live end-to-end across 31 edge cases (empty/binary/unicode/chunk-boundary files, shell-metachar content, error paths, list/glob/grep, warm-pool reclaim, concurrency). * fix(sandbox): address Tenki provider review feedback - Use Tenki's native sandbox.fs API for all file transport (read_text, read_bytes, write_stream, mkdir) instead of cat/chunked-base64 over shell. Uploads stream in 1 MiB frames; append is read-modify-write because the write stream has no append mode (same approach as community/e2b_sandbox). - download_file streams via fs.read_stream and enforces the 100 MB cap on bytes actually received, closing the TOCTOU window between the old wc -c size probe and the read. - list_dir/glob/grep report paths back under /mnt/user-data instead of the sandbox-internal home dir, so results feed straight into the file APIs. - Create with wait=False and await wait_ready() here: create(wait=True) raises with the session handle still inside the SDK, leaking a running microVM this provider could never terminate. - Configure the sandbox lifetime (max_duration, default 4h) and expose sticky; without it Tenki reaps a reused thread's sandbox after ~30 min. - close() terminates before marking the adapter closed and re-raises real failures, so a failed termination stays retryable instead of silently leaking a billed microVM; an already-gone session still counts as closed. - Bump the optional extra to tenki-sandbox>=0.4.0 and commit backend/uv.lock. * fix(sandbox): scope tenki grep() glob filter to its directory prefix Mirrors #4168, which fixed the same defect in the E2B provider. The tenki adapter reduced a directory-scoped pattern like "src/*.js" to its basename before filtering, so the search silently broadened to every matching-extension file in the tree. Post-filter grep's hits through path_matches() against the path relative to the search root, the same way glob() already does, so both agree on what a directory-scoped pattern means. * fix(sandbox): address Tenki provider review — eviction, id width, write lock, grep -H Four fixes from the upstream review: download_file no longer swallows terminal transport errors. The broad `except OSError: raise` re-raised ConnectionError/BrokenPipeError/EOFError (all OSError subclasses that _is_terminal_failure treats as terminal) before _note_failure ran, so a session that died mid-download was never evicted. Only our own EFBIG size-cap now passes through without eviction. Sandbox id widened from 32 to 64 bits (`[:8]` to `[:16]`), matching community/e2b_sandbox. The warm pool is keyed by this id with no full-seed fallback, so a collision could let one user reclaim another's parked sandbox on a multi-tenant gateway. _fs_op now holds the lock across the op, not just the fs lookup, so concurrent calls on the same sandbox serialise over the SDK's shared connection. The eviction callback runs after the lock is released to avoid a lock-order deadlock with the provider. The append read-modify-write is serialised by a dedicated _write_lock so two concurrent appends can't clobber each other. grep passes -H so a search whose path resolves to a single file still prints the filename; without it the file:line:text unpack dropped every match. * fix(sandbox): address Tenki provider review round 2 - validate config `environment` at load time (_validate_extra_env) so a bad key fails fast instead of surfacing as an SDK error mid-command - document the deliberate lock decision in download_file: the instance lock is dropped before streaming so a 100 MB download can't block every other tool; terminal transport errors still evict via _note_failure - tighten the terminal-error comment to note ConnectionError/BrokenPipeError/ EOFError are also treated terminal via isinstance - document TenkiSandboxProvider in backend/AGENTS.md (provider detail, warm-pool destroy hook, community provider list) - add a commented Tenki block to config.example.yaml for parity with AIO/BoxLite - tests: config env validation, grep -F/case-sensitive flags, glob include_dirs, list_dir max_depth, bootstrap-failure warning branch * fix(sandbox): make Tenki bootstrap non-interactive and time-bounded The create-time bootstrap runs under the per-scope acquire lock, so a hang would stall acquire for that scope indefinitely: - use `sudo -n` so a password-requiring sudoers entry fails fast (swallowed by the existing `|| true`) instead of blocking on a tty password prompt - pass a timeout to the bootstrap `remote.exec` so any other stall drops to the existing warning path rather than wedging acquire Best-effort by design; the file APIs still work via the home remap on failure. * test(sandbox): pin Tenki bootstrap timeout to its actual value Assert bootstrap["timeout"] == _BOOTSTRAP_TIMEOUT instead of `is not None`, so a regression to timeout=0 (treated as no timeout by some SDKs) or an unrelated value is caught rather than passing a weaker non-None check. --------- Co-authored-by: Willem Jiang <willem.jiang@gmail.com>
5.1 KiB
Tenki backend
Runs each DeerFlow sandbox as a Tenki cloud sandbox — an isolated microVM created from a stock base image, with no daemon or local virtualization to manage. A cloud-hosted alternative to the container-based AIO sandbox and the local-virtualization BoxLite backend.
Configuration
sandbox:
use: deerflow.community.tenki:TenkiSandboxProvider
api_key: $TENKI_API_KEY # falls back to TENKI_API_KEY / TENKI_AUTH_TOKEN env var
base_url: https://tenki.cloud # optional; SDK default when omitted
image: my-base-image # optional; Tenki account default base image when omitted
project_id: proj_... # optional; auto-selected if the account has exactly one
workspace_id: ws_... # optional; auto-selected if the account has exactly one
cpu_cores: 2 # optional per-sandbox vCPUs
memory_mb: 2048 # optional per-sandbox memory
replicas: 3 # active + warm microVM cap per gateway process (default: 3)
idle_timeout: 600 # warm microVM idle seconds before terminate; 0 disables
max_duration: 14400 # Tenki sandbox lifetime in seconds (default: 4h); 0 uses the account default
sticky: false # pin the microVM to its host (only matters with pause/resume)
home_dir: /home/tenki # writable dir backing /mnt/user-data (default: /home/tenki)
environment: # injected into every command (and as create-time env)
PYTHONUNBUFFERED: "1"
Install the optional SDK before selecting this provider:
pip install "deerflow-harness[tenki]"
The tenki-sandbox package is an optional DeerFlow harness extra, not part of
the default install. Get an API key from https://tenki.cloud/docs/sandbox/sdk.
Design
Tenki's Python SDK is synchronous, so — unlike BoxLite — the adapter calls the
SDK directly with no event-loop bridge. Sandboxes are named deterministically
from user_id:thread_id, released into an in-process warm pool after each agent
turn, and reclaimed (after a liveness health check) by the same thread on the
next acquire. A terminal session error evicts the sandbox and the next acquire
rebuilds it; other transport errors surface to the caller (exec is never
auto-retried — it is not idempotent, so re-running could double a command's side
effects). Sandboxes are created with wait=False and awaited via wait_ready()
so a readiness failure still leaves this provider holding the handle to
terminate, and with an explicit max_duration so a long-lived thread does not
lose its sandbox to Tenki's default lifetime mid-conversation.
| File | Role |
|---|---|
provider.py |
SandboxProvider lifecycle, warm pool, scope resolution |
sandbox.py |
Sandbox adapter; execute_command + file ops |
Contract coverage
The full Sandbox surface is implemented. File transport uses Tenki's native sandbox.fs
API; directory and content search shell out and reuse deerflow.sandbox.search,
mirroring e2b_sandbox:
execute_command—sh -lc, with per-call env and timeout.read_file/write_file/update_file— nativefs.read_text/fs.mkdir/fs.write_stream(binary-safe, streamed).download_file— nativefs.read_stream, restricted to the/mnt/user-dataprefix; the 100 MB cap is enforced on bytes actually received, so a file growing mid-transfer cannot slip past it.list_dir/glob/grep—find/grepwith busybox-portable flags (the fs API is single-level and has no content search); results filtered/capped in Python and reported back under/mnt/user-data.
Tenki sandboxes run as the unprivileged tenki user with /mnt root-owned, so
DeerFlow's /mnt/user-data virtual prefix is remapped under the writable
home_dir (like e2b_sandbox). The provider also best-effort sudo-symlinks
/mnt/user-data → home_dir at create time so agent shell commands using the
literal /mnt/... path still work; if sudo is unavailable the file APIs keep
working via the remap.
Warm-pool capacity is governed by sandbox.replicas across active + warm
sandboxes. sandbox.idle_timeout controls how long released warm sandboxes stay
running; 0 disables idle reaping. Active sandboxes are never evicted to satisfy
the cap.
Scope: stable features only
Only the stable Tenki surface is used — sandbox create/terminate plus exec/shell/filesystem over shell commands. Volumes, snapshots, and template builds are intentionally not used, so no prebaked image or unstable Tenki feature is required and any stock base image works.
Not yet implemented (follow-ups): cross-process orphan reconciliation (adopting sandboxes left by a previous gateway process) and a preview-URL surface.
Status
Verified end-to-end against live Tenki sandboxes: provider resolution →
execute_command → full file-op surface (read/write/update/download,
list_dir/glob/grep) → warm-pool reclaim → terminate, plus the
/mnt/user-data sudo symlink. Unit tests run in CI without tenki-sandbox
installed; test_integration_real_sandbox exercises a real microVM when
TENKI_API_KEY is set.